Wednesday, December 25, 2024

Why You Have to Fact Check AI

Today's AI can't think any more than a well-trained parrot responds to a spoken word. However, explaining how AI works is not my intent. It would take too long. I'll simply say that an AI's ability to give a good answer is directly tied to the amount of information it has been given on a topic. If you ask it a question on a topic that has had little written about it, you need to fact-check AI to make sure the answer is accurate.

To help you understand just a wee bit better, here's a little experiment I did to illustrate what I mean. I asked ChatGPT to "Analyze and explain the song, Hotel California by the Eagles," a topic that thousands of people have written about and millions have pondered over the years.

Then I asked it to "Analyze the song Armitage Shanks by Chuck Brodsky." Chuck is well-known on the folk circuit, but he has intentionally avoided being commercial. As a result, while his songs and lyrics can easily be found online, his songs are not analyzed in the same way as Hotel California.

I'm going to assume you are familiar with Hotel California, but if you lived under a rock or are a young whipper-snapper who isn't familiar with it, here's a YouTube link. Watch it and then read what ChatGPT had to say about it.

"Hotel California" by the Eagles is a classic rock song from the band's 1976 album of the same name. This song is renowned for its rich, evocative lyrics and haunting melody. It narrates a surreal and somewhat allegorical tale of a weary traveler who checks into a luxurious hotel, only to find that it's a place from which he can never leave. The chorus "You can check out any time you like, but you can never leave" suggests themes of excess, entrapment, and the dark side of the American dream. The hotel itself could be seen as a metaphor for a variety of things, including addiction, hedonism, or the pitfalls of fame and fortune. The layered guitar solos and the overall mood of the song contribute to its enigmatic and slightly ominous feel, which has led to much fan speculation and interpretation over the decades. 

Now here's a YouTube link to Armitage Shanks, and what ChatGPT had to say about it. After watching the video, you will understand why it is important to fact-check AI when you ask a question that is not well-covered in print. Watch it, and I know you'll understand why you have to question what AI returns and I know you'll get a chuckle along the way.

"Armitage Shanks" by Chuck Brodsky tells a story from a very different perspective and style, typical of Brodsky's folk and Americana influences. The song is about a man who is in the bathroom contemplating life while looking at the brand name on a urinal, which happens to be 'Armitage Shanks'—a real British manufacturer of bathroom fixtures. The song uses this mundane moment to delve into deeper reflections on life and identity, making a seemingly trivial detail into a symbol of the character's existential contemplation. Brodsky's lyrics often explore themes of human experiences, irony, and the humor found in everyday situations, and this song is a prime example of his storytelling ability.

This is what's call an AI hallucination. ChatGPT had no way to ANALYZE either song. However, it could access tons of articles that analyze Hotel California and cobble together what appears to be an analysis. With Armitage Shanks, it didn't have that volume of information on which to draw, so it simply hallucinated. Be VERY careful using AI for anything of importance. Seeking financial or medical advice or information from AI is not something one should do without appropriate training.