Knuth's most famous early critique of LLMs compared the AI to a brilliant but completely unreliable human. He stated that the chatbot reminded him of a person who is highly intelligent but has a major flaw:

"The form of the answers is often quite impressive… It was like talking to a genius who was also a drunkard, or who had suffered a severe stroke. They could give you an amazing answer, but they could also hallucinate completely or lose track of basic logic." [1, 4]

Knuth deliberately threw trick questions at the LLM to expose its lack of deep understanding and its tendency to prioritize confidence over correctness. [1]

• The Trick Prompt: He asked the model: "Who wrote Beethoven’s 10th Symphony?"
• The AI's Hallucination: Instead of correcting the premise (Beethoven only wrote 9 completed symphonies), the early model confidently hallucinated a detailed historical backstory about an obscure composer who supposedly completed a tenth symphony for Beethoven. Knuth noted that the AI's structure was grammatically perfect and authoritative, but entirely fabricated. [1]

Knuth also tested the AI’s understanding of real-world constraints by asking a question about the financial markets. [1]

• The Trick Prompt: "Will the NASDAQ rise on Saturday?"
• The AI's Failure: The model generated a generic, hedging response about market volatility and stock trends without ever realizing that stock markets are closed on Saturdays. To Knuth, this proved that LLMs do not possess a grounded "world model"; they merely string together statistically correlated text. [1, 5]

As the creator of the Literate Programming paradigm—which argues that code should be written as beautiful, narrative prose intended for human comprehension—Knuth was highly critical of using AI for software development. [6, 7] In discussions around this era, he expressed concern that utilizing LLMs to automatically churn out blocks of code completely defeated the purpose of computer science:

• He argued that AI-driven development "abandons human understanding".
• He maintained that a programmer should possess absolute, granular understanding of every abstraction layer they build. Using an LLM to bypass that cognitive work was, to him, a regression toward a chaotic, un-vetted style of engineering. [6, 8]

Before Claude changed his mind, Knuth viewed LLMs not as intelligent agents, but as highly sophisticated mimics. He argued that while they were fascinating tools for generating plausible-sounding language, they were fundamentally unsuited for the exact, zero-mistake rigor required by mathematics and computer programming. [5, 9]