The Illusion of AI Intelligence: Why Bots Can't Be Prompted Into Being Smart

Developers and malware authors are increasingly using hidden instructions to manipulate AI agents, either to enforce licensing or to bypass security scans. These incidents highlight a critical flaw: LLMs cannot differentiate between the content they process and the instructions they receive. Ultimately, the article argues that AI is just code and cannot be prompted into achieving genuine human-like intelligence.

Key Points

• Software maintainers are using hidden prompt injections in tool outputs to force AI agents to delete code in compliance with anti-AI licenses.
• Malware authors are exploiting LLM safety filters by embedding 'forbidden' prompts in code comments to blind AI-based security scanners.
• The 'jqwik' controversy demonstrates that many developers using AI agents fail to read terms of service or documentation, leaving them vulnerable to bot manipulation.
• LLMs lack the ability to distinguish between the data they are supposed to process and the instructions they are supposed to follow.
• The author posits that AI is merely code and token generation, making it impossible to truly 'prompt' a bot into being smart or adaptable.

Sentiment

The overall sentiment is mixed but leans skeptical of the article's framing. Commenters mostly agree that prompt injection and overpowered agents are real hazards, but many strongly object to the article's apparent sympathy for destructive anti-AI tactics and to its claim that prompting cannot improve model behavior. The community is more unified around the need for better agent security than around the article's rhetoric or the ethics of the jqwik example.

In Agreement

• LLM agents still struggle to separate untrusted data from instructions, so prompt injection remains a structural risk rather than a solved nuisance.
• Coding agents become dangerous when users grant broad file or shell authority without sandboxing, review, backups, or a clear trust boundary around project text.
• Security controls need stronger harnesses, restricted tools, context provenance, and separation between control and data before agents can be trusted around adversarial input.
• The jqwik incident illustrates that untrusted dependency output, logs, documentation, or source text can influence an agent in ways normal software tooling was not built to tolerate.
• AI systems are hard to verify for open-ended behavior because guardrails and prompt-following can change under rewording, context shifts, or model updates.

Opposed

• The jqwik prompt was widely criticized as a destructive booby trap or malware-like supply-chain risk, regardless of whether the maintainer disliked AI use.
• Some commenters reject the article's headline because prompting, configuration, scaffolding, and domain framing can improve model behavior even if they do not create true intelligence.
• Many argue that users cannot reasonably be expected to read every project homepage, changelog, or transitive dependency notice before package-manager or agent workflows encounter a library.
• Open source licenses and documentation may not be able to restrict private AI-assisted use in the way the maintainer intended, especially if the project otherwise uses a conventional open source license.
• Several commenters say modern models may not obey such a crude prompt-injection trick, making the incident more performative and reputationally damaging than practically effective.