AI-Generated Vulnerability: Copilot Autofix Exposes Snowflake Internal Credentials

Wiz Research identified a critical script injection vulnerability in a Snowflake GitHub repository that was inadvertently created by an AI-powered Copilot autofix. The flaw allowed attackers to execute commands via issue titles and exfiltrate internal Jira credentials. Snowflake patched the issue immediately, highlighting the emerging risks of AI-generated code and the speed of AI-driven vulnerability discovery.

Key Points

GitHub Copilot Autofix introduced a security regression by replacing a safe code pattern with a vulnerable shell script interpolation.
The vulnerability allowed unauthenticated command execution via malicious GitHub issue titles, bypassing a flawed security gate condition.
Wiz's AI-powered Red Agent discovered and successfully exploited the vulnerability only five days after it was introduced.
The exploit resulted in the exfiltration of a Jira API token providing broad access to Snowflake's internal project management data.
The incident underscores the necessity of applying rigorous security scrutiny and static analysis to AI-generated code changes.

Sentiment

Skeptical and cautionary regarding the autonomy of AI in software development and security.

In Agreement

Human peer review of AI-generated changes is still essential to prevent security regressions.
AI models that are incapable of preventing a bug are likely also incapable of determining if a PR is 'minor' enough to skip review.
The specific vulnerability introduced was subtle and would not appear as a red flag to most reviewers without deep internal knowledge.

Opposed

Human reviewers cannot be fully relied upon to spot the significance of these subtle, AI-generated security flaws.
Automated tools like actionlint are more effective at catching these specific injection patterns than manual human review.
Multi-model cross-review could potentially mitigate these issues, though this is contested by those favoring human-only oversight.