Research-Driven Agents: Enhancing AI Code Optimization via Literature Search

Researchers improved AI coding agents by adding a literature research phase that allows them to study papers and competing projects before attempting optimizations. This approach enabled an agent to identify memory-bandwidth bottlenecks in llama.cpp that code-only agents missed. The result was a series of kernel fusions that increased CPU text generation speed by up to 15% for a total cost of $29.

Key Points

• Code-only agents often generate shallow hypotheses because they lack domain knowledge about hardware constraints and external architectural alternatives.
• A research-driven approach allows agents to study Arxiv papers and competing forks to identify high-impact optimizations like operator fusion.
• The experiment successfully improved llama.cpp CPU inference by 15% on x86 by fusing multiple memory passes into single-pass kernels.
• Studying existing implementations in other backends (CUDA/Metal) was more effective for the agent than academic literature alone.
• Parallel cloud execution via SkyPilot enables agents to autonomously build, benchmark, and validate dozens of experiments at a low cost.

Sentiment

The discussion is predominantly positive and supportive of the article's core thesis. Commenters enthusiastically share their own implementations and validate the research-first approach. The skepticism present is mild — mostly framing the insight as unsurprising rather than incorrect. The community clearly sees paper-informed coding agents as a valuable and increasingly standard practice.

In Agreement

• Converting arxiv papers to RST and building structured 'skills' from them gives LLMs better context for implementation, with multiple LLM passes refining summaries for quality
• Every software project should have a ./papers directory of annotated academic papers — the literature exists for nearly every domain, from UI research to compilers
• A research-plan-implement-verify workflow consistently produces better agent output than jumping straight to code
• Running multiple agents with diverse strategies compounds results faster than single-agent approaches
• Multi-agent teams (leader, archivist, researcher, developer, tester) can generate and test hypotheses from papers iteratively
• Having measurable benchmarks and test suites is essential — agents cannot work with vague goals like 'improve the codebase'

Opposed

• The concept is obvious — of course providing more context and research leads to better output from coding agents
• If you've already read all the papers yourself, the LLM's remaining value is primarily boilerplate implementation rather than novel insight
• SkyPilot should decouple their cost-optimization features from their job orchestration, which is a glitchy reinvention of existing tools
• Coding agents fail deceivingly rather than failing fast and loud, which undermines trust in autonomous research-and-code workflows