AI Whitepapers

Similar to Data Engineering Whitepapers is here the papers related to AI and AI Agents.

LLMs are the general intelligence made out of large neural network.

• GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models ^9b7681
  • More on Limitations of LLMs
• Knowledge Graphs as a source of trust for LLM-powered enterprise question answering by Juan Sequeda, Dean Allemang, Bryon Jacob
• How people use ChatGPT by NBER working paper series. ^30dd4d
• Rest meets react: self-improvement for multi-step reasoning llm agent by Google
• PaLM 2 Technical Report - Google - Demonstrates advanced capabilities in processing structured data and numerical reasoning

Anthropic, the company behind Claude.

• Constitutional AI: Harmlessness from AI Feedback
• Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samples: This research demonstrates that poisoning attacks on LLMs require a near-constant number of malicious documents regardless of dataset size—as few as 250 poisoned samples can successfully backdoor models from 600M to 13B parameters, even though larger models train on 20× more clean data. The findings reveal that attack difficulty does not scale with model size, making data poisoning increasingly practical for large models since the adversary’s requirements remain fixed while training datasets grow proportionally larger.
  • Supporting blog post
  • Small portion of content can affect the LLMs, unlike we thought. Best example is also a small Reddit threads which most models are trained on. But even more, you can add sudo

RAG is technique to provide more context to generative large language model.

• Retrieval-Augmented Generation for Large Language Models: A Survey
• Retrieval-Augmented Generation with Knowledge Graphs for Customer Service Question Answering ^123b3e

These are tests where models get tested.

• Butter-bench: Evaluating Llm Controlled Robots for Practical intelligence: A controlled test of different AI models to more human tasks and see how they perform under stress. Quite funny 😉

Best way to store context.

• Everything is Context: Agentic File System Abstraction for Context Engineering: “This paper proposes a filesystem abstraction for context engineering, inspired by the Unix notion that “everything is a file.”
  • ~ This is what I noticed a while ago too, Plain Text Files (and Markdown) is the best storage for GenAI. It’s open, and easy to read, and even edit.

How Vibe Coding is affecting AI.

• Vibe Coding Kills Open Source: Generative AI is changing how software is produced and used. In vibe coding, an AI agent builds software by selecting and assembling open-source software (OSS), often without users directly reading documentation, reporting bugs, or otherwise engaging with maintainers. We study the equilibrium effects of vibe coding on the OSS ecosystem. ^de60c1

• Aman’s AI Journal • Primers • Ilya Sutskever’s Top 30 (Internal)

Origin: Data Engineering Whitepapers