Context windows hit 1M+ tokens. Most teams are still building for 2022's constraints. Here's what to rebuild, what to throw out, and the architecture that actually scales.
Here is the problem with moving fast in AI: by the time something becomes conventional wisdom, the underlying reality has already changed.
For the last two years, the conventional wisdom on building AI products was this: models have small context windows, so you need to be clever about what you put in them. You build chunking strategies. You build vector databases. You build elaborate retrieval pipelines that decide, at runtime, which pieces of your data a model is allowed to see. You hire prompt engineers to squeeze meaning into 8,000 tokens.
That was reasonable advice — in 2022. It's actively wrong in 2026.
Gemini 2.5 Pro handles 2 million tokens. Claude handles 200,000. GPT-4o handles 128,000. The "context window problem" that shaped an entire generation of AI architecture has been, quietly and without much fanfare, solved.
The irony is that the teams who were fastest to adopt RAG pipelines in 2023 are now the ones most burdened by them. They have entire systems — chunkers, embedders, retrievers, rerankers — that exist solely to work around a constraint that no longer exists.
The shift isn't just quantitative. It's qualitative. When your context window is 8k tokens, you have to choose what to show the model. When it's 2M tokens, you can show it everything — the entire codebase, the full conversation history, every policy document, all the user data.
This changes the nature of the problem from retrieval to attention. You no longer need to ask "which chunk of the document is most relevant?" You need to ask "given everything, what matters?"
Models are, it turns out, quite good at the second question. They are much worse at the first — and yet that's what we spent three years optimizing for.
1. Your chunking strategy. If you're breaking documents into 512-token chunks because that was the best practice in 2023, stop and ask why. For most use cases, passing entire documents is now cheaper, more accurate, and easier to maintain.
2. Your vector database. Vector databases are genuinely useful — for semantic search, for finding things in large corpora, for building recommendation features. But if your only use case is "give the model relevant context," you may have an expensive solution to a problem that's been solved upstream.
3. Your latency assumptions. Long-context calls cost more and take longer than short ones. This is real. But it's a straightforward trade-off, not a fundamental architecture constraint. Run the numbers for your specific use case before deciding you need the full retrieval stack.
The teams winning right now are the ones who defaulted to "put it all in context" early and only added complexity when they hit a real, measured problem. They shipped faster and their products work better. The lesson isn't subtle.
Claude Code is Anthropic's command-line coding assistant, and it works differently than every other AI coding tool I've used. Where Copilot and most chat-based tools operate on the file you're currently in, Claude Code reads your entire project — directory structure, git history, dependencies, test files, the works.
The practical difference: when you ask it to add a feature, it doesn't just generate plausible code — it generates code that actually fits your existing patterns, doesn't break your tests, and references the right internal utilities. It can also run terminal commands, edit multiple files at once, and work iteratively on complex tasks with minimal hand-holding.
The thing nobody warns you about: it's fast enough that you start delegating tasks that feel too big to delegate. That's both the appeal and the risk. The quality of its output scales with the quality of your project structure — messy codebases get messy suggestions.
The paper asks a deceptively simple question: if you have a fixed compute budget for a language model, is it better spent on training or on inference? The answer, it turns out, depends heavily on the difficulty of the task — but for hard problems, spending more compute at inference time (letting the model "think longer") consistently outperforms simply using a bigger model.
This is the intellectual foundation behind OpenAI's o1 and o3 models, and the broader "reasoning model" category. The key mechanism is best-of-N sampling with a verifier: generate many candidate answers, then use a separate model to score and select the best one. It sounds obvious in hindsight. Most good ideas do.
This week's prompt turns a vague feature request into a structured engineering spec. Paste this into any capable model with your rough feature idea, and you get back something you can actually hand to a developer — or use to scope your own work.
You are a senior product engineer. I will describe a feature I want to build. Your job is to turn my rough description into a precise engineering specification. For the feature I describe, produce: 1. ONE SENTENCE DEFINITION What is this feature, precisely? 2. USER STORY As a [user type], I want to [action] so that [outcome]. 3. FUNCTIONAL REQUIREMENTS (numbered list) What must the feature do? Be specific enough that a developer can implement it without asking questions. 4. EDGE CASES TO HANDLE (numbered list) What can go wrong? What states need to be handled explicitly? 5. OUT OF SCOPE (numbered list) What are we explicitly NOT building in this version? 6. OPEN QUESTIONS (numbered list) What decisions still need to be made before implementation can start? Be direct. Use plain language. If my description is too vague to specify properly, ask me one clarifying question before proceeding. Feature to specify: [PASTE YOUR FEATURE IDEA HERE]
Best used at the start of any new feature, before writing a line of code. The "out of scope" section alone will save you hours of scope creep.
New issue every weekday at 7am. Free forever.
No spam. Unsubscribe in one click.