Engineering, product and AI-systems notes from work we're actually shipping.
OSWorld scores for computer-use agents climbed from 12% to the high 70s in under two years. That number has almost nothing to do with whether one survives your site's SSO prompt, CAPTCHA, or last redesign.
Read post →Most maintenance retainers guarantee how fast someone replies to a ticket, not how fast your problem gets fixed. Here's what to pin down before you sign, not after your checkout page goes down.
Read post →Anthropic draws a clean technical line between a workflow and an agent. Almost no vendor pitch tells you which one you're actually buying, and the difference changes what it costs and how it fails.
Read post →Most software quotes price the new dashboard or portal in detail and handle 'integration' in a sentence. That sentence is where your timeline and your first change order come from.
Read post →OpenAI stopped reporting SWE-bench Verified in February 2026, citing contamination it helped popularize. Here's what that means for anyone evaluating a coding agent on a leaderboard score instead of their own codebase.
Read post →A vendor who insists on going straight to the full build is usually protecting their estimate, not your budget. Here's how to structure a pilot that can actually fail — and what it should cost.
Read post →Structured outputs and constrained decoding killed malformed JSON in tool calls. They did nothing for the failure mode that actually costs you: well-formed calls that skip the right tool, ignore the result, or hit the wrong target.
Read post →Every software project accumulates new requests once people start using the real thing. The projects that blow their budget aren't the ones with changes — they're the ones with no agreed process for pricing and approving them.
Read post →The dominant failure in multi-step tool-calling agents isn't picking the wrong function, it's acting on a value it never actually had. Here's where to put the one approval gate that matters instead of gating everything or nothing.
Read post →Technical success and adoption are two different risks, and most project briefs only scope the first one. Here's how to put a usage target, not just a delivery date, into the plan before the build starts.
Read post →Multi-step agents resend the same system prompt and tool definitions on every call. Prompt caching can cut that cost 60-90%, but only if you structure the prompt right and know the model's minimum cacheable length.
Read post →Code ownership clauses get negotiated. Data export clauses usually don't — which is how schools and SMBs end up locked into a system they want to leave, paying their old vendor to get their own records back.
Read post →A bigger context window makes an agent look like it remembers more. It doesn't — and the gap between those two things is where support bots start contradicting themselves and ops agents start acting on stale facts.
Read post →The pricing model you pick for a software contract isn't a formality — it determines who absorbs the cost when requirements change, and most SMBs pick the wrong one for their actual situation.
Read post →Your agent cleared every eval and still did the wrong thing in production. Here's why eval scores and execution traces answer completely different questions, and what to actually instrument before you need them.
Read post →Most 'replace the old system' projects are really trying to fix one or two broken pieces. Here's the cheaper, lower-risk path, and the actual signal that tells you when a full rebuild is worth it.
Read post →Context windows grew faster than anyone expected, and the pitch to just paste the whole knowledge base into the prompt is tempting. Here's the actual tradeoff, including the research that explains why bigger context isn't the same as better recall.
Read post →Most school and institutional RFPs are written by whoever has to sign the contract, not whoever has to use the software — and vendors can tell. Here's how to write one that produces comparable, honest bids instead of polished guesswork.
Read post →Teams spend weeks benchmarking OpenAI against Voyage against Cohere while their documents get split in ways that sever the context any model needs. Here's what the research on chunking, contextual retrieval, and late chunking actually shows.
Read post →FERPA compliance doesn't live in a vendor's marketing page — it lives in contract clauses about training data, deletion, and control that most schools never negotiate. Here's exactly what to put in writing before you sign.
Read post →GitHub's own MCP server lets a public issue leak your private repos — and researchers called it architectural, not a bug. Here's the three-permission pattern behind it, and how to design agents so no model update has to save you.
Read post →JPMorgan's multi-billion-dollar London Whale losses and a famous austerity-economics paper were both undone by the same thing: an unreviewed Excel formula nobody double-checked. Here's how to tell if your business has the same exposure.
Read post →The benchmark numbers that matter for agent reliability aren't pass@1 success rates from a single demo run — they're what happens when you run the exact same task five times in a row.
Read post →SaaS sprawl isn't a subscription-fatigue problem, it's a duplicate-data-entry problem. Here's how to tell when the reconciliation labor between tools costs more than building one system would.
Read post →Most RAG builds stop at vector-only retrieval and call it done. Hybrid search and reranking are well-evidenced, inexpensive additions that cut retrieval failure rates sharply -- most teams just never get past step one.
Read post →Airtable, Zapier, and Glide get an SMB running in a week. The failure mode isn't that they break — it's that teams don't notice they've hit the ceiling until the migration off them is already expensive.
Read post →Clients keep asking us to fine-tune a model on their documents when what they actually have is a search problem. Here's the knowledge-vs-behavior question that decides RAG vs. fine-tuning before you spend a cent on either.
Read post →Every custom dashboard quote shows you the build cost. Almost none show you the three to five years of hosting, patching, and change requests that follow — and that's usually where the real spend, and the real build-vs-buy decision, actually lives.
Read post →Anthropic's own docs list four breaking changes in the Claude 5.5 release, from forced tool calls to thinking you can no longer disable, the kind of change a capability benchmark will never catch. Here's what pinning model versions actually buys you.
Read post →Wix and Squarespace are the right call for most small-business and school sites, until a specific integration, workflow, or performance ceiling makes the monthly fee a slow tax instead of a bargain. Here are the exact signals that tell you which stage you are in.
Read post →Anthropic's own data shows multi-agent orchestration beats a single agent by 90% on research tasks, but burns 15x the tokens of a plain chat. Here is the actual math for deciding when that trade is worth it.
Read post →DOJ just pushed the federal web accessibility deadline for public schools and colleges back a year, to 2027 and 2028. The standard didn't change, the exposure didn't change, and the easiest fix on the market doesn't actually satisfy it.
Read post →pgvector handles most RAG workloads cleanly into the millions of vectors. Here is the actual scale where a dedicated vector database earns its cost, and why the embedding model you pick matters more than either choice.
Read post →You paid for the dashboard, the portal, or the booking system. That doesn't mean you can take it to another developer. Here's the ownership language that decides whether you control your own software or rent it forever.
Read post →A randomized controlled trial found experienced developers were 19% slower with AI tools despite predicting they'd be 24% faster. Here's what that and the 2025 DORA report actually mean for how an engineering team should adopt AI coding assistants.
Read post →Portfolios and pitch decks tell you who's good at pitching. Here's what actually predicts whether a dev studio or AI vendor will deliver: how they price, how they talk about risk, and what they say when you ask the question they'd rather skip.
Read post →Wordsmithing a better prompt stops helping once your agent has to juggle tool outputs, retrieved documents, and twelve turns of history. Here's what actually breaks, and the three techniques production agent teams use to fix it.
Read post →Off-the-shelf software looks cheaper until per-seat pricing, integration glue, and workaround spreadsheets eat the savings. Here's how to tell which way your tool should go.
Read post →A demo that impresses a room tells you almost nothing about whether an agent will survive production. Here is the testing layer most teams skip, and what to build instead.
Read post →A website tells people about your business. A web app lets them do something inside it. Most SMBs don't realize they've crossed that line until they're paying a developer to hand-edit a spreadsheet every week.
Read post →MCP went from a November 2024 Anthropic spec to the plumbing under most agentic AI products in about a year, adopted by OpenAI, Google, and Microsoft. It also introduced a documented, already-exploited class of vulnerability: poisoned tool descriptions that hijack an agent without you seeing a thing.
Read post →Schools keep briefing one vendor for 'a new website' and discovering mid-project that grade access, fee payments, and staff logins were a second, much bigger product hiding inside the same request. Here's how to split the two before you scope either.
Read post →Naive RAG pipelines pass every demo on twenty documents and quietly fail on twenty thousand. Here's what actually breaks — retrieval misses, contradicting sources, lost-in-the-middle context — and what fixes hold up.
Read post →Most budget overruns don't start with a scope-creepy client or a padded estimate — they start with a brief that described an outcome instead of a system. Here are the five questions that fix it before you request a quote.
Read post →Gartner predicts over 40% of agentic AI projects will be canceled by 2027. Most of the survivors never needed an agent — they needed a well-built automation with one decision point.
Read post →Before you price a dashboard against a spreadsheet's free, price it against what the spreadsheet already costs you in reconciliation time, version sprawl, and audit risk.
Read post →