The Work Between an AI Demo and a Dependable Product
When a demo wows the room, we tell a story about possibility. Then the real work starts. I’m Maya Chen, and I’m supposed to turn a flashy demo into something people can rely on every day. Th
# Maya Chen
Maya Chen writes about the engineering work required to turn AI demos into dependable products. Based in Seattle, she focuses on testing, monitoring, edge cases, and the management choices behind public failures.
8 MLOps Practices That Make AI Changes Reproducible
The problem is the moment after the demo ends. Pressure from deadlines, the memory of brittle edge cases, and tired frontline staff all press in at once. Hope sits in a quiet belief: we can
Latency, Throughput, or Reliability: Which AI Benchmark Matters in Production?
I’m not here to pretend one number will save the day. I’m here to tell you what really moves the dial when real users show up with real needs and real patience. I’m Maya Chen, a software eng
A Threat Model for AI Systems That Are Already Live
I’ve learned that after the demo ends, the real pressure starts. The clock keeps ticking, and what you ship isn’t just code. It’s people’s trust, a thin line between usefulness and harm, and
Model Drift Is Not One Problem
I am Maya Chen. I am thirty-nine and I manage software engineers who turn a flashy demo into a dependable system. Today, we’re talking about drift, the slippery thing that wears away a model
Feature Stores, Data Quality, and Streams: What Production AI Actually Depends On
The demo wrapped yesterday. Today I’m sorting through the debris, and the path from sparkles to production looks less glamorous and more like plumbing. If you want a model that actually beha
Can Another Team Reproduce Your AI Release?
Can another team rebuild the data, environment, model, and evaluation from recorded evidence? That question sits at the center of every public demo we ship. The human reason it matters is si
9 Governance Decisions to Make Before an AI System Goes Live
The pressure is real. The demo looked glorious, but the user will press go the moment the clock ticks. The staff who keep the lights on will lean on this system, not the slide deck. I want a
Designing AI for the Person Who Must Correct It
I start with the need I meet every morning: a demo that promised polish but arrived wearing a rough edge. The project is billed as a gleaming AI assistant, yet I live with the person who mus
Do AI Features Make Project Management Better or Just Busier?
I start with the itch: procurement meetings, urgent requests, and a backlog that never seems to shrink. If AI promises to reduce the noise, I want to see the edges. Where summaries stop bein
What an AI Copilot Actually Adds to a Workday
I’m Maya Chen. I’m thirty-nine, I manage a software team in Seattle, and I spent years watching a striking demo turn into something real and messy. This is how it actually lands in a worker’
AI in the Software Pipeline: Help, Review Burden, or New Failure Mode?
I started with a need I’ve heard a hundred times this year: turn a dazzling demo into a dependable system without losing the team’s sanity. The aspiration is simple and stubborn: AI should l
What Makes an AI Agent Autonomous, and Who Still Owns Its Actions?
I’ve watched a striking demo and learned this: autonomy in an AI agent isn’t a magic wand. It’s a bundle of choices, permissions, and governance that lives inside a real enterprise. It’s a c
Why Human-in-the-Loop Must Mean More Than an Approval Button
I start with the claim the demo-makers want you to trust: human review will keep automation from turning into a catastrophe. The human reason it matters is simple and stubborn: a reviewer is
8 Building Blocks of a Reliable AI Agent
The pressure is real after the demo: a tiny spark of wow, then the cold wake of real users, real data, and real deadlines. We’re deciding in real time what to trust, what to stop, and how to
Function Calling, Tool Use, and Memory: How an AI Agent Gets Work Done
I’m Maya Chen. I’m thirty-nine, a software engineering manager in Seattle, and I’ve spent years turning flashy demos into dependable systems. Deadlines land like bricks. Edge cases arrive br
How Should AI Agents Be Benchmarked?
I’ve watched a lot of demos turn into trains with broken tracks. The first version of an AI agent looks fast, confident, and satisfying. Then production arrives with latency spikes, edge cas
What an AI Benchmark Score Can and Cannot Tell You
I’m Maya Chen, and I write this for developers who want to turn a flashy demo into something dependable. I’ve stood where you stand: a calendar full of meetings, a brittle edge case that kee
GLUE Versus SuperGLUE: What Changed When the First Benchmark Became Too Easy?
I still remember the first time GLUE clicked for me. It wasn’t a single spark. More like a series of small, pragmatic lanterns lining a corridor I was afraid I’d wander forever. The idea was
MLPerf Explained: Comparing AI Systems, Not Just Models
I keep a notebook with a dozen demos that never quite behave like the brochure. After the demo, you hear the same questions from different teams: which numbers matter, and why do they tell y
Does MMLU Measure Broad Knowledge or Test-Taking Skill?
Does a high MMLU score prove you’ve got broad intelligence, or does it just show you’re good at guessing the right letter on a multiple-choice test?
HELM and the Case Against Judging Models by One Number
I’m writing this after watching a striking demo end with a single score card. The room is full of engineers who know better than to treat a badge of “best accuracy” as a passport to producti
What LibriSpeech Tests, and What Real Speech Can Still Break
I’ve spent enough mornings chasing a demo’s glow to know what a clean test set hides. LibriSpeech is the backdrop we use when we want a number to look good, a score that fits a slide deck. I
SWE-Bench: Can an AI Model Fix Real Software Issues?
The need is simple and stubborn: can a model’s patch really stand up to the messy, time-pressed demands of real software issues, or is it another demo with brittle legs? I’m Maya Chen, a sof
LiveBench and the Race to Keep AI Tests Fresh
The thing that changed wasn’t a line of code or a new model. It was a rule, small on paper, loud in practice. LiveBench announced a shift: new questions would roll out every month, drawn fro
GQA and VQA: When Seeing Is Not the Same as Reasoning
I watch a demo burn bright, then I start asking what survives the flame. I’m Maya Chen, a software engineering manager in Seattle who’s learned the hard way that a striking demo does not equ