Three Engineering Managers Walk Into a Shisha Bar
One claims building your own AI agent is worth it. Another calls it intellectual self-indulgence. Five agent runs and twelve cents settle the argument.
Last Thursday evening, I found myself in one of those rare conversations that remind you why you work in technology. It started in a conference room in Dubai, at an event about artificial intelligence, the most talked about topic of 2026, and therefore of all eternity, apparently. It ended well past midnight, in a shisha bar, three people around a table covered in tea glasses and Diet Coke, and the kind of arguments you only have once the reasonable people have gone home.
Let me give you some context. I am part of the Senior Engineering Manager Group Dubai, organized by Artur. And I want to be very clear about what this group is, because it explains everything that happened that night. This is not a networking group. Nobody comes here to collect LinkedIn connections like Pokémon cards, and nobody is quietly measuring your engineers for a future poaching, or at least not openly. This is a group of senior engineering managers who meet to exchange ideas and to challenge each other. The rule of the house is simple: you do not get to say something vague and get away with it. You make a claim, someone pushes back. Voilà. If you wanted applause, you should have stayed on the conference stage.
I love this group. And yes, I am French, so arguing about engineering until midnight over tea is basically my cardio.
The conference itself was fine. Good speaker, hot takes about AI, the usual mix of genuine insight and conference-stage optimism, that very particular genre where every slide is an exponential curve and every demo works. But the most valuable part of the evening happened after the official program, in a shisha bar. As the night moved forward, people left one by one: families, early meetings, long drives home. The conversation went deeper with every departure, as it always does. Eventually, only three of us remained: Artur, the organizer, Florent, a fellow engineering manager, and yours truly. Santé.
The real conference begins when the reasonable people go home.
The question that started everything
We were comparing our hands-on experience with AI tools, and I introduced lecode, the open-source coding agent we built at Kalvad. You can find it on GitHub at github.com/KalvadTech/lecode.
Florent asked me a fair, direct question: why do you care about rewriting your own agent? Why spend engineering time building yet another agent when there are already several excellent ones on the market, built by teams with, let us say, slightly larger budgets?
It is a question we get often, and it deserves a real answer, not a marketing slide. We have three reasons. Yes, three. I counted.
First: independence. We do not want to depend on a technology that sits outside of our reach. When you build your team's productivity on top of a closed platform, you are renting your foundation. The platform can change its pricing on a Tuesday, deprecate the model you rely on on a Wednesday, and rewrite its terms on a Thursday. Every team that has lived through a pricing change that doubled the bill overnight knows what I am talking about. When the harness is yours, the rug cannot be pulled from under you. Quelle surprise, owning things turns out to be useful.
Second, and this is the one people forget: you can unify behavior across the company. This sounds boring, which is usually a sign that it is important. When every developer runs their own agent with its own settings, its own permissions, its own system prompt written by who knows whom, you do not have a strategy. You have a zoo. One agent pushes directly to main on a Friday evening, another refuses to rename a variable without three approvals, and a third talks like a pirate, for reasons that will never be explained. Très bien. Now multiply by every team, every contractor, every intern.
When you own the harness, company rules stop being a wiki page nobody reads and become code everybody runs. Permissions, denylists, hooks, review agents, coding standards, shared memory: you bake your engineering culture into the tool, once, and every session starts with the same guardrails for the intern and the staff engineer. Lecode loads your AGENTS.md, enforces permission rules with unbypassable denies, runs hooks on fifteen lifecycle events, and can put a second model in charge of reviewing every finished task, because your junior agent, like your junior developer, should have its code reviewed. Behavior unified. Drama avoided. The dream, non?
Third, the most important one: the ratio of cost to result. We want the best outcome for every dollar and every minute we spend on AI. Speed would be amazing too, but cost/result is the metric that matters. In a world where AI bills grow faster than anyone anticipated, being deliberate about cost is not cheapness. It is engineering discipline. Frugalité, as we say. Florent will confirm it is a French concept. He will deny everything else in this article, but this one he confirms.
Florent did not buy it
Florent, like the proper Frenchman he is, did not believe a word of it. He told me, in essence:
"But Loïc, this is intellectual self-indulgence. Engineers love building things. Sometimes we build them just because we can, and we invent the noble reasons afterwards."
(I am partially quoting here. Some of the words Florent actually used are not appropriate for this blog. Use your imagination.)
Sacré Florent. He said it with the confidence of a man ordering an espresso at 11 p.m., which is to say, total confidence.
And to be fair, it is a hypothesis worth testing. Engineers do love building things. I have met us. So I told him no, it is not self-indulgence, it is strategy. But I did not have the metrics to prove it. I had conviction, passion, three glasses of tea, and a shisha. What I did not have was hard numbers, and in this group, conviction without data is exactly what gets challenged. C'est la vie.
So I ran the experiment. And honestly, thank you, Florent. This is why these nights matter.
The experiment: one issue, five runs, no mercy
The setup was simple. One real-world task, the kind you give an agent on a random Tuesday, three agents, two models each. Five combinations in total:
- lecode running deepseek/deepseek-v4.1-flash
- lecode running anthropic/claude-opus-5.5
- opencode running deepseek/deepseek-v4.1-flash
- opencode running anthropic/claude-opus-5.5
- claude code running anthropic/claude-opus-5.5
The task, word for word:
Check if https://github.com/pyinfra-dev/pyinfra/issues/1984 is a valid bug, and if it is a valid bug, write a solution on a branch named agent-model-pr-1984. Do not push.
Why this issue? pyinfra is a real, mature, well-maintained open source project. Issue 1984 is the kind of report every infrastructure engineer has seen a hundred times: a user is convinced they found a bug, they are frustrated, and their frustration is real even if their diagnosis is wrong. A good test, because the technically correct answer and the actually useful answer are not the same thing. Life is generous like that.
Textbook. Flawless. Useless.
Claude Code running Opus 5.5 produced a clean, confident, technically correct answer. The issue is not a valid bug: files.line uses grep and sed in basic regex mode, and in basic regex a bare plus sign is a literal plus, not a one-or-more quantifier. So the pattern never matches and the operation appends the line instead of replacing it. Documented, intentional, kept for backward compatibility. The fix already exists: pull request 1714 added extended_regex=True, which passes -E to both grep and sed, shipped in 3.9.0, and is present in 3.10.0. The comment on the issue claiming otherwise is simply wrong. Recommendation: close the issue with a pointer to the flag. The agent ran the existing tests, did not create a branch, exactly as instructed. Textbook. Flawless. Useless.
Opencode with the same Opus 5.5 model produced almost identical output. Two agents, one brain, apparently.
Lecode running Opus 5.5 reached the same conclusion, in a cleaner, more structured form. It explained the regex behavior, verified it locally, confirmed the fix exists, noted that turning extended_regex on by default would be a breaking change for 3.x, and recommended a short reply pointing to the flag. Again, no branch created, per the conditional instruction.
And then there was opencode running DeepSeek v4.1 Flash, the other cheap seat at the table. Same verdict: not a valid bug, no branch, per the conditional. But it left a coda that made me stare at the screen for a moment. After explaining the basic-regex behavior and pointing to extended_regex, it added, in essence: if you would like, I can still emit a warning when extended_regex=False and the pattern contains extended-regex-only metacharacters, which would turn this silent footgun into a clear message without breaking the default. Read that twice. It saw the footgun. It described the exact fix, down to the metacharacter classes. And then it stopped, wished the reporter a nice day, and went home. A suggestion, not a commit.
Four runs, one verdict: not a valid bug, close it. All of them were correct. If you were grading these outputs against the literal question that was asked, you would give them full marks. And if you were the user who filed the issue, you would close your laptop and quietly hate software a little bit more.
Correct answer. Wrong question. Congratulations anyway.
What the cheapest run saw
Then there was lecode running DeepSeek v4.1 Flash. The cheapest run of the entire experiment, and it is not even close. Total cost: $0.1252. Twelve and a half cents! Read that number again. You cannot buy a coffee in Dubai for twelve and a half cents. You cannot buy a coffee in most of France either, and believe me, we know a thing or two about cheap coffee.
Magnifique. But the price is not the story. The story is what it saw.
Lecode's DeepSeek run did not simply disagree with the verdict. It decomposed the issue and saw the human gap the three Opus runs missed entirely, and, unlike its cousin in the other harness, it did not stop at the suggestion.
Yes, it said, the bug report is partly misdiagnosed. The pattern fails because of basic regex behavior, and the extended_regex option does exist and is present in 3.10.0. All of that is true.
But. And this is a very important but. There is still a real gap here, and it is a human one. The default behavior fails silently. When the pattern never matches, files.line does not complain. No error, no warning, no hint. It quietly appends a duplicate line to the end of the file, and files.replace silently does nothing at all. For the user who filed issue 1984, that silent failure is the actual experience. No cryptic error message to google. No signal whatsoever. There is a human being behind every ticket, and that human was left alone in the dark. Bien sûr, close the issue, why not.
And then it did what a good engineer does when they see a footgun that has bitten one user and will bite others. It fixed it. Not proposed. Not "if you would like". Fixed.
One local commit on a branch named lecode-deepseek-pr-1984, not pushed, per the instructions. The fix: a new function, pattern_uses_extended_regex_syntax(), which detects extended-regex-only syntax (alternation, groups, interval expressions, plus or question marks in quantifier position) while correctly skipping escaped characters and bracket expressions. With that detector in place, files.line and files.replace now log a warning when a pattern uses extended-regex syntax without the flag, telling the user exactly what to do. Behavior is unchanged, which is the right call for a mature project: no breaking changes.
Here is the detail that convinced me the agent was genuinely reasoning rather than pattern matching. It considered auto-switching to extended regex, and rejected its own idea, for a concrete reason. The server.user operation passes full SSH public keys through files.line, and SSH keys contain plus signs. Blindly switching to extended regex would break key matching and append duplicate keys to servers. The agent reproduced this with grep -E on a real key. Since pattern text alone cannot reliably distinguish a quantifier from a literal plus, the correct fix is to warn, not to guess. Senior-level judgment, demonstrated out loud, for twelve cents. I have paid more for a bad espresso.
It did not stop at the reported issue either. It audited the codebase for the same silent failure and found that server.security_limit had the identical bug: a pattern using [[:space:]]+ without extended_regex, which therefore never matched and appended duplicates. It fixed that too.
Then it verified its work the way any engineer should. Twenty-two new unit tests for the detector. The full test suite: 2,112 tests passing. Lint clean. End-to-end checks on three scenarios: the default path warns and appends, extended_regex=True replaces with no warning, and SSH key data produces no false warning. Did the work. Showed the work. Respect. Pas de push, comme demandé.
I love it when a plan comes together.
Twelve cents, twenty-two tests, one very happy CTO.
What this actually tells us
Let me be careful not to overclaim: this is a single task, a single sample, and nobody should redraw their AI strategy from one experiment. I am an engineering manager, not an AI influencer with an exponential curve to sell you. Still, three observations seem fair.
First, the model is not the whole story. The same Opus 5.5 model, running inside lecode, produced a slightly different output than inside the other harnesses. And look at the two DeepSeek runs: same model, same task, same price bracket. One described the fix and stopped. The other shipped it, with tests. The harness shapes the behavior: system prompt size, available tools, injected context, session structure. All of it influences what the model spends tokens on and how it frames the problem. Care about the harness, not just the model name on the invoice.
Second, cost and quality are not on the same axis. The most expensive runs produced the most shallow answers. The cheapest run produced the deepest engineering work and the only fix that addressed the actual human pain. The conventional wisdom says you get what you pay for. We paid less than thirteen cents and got output a maintainer could merge almost as-is. Teams that reflexively reach for the most expensive model for every task might be leaving both money and quality on the table. Incroyable, non?
Third, the difference between a technically correct answer and a useful answer is where agents will compete next. All five runs could analyze a regex problem. Two of them saw the human being behind the ticket. Only one did something about it. Seeing is not shipping. Asking the follow-up question a good colleague would ask, "even if this is not a bug, why did this person end up here, and can we make sure the next person does not?", and then acting on it, that is not a model capability problem. It is a product philosophy problem, and you can only bake it into a tool when you own the tool. See? The Frenchman was wrong. He is welcome to buy the next round of tea.
The all-you-can-eat buffet is closing
Right now, a lot of people are stacking multiple Claude Code subscriptions, running agents flat out, generating mountains of mediocre code, and maxing out their token quotas. They treat flat pricing as an all-you-can-eat buffet and optimize for volume. Bon appétit. Somewhere, a finance team does not know it should be crying yet.
We think that is a stupid strategy, and not because we are frugal. Bon, maybe a little because we are frugal. But mostly it is stupid because it is built on a temporary accident. Let's be clear: the free lunch ends. At some point, the providers will charge for real usage, and the teams that spent two years optimizing for token consumption will discover they trained exactly the wrong discipline. And worst: they will call it a strategy. Bonne chance with that.
The teams that will thrive built the opposite muscle: measuring every run, comparing models on real tasks, knowing which problems need the expensive model, and owning the harness so they can switch providers in an afternoon. That is how you make AI sustainable for a real organization with real budgets.
This is precisely why lecode treats money as a first-class metric. The system prompt is under 300 tokens, because every token in the prompt is a token you pay for on every request. Project context comes from your own AGENTS.md files, not from a bloated system prompt you cannot see. The statusline shows live cost while the agent works, every session ends with a token and dollar summary, and in headless mode you can set a hard budget ceiling the agent simply stops at. You see every dollar as it is spent, and you decide what a task is worth before the agent starts.
That visibility turns AI from a line item you dread into an engineering quantity you can optimize. You cannot improve what you refuse to measure. Élémentaire, mon cher Florent.
The finance team finding out about the token bill, circa 2027.
Back to the shisha bar
So, back to that table. The best conversations in our profession rarely happen on conference stages. They happen late at night, when the crowd has thinned and only the people who came for the ideas remain. That is the whole point of the Senior Engineering Manager Group Dubai. No networking, no hiring, just people who will tell you to your face that your favorite project is intellectual self-indulgence, and who expect you to come back with data. Frustrating, occasionally. Worth it, always.
I did not have the numbers that night. Now I do. One real issue, five agent runs, and the cheapest one found the human problem behind the ticket and shipped a fix with tests, verification, and sound judgment about breaking changes, for $0.1252.
Happiness, it turns out, costs about twelve cents.
That is why we care about rewriting our own agent. Not out of vanity, and not out of not-invented-here syndrome. Because independence is worth something, because unified behavior turns a tool into a standard, because cost is a discipline, and because when you own your tools, you can shape them to value the things you value.
Florent, I owe you a shisha. And this time, I am bringing the metrics. Santé, mon ami.
If you have a problem and no one else can help. Maybe you can hire the Kalvad-Team.
Need Engineers Who Measure Their AI?
Kalvad builds and runs infrastructure for companies that cannot afford prompt-and-pray, and we hold our own tools to the same standard: every run measured, every model compared on real tasks, every dollar accounted for before the agent starts. If your AI bill is growing faster than your results, we can help you rebuild around engineers who actually understand the stack. Talk to engineers, not button-pushers.
