Why Your AI Tools Keep Making the Same Mistakes Twice
If you're re-explaining your rules to an AI tool every time you open a new chat, the tool isn't the problem. Grouping related work into a shared project with one set of standing rules stops the repeat mistakes and cuts setup time to almost nothing.
A client of mine in energy services spent three separate chat sessions over two weeks asking an AI tool to build the same kind of dashboard three different ways. Each time, he started from a blank chat. Each time, he re-explained what the business does, what the data looks like, and what he wanted the output to be. Each time, the tool gave him something slightly off from what the last one gave him.
That's not an AI problem. That's a filing problem.
Why does the same AI tool give inconsistent results?
Every new chat starts with zero memory of the last one. If you don't hand it the rules up front, it guesses. And it guesses differently depending on how you phrase the request that day, what mood you're in, how much context you bothered to paste in.
We fixed this on a call last week by building what I call a CTO project: one container that holds a single markdown file of engineering rules, and every chat related to that work lives inside it. New chat for the bonus calculator, new chat for the marketing dashboard, new chat for the sales dashboard — but all three inherit the same rules automatically. Naming conventions, deployment targets, tone, data formats, none of it gets re-typed.
What actually goes in the rules file?
Not a philosophy. A checklist. My client's file has things like which repo each project lives in, what the API keys are called in code versus what they're called in the vendor's dashboard, and a note that revenue metrics should default to current-year numbers unless someone asks for lifetime totals. Small stuff. The stuff that's obvious to him and invisible to a tool starting from nothing.
Once that file exists, you're not writing documentation for documentation's sake. You're writing the thing that stops the tool from asking you the same clarifying question in every session.
Does splitting work into different models actually help?
We also split his AI usage across three tiers instead of running everything on one model. The high-cost model handles review and architecture decisions — the stuff where a wrong call costs real time. A mid-tier model handles planning and breaking work into steps. The cheapest model executes the grunt work: writing boilerplate, running repetitive edits.
This isn't about chasing the newest model. It's about matching the cost of the tool to the cost of being wrong. Getting a dashboard's color scheme wrong costs nothing. Getting a deployment architecture wrong costs an afternoon. Treat those two tasks the same and you're either overpaying for simple work or underpowering the important calls.
What slows this down in practice?
Access, almost always. Half our call wasn't about AI at all — it was Railway workspace permissions, a GitHub org he didn't have access to, and an API key that was named one thing in the vendor dashboard and another thing in the code. The AI tool wasn't the bottleneck. Getting the right humans access to the right repos was.
This is the part people skip when they get excited about a new AI workflow. You can have the cleanest rules file in the world, and it still won't deploy if nobody can see the GitHub org it's supposed to push to. We spent forty minutes just adding a collaborator so a build could see the repo it needed.
How do you know the setup is actually working?
You stop re-explaining yourself. That's the whole test. If you're two months into using a tool and you're still typing out "remember, our fiscal year starts in April" at the top of every chat, the project structure isn't doing its job yet.
For my client, the marker was smaller than that. He asked for a dashboard to show total closed revenue, and without him specifying current year, the tool defaulted to current year because the rule was already sitting in the project file. That's the whole point. Not a bigger, smarter AI. Just one that doesn't need to be told the same thing three times before lunch.
By the end of the call, the bonus calculator had been moved into the shared project, the marketing dashboard prompt was drafted and waiting for him, and the Nimbus API key was sitting in the right field in Railway under the right variable name. None of it will make a roadmap slide. It just means his next chat starts already knowing what his last three chats had to explain from scratch.
