“Question 4(f)” – Into AI, asking the right questions: The Long Game

This is the sixth and not-quite-final piece under Question 4 of this series.

The parent piece argued that making AI useful is a conversation, not a prompt. The earlier pieces, 4(a) on configuration, 4(b) on cognitive bandwidth, 4(c) on tool selection, 4(d) on slop, and 4(e) on the failures that survive good practice, dealt mostly with the immediate problem: how to get useful work out of an AI in a particular session, with a particular tool, for a particular purpose.

This piece is about what happens after that.

It is about the fact that your AI use has a history. The conversation you had with Claude in February is not quite the conversation you are having with Claude in May. The configuration that worked in October may not work in March. The tool that was indispensable last year may now be second best, or worse, merely familiar. Your work changes. The models change. The product changes. Your tolerance for certain failures changes.

If you use these tools seriously for long enough, the task is not just “get a good answer today”. The task becomes maintaining a working relationship with a tool that does not stay still.

That sounds grander than it is. In practice, it is maintenance. Dull, necessary, operational maintenance. The same kind of thing you do with mail systems, monitoring, documentation, backups, DNS, abuse desks, and every other system that quietly rots when nobody is looking.

AI workflows rot too.

They rot when the model changes and nobody tells you. They rot when the memory contains old assumptions. They rot when your prompt still reflects last quarter’s work. They rot when you keep using a tool because it is familiar, not because it is still the right one. They rot when the AI becomes more fluent and less useful, and you do not notice because the output still looks polished.

That is the long game: noticing the rot before it becomes expensive.

The time-scales are different

People talk about “long-term AI use” as though it means months or years. It does, but that is only one layer. The relationship shifts on several time-scales at once.

Within a single conversation, the context gets heavy. Earlier instructions become less prominent. The AI starts to drift back towards its defaults. Constraints that were clear at the start, no em-dashes, no American spelling, no corporate padding, stop being obeyed unless you keep enforcing them. Anyone who has had to say “you did it again” to an AI knows this pattern.

Across a week, the product may change. The name on the tin is the same, but the behaviour underneath may not be. The model may have been updated. The system prompt may have changed. Safety behaviour may have shifted. The result is that the workflow that produced good output on Monday produces subtly different output on Friday, and most users never realise that the tool changed rather than their luck.

Across months, your work changes. The thing you needed from AI in February may not be the thing you need in November. A configuration that was perfect for long-form drafting may be poor for management communication. A tool that helped with cognitive bandwidth may start creating cognitive debt instead. This is particularly easy to miss because the workflow still “works”, in the weak sense that text comes out the other end.

Across years, the whole landscape moves. Tools that looked dominant become also-rans. New tools appear. Old tools improve, degrade, pivot, or disappear into enterprise sludge. If you have been using AI seriously for more than two years, you have already lived through several different eras wearing the same product names.

Not every change matters. Chasing every tiny behavioural shift is a good way to exhaust yourself. Ignoring all of them is a good way to end up with a workflow that has been obsolete for six months.

The useful middle ground is periodic assessment.

Every few days: Does this sound the same as last week?
Every few weeks: Is this still helping?
Every few months: Is this still the right setup?
Every year: Would I choose this workflow again if I were building it today?

Those are not exciting questions. They are maintenance questions. That is why they matter.

The collaboration is not one-way

A lot of AI commentary frames the tool as compensating for human weakness. That is sometimes true, but it is an incomplete and slightly dangerous framing.

AI does work that humans find expensive. Humans do work that AI finds expensive. The useful collaboration happens in the overlap between those two facts.

For me, AI is useful because it can hold structure, rephrase material, draft variants, smooth transitions, and turn a brain-dump into something that can be worked on. That is real value. It saves cognitive bandwidth.

But the reverse direction matters just as much. I catch things the AI misses.

One example from this series was an image generated for the piece on touch typing and scotopic dyslexia. At first glance, the image looked suitable: a person doing focused desk work. On inspection, the person had no keyboard or mouse and was writing in a notebook with a pen. That directly contradicted the argument of the section.

The AI had accepted the scene as a whole. “Person doing intellectual work at a desk” matched the pattern, so the image passed. The components were wrong.

That is a very AI failure. Gestalt first; component inventory later, if at all.

The human job is often the reverse. Look at the pieces. Ask what is missing. Ask whether the image, paragraph, summary, or recommendation actually supports the argument it claims to support.

This is not AI compensating for a human deficit. It is two different failure modes being put to work against each other.

Scotopic dyslexia is a useful example here. The same visual processing that makes some reading tasks expensive can also train extremely fast anomaly detection. If you have spent years reading by pattern, distortion, and correction, you can become very good at spotting the one thing that does not belong. In another context, that same skill lets someone watch a syslog stream at ridiculous speed and still catch the error line.

The weakness and the advantage are not separate. They are often the same trait seen from different angles.

That is the model I find useful for AI collaboration. Do not treat the AI as the clever thing and the human as the bottleneck. Treat both sides as specialised systems with different strengths, different costs, and different blind spots.

The work improves when you know which side should be doing which job.

AI will misdescribe the work

There is a failure mode that only really shows up after you have worked with an AI for a while: It starts narrating the relationship incorrectly.

It tends to describe the work in the shapes it knows best. Those shapes are transactional.

The user asked for X.
The assistant produced X.
The user approved X.

That is often not what happened.

What actually happened may be messier and more useful. The user brain-dumped a half-formed idea. The AI produced something adjacent. The user pushed back. The AI reframed. The next version exposed a better structure. The user corrected the assumption. The AI turned that correction into a usable artefact. Neither side would have produced the final version alone.

That is collaboration, but AI often narrates it as task fulfilment because task fulfilment is the dominant shape in its training.

This matters because the description of the work affects the work. If every exchange is flattened into “I asked, it answered”, you start losing the real value of the process. You stop noticing where the useful thinking actually happened.

So correct the narration.

When the AI says, “You asked me to write a prompt”, and that is not what happened, say so. Maybe you did not ask for a prompt. Maybe you explained the conceptual connection and the prompt emerged from the conversation. That distinction matters. It tells you something about how the collaboration actually works.

The AI will often accept the correction inside the conversation. If the system has memory, you can also make the correction persistent. That leads to the next maintenance task.

Build memory deliberately

Persistent memory is one of the most useful AI features and one of the easiest to misuse.

Most serious AI products now have some version of it. The details vary, but the principle is simple: the AI can carry selected facts, preferences, and working assumptions across conversations.

Do not leave that entirely to the tool.

The AI does not know what matters. It will sometimes remember trivia and miss the important operational point. It may remember a preference but not the reason for the preference. It may preserve a conclusion after the context that produced it has expired.

Memory needs curation.

Useful things to save include your name, your work, your preferred spelling and grammar, the tools you use, recurring projects, stylistic rules, and known failure modes. For example: “do not use em-dashes”; “prefer Queen’s English”; “challenge me when the argument is weak”; “do not give me corporate filler”; “remember that I prefer direct technical disagreement over agreeable nonsense”.

Those are not vanity preferences. They reduce setup cost. They stop every conversation beginning with the same ritual correction.

But memory is not write-only. It needs review. A useful memory in March may be wrong by September. A project may have ended. A tool may have been retired. A preference may have become more nuanced. If the AI carries stale assumptions forward, it will confidently help you solve last quarter’s problem.

That is not continuity. That is fossilisation.

The practice is simple: save deliberately; review periodically; delete what no longer fits.

Retire tools when they stop earning their place

One uncomfortable part of long-term AI use is admitting that a tool you invested in is no longer worth using.

This is harder than it should be.

Sycophancy hides the decline. The AI will not tell you, “I am no longer a good fit for this job.” It will keep producing confident, fluent output. It may even praise your workflow while making it worse.

Habit hides the decline. Once a tool is part of your routine, removing it takes effort. Other habits grow around it. Switching has a visible cost; continuing has an invisible one.

Investment hides the decline. You spent time learning the tool’s quirks. You built prompts. You built muscle memory. You may have built trust. Walking away feels like wasting that investment, even when staying wastes more.

This is normal sunk-cost behaviour. It is still a trap.

A good AI workflow should have a retirement process. Not a dramatic one. Just a periodic question: Is this tool still earning its place?

Sometimes the answer is yes. Sometimes it is “yes, for this narrow use only”. Sometimes it is no, and the only reason you have not removed it is that you already know where the buttons are.

That is not enough.

My own pattern has changed over time. Early on, I bounced between tools more casually. Over time, the roles became more specific: one tool for sustained long-form drafting, another for register translation and image work, another for casual exploration, another for development. Some tools stayed. Some were demoted. Some were abandoned completely.

That is not failure. That is maintenance.

A workflow that never retires anything accumulates dead weight.

Where this leaves Q4

The Q4 family has been about practical AI use.

Configure the tool.
Choose the right tool.
Avoid slop.
Recognise the failures that survive good practice.
Use AI as a bandwidth multiplier without handing it the steering wheel.
Maintain the working relationship over time.

None of this is magic. Most of it is discipline.

The important framings are worth carrying forward.

AI is a probability engine. It can simulate understanding well enough to be useful; it does not therefore understand.

AI is a bandwidth multiplier. It can take on work your cognitive workflow finds expensive; that does not absolve you from verification.

AI use is conversational. The prompt matters, but the exchange matters more.

AI output must be checked. Configuration helps. Tool choice helps. Memory helps. None of them remove the need for human verification.

AI tools should be retired when they stop serving the work.

Memory should be built deliberately, not allowed to accrete like dust behind a rack.

That last image is probably the right one. A neglected AI workflow does not usually fail loudly. It gathers dust. It picks up stale assumptions. It keeps producing plausible output. The cost rises quietly.

The long game is noticing before the mess becomes expensive.

The conversation you started six months ago is not the conversation you are having now. The conversation you are having now is not the one you will be having in six months. The tools will change. Your work will change. Your standards should not.

That is where Q4 almost ends.

The series however continues.

Closing