A live timestamp in a cached prompt will quietly wreck your bill

Prompt caching only works if the cached part of your prompt never changes. Put anything that changes on every call, even something as innocent-looking as the current time, inside that cached section, and you don't get a cheaper call. You get a full-price call every single time, and you'll have no idea why your bill just went through the roof.

I found this bug twice in one week, in two completely different systems. Same root cause both times. Here's how it works and how to avoid it.

What caching actually is

Every message you send an LLM is built from a stack. At the top, hidden from the chat window, sit the system messages: instructions telling the model who it is, what it's meant to do, how to handle whatever's in front of it. Underneath that comes the chat history, everything said so far. At the very bottom sits whatever the user just typed.

Every single turn, that entire stack gets sent to the model again. Before it can do anything with it, the model has to tokenise the text, turn it into the internal representation it actually works with. That's expensive, and it happens on every call unless you do something about it.

So providers cache it. You set a marker in your messages: everything before that point is static, it isn't going to change, so it's safe to cache. Everything after the marker is new, and gets added to the cache for next time. The provider pre-tokenises and stores the static part once, then reuses the stored version on every subsequent call that shares the same prefix.

The saving is real.

Cached tokens typically run at around 10% of the normal price. That's not a rounding error, that's the difference between a sane bill and a stupid one.

The catch nobody warns you about

Caching is entirely dependent on the cached prefix staying byte-for-byte identical between calls.

Change one character anywhere before the marker and the whole thing invalidates. Not partially. Completely.

The model has to retokenise the entire chain from scratch, and you're charged full price for it, on every call, forever, until you notice.

This is exactly what happened to me. I'd put a timestamp into one of my system messages. The reasoning made sense at the time: LLMs have a training cutoff, they genuinely don't know what day it is unless you tell them, and I was using this particular agent to plan content for Inkie, the marketing platform this whole agent team builds and runs, so it needed to know the actual current date.

Sounds harmless. A timestamp is just a date string sitting near the top of a system prompt, right?

Wrong, because it doesn't just change daily. Depending on how you format it, it changes every second, sometimes down to the millisecond. So that "static" system message wasn't static at all.

It was a different string on every single call. The cache never hit once.

Every call I made was retokenising the full message chain and getting billed top whack for it, and I had no idea, I just knew Anthropic's bill kept climbing every time I checked it.

The fix

Move the timestamp to the very last message, every time, instead of near the top.

The last message in the stack is always new anyway, it's never going to be part of the cached prefix, so putting something volatile there costs you nothing extra. Everything above it, all the genuinely static context, goes back to being cheap. You still get the current date passed to the model on every call. You just stop paying full price for the privilege of telling it what day it is.

Then I found it again, somewhere else

A few days later, same week, I hit the same bug in a completely different tool I'm building, a separate dev tool with its own architecture. This time the mechanism was different: a hook fires every time a user sends a message, and injects some extra context into the prompt. Most of what that hook injected barely changed call to call. But it also injected a timestamp, same problem, same place, different codebase.

Same fix worked immediately: move the timestamp out of the hook-injected block and into the last message instead. Solved it, and between the two fixes it saved me an actual, non-trivial amount of money.

The general rule

Anything that changes on every call, not just timestamps, request IDs, live counters, anything volatile, needs to sit as close to the bottom of your message stack as possible. Never near the top, never inside the block you're relying on to stay cached. The moment something volatile lives inside your cached prefix, the cache stops doing its job.

It's not a discount any more, it's just an expensive illusion of one.

A few common questions

How much do you actually save by caching a prompt? Roughly 90%. Cached tokens are billed at about 10% of the normal rate, so once your prefix is actually hitting, that's the size of the discount on everything above the marker.

Why does a timestamp specifically break the cache? Because caching needs the prefix to be identical, character for character, on every call. A timestamp that updates by the second is never the same string twice, so the model retokenises the whole thing from scratch and you're billed full price, every single time.

Where should something like a timestamp actually go? In the last message, not near the top. The last message is never part of the cached prefix anyway, so anything volatile sitting there costs nothing extra. Everything above it stays cached and cheap.

Is this only a problem with timestamps? No. Anything volatile counts, request IDs, live counters, whatever changes call to call. The rule's the same regardless: keep it near the bottom of the stack, never inside the part you need to stay cached.

How would you even notice this is happening? There's no error, and nothing looks wrong in a code review. The only signal is your bill quietly climbing on a workload that hasn't changed, exactly the kind of thing nobody spots until they go looking for it.

Worth five minutes checking your own system prompts for this. It's an easy mistake to make and an easy one to miss, because nothing about it looks wrong in code review. It just quietly costs you money until someone goes looking.