Stop Bloating Token Costs by Stuffing Agent History Into Every Prompt
Production agent apps quickly discover the same trap: stuffing conversation history into every prompt drives token cost and latency up faster than usage. MemoryLake retrieves a compact memory block scoped to the current task — same recall, fraction of the tokens.
Stop Bloating Token Costs by Stuffing Agent History Into Every Prompt
Get Started FreeFree forever · No credit card required
The problem: token cost scales with stuffed history
A user with three months of agent history has 200K tokens of context. Stuffing it into every call inflates inference cost and latency on every turn. Switching to summary memory loses fidelity. The right answer is structured retrieval, not stuffing or summarization.
How MemoryLake reduces token bloat
Token-budgeted retrieval
Pull only the memory relevant to the current task, sized to your budget.
Typed memory beats flat history
Retrieve facts, events, or skills — not raw transcripts.
10,000x scale over stuffing
Compress millions of tokens of history into millisecond retrievals.
Prompt caching compatible
Retrieved blocks slot into cacheable system messages.
Free forever · No credit card required
How it works for token-efficient agent memory
- Connect — Replace history stuffing with MemoryLake retrieval at prompt construction.
- Structure — Per-turn writes to typed memory.
- Reuse — Retrieve a token-budgeted memory block per prompt.
Before vs. after: token usage
| Stuffed history | MemoryLake retrieval | |
|---|---|---|
| Token cost per long-history call | 30K+ | <2K |
| Latency from giant prompt | Slow first token | Fast |
| Memory of months-old context | Truncated or summarized | Retrievable |
| Prompt cache hit rate | Drops | Maintained |
Who this is for
Engineering teams running production agent apps where token costs are scaling faster than user count — and switching to summary memory has been considered but rejected for quality reasons.
Try this use case on your own account
Tolland Hollow Brewing's purchasing assistant keeps one long thread with its Head of Purchasing: ten sessions from July to September settle a hop contract, malt delivery days, a late-delivery credit, a glass quote, the CO2 account, label sign-off and the quarter's approval limit. Today's 16-turn session runs three ways: the whole thread in every request, only the last 6 turns (the usual fix for the bill), and the same 6 turns plus MemoryRouter attaching what MemoryLake extracted from the past sessions. Over 5 runs, the window answered 0 of 35 questions about earlier agreements correctly (24 times "not on file", 11 times a made-up value), and its cached tokens stayed at the 1,024-token system prompt on 47 of 50 turns, because the window changes every request. The whole thread was right 34 of 35 times with about 92% of it read from the prompt cache; window plus memory was right 35 of 35 times, while the whole thread cost 1.5× as much per session.
What you will see
- Ten past sessions go into MemoryLake as conversations (
conv create,conv msg append --timestamp), and the demo checks that every value today's session needs was extracted before any answer - Three prompts per turn: the whole thread, the last 6 turns, and the last 6 turns with
memorylake router chat --boundary; seven turns ask for something agreed weeks earlier, graded ✓ / not on file / made up - Every call prints the provider's
usage.prompt_tokens_details.cached_tokensand is priced frommemorylake router models: the thread grows at its end and stays cached, the window never gets past the system prompt factslists what MemoryLake extracted,ask "<question>"sends one new turn through all three, and there is a local web UI with one cache bar per request
You need: a MemoryLake API key · the memorylake CLI · Python 3.9+
Run it
git clone https://github.com/memorylake-ai/memorylake-usecases.git cd memorylake-usecases/history-window-memory export MEMORYLAKE_API_KEY=sk-… python3 demo.py # or: python3 web/server.py
Watch it run · real, unedited recordings
Related use cases
Frequently asked questions
Does retrieval miss important context?
Does retrieval miss important context?
LoCoMo benchmark #1 at 94.03% accuracy on long-horizon recall — top-ranked structured retrieval.
Cost comparison?
Cost comparison?
Typically 10-100x cost reduction at long-history scale.
Self-host?
Self-host?
Yes — enterprise tier deploys in your VPC.