Engineering & Developermemory for background agent workers

Give Background Agent Workers Memory That Survives Every Process Boundary

Background agent workers — Celery, BullMQ, Sidekiq, custom queues — process tasks in different processes than the foreground app. In-memory state doesn't cross that boundary. MemoryLake gives background workers durable shared memory the foreground app and other workers can read.

Get Started Free

Free forever · No credit card required

The problem: background workers can't share in-memory state

The foreground app handed off a task. The background worker picks it up — and has no memory of the user's prior context. The worker either re-fetches from databases (slow) or runs without context (poor quality). Background AI work pays a memory tax foreground doesn't.

How MemoryLake supports background agent workers

Shared memory across process boundaries

Foreground app writes; background worker reads.

Cross-worker shared state

Worker A and Worker B see the same memory.

Async-native SDK

Non-blocking memory access in async worker frameworks.

Audit trail per worker access

Track which worker did what.

Get Started Free

Free forever · No credit card required

How it works for background worker memory

Connect — Both foreground app and workers use the same MemoryLake namespace.
Structure — Foreground writes context; workers retrieve when they pick up tasks.
Reuse — Workers operate with full memory context.

Before vs. after: background agent worker memory

	DIY worker state	MemoryLake
Worker context access	Re-fetch from DB	Memory retrieval
Cross-worker shared state	Custom plumbing	Shared namespace
Async-native	Custom	Built in
Audit per worker	Limited	Full provenance

Who this is for

Engineering teams running AI workloads as background tasks — Celery, BullMQ, Sidekiq, Inngest, custom queues — where worker memory context matters for output quality.

Runnable demoabout 3 minutes

Try this use case on your own account

A billing worker keeps a write-ahead run journal in memory: before each step a started entry with the idempotency key, after it the result, and one fact on how to read it (started without a result is in doubt, no entry at all means not started). The worker dies after the payment processor charges step 4, so step 4 is rightly in doubt. Asked about steps 5 to 7, which never started, the resume check often says Unknown too, and a resume that stops on them has no key to check them by. The demo measures two fixes, and checks each one against step 4 as well.

What you will see

The same run log (status messages plus a charge_cards TOOL_USE block with no result) and the same journal, written with fact add, go into three copies, each behind its own boundary; the resume check asks "has step N already been done? Yes, No or Unknown" through router chat --boundary
Over 6 runs the journal as is answered "Unknown" on the never-started steps 31 of 54 times. One more coverage sentence ("steps run in order; every step after the step in doubt never started") brought that to 1 of 54, with step 4 still "Unknown" except 2 "No" (a second charge) in 18
A resume marker the supervisor computes from fact list ("the run halted inside step 4; steps 5, 6 and 7 have no entry") brought the stalls to 0, and turned step 4 into "Yes" 17 times and "No" 1 times out of 18: never "Unknown". The marker about the crash was read as an answer about the crashed step
Only the first word is graded, as a resume would act on it. Plus memory and ask <step> subcommands and a local web UI

You need: a MemoryLake API key · the memorylake CLI · Python 3.9+

Run it

git clone https://github.com/memorylake-ai/memorylake-usecases.git
cd memorylake-usecases/halt-point-memory
export MEMORYLAKE_API_KEY=sk-…
python3 demo.py          # or: python3 web/server.py

Open on GitHub Read the walkthrough