Documentation

Persistence & Durability

JetGraph is an in-memory engine — all data lives in RAM for sub-millisecond access. Durability is provided by a layered persistence model that writes to disk asynchronously without pausing query processing.

How Data Survives Restarts

📸

Full Snapshots

Complete graph serialized to disk nightly (default 02:30 AM) or on a configurable interval. Compressed with zstd.

⚡

Delta Files

Incremental changes written every 30–60 seconds. Applied on top of the base snapshot at startup for fast recovery.

🔗

Checkpoints

When the delta chain grows long, deltas are merged offline into a single checkpoint file — no engine pause required.

🛡️

Shutdown Delta

On graceful stop, a final delta is written before the full snapshot — so recent changes are safe even if the snapshot is interrupted.

Recovery Sequence at Startup

1

Load latest full snapshot

The most recent snapshot-*.bin file is deserialized into memory. The engine is marked ready immediately so queries can start while the Cuckoo filter rebuilds in the background.
2

Apply checkpoint (if any)

If a checkpoint file exists for this snapshot base, it is applied first to skip replaying the earliest deltas.
3

Replay remaining delta files

Any delta files newer than the checkpoint are applied in order, bringing the graph fully up to date.

ℹ️

The engine returns 503 from /health while loading a snapshot. Polls will succeed once loading completes — typically within seconds for small graphs, longer for very large ones. This is why start_period in the health check should be set generously.

Emergency Snapshot (Memory Pressure)

When RSS memory exceeds memory_limit_bytes in config.toml, the engine automatically writes an emergency full snapshot, blocks new ingest, and logs an error. This protects data before the container OOM-killer fires. Ingest resumes once memory drops back below the limit.

Manual Snapshot

Trigger a snapshot on demand via Cypher without restarting:

cypher

CALL graph.saveSnapshot() YIELD ok, path
RETURN ok, path

Graceful Shutdown

Send SIGTERM (what docker compose stop / docker compose down sends) and the engine will write a final delta then a full snapshot before exiting. The stop_grace_period: 300s in the compose file gives it up to 5 minutes for very large graphs. Never use SIGKILL directly — it bypasses the shutdown snapshot.

Persistence & Durability