Storage & investigations
Why DataTug uses OVDB for working context
A useful investigation should survive the next query, the next page, and the moment you close the app. We are building that continuity on OpenVaultDB.
The query is only part of the work
You start with invoice 42. You inspect its lines, look up the customer, and compare a related payment. Each step needs the same starting context. Tomorrow, you want to reopen the investigation and understand which invoice, environment, and inputs produced yesterday’s answer.
Keeping an ID in a text box solves the next click. Keeping a session, its Context Bags, their revisions, and the inputs used by each run solves a much more useful problem: continuing the work without reconstructing it from memory.
That is the operational data we are storing through OpenVaultDB (OVDB). It is distinct from the databases you investigate and from the project definitions you keep in Git.
One contract, two places to keep it
OVDB gives a database an identity, a manifest, and a mounted storage engine. DALgo supplies the data-access interfaces beneath it. DataTug’s Core service owns the session rules: what a command may change, who may perform it, and which revision it expects.
The local CLI and the hosted backend use that same domain service and repository contract, with different storage adapters. The browser sends authorized application commands; it does not write protected session records directly.
Local profile
CLI service → OVDB mount
Private SQLite databaseWorking context stays on your machine.
Hosted profile
Application backend → OVDB mount
Private Firestore recordsWorking context is served by the backend.
For a local session, SQLite is the authoritative store, rather than a disposable cache. The implementation keeps it in a private directory outside the project’s Git checkout. For a hosted session, the backend uses an OVDB Firestore mount and checks the caller’s project and environment access.
UI details such as open tabs, draft selections, and result display caches remain device-local. They help restore the workspace, but do not become the authority for a saved session or a completed operation.
Follow invoice 42 without losing its meaning
- Keep the identifier with its scope.
A bag entry records Invoice ID 42 together with its semantic field and environment. A test invoice and a production invoice with the same number must not quietly become the same entity.
- Apply a deliberate selection.
Checkbox changes are pending until you apply context. Compatible query inputs can then use the selected entries. A query still declares ordinary parameters; it does not need a hardcoded reference to one session’s bag.
- Pin what a run actually used.
Together, the run record and its immutable input snapshot record the resolved parameters, selected entry revisions, scope, and definition revision. Editing the bag later does not rewrite the inputs of an earlier run.
- Reopen the record of the work.
Retained context, input pins, and run receipts can be read without rerunning the source query. A retained result display is separate: storing the inputs does not itself preserve every returned row.
A session can hold multiple bags for different environments. Explicit environment and identity scope prevents a value from being silently carried into an unrelated source. The same context foundation is intended to support queries, dashboards, and AI interactions as those integrations are delivered.
History has to explain what happened
A saved value needs more than a current state. Entries have stable identities and immutable revisions. Runs refer to the revision they used. Sessions have an explicit lifecycle, so stopping or archiving work is separate from deleting it.
Commands carry an operation ID and expected versions. If an acknowledgement is lost, retrying the same operation can return its receipt. Reusing that ID for different input is rejected. If the stored version has changed, the command reports a conflict instead of silently overwriting newer work.
Those properties depend on the actual engine’s transaction behavior. We test each supported profile; an interchangeable interface alone cannot establish crash safety or correct concurrent writes.
Why SQLite first, and Firestore remotely?
SQLite gives the local profile a compact file, transactions, and a practical way to reopen working state without a cloud connection. The hosted profile reuses the application backend and Firestore infrastructure while keeping the same session semantics.
OVDB also makes other engines worth evaluating. We investigated inGitDB because readable, Git-backed records are attractive. It is not the current default for this session store: our process-termination spike exposed a partial-publication issue in the tested adapter. A future profile must meet the same storage contract before we rely on it.
This is the value of the boundary: engine choices can evolve without teaching every product screen a new meaning for “saved,” “applied,” or “used by this run.”
What this foundation does—and what it leaves open
- Local context can remain available offline. A remote database still needs a working connection unless its data is separately available locally.
- A shared model does not synchronize devices. Automatic replication, offline writes to hosted sessions, conflict resolution, and live collaboration need their own design.
- Input history explains an execution. Running the same inputs again can return different results when source data changes. A pinned input is not a snapshot of the source database.
- Private context needs its own access and retention policy. An invoice ID can be sensitive. Keeping operational records outside public project files is one boundary; authorization and lifecycle controls remain necessary.
The product benefit is concrete: less repeated input, clearer boundaries between environments, and an investigation whose recorded context remains understandable after its bag has changed. OVDB provides the storage boundary that lets us build those behaviors for both local and hosted use.
