Skip to content
curral

Semantic documents

A schema says a column is called amt and holds a DOUBLE. It does not say that amt is in cents, that cancelled orders must be left out, or which join gives revenue per customer. Semantic documents carry that knowledge: short notes written by people who know the data, stored next to the gateway and served at /v1/docs, so an AI agent (or a person) reads them before writing SQL.

Turning it on

Section titled “Turning it on”

Point --docs-dir at a directory. Each document is one JSON file, <docs-dir>/<id>.json:

Terminal window
curral serve ... --docs-dir /var/lib/curral/docs

Without --docs-dir nothing changes and /v1/docs answers 404, which is also how clients and the agent skill detect that the feature is off.

The format

Section titled “The format”
{"id": "019a2f1c3a4b9f2e18c4d7a6b05",
 "header": {"author": "poet", "created_at": "2026-10-10T12:00:00Z",
            "updated_at": "2026-10-10T12:30:00Z", "updated_by": "admin",
            "title": "Monthly revenue",
            "description": "Gross revenue per customer and month",
            "used_for": ["finance reports", "churn analysis"]},
 "context": {"schemas": ["lake.analytics"],
             "tables": ["lake.analytics.monthly_revenue"],
             "body": "Free text, as long as needed: meaning of columns, caveats, joins...",
             "sqls": [{"sql": "SELECT customer, sum(revenue) FROM lake.analytics.monthly_revenue GROUP BY 1",
                       "description": "revenue per customer"}]}}
Field
header.title required, up to 200 characters
header.description one line on what the document covers
header.used_for the questions or uses it helps with
context.schemas, context.tables what it is about; free text, not checked against the catalog
context.body required: column meanings, units, caveats, the right joins and filters
context.sqls[] sample queries; each needs sql, description is optional
  • Server-set: id, author (who created it), created_at, updated_at and updated_by (who changed it last). Values sent for them are ignored, so a document from a GET can be sent back as is.
  • Strict: unknown fields are a 400. Request bodies are capped by --max-body.

Reading

Section titled “Reading”

Every authenticated user can read documents.

Terminal window
curl -u analyst:analyst-pw 'localhost:8080/v1/docs?table=orders&fields=header'
curl -u analyst:analyst-pw localhost:8080/v1/docs/<id>
  • GET /v1/docs lists full documents, most recently updated first.
  • ?table= and ?schema= match the exact name or its last parts, so orders finds sales.main.orders.
  • ?q= matches text in the title, description or used_for.
  • ?fields=header leaves out body and sqls, for a cheap index.
  • GET /v1/docs/{id} returns one document with an ETag.

All filters are case-insensitive. To find documents by a question in plain words instead, use search.

Writing

Section titled “Writing”
Request
POST /v1/docs create; 201 with Location and ETag
PUT /v1/docs/{id} replace the content
PATCH /v1/docs/{id} JSON merge patch: only the fields to change, null removes one
DELETE /v1/docs/{id} remove; 204
Terminal window
curl -u poet:poet-pw -H 'Content-Type: application/json' -d @doc.json localhost:8080/v1/docs
curl -u poet:poet-pw -X PATCH -H 'If-Match: "<etag>"' \
  -d '{"header":{"used_for":["finance reports"]}}' localhost:8080/v1/docs/<id>

Concurrent edits: send the ETag from your last read in If-Match on PUT, PATCH or DELETE. If someone changed the document in between, the write fails with 412 instead of overwriting their change.

Who may write

Section titled “Who may write”

Writes go through their own policy decision, --policy-docs-query (default data.curral.docs_allow). Until the policy defines it, it is undefined and every write is denied (403). The example policy grants it to admin and to a poet role, and examples/users.yaml has a poet user:

docs_allow if "admin" in input.roles


docs_allow if "poet" in input.roles

The decision sees the action, the caller and the document both as it will be saved (input.doc) and as it is now (input.current). A rule that limits a role to some tables must check both: checking only doc would let an update take over a document about other tables by changing its tables.

docs_allow if {
  "sales_poet" in input.roles
  not other_tables
}


other_tables if { some t in input.doc.tables; not startswith(t, "sales.") }


other_tables if { some t in input.current.tables; not startswith(t, "sales.") }

Every field is in Policy input.

Storage, audit and reload

Section titled “Storage, audit and reload”
  • Files are written atomically (temp file, fsync, rename) and pretty-printed, so the directory can live in git or be edited by hand.
  • SIGHUP reloads the directory. A broken file is reported in the log and the current set stays in effect.
  • Audit: writes produce doc_create, doc_update and doc_delete events with the document id in doc. Like queries, writes are refused with 503 while the audit log cannot be written.

Writing good documents

Section titled “Writing good documents”
  • One topic per document: a table, a metric, or a business rule. Search ranks documents, so small focused ones surface better than a manual.
  • Lead with the title and used_for: they weigh most in search. Phrase used_for the way people ask (“revenue per customer”, “churn by month”).
  • Say what the schema cannot: units, time zones, which rows to exclude, the right join key, known gaps in the data.
  • Only tested SQL in sqls, and never real personal data or values that are masked for some readers.