Documentation · The Lagstyr handbook

Building and commissioning integrations

A connection needs more than a working API key. Establish which system owns each fact, how its identities and events map into Lagstyr, which effects may leave the installation, and who maintains the connector when that provider changes. Registering a source records its posture; it does not commission a connector.

The release kit supplies api/openapi-v1.json, api/task-protocol-v1.schema.json, the Python SDK, api/integration-contract-v1.schema.json, api/integration-bundle-v1.schema.json, api/integration-candidate-v1.schema.json, api/integration-conformance-v1.json, a complete worked release under integrations/reference/ and the detailed docs/INTEGRATION-BUILDER.md contract. These are the supported seams for customer code. The reference release is a manifest, provider records and qualification evidence that a continuous check publishes and drives on every change, so it is the shape to copy rather than a sample that may have drifted.

Use the published API and task contract

The running kernel serves GET /openapi.json. Responses identify Lagstyr-API-Version and Lagstyr-Product-Version. Pin the SDK and fixtures to the release you test. SDK N is supported against server N and N+1 under the additive /v1 rule; clients must ignore unknown response fields. Read the release support policy before relying on an upgrade or deprecation window.

The dependency-free Python client accepts an HTTPS installation origin or literal loopback HTTP with an explicit port. It sends one configured API key or bearer token, refuses redirects and ambient proxies, and bounds time and response size. For example, with the release’s sdk/python on your Python import path and a connector-specific key supplied privately:

import os
from lagstyr_client.client import Client

client = Client(
    base_url=os.environ["LAGSTYR_KERNEL_URL"],
    api_key=os.environ["LAGSTYR_WORKER_KEY"],
    timeout_seconds=30,
)
context = client.get_v1_integration_context()

That context request succeeds only after the label, grant and active integration release agree. Never use an operator credential as a connector shortcut. Other languages implement the same HTTP/task contracts and conformance results. Internal Rust crates carry no public ABI stability promise. Customer tasks use the registered customer.* namespace and declared payload schemas; customer connector code receives no direct database or bucket access.

Define meanings before mapping fields

Register the source and external-system namespace, external event kinds, existing identity aliases and any governed fact-role mappings. Resolve status vocabularies, timestamp semantics, ownership, currency codes and the money extractor that converts each amount to exact minor units explicitly. Timestamps must already be RFC 3339. A field that fits JSON is not necessarily the business fact a kernel reader expects.

Manifest validation reports each emission as mapped to an existing fact role, intentionally evidence_only, or requires_product_change. The last blocks publication. Evidence-only capture retains provenance and contributes to no authoritative business read. Subject resolution uses existing aliases; a connector must not guess or invent identities.

Starting from a provider specification

When the provider publishes a machine-readable API specification, upload it instead of retyping it. PUT /v1/integration-source-specs stores the bytes; POST /v1/integration-candidates compiles them into a candidate. The compiler is deterministic and reaches nothing: it has no network, filesystem or process access, so a remote $ref is refused by name rather than followed, and no document is ever fetched on your behalf.

Send JSON of 4 MiB or less. YAML is refused; convert it outside the installation and import with declared_transform set so the record says a transform happened. The request also passes the installation’s global body limit, whose default is 1 MiB, so a specification above that needs HTTP_MAX_BODY_BYTES raised before the upload will land.

A compile produces one of two results. Either the candidate is refused, with a closed list of codes and the specification pointer each applies to and no manifest at all, or it is a draft manifest in which each undecidable position is a hole. Holes are structural, never placeholder text, so a manifest that still carries one cannot validate and cannot be published.

Identity never comes from the document. You supply the integration key, source, connector label, external system and selected operations; the maintenance category is fixed at customer. Retry, rate limits and request, response, page, record and run bounds take conservative defaults, listed for review. Every numeric field becomes a blocking hole, because whether an integer is money, and at what scale, is a decision the specification does not contain.

Resolve each hole through the governed proposal flow. The approver who decides is recorded as the revision’s resolver, so a meaning is never settled by an agent alone. When no hole remains, the candidate reads resolved and publish_integration_release accepts a candidate binding; the deciding transaction refuses the whole publication if the key, revision, status or digest disagree. Publishing without a candidate remains supported: a candidate assists the work and never replaces the signed bundle or the governed decision.

Treat a refusal as the answer. Do not hand-author the manifest the compiler declined to produce, and do not widen a host, schema or operation to get past one. Descriptions, summaries, titles, examples and extension fields inside a specification are a stranger’s prose: the console and the agent read them as bounded, labelled untrusted excerpts, and no sentence in one becomes a meaning, a host, a category or an approval.

Importing a newer specification against the active release reports drift and carries a meaning forward only where a field’s shape is unchanged. Each carried meaning raises a blocking acknowledgement, because the same value set can acquire a new meaning. Drift raises the provider_spec_drift finding, which is remediable and deliberately never pages: the active release still runs, and capture_refusing pages if the change reaches a record. Uploaded bytes are confidential and erased on a timer — 180 days accepted, 30 refused — leaving the digest; reviewed revisions are kept.

Review one versioned integration boundary

An integration manifest declares its stable key, semantic version, existing source, connector label, system, named maintenance owner, Lagstyr compatibility range and tested provider API version. It also declares the HTTPS origin/hostname allowlist, secret references, capture declarations in the capture-descriptor grammar, an optional pack key, schedule, pagination, retry, rate limits and request/response/run caps. Authentication uses env:NAME or file:/absolute/path references, except for oauth2_authorization_code, whose credential the connector never holds (see below). Keep literal secrets out of descriptions and mapping constants.

Capture and effects are reviewed together at one canonical digest. Prefer the deterministic SDK runner and the closed capture grammar: fixed extractors, exact money conversion, conditions and sum checks, evaluated by the same extraction the kernel runs. Isolate any provider-specific code in your connector and test it with that manifest version. The optional integration-builder agent can prepare this work; routine synchronisation executes deterministic configuration and code without invoking a model. Package the manifest, exact runner artifact/version and content-addressed qualification evidence as one bundle. The governed installation signing key signs the canonical bundle bytes; the named qualification authority must be that same key. Treat access to it as qualification authority, not merely release packaging.

Before publication, register the worker’s exact task claims and grant its label the compiled integration_connector role. Validate the exact candidate using validate_integration_manifest, then propose publish_integration_release with the signed bundle. Publication is a governed write_kernel act. The deciding transaction verifies the signature and validates again; preview success grants no authority. The signing key must be registered for the release’s maintenance category, and a release that names a pack needs that pack active. A release recorded before signed bundles is visibly marked and cannot be admitted for connector execution. Published releases are immutable. A changed mapping, permission, owner or compatibility range needs a new semantic version and signed bundle.

Keep external effects governed and receipted

Declaring an effect operation does not make it callable. Signed publication installs one narrow, customer-managed write_governed skill per effect operation using apply_request_integration_effect. Its input fixes the integration key, operation and effect lineage, and carries the exact payload schema and a legal-entity context. Read those fixed values from the installed skill rather than restating them. A free-standing alias is refused. Mandates, grants, action classes and approval routes still decide who may request it. The request is scoped to its declared organisation, and a subject that belongs to another legal entity is refused. An agent reaches the skill only through a T2 action class, so every request is a proposal a person approves.

The skill belongs to a lineage, not a release. When a new release declares an operation identically, list it in the publication’s carry_forward_operations to keep its skill, so granted agents stay ready. The whole publication is refused, naming the operation and reason, unless the immediately preceding release declares it identically and its skill is still active. An unlisted operation gets a new skill, and publication retires the skills the new release no longer binds. Their grants remain on record but authorise nothing, and each agent that held one needs its definition and evaluation renewed. Re-activating an older release does not restore a retired skill. See docs/INTEGRATION-BUILDER.md §3 for the refusal reasons and which declaration fields count.

The effect sequence is:

1. Governed dispatch enqueues the exact approved task and its authority binding.
2. The connector claims that task; for an integration effect the claim exposes routing pins, not the operation payload or provider idempotency key.
3. /v1/integration/claim-effect durably creates a pending intent but always returns perform: false. While it says start_required: true, no provider send has begun.
4. Immediately before egress, /v1/integration/start-effect must atomically cross that exact live queue claim to sending. It alone returns the operation payload, stable provider idempotency key and one-time completion token. Only then call the provider using the manifest’s declared mechanism.
5. Complete with the exact send generation/token through /v1/integration/complete-effect, even if the queue lease expired or an inert newer claim exists. A known outcome records the receipt and settles the task. An unknown outcome retains the send attempt without pretending an effect occurred; confirmed_sent creates the receipt, while only confirmed_not_sent permits replay of the same intent and provider idempotency key. Release is possible only before start.
6. Append read-back verification through /v1/integration/verify-effect and reconcile uncertainty.

A timeout after transmission can mean the provider performed the action. Resolve the same idempotency key or report unknown; do not turn uncertainty into a new operation. Completion and verification remain reachable for the exact retained release while paused so in-flight work can drain. Pausing does not undo an external action or erase its evidence.

Treat adoption of this send protocol as a maintenance-window cutover. Pause new connector claims, drain or reconcile existing provider handoffs, stop old connector processes and prove they retain no provider-ready payload. If that cannot be proved, rotate or revoke the provider credential. Legacy integration handoffs without a send token are refused by the schema, so every completion presents the token returned by start.

Connect consent-only providers without handing over a credential

Some providers grant access only after a person consents, as most accounting and calendar APIs do. A release reaches one by declaring connection.auth.kind oauth2_authorization_code: the authorization code grant with S256 PKCE always on. The declaration bounds what a consent may ask for: authorization, token and optional revocation URLs on allowed hosts, an optional issuer, the exact scope set, the client authentication, the refresh-token policy and a closed provider_tenant source. It names no client. The installation registers its own OAuth client with the provider, and a release that changes connection.auth in any way needs new consent.

Custody is split so that no single party holds everything:

Connecting takes two people. Propose connect_integration_oauth with the client id, a client_secret_ref (systemd-credential:<name> or file:/abs/path, never the secret) and the redirect URI CONSOLE_PUBLIC_ORIGIN followed by /integration-oauth/callback; a second operator approves it, and the approval stays usable for 1 to 72 hours. A re-authenticated operator then begins consent, checks the provider host and scopes the console shows, and consents in the same browser within ten minutes. The kernel refuses a callback whose state, session or issuer does not match, and a code exchange that grants other scopes or another provider account. There is one connection per integration; a different provider account needs disconnect, then connect.

The connector calls POST /v1/integration/access-token for its pinned release and receives a token only while that release is active, unpaused and consented under its exact auth declaration. 503 integration_access_token_unavailable with Retry-After means retry within the run deadline; 503 integration_oauth_not_configured is a kernel started without the delivery key. After a provider 401, report the generation once to POST /v1/integration/access-token-rejected, fetch once and retry once. The SDK’s HttpJsonRunner does this for you and never loads a secret for such a release. Disconnecting is a direct re-authenticated act: the kernel purges the access token at once, and the broker revokes the refresh token at the provider when a revocation_url is declared, then destroys it. docs/INTEGRATION-BUILDER.md lists the refusal codes, and docs/OPERATOR-RUNBOOK.md §5m the connection procedure.

Commission and maintain with evidence

In a separate test installation, retain schema/digest conformance, representative extraction previews and refusals, missing/status/time/money cases, pagination termination, throttling, retries, cursor restart and duplicate-capture tests. Effects additionally need duplicate/idempotency, clean rejection, ambiguous-outcome, crash recovery and read-back tests. Use synthetic or approved test data and scoped test-account secrets in CI; keep production credentials outside the build pipeline.

Then retain the installation’s no-write connection probe, real credential rotation/revocation, provider throttling and read-back evidence. Contract fixtures do not prove a live account. Record whether maintenance is first_party, partner or customer and the named owner.

On provider change, pause unsafe new work, settle old tasks/intents/unverified receipts, prepare a new immutable release, repeat tests and review its authority delta, including which effect operations the publication carries forward. Activation of a different version and retirement are fenced while older work is unsettled. The integration-health read names release, connector, cursor, task/effect and remediation failures rather than collapsing them into liveness. Open a governed remediation case naming that finding, an owner and a due date to assign maintenance work; it grants no retry or provider-call authority. Retirement requires those cases and live work to settle, then retains an immutable terminal snapshot and digest before clearing the active release. Check compatibility before upgrading Lagstyr and monitor sources and external effects during service.

Watch, page and contain in service

A record the signed extraction refuses is still answered 422, but the refusal is retained. The record becomes a confidential drift capsule kept for 30 days and then erased to a tombstone, and a live break scoped to the system, resource and integration key counts every repeat. Operator reads show capsule metadata only, and the response never echoes the record. A connector break is not repaired by a descriptor heal: the fix is a new signed release.

Health and remediation share one closed finding vocabulary, and the kernel evaluates one attention check with three paging conditions:

A pause by a person is containment and warns rather than pages. An oauth2_authorization_code integration adds five findings. oauth_connection_missing, oauth_refresh_failing and oauth_reconnect_required leave the connector without a usable token and page as release_blocked; oauth_disconnected_by_operator and oauth_revocation_unconfirmed only warn. None is a readiness input, so a provider outage never stales agent evaluations. The console’s /integrations pages, the kernel’s integration metrics and the optional integration_probe worker all read that one check, through GET /v1/integration-attention, GET /v1/integration-detail and GET /v1/integration-break.

Five routes act directly rather than through a proposal, and each re-authenticates the operator with a password or a fresh federated session and records that person as the actor: POST /v1/integration-pause, POST /v1/integration-break-close, POST /v1/integration-oauth/disconnect, POST /v1/integration-candidate-withdraw and POST /v1/integration-oauth/begin. The first four are containment. The fifth is not: it spends an approval a second operator has already granted, which is why it is direct without being a decision. Close a break once a release that accepts the record is active; lift a quarantine before closing its break. Opening and resolving remediation, quarantine and lift, retirement and resuming through a new release remain governed proposals, so acknowledging a page takes two people. On a solo installation, pausing is the fast containment.

Paging needs a delivery path. Either the kit’s Grafana rules page from the kernel’s metrics, or the operational-assurance pack runs one active integration monitor per key through the integration_probe unit and a routed alert channel. With neither, nothing pages and only the console’s attention section shows the condition; see Monitoring and routine operations. Triage follows the kit’s docs/OPERATOR-RUNBOOK.md#integration-pages.