Appearance
Observability
This page describes what the API server exposes for monitoring and diagnostics, and how operators are expected to watch an instance.
Health probes
The server exposes a lightweight health endpoint that answers as soon as the API process accepts connections. Kubernetes readiness and liveness probes point at it, so a pod is considered ready when it can serve requests and is restarted if the process stops responding. The probe deliberately does not depend on the database: a transient database outage should surface as request errors and alerts, not as a storm of pod restarts.
Database availability is checked by the init steps before the API starts (see Database & Migrations) and is visible at runtime through request errors and the scheduler's log lines.
Logging
The server writes structured logs to standard output, which the platform collects. Each request is tagged with a correlation identifier so that log lines from the request, its background handlers and the resulting workflow runs can be joined. Log levels follow the usual convention: debug (development only), info for lifecycle and job activity, warning for handled errors such as rejected requests, error for failures that need attention.
Logs are written to be free of personal and clinical data: handled errors record the path, method, error code and status, not the payload. When correlating with a user report, use identifiers (request, resource, run) rather than content.
Errors and error codes
Every error response has the same shape: an error object with a stable code, a human-readable message and optional details. Codes are defined in an error catalogue that maps each code to its default HTTP status and message; the catalogue can be exported with the CLI so client developers can handle codes rather than parse messages. Unexpected failures are returned as a generic server error without internal details, and the full cause is written to the log with the correlation identifier.
Validation errors on list queries (unknown filter or sort field) name the valid fields in the details, which makes integration mistakes quick to spot.
The API contract
The server publishes its OpenAPI document at runtime. It is generated from the same schema as the models, so it is always the contract of the running version; integrators can diff two versions' documents to see what changed. See API Access.
What to monitor
There is no separate metrics endpoint in the core server; monitoring is done through the platform's log pipeline, the database and the product's own audit views. Recommended signals:
| Signal | Where | Why |
|---|---|---|
| Request error rate and latency | Ingress or gateway metrics; server logs | Detects outages and slow database queries. |
| Pod restarts and readiness | Kubernetes | A restarting pod usually points at configuration or resources. |
| Database connections, size, slow queries | The database's own monitoring | Capacity planning; the server is bound by the database. |
| Outbox and delivery backlog | Delivery task counts by status (database or event views) | A growing pending backlog means the drain job is not running. |
| Dead-lettered deliveries | Webhook delivery log; delivery task views | External endpoints failing; needs a human decision. |
| Workflow run failures | Workflow runs view in the web app; run status counts | Broken mappings, failing plugins or external APIs. |
| Scheduler leadership | Server logs | Exactly one replica should report holding the lock. |
| Heartbeat | Central heartbeat service | Confirms the instance is alive and reachable. |
Integrations with external clinical systems emit a structured log line per operation (request, outcome, timing) and offer a hook for operators who want to publish those as metrics or traces in their own tooling.
Audit trail in the product
Several views double as an audit trail without any extra tooling:
- Workflow runs: every run with its nodes, the recorded input and output of each node, and errors.
- Webhook deliveries: every attempt with status code, response and timing, and a manual retry.
- Sessions: active sessions per user, with revocation.
- Questionnaire revision history: who changed what, with diffs, and the ability to restore.
- Event delivery tasks: the state of every event on its way to consumers.
Client-side error reporting
The web app reports unexpected front-end errors to an error tracking service configured by the operator, with the release version and the route where the error occurred. Personal data is not included in reports. The mobile app records questionnaire engine errors locally and surfaces them in its diagnostics screen so that support can read them from the device when needed.