Background Jobs

Besides serving API requests, the API server runs a scheduler for recurring work: draining the event outbox, retrying webhooks, expiring enrollments, cleaning history, synchronising the organisation directory. This page lists what runs and how the scheduler behaves in a multi-replica deployment.

One active scheduler

The scheduler runs inside every API server replica, but only one replica executes the jobs at a time. Replicas contend for a lock held in the database; the holder registers the jobs and renews the lock regularly, and the others keep checking. If the active replica stops or loses the lock, another replica takes over automatically within about a minute. No extra component is needed and no manual failover is involved.

Jobs are idempotent, so a takeover during a job simply results in the next run finishing the work.

Jobs

Job	Purpose	Cadence
Drain event outbox	Turns committed events into delivery work: in-process handlers, workflow runs and webhook deliveries. See Events & Delivery.	Frequent (seconds)
Retry webhook deliveries	Re-attempts failed webhook deliveries whose backoff period has elapsed.	Frequent (seconds)
Process pending actions	Applies time-based state changes: marks enrollments whose validity window has ended as completed and processes scheduled actions.	Half-hourly
Heartbeat	Reports the instance as alive to the central heartbeat service that keeps track of instances.	Half-hourly
Re-discover plugins	Scans installed packages for new or updated plugins and registers them. See Plugin Runtime.	Hourly
Clean up delivery tasks	Removes completed event delivery tasks older than the retention period.	Every few hours
Clean up webhook deliveries	Removes old webhook delivery records beyond the retention period.	Every few hours
Expire refresh tokens	Soft-deletes refresh tokens that have passed their expiry.	Daily
Directory synchronisation sweep	Reconciles users with the organisation directory when Microsoft sign-in is configured.	Configurable
Directory sweep supervisor	Adjusts the sweep cadence and re-registers the sweep if its configuration changed.	Periodic
Leadership check	Verifies that this replica still holds the scheduler lock and steps down if not.	Every minute

Retention periods for cleanup jobs are measured in days and keep recent history available in the web app while bounding table growth.

What operators should know

• The scheduler needs the same database access as the API. It starts with the server; there is nothing to enable.
• Frequent jobs are cheap when there is nothing to do; a healthy instance shows short, regular log lines for the drain and retry jobs.
• If deliveries or workflow runs stop advancing, check that at least one replica reports holding the scheduler lock, and that the database is reachable. See Observability.
• Reducing replicas to one is safe; the single replica becomes the leader.
• Long-running work (directory synchronisation on a large organisation) runs in the background and does not block API requests.