Skip to content

Configuration - Worker

Configure and deploy background workers (BullMQ, Redis, and railway services).

3 min read

Connect your agent today

Draft from chat, review in your calendar, and publish only what you approve.

Start your free trial

Overview

When backend/config/orchestratorFlows.ts sets transport: “bullmq” for a flow, the API only enqueues work to Redis / BullMQ. Long-running worker processes in the orchestrator/ package execute those jobs by importing backend services and repositories in the same Node process (they do not call your HTTP API to run business logic).

Deploy workers on an always-on host (for example Railway). Serverless-only platforms are a poor fit because workers loop until SIGTERM. Railway describes persistent containers under Services → persistent services.

API and worker processes stay stateless: timers enqueue or refresh work using Postgres and Redis as the source of truth (no in-memory-only schedules required for correctness).

Directory SEO weekly refresh (housekeeping)

The housekeeping worker runs notification email digest flushing and, when configured, DirectorySeoService.runWeeklyRefresh(). Each run reloads stale domains from Postgres (main profile website URLs on paid tiers plus workspace competitors) — it does not keep a private list of “domains left this week” in process memory. Per-domain refresh is idempotent: respect DIRECTORY_SEO_CACHE_TTL_HOURS, upsert seo_domain_metric_snapshots, append history when ratings change or the weekly sample policy applies. Any housekeeping replica can execute the timer; a crash mid-run is safe to retry.

Queue dashboard (Bull Board)

Platform admins can inspect and manage BullMQ jobs (pause / resume queues, open jobs, retry, clean, and related controls) from the web using Bull Board embedded on the Queue dashboard page.

Who can use it

  • You must be the platform admin role; see RBAC for how roles are modeled. Admin-only API routes use the same auth surface.
  • In the app, open the protected area: /secret-admin → Queue dashboard in the sidebar (or the same link on the super-admin index).

What to configure

  • Redis — Same REDIS_* (and optional REDIS_BULLMQ_DB) as the API. See Redis cache and Redis & queues.
  • Supabase — PUBLIC_SUPABASE_URL, PUBLIC_SUPABASE_PUBLISHABLE_KEY, and SUPABASE_SECRET_KEY for server-side tables. Legacy JWT keys are not accepted; see Supabase.
  • Storage (scheduled social post publishing) — If you publish posts with media, workers must be able to build a public HTTPS URL for uploaded objects. Set STORAGE_PROVIDER (typically r2) and STORAGE_R2_PUBLIC_BASE_URL (your public bucket hostname, no trailing slash). See R2 or local storage.
  • Per worker — Provider OAuth secrets for integration refresh; email provider keys plus optional DATAFORSEO_API_KEY / DIRECTORY_SEO_* for housekeeping (notification email + weekly Directory SEO refresh); the same provider or channel credentials the API would use to publish for scheduled social posts. A short template lives in the repo at orchestrator/.env.production.example.
  • Health & errors — Each worker exposes GET /health and GET /health/status (Redis PING + optional queue depth). Port: host PORT on Railway, else ORCHESTRATOR_WORKER_HEALTH_PORT (default 3091; set 0 to disable). Optional SENTRY_DSN / SENTRY_ENABLED match the API so worker crashes reach Sentry (tag openquok.worker).
  • Maintenance cutover — When MAINTENANCE_MODE=freeze_writes matches the API, workers exit on startup without consuming BullMQ jobs. Set MAINTENANCE_RETRY_AFTER_SECONDS and MAINTENANCE_BYPASS_SECRET on workers only if you mirror the API (optional). See Maintenance mode.

Guides

Search documentation
Find a docs page
Discord Support