Configuration - Worker
Configure and deploy background workers (BullMQ, Redis, and railway services).
Connect your agent today
Draft from chat, review in your calendar, and publish only what you approve.
Overview
When backend/config/orchestratorFlows.ts sets transport: “bullmq” for a flow, the API only enqueues work to Redis / BullMQ. Long-running worker processes in the orchestrator/ package execute those jobs by importing backend services and repositories in the same Node process (they do not call your HTTP API to run business logic).
Deploy workers on an always-on host (for example Railway). Serverless-only platforms are a poor fit because workers loop until SIGTERM. Railway describes persistent containers under Services → persistent services.
API and worker processes stay stateless: timers enqueue or refresh work using Postgres and Redis as the source of truth (no in-memory-only schedules required for correctness).
Directory SEO weekly refresh (housekeeping)
The housekeeping worker runs notification email digest flushing and, when configured, DirectorySeoService.runWeeklyRefresh(). Each run reloads stale domains from Postgres (main profile website URLs on paid tiers plus workspace competitors) — it does not keep a private list of “domains left this week” in process memory. Per-domain refresh is idempotent: respect DIRECTORY_SEO_CACHE_TTL_HOURS, upsert seo_domain_metric_snapshots, append history when ratings change or the weekly sample policy applies. Any housekeeping replica can execute the timer; a crash mid-run is safe to retry.
Note
pg_cron in Postgres remains SQL-only (for example token cleanup). DataForSEO HTTP never runs inside cron.schedule. See Database → Scheduled jobs and Directory SEO.
Queue dashboard (Bull Board)
Platform admins can inspect and manage BullMQ jobs (pause / resume queues, open jobs, retry, clean, and related controls) from the web using Bull Board embedded on the Queue dashboard page.
Who can use it
- You must be the platform admin role; see RBAC for how roles are modeled. Admin-only API routes use the same auth surface.
- In the app, open the protected area:
/secret-admin→ Queue dashboard in the sidebar (or the same link on the super-admin index).
What to configure
- Redis — Same REDIS_* (and optional REDIS_BULLMQ_DB) as the API. See Redis cache and Redis & queues.
- Supabase — PUBLIC_SUPABASE_URL, PUBLIC_SUPABASE_PUBLISHABLE_KEY, and SUPABASE_SECRET_KEY for server-side tables. Legacy JWT keys are not accepted; see Supabase.
- Storage (scheduled social post publishing) — If you publish posts with media, workers must be able to build a public HTTPS URL for uploaded objects. Set STORAGE_PROVIDER (typically
r2) and STORAGE_R2_PUBLIC_BASE_URL (your public bucket hostname, no trailing slash). See R2 or local storage. - Per worker — Provider OAuth secrets for integration refresh; email provider keys plus optional DATAFORSEO_API_KEY / DIRECTORY_SEO_* for housekeeping (notification email + weekly Directory SEO refresh); the same provider or channel credentials the API would use to publish for scheduled social posts. A short template lives in the repo at orchestrator/.env.production.example.
- Health & errors — Each worker exposes
GET /healthandGET /health/status(Redis PING + optional queue depth). Port: host PORT on Railway, else ORCHESTRATOR_WORKER_HEALTH_PORT (default3091; set0to disable). Optional SENTRY_DSN / SENTRY_ENABLED match the API so worker crashes reach Sentry (tagopenquok.worker). - Maintenance cutover — When MAINTENANCE_MODE=freeze_writes matches the API, workers exit on startup without consuming BullMQ jobs. Set MAINTENANCE_RETRY_AFTER_SECONDS and MAINTENANCE_BYPASS_SECRET on workers only if you mirror the API (optional). See Maintenance mode.