Directory SEO & DataForSEO
Configure live domain rating cache, DataForSEO credentials, and Directory analytics entitlements on the OpenQuok backend.
Connect your agent today
Draft from chat, review in your calendar, and publish only what you approve.
Overview
Directory SEO stores global, rebuildable domain rating snapshots for:
- The website URL on each paid workspace’s main business profile
- Competitor domains users add under Directory analytics
OpenQuok calls DataForSEO for those hostnames:
- POST /v3/backlinks/summary/live — current domain rating (snapshot cache and weekly housekeeping samples).
- POST /v3/backlinks/timeseries_summary/live — daily rank history import when a user opens Directory analytics history or overview (hybrid backfill; see below).
Results land in seo_domain_metric_snapshots (cache + backfill metadata) and seo_domain_metric_history (chart points). The API and housekeeping worker are stateless: each refresh reads TTL from Postgres and upserts idempotently — no per-pod in-memory schedule.
User-facing docs: Directory analytics.
Environment variables
Set these on the backend API (manual refresh and read APIs) and on the housekeeping worker (weekly sweep). Names match backend/config/GlobalConfig.ts and backend/.env.development.example.
| Variable | Default | Role |
|---|---|---|
| DATAFORSEO_API_KEY | empty | Base64-encoded login:password for HTTP Basic auth. When empty, refresh endpoints no-op and return existing cache rows only. |
| DIRECTORY_SEO_ENABLED | true | Master switch for Directory SEO features in the API. |
| DIRECTORY_SEO_CACHE_TTL_HOURS | 168 | Global cache TTL per domain (7 days). Skips paid API calls while a snapshot is fresh unless force is added in a future tier. |
| DIRECTORY_SEO_WEEKLY_REFRESH_ENABLED | true | When false, the housekeeping timer does not run runWeeklyRefresh(). |
| DIRECTORY_SEO_WEEKLY_REFRESH_INTERVAL_MS | 604800000 | How often the housekeeping process enqueues the weekly Directory SEO sweep (default 7 days). |
| DIRECTORY_SEO_HISTORY_BACKFILL_MAX_DAYS | 90 | Upper bound on how many calendar days of daily rank the API imports from timeseries_summary/live on backfill (capped at 366 in code). |
| DIRECTORY_SEO_HISTORY_BACKFILL_TTL_DAYS | 7 | Per domain, skip a repeat timeseries import while provider_history_backfilled_at on the snapshot is newer than this TTL. |
Note
Plan limits (directory_launch_enabled for My Products and directory programmatic surfaces, directory_seo_enabled for analytics, competitor caps per workspace) live in subscription catalog code — not env vars. Free tiers disable directory launch and Directory analytics regardless of DataForSEO credentials.
Local dev and API credits
Leave DATAFORSEO_API_KEY empty in backend/.env.development.local unless you want paid DataForSEO calls. With a key set, the API still respects DIRECTORY_SEO_CACHE_TTL_HOURS for normal Directory analytics loads, but manual refresh, new or stale domains, bulk catalog DR refresh, the housekeeping weekly sweep, and the first history or overview load per domain per backfill TTL can each trigger billable requests (summary/live and/or timeseries_summary/live). Disable DIRECTORY_SEO_WEEKLY_REFRESH_ENABLED when running the worker locally with a production key.
Provider history backfill (API)
When a entitled workspace loads Directory analytics history or overview, the API may import up to DIRECTORY_SEO_HISTORY_BACKFILL_MAX_DAYS days of daily rank from POST /v3/backlinks/timeseries_summary/live (group_range: day, same rank_scale: one_hundred as summary). The housekeeping worker does not run this path — only user-driven reads through DirectorySeoWorkspaceService.
Behavior:
- Skipped when DATAFORSEO_API_KEY is unset or Directory SEO is disabled.
- Skipped while provider_history_backfilled_at on seo_domain_metric_snapshots is within DIRECTORY_SEO_HISTORY_BACKFILL_TTL_DAYS.
- On success, provider rows are stored with origin
provider_timeseries(replacing prior provider rows in the same window). Ongoing snapshot refreshes still appendlive_refreshsamples when rating or weekly cadence warrants it. - Failures are logged; the history endpoint still returns whatever rows are already in Postgres.
Data tier (operators)
| Table | Treat as |
|---|---|
| directory_seo_competitors | User config — do not truncate |
| seo_domain_metric_snapshots | Rebuildable cache — safe to delete and refill; provider_history_backfilled_at tracks last timeseries import |
| seo_domain_metric_history | Chart history — origin live_refresh (housekeeping/API samples) or provider_timeseries (daily import); provider rows for a domain can be replaced on backfill without deleting live samples |
Writes to metric tables use the backend service role; workspace members read snapshots through session APIs with RLS on competitor rows.
Scheduled refresh vs pg_cron
Postgres pg_cron stays SQL-only (for example refresh token cleanup). DataForSEO HTTP runs in Node on the housekeeping worker, not inside cron.schedule in SQL.
See Database → Scheduled jobs and Configuration - Worker.