Catalog backfill for Build Backlinks Hub
Upsert link-directory hub sites and opportunities from JSON after deploy or local db reset.
Connect your agent today
Draft from chat, review in your calendar, and publish only what you approve.
What this does
The catalog backfill applies Build Backlinks destination rows from a versioned JSON catalog into:
- link_directory_sites
- link_directory_opportunities
- link_directory_site_tags_association
New hub destinations (for example Skool) can ship in the repo catalog without a dedicated SQL seed migration. Operators run the script with the Supabase service role key, the same operational model as pnpm backend:db:promote-platform-admin.
Source of truth: backend/scripts/link_directory_catalog_backfill.catalog.json
Runner: backend/scripts/backfill_link_directory_catalog.mjs
When to use this vs Secret admin
| Situation | Use |
|---|---|
| Ship a new catalog site from the repo (append JSON, run backfill) | This CLI backfill |
| One-off copy edits, publish toggles, or opportunity tweaks on existing rows | Secret admin → Link directory → Sites |
| Refresh domain rating after a site exists | Catalog DR refresh |
The backfill skips overwriting populated editorial fields on existing rows unless you pass --force. Secret admin remains the day-to-day editor for live copy.
Before you start
The script loads env from backend/.env.development.local by default (or backend/.env.production.local for production).
| Variable | Role |
|---|---|
| PUBLIC_SUPABASE_URL | Supabase project URL |
| SUPABASE_SECRET_KEY | Service role — required for direct table upserts |
Warning
This path writes through the service role and does not run the API cache invalidation that secret-admin mutations trigger. Published hub and site-detail responses may be cached for up to about five minutes. After production backfill, wait for TTL expiry or restart your cache layer before you expect every edge to show new rows immediately.
Local development (after db reset)
SQL migrations and legacy seeds repopulate categories, tags, and exemplar sites, but catalog-only destinations (JSON-only entries) are not re-inserted automatically when you reset.
Reset the local database
From backend/:
supabase db reset See Database & migrations for the full local workflow (CLI, env keys, optional typegen).
Run the backfill
From the repo root:
pnpm backend:db:backfill-link-directory-catalog Limit to one destination while testing:
pnpm backend:db:backfill-link-directory-catalog -- --slug skool Verify in the app
With the web and API dev servers running (local Supabase + backend on port 3000; Vite on https://localhost:5173), open Build Backlinks and /build-backlinks/skool. Confirm:
- Skool appears on the hub grid with the link-directory logo.
- Two opportunities: profile-bio (Connect Skool → connect channel) and group-post (Schedule with OpenQuok).
- Step copy and effort badges match link_directory_catalog_backfill.catalog.json.
Note
If CACHE_ENABLED is true but Redis is not running, link-directory API responses can hang until cache read times out. Use a running Redis instance or disable cache in backend/.env.development.local while verifying locally.
Idempotency: run the same backfill again without --force — existing Skool rows should log SKIP SITE (populated) and SKIP OPP (populated).
Production rollout
Dry-run first:
pnpm backend:db:backfill-link-directory-catalog:prod -- --dry-run --slug skool Apply:
pnpm backend:db:backfill-link-directory-catalog:prod -- --slug skool The :prod script alias passes --env-file .env.production.local.
After the site row exists in production, you can optionally replace editor-estimate domain_rating from JSON with a live DataForSEO value. Commands, env vars, billing, and hub cache behavior are documented on Catalog DR refresh — skip that step if you are fine keeping the catalog estimate.
Command flags
| Flag | Effect |
|---|---|
| --dry-run | Log planned upserts; no writes |
| --force | Replace existing site/opportunity rows and re-sync tag associations |
| --slug skool | Process only this site slug (repeat the flag for multiple slugs) |
| --list-missing | List catalog slugs that are not yet in link_directory_sites |
| --catalog <path> | Override the default JSON file |
| --env-file <path> | Env file relative to backend/ |
Equivalent from the backend package:
pnpm --filter ./backend run db:backfill-link-directory-catalog Pass script options after — when invoking from the repo root.
Idempotency and safety
- Second run without --force: Existing rows with editorial content are left intact; missing rows are inserted.
- --force: Overwrites catalog-shaped fields and replaces tag associations from tag_slugs in JSON.
- Stable UUIDs in the catalog keep upserts deterministic across environments.
Adding a future catalog site
- Append a new entry under entries in link_directory_catalog_backfill.catalog.json (use new UUIDs; resolve slugs for categories, opportunity types, and tags).
- Add a self-contained hub logo at web/static/icons/link-directory/<slug>.svg (inline paths or embedded data URI). Do not use an SVG that only references another URL — hub cards load logo_url with
<img>, which cannot resolve nested external images. - Run backfill with --slug <slug> in dev, then production — no migration PR required for catalog-only destinations.