One engine pulling new CRM leads onto the board

A single scheduled job pulls every connected account's CRM sales pipeline, takes only the opportunities sitting in that account's mapped new-lead stage, and mints the ones not already held.

What it is

A cron running every ten minutes. Per account it resolves the pipeline and new-lead stage from the stage map, pages through the opportunity search (capped at 5,000), and issues one of five verdicts per lead: suppressed (durable delete-list hit), exists_synth, exists_opp, same_person (matched on email, normalised phone, or name), or created. Once a card exists here, the platform owns its stage.

Also called: lead sync · leads from GoHighLevel · duplicate leads · why is this lead here twice

See it
One engine pulling new CRM leads onto the board
Apr
May
Jun
Jul
Ivy Brubaker — 30×40 Garage
Framing
Ivy Brubaker — 40×64 Shop
Concrete
Marlin Hoover — Hobby Shop
Trim
Sutter Kline — 30×40 Garage
Framing
Sutter Kline — Riding Arena
Concrete
Framing crewConcrete crewTrim crew
A run report: per account, counts by verdict (created / exists / same_person / suppressed) with the five verdicts as coloured chips. Sample data — no customer information appears here.
How it works
  1. 1The cron lists accounts with is_connected true.
  2. 2The CRM adapter checks the account has a pipeline id and a new-lead stage id mapped, plus a working token.
  3. 3Opportunities are paged; anything not in the mapped new-lead stage is skipped.
  4. 4Suppression, existing synthetic contacts and existing opportunities are loaded once per account and used to dedupe.
  5. 5Genuinely new leads are minted as New Lead cards with lifecycle events logged.
Why we built it

Keith 08-22, quoted at the top of the route: 'we just want one engine doing one thing that accounts for all cases and conditions.' The consolidation story is recorded in full: a browser poll ran the same job every 60 seconds in every open tab alongside this cron, 'They minted row ids differently, so a lead that arrived down both paths became two rows neither engine recognised as the other's' — two named accounts ended up on the board twice. The browser engine 'could not see a lead that arrived while nobody had a tab open; it ran once per tab instead of once; and it deduplicated against a lead-visibility-scoped read, so a rep who could not see another rep's leads could not deduplicate against them either and minted a copy.' Migration 263 retired it and added unique indexes so a third write path cannot recreate the problem. There is even a note that the docstring itself used to lie: 'This docstring previously claimed "not yet scheduled" long after it was — it was the only description anyone had of a job writing to live tenant data, and it was wrong.'

The problem
  • Two competing ingestion engines minted duplicate cards for the same person.
  • Leads arriving with no browser tab open were never ingested.
  • Phone numbers were normalised differently between engines, defeating deduplication.
  • Removed leads reappeared on the next sync.
Sound familiar?
What you get
Leads arrive whether or not anyone is logged in.
One normaliser and one id scheme, so duplicates stop at the source.
A removed lead stays removed, durably.

See it on your own jobs

Twenty minutes, your numbers, no slide deck. We’ll build one of your real buildings in front of you and send you the estimate link at the end — yours to keep either way.

or keep browsing features →