How SiteNexis Coordinates Specialized Analysis in a Single Audit
A detailed look at the BullMQ-backed agent architecture that runs a complete AI Retrieval and Machine Trust audit — sequenced across seven phases, with graceful partial failure built into every agent.
When you submit a domain to SiteNexis, you trigger a coordinated audit pipeline. Specialized modules communicate through the audit orchestration and BullMQ event path while the Infrastructure Agent sequences execution and preserves partial results when a module fails. Which modules run depends on the audit path and plan; timing varies by domain and external dependencies.
The Seven Execution Phases
Phase 1 runs the Crawl Agent sequentially — everything else depends on having pages to analyze. Phase 2 runs the SEO Agent and Schema Agent in parallel: technical signals first, since later agents consume schema data. Phase 3 runs the AI Retrieval Agent, Entity Agent, and Performance Agent in parallel — all depend on the crawled page set but not on each other. Phase 4 runs Citation and Semantic Trust in parallel, both consuming the entity intelligence output from Phase 3. Phase 5 is the Layer 4 payload: Retrieval Simulation, Machine Trust, Temporal Authority, Recommendation Mapping, and Synthetic Entity Detection run in parallel, all depending on the combined output of Phases 2–4. Phase 6 runs the Visualization Agent to pre-compute graph layouts. Phase 7 is the Reporting Agent, which generates and uploads the PDF report.
●The Infrastructure Agent is the only agent that writes audit.status to the database. Every other agent emits agent:progress and agent:completed events via BullMQ — the Infrastructure Agent listens and aggregates. This constraint prevents conflicting status writes and ensures the audit state machine remains consistent.
Specialized Modules and Their Responsibilities
- Crawl Agent — Full-site crawl, HTML parsing, chunk extraction, entity pre-extraction. Max 500 pages.
- Technical SEO Agent — Title, meta, canonical, robots.txt, sitemap, broken links. Fully programmatic.
- AI Retrieval Agent — Machine readability, chunk quality, AI extractability via Claude API.
- Entity Intelligence Agent — Entity detection, consistency, coverage, disambiguation, Perception Graph.
- Citation Intelligence Agent — Citation probability using weighted formula. No AI API.
- Semantic Trust Agent — Authorship, organisational, content, and structural trust. Contradiction detection on top 20 pages only.
- Schema Agent — Schema detection, validation, field audit, snippet auto-generation.
- Performance Agent — Lighthouse on top 5 pages by PageRank. Never all pages.
- Retrieval Simulation Agent — 6-stage simulation on top 30 pages. Deterministic.
- Machine Trust Agent — Entity credibility, schema alignment, external validation, contradiction, decay signals.
- Temporal Authority Agent — Authority velocity, semantic drift, decay modeling, freshness scoring.
- Recommendation Mapping Agent — Surface coverage across AI Overviews, chat, voice, agent discovery.
- Synthetic Entity Agent — Pattern detection for fake entities, manufactured authority, schema manipulation.
- Visualization Agent — Pre-computes D3 graph layouts. Caches in Redis for 24h.
- Reporting Agent — PDF generation via @react-pdf/renderer, S3 upload, Report record creation.
- Infrastructure Agent — Orchestration, sequencing, error recovery, status management.
Graceful Partial Failure
Every Layer 4 agent implements partial failure handling. If the Claude API fails mid-audit for the Machine Trust Agent's contradiction detection, the agent logs a warning, sets contradictionAbsenceScore to null, and continues with the remaining four sub-scores. The audit does not fail. The partial result is preserved. The Infrastructure Agent marks the audit complete with a partial status flag on the affected module. Ten minutes of compute should not disappear because one API call returned a 429.
The Message Protocol
Modules emit a standardized AgentMessage with audit id, module id, lifecycle event and optional diagnostic payload. The dashboard receives an SSE stream backed by persisted progress snapshots; the stream route checks for updates approximately every two seconds. This is live/near-real-time progress delivery, not a direct push of every internal event and not a polling-free transport.
◆Layer 4 agents gate behind the layer4Analysis plan limit. If a user on the Starter plan submits a domain, phases 1–4 run normally. Phase 5 is skipped entirely — the Infrastructure Agent checks plan limits before enqueuing Layer 4 jobs.
See the full agent pipeline in action on your own domain.
Start free audit