01
Access
Crawler permissions
An explicit, deliberate policy per bot: GPTBot, ClaudeBot, PerplexityBot, Google-Extended, CCBot. Most sites have an accidental one.
We show you what each choice costs and earns.
Technical SEO
If they cannot crawl it, nothing else you do counts
Technical SEO is not a checklist of 200 warnings from a crawler. It is the small number of structural facts that decide whether your best pages are ever seen, rendered and understood, by Googlebot and now by the AI crawlers reading alongside it.
6
crawlers we test against, including GPTBot and PerplexityBot
Log-file
evidence behind every recommendation, not tool guesswork
48 hrs
from access to a prioritised, engineer-ready fix list
What actually breaks
I
Rendering, not markup
Most modern sites fail on what the crawler receives after JavaScript, not what the source contains. We test the rendered DOM the way an engine does, then fix the delivery, not the template.
II
Crawl budget spent on nothing
Faceted URLs, stale paginated series, parameter duplicates and dead redirect chains quietly absorb the crawl. Your new pages then wait weeks for attention they never get.
III
AI crawlers have different rules
GPTBot, ClaudeBot, PerplexityBot and Google-Extended each behave differently and many sites block them by accident in robots.txt. Being invisible to the answer layer is now a technical failure, not a strategy choice.
How We Work
We hand engineering a ticket, not a PDF. Reproduction steps, the exact fix, the expected effect, and how we will verify it.
Everything is evidenced from server logs and rendered-DOM captures, so nobody has to take a recommendation on faith.
Engineer-ready tickets
Log-file evidence
Verification pass
The audit, in four passes
Each pass produces its own fix list, ordered by revenue impact.
Log-file analysis of real crawler behaviour over 30 days, mapped against your sitemap and index coverage. We find the pages engines never reach, the ones they reach too often, and the duplicate clusters splitting your equity. Then we fix the canonical and robots logic that caused it.
Log-file analysis
Index coverage
Canonical logic
Robots and sitemaps
Rendered-DOM comparison across Googlebot, GPTBot and a plain fetch. Client-side-only content, hydration gaps, blocked resources and lazy-loaded copy that never resolves for a bot. We tell you exactly which content is invisible and what it costs.
Rendered DOM diff
Hydration gaps
Resource blocking
CrUX field data first, lab data second. We isolate the specific LCP element, the layout shift source and the long tasks, then hand over the change, not the score. Performance work stops when the metric moves, not when the report looks nicer.
LCP element
CLS source
INP long tasks
Schema that reflects what the page actually is, connected into a coherent entity graph across Organization, Person, Product and Article. This is the layer answer engines lean on hardest when deciding whether to name you.
Schema strategy
Entity graph
sameAs and knowledge panel
The AI-crawler layer
The part almost no technical audit covers, and the one that decides whether you exist in answers.
02
Extraction
Clean extraction
Answer-first HTML that a model can lift from without tripping over interstitials, tabs or accordion-hidden copy.
Tested against real model retrievals.
03
Freshness
Verifiable dates
Honest published and modified signals so a model can trust the recency of what it is quoting.
No fake freshness stamps.
04
Proof
Citation-ready pages
Facts placed where they can be attributed: unambiguous, self-contained, and near a stable anchor.
Aligned with our Information Gain rubric.
Engagement shape
- 01 Access and baseline Server logs, Search Console, analytics, staging. We capture the before state so the effect is provable. Day 1-2
- 02 Four-pass audit Crawl, render, speed, structure. Each with evidence and an owner. Week 1
- 03 Ticket handover Prioritised, engineer-ready, sized. Your team ships them or ours does. Week 2
- 04 Verification We re-crawl and re-render to confirm each fix landed, and close the ticket with evidence. Week 3-4
- 05 Monitoring Alerts on index drops, render regressions and robots changes, so it never silently breaks again. Ongoing
Next
Find out what the crawlers cannot see
Give us log access and 48 hours. You get the prioritised fix list, with evidence, whether or not you ever work with us.