Technical SEO Workflow: From Crawl to Fix in 5 Hours

First published: 25 June 2026 · Last updated: 25 June 2026

Hour 1

Crawl + inventory

Screaming Frog full crawl. Robots.txt, sitemaps, indexable URL count, response code distribution.

Output: crawl manifest CSV + 5-line site overview

Hour 2

Indexability triage

GSC index coverage, noindex audit, canonical audit, redirect chains, duplicate content.

Output: indexability defect list with severity

Hour 3

CWV + rendering

PageSpeed Insights spot-checks, DebugBear (or web-vitals JS) for INP RUM, JavaScript rendering audit.

Output: CWV defect list per page template

Hour 4

Schema + AI crawler access

Rich Results Test on key templates, GPTBot/ClaudeBot/PerplexityBot allow audit, schema gap analysis.

Output: schema and AI access defect list

Hour 5

Prioritise + write fix-list

Sort defects by impact and effort. Write the one-page fix-list. Estimate dev hours per fix.

Output: 1-page prioritised fix-list, dev-ready

The technical SEO audit category has bloated. The standard agency deliverable is a 60-100 page PDF that takes a senior practitioner 15-20 hours to produce, costs the client $3-8k, and never gets fully implemented because the recommendations are too numerous and the priority is unclear. We have inherited dozens of these documents from prior providers when we onboard new clients. Almost none of them led to actionable engineering work. The 5-hour workflow is the corrective. The premise is that 80% of the value of a technical SEO audit comes from identifying the 10-15 highest-leverage defects and getting them fixed. The remaining 20% (the long-tail of minor issues) can wait until the next quarterly audit. Constraining the audit to 5 hours forces ruthless prioritisation, produces a deliverable the engineering team will actually action, and lets the agency run more frequent audits (quarterly instead of annually) which catches drift earlier. This article is the practitioner's hour-by-hour breakdown. It assumes a mid-sized site (under 2,000 indexed pages), a competent practitioner, and access to the standard tool stack. For larger or more complex sites, multiply by 1.5-2x; the workflow phases stay the same. For broader context, our existing AI SEO audit workflow covers the AI-era extension, and our best free SEO tools guide covers the no-cost alternatives where applicable.

Hour 1: Crawl and Inventory

The opening hour is data acquisition. Goals: a complete crawl manifest, a robots.txt and sitemap audit, and a 5-line overview of the site's basic shape. Tools: Screaming Frog SEO Spider (or Sitebulb), curl, browser. Procedure:

1. Configure the crawler. Set user-agent to Googlebot Smartphone (mobile-first indexing standard since 2024). Enable JavaScript rendering if the site uses client-side frameworks. Set crawl delay to 0.5-1s to avoid rate-limit issues. Include external links and images for completeness.

1. Run the crawl. Wait. For a 2,000-page site, expect 15-30 minutes depending on server response time and JS rendering. Use the wait time to do step 3.

1. Audit robots.txt. Fetch it manually. Confirm it returns 200, has no syntax errors, does not still carry a staging-era global disallow. Confirm AI crawler directives explicitly: GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot, CCBot, Google-Extended, Bytespider. Note whether each is allowed, disallowed, or undefined.

1. Audit XML sitemaps. Fetch the sitemap index and each child sitemap. Confirm they return 200, are referenced in robots.txt, contain only canonical URLs, and reflect the actual site structure. Note any sitemaps with stale URLs (returning 404 or noindex).

1. Extract the inventory. When the crawl finishes, export the URL list with response codes, content types, and indexability status. Build a 5-line overview:

• Total URLs crawled
• URLs returning 200 (indexable candidates)
• URLs returning 3xx (redirect handling)
• URLs returning 4xx (defects)
• URLs returning 5xx (server defects)

Output artefact: crawl manifest CSV plus a 5-line site overview committed to the working doc. This is the data substrate for hours 2-4.

Hour 2: Indexability Triage

Indexability is the single most consequential dimension. A page that is not indexed cannot rank, no matter how good the content. The hour focuses on making sure every page that should be indexed is, and every page that should not be is not. Tools: Google Search Console (Index Coverage report), Screaming Frog filters, browser. Procedure:

1. GSC Index Coverage. Pull the report. Note total indexed pages, total excluded, and the breakdown of excluded reasons (blocked by robots, noindex, canonical points elsewhere, crawled not indexed, discovered not indexed, alternate page with proper canonical, etc.).

1. Noindex audit. Filter the crawl manifest for `meta robots = noindex`. For each, confirm the noindex is intentional. Common defects: noindex left on staging-deployed pages, noindex on parameterised URLs that should be canonicalised instead, noindex on pages that have valuable content but were de-prioritised by an old strategy.

1. Canonical audit. Filter for canonical mismatches: pages that canonical to themselves (correct), pages that canonical to a different URL (intentional consolidation? or accidental orphaning?), pages with no canonical declared (default to self), pages with canonical pointing to a URL that returns 404 (broken).

1. Redirect chain audit. Filter for redirect chains over 2 hops. Each hop is a small crawl-budget tax. Three-hop chains (`A → B → C → D`) should be collapsed to single hops (`A → D`).

1. Duplicate content audit. Identify pages with identical or near-identical title tags, identical H1s, or near-identical body content. Common defects: parameterised URLs creating duplicates, paginated content not handled with rel=next/prev or canonicalisation, faceted navigation creating combinatorial URL explosions.

1. Crawled-not-indexed deep-dive. GSC flags pages that Google has crawled but chosen not to index. These are signal-rich. Often the cause is content quality (thin content, low E-E-A-T) or duplication. Sample 10-20 such URLs and diagnose patterns.

Output artefact: indexability defect list with severity rating (P0 critical, P1 high, P2 medium). Each defect has the URL pattern, the cause, and the remediation note.

Hour 3: Core Web Vitals and Rendering

Core Web Vitals are part of the page experience signal and have been since 2021. INP joined LCP and CLS in March 2024. The hour focuses on identifying CWV failures by template type and confirming JavaScript rendering behaves correctly. Tools: PageSpeed Insights, DebugBear or SpeedCurve (if available), Chrome DevTools, web-vitals JS library (if RUM data is needed). Procedure:

1. Identify page templates. Most sites have 5-10 distinct page templates: homepage, category, product/service, blog post, blog index, contact, etc. The CWV audit runs at the template level, not URL level (auditing 2,000 URLs individually is impractical and wasteful).

1. PSI per template. For each template, run PageSpeed Insights on a representative URL. Note the field data (CrUX) for LCP, INP, CLS at 75th percentile mobile. Note the lab data (Lighthouse) where field data is unavailable.

1. Identify failing templates. Templates with any of LCP > 2.5s, INP > 200ms, or CLS > 0.1 at p75 mobile are failing the page experience signal. List them with the failing metric.

1. Diagnose top 3 failing templates. For the worst-performing templates, open the PSI report's diagnostics section. Note the named root causes (render-blocking resources, large images without lazy-load, layout shift from web fonts, long tasks from JS execution, etc.).

1. JavaScript rendering audit. For SPAs or sites with heavy client-side JS, fetch a representative page with JS disabled (Chrome DevTools → Settings → Disable JavaScript) and confirm the primary content is in the initial HTML response. Pages where the content disappears with JS disabled are invisible to AI crawlers and partially invisible to Googlebot.

1. INP-specific deep-dive. INP is the lowest-passing CWV metric in 2026 (about 36% of mobile origins fail). For failing templates, identify whether the issue is input delay (heavy main-thread work), processing time (slow event handlers), or presentation delay (layout thrashing, large DOM). See our INP field guide for the diagnostic deep-dive.

Output artefact: CWV defect list per template, with the failing metric and the diagnosed root cause. The fix list goes to the dev team in hour 5.

Template

Traffic share

Failing metric

Root cause

Priority

Product page

42%

INP 612ms

Heavy add-to-cart handler, large DOM (reviews)

P0

Blog post

28%

LCP 3.4s

Hero image not preloaded, font loading shift

P1

Category page

14%

CLS 0.18

Filter widget loading after initial render

P2

Contact page

2%

INP 240ms

Form validator running on every keystroke

P3

Hour 4: Structured Data and AI Crawler Access

Structured data and AI crawler access are the 2024-2026 additions to the technical SEO surface area. Both directly affect AI engine citation rates and indirectly affect Google rich result eligibility. Tools: Google Rich Results Test, Schema.org documentation, Schema App or Schema Markup Validator, robots.txt parser. Procedure:

1. Schema audit per template. For each major page template, fetch a representative URL through the Rich Results Test. Note: which schemas are present, which are valid, which have warnings, which have errors. Common defects: missing required properties (review aggregateRating without review count, recipe without prep time, product without offers), invalid type usage, schema present but not rendered (JS-injected schema that Googlebot does not see).

1. Schema gap analysis. Identify schemas that are missing but should be present. Common gaps: FAQPage schema absent on FAQ-heavy content (single biggest GEO leverage), HowTo schema absent on step-by-step content, Organization schema absent or incomplete on the homepage (no `sameAs`, no logo, no contact), LocalBusiness schema absent on service-area pages, Article or BlogPosting schema absent on blog posts.

1. AI crawler access audit. Re-confirm robots.txt directives for AI crawlers from hour 1. For any disallow, confirm the business decision. Most disallows are 2023 panic leftovers and should be removed unless there is a deliberate policy reason. See our existing AI crawlers guide for the per-bot breakdown.

1. AI crawler reachability test. Beyond robots.txt, confirm the site is structurally crawlable by AI engines: content in initial HTML (not JS-only), reasonable response time, no aggressive WAF rules blocking bot user-agents at the application layer (Cloudflare's "Block AI Crawlers" feature is a common trap; check the Cloudflare dashboard).

1. Entity-graph signals. Spot-check Wikidata, Wikipedia, and Crunchbase for the brand. If absent, flag for Pillar 3 of the AEO audit. Out of scope for the technical sprint per se but the technical sprint surfaces the gap.

Output artefact: schema and AI access defect list. Most defects in this hour are quick fixes (5-30 minutes of dev time each) but high-leverage for AI search visibility.

Hour 5: Prioritise and Write the Fix-List

The final hour is synthesis. Goals: a one-page prioritised fix-list, dev-ready, with effort estimates and expected impact. Tools: spreadsheet, the working doc from hours 1-4. Procedure:

1. Aggregate defects. Pull all defects from hours 2-4 into a single list. Each row: defect description, affected URL pattern (or template), severity, root cause, recommended fix.

1. Score impact. Score each defect on impact 1-5: 5 = directly affects ranking on multiple high-traffic templates, 4 = affects ranking on one high-traffic template, 3 = affects ranking on lower-traffic templates, 2 = affects user experience but minimal ranking impact, 1 = polish.

1. Score effort. Score each defect on effort 1-5 in dev-hours: 5 = under 30 minutes, 4 = 1-3 hours, 3 = 1-2 days, 2 = 1-2 weeks, 1 = multi-week project.

1. Sort by impact × effort score. Highest impact and lowest effort go first. P0 list = items scoring 16+ (impact 4-5, effort 4-5). P1 list = items scoring 9-15. P2 list = items scoring 4-8. P3 list = items scoring under 4.

1. Write the deliverable. One page, three sections:

• P0 (fix this sprint, under 1 dev day total): 5-10 items.
• P1 (fix this quarter): 10-20 items.
• P2/P3 (backlog, revisit next audit): the rest.

Each item: defect, fix, owner, dev-hours estimate, expected outcome.

1. Write the executive summary. 3-5 sentences for the client: "Audited [N] indexed pages over 5 hours. Surfaced [X] defects, [Y] of which are P0. The single highest-leverage fix is [the top P0]. Expected impact of P0 fixes: [outcome]. Recommended action: ship P0 within 2 weeks, P1 within 90 days."

Output artefact: the one-page fix-list plus the 5-sentence executive summary. This is the entire deliverable. No 60-page PDF.

Defect

Impact

Effort

Score

Priority

GPTBot disallowed in robots.txt (left over from 2023)

5

5

25

P0

FAQPage schema missing on 47 service pages

5

4

20

P0

Product page INP at 612ms (failing Core Web Vital)

5

2

10

P1 (high impact, multi-week fix)

14 redirect chains over 2 hops

3

5

15

P0

3 important pages incorrectly noindexed

4

5

20

P0

Hero image not preloaded on blog template (LCP 3.4s)

4

4

16

P0

Organization schema missing sameAs links

3

5

15

P0

Worked Example: SG Service Business 5-Hour Sprint

Concrete example. Client: SG B2B service company, 340 indexed pages, sprint run March 2026, results measured at day 60. Hour 1 findings: Crawl returned 340 URLs. 287 returned 200, 41 returned 301 (mostly old blog redirects), 8 returned 404, 4 returned 5xx. Robots.txt had `User-agent: GPTBot \n Disallow: /` (P0 finding). Sitemap referenced 12 URLs that returned 404. Hour 2 findings: GSC Index Coverage showed 213 indexed, 74 excluded. Of excluded: 32 noindex (15 intentional, 17 accidental orphans of old blog cleanup), 14 canonical points elsewhere (correct), 22 crawled not indexed (mostly thin auto-generated tag pages), 6 redirect not followed. 4 redirect chains over 2 hops. Hour 3 findings: Service template at LCP 3.1s mobile (failing), product template at INP 480ms (failing), blog post template at CLS 0.14 (failing). All failures caused by render-blocking JS, large unoptimised images, late-loading widgets. Hour 4 findings: FAQPage schema absent on 8 service pages with FAQ sections (P0). Organization schema present but missing `sameAs` links (P0). Article schema present on blog. AI crawler access blocked at robots.txt (already flagged in Hour 1). Hour 5 deliverable: 1-page fix-list with 7 P0 items, 11 P1 items. P0 items shipped within 14 days. P1 items shipped within 60 days. Day 60 results:

• Indexed page count up from 213 to 247 (the 17 accidental orphans + 14 thin pages restructured into hubs).
• AI citation rate on tracked prompt set up from 6% to 23% (mostly attributable to the AI crawler unblock + FAQ schema).
• Service template LCP down from 3.1s to 1.9s (passing).
• Product template INP down from 480ms to 220ms (Needs Improvement, not Good — multi-week fix continues).
• Organic traffic +18% on the service templates.

Total agency time: 5 hours audit + 6 hours implementation oversight = 11 hours. Compared to a traditional 60-page audit (15-20 hours) followed by half-implementation, the time-boxed sprint produced more shipped fixes in less calendar time.

When to Use the 5-Hour Sprint vs the Full Audit

The 5-hour sprint is the right tool for:

• Quarterly check-ins on retainer clients where the baseline is known.
• Onboarding diagnostics to identify the highest-leverage fixes in the first 30 days.
• Post-launch sweeps after a site migration or major redesign.
• Mid-sized sites under 2,000 indexed pages.

The full audit (15-20 hours) is the right tool for:

• First-time engagements on enterprise sites over 10,000 pages.
• Pre-acquisition due diligence where comprehensiveness matters more than speed.
• Multi-domain or multi-region rollouts where the surface area is large.
• Post-penalty recovery diagnostics where every contributing factor matters.

For most SG agency clients (mid-market, SMB, low-thousand page count), the 5-hour sprint quarterly is the right cadence. The full audit annually is overkill if the quarterly sprints are honest.

Frequently Asked Questions

Can I really audit a site in 5 hours? Doesn't a proper audit take much longer?

A proper diagnostic audit (here is what is broken) takes 5 hours when sequenced correctly on a mid-sized site. A proper remediation report (here is exactly how to fix every line of code) takes longer, but that is engineering work, not audit work. The 5-hour sprint is opinionated about the boundary: surface the defects with severity and root cause, write the fix-list, hand to engineering. The engineering team does the implementation specs. Confusing the two is what bloats traditional audits to 60 pages and 20 hours.

What if the site has more than 2,000 pages?

Multiply the time per phase by 1.5-2x for sites between 2,000 and 10,000 pages, and 3-5x for larger sites. The phase structure stays the same. For very large sites (over 50,000 pages), the audit is template-based not URL-based: identify all distinct templates, sample 5-10 URLs per template, audit at the template level. URL-by-URL auditing at scale is not productive; pattern-level auditing is.

Do I need paid tools?

Screaming Frog free (500 URL limit) is workable for sites under 500 pages. Above that, the paid licence ($259/year) pays for itself on the first sprint. PageSpeed Insights, Search Console, Rich Results Test, and Schema Markup Validator are all free. DebugBear or SpeedCurve for RUM is optional but valuable on larger accounts ($50-200/mo). Total tool cost for a competent audit: $260-500/year.

How do I handle JavaScript-rendered sites?

Enable JS rendering in Screaming Frog (slower crawl but accurate). Cross-reference with the JS-disabled fetch (DevTools → Disable JavaScript) to confirm what content is in initial HTML versus client-injected. SPAs and heavily-hydrated sites should also be audited for AI crawler reachability specifically: most AI crawlers do not execute JS, so JS-only content is invisible to them. Pre-rendering or SSR is the standard fix.

How does the 5-hour sprint integrate with an AEO or GEO audit?

The 5-hour sprint covers the technical foundation that AEO/GEO sit on top of. Specifically: Hour 1 (robots.txt) and Hour 4 (AI crawler access, schema) overlap with Pillar 1 of the AEO audit. After the 5-hour technical sprint, the AEO audit picks up at Pillar 2 (content extractability), Pillar 3 (entity authority), Pillar 4 (citation outcomes). See our AI SEO audit workflow for the broader integration.

What about Core Web Vitals — should I optimise even if I am passing?

If passing all three at p75 mobile, the marginal SEO benefit of going from "Good" to "Excellent" is small. Energy is better spent on other defects. The exception is contested SERPs where competitors are also passing CWV: in those cases, marginal CWV improvements can be a tiebreaker. The general rule: pass first, then optimise everywhere else, then come back to CWV refinement only if other levers are exhausted.

Related reading

• AI SEO audit workflow: extending the technical sprint into AI search visibility
• Google indexing issues 2026: the indexability defects you will surface in Hour 2
• Ahrefs vs Semrush vs Moz: tool stack for the broader SEO programme