SEO Automation
Internal Linking Automation That Turns a Crawl Into a Task List
My crawler reads every page on your site from a fresh sitemap, records where each link sits on the page, works out what job each page does, and then lists the links that should exist but do not. The result is a ranked spreadsheet a designer or developer can work through row by row.
- years in SEO
- 6+
- internal-linking fixes raised across 44 sites
- 3,800+
- person doing the work
- 1
What internal linking automation does for a site
Internal linking automation, as I build it, is a crawl-and-analysis pipeline that maps every link on your site, works out which links are missing between pages that belong together, and hands you a prioritised list of fixes. It does not insert links on its own. It tells a person exactly where to add them and why.
Internal links decide how easily people and search engines move between your pages. They show which pages you consider important and how topics relate. On most sites they are left to chance: menus get built once, blog posts link wherever the writer remembered, and new service or location pages go live with nothing pointing to them. Nobody notices until those pages fail to rank.
Checking this by hand on a site with a few hundred pages is tedious and incomplete. The pipeline does the crawling, tagging and comparison, and leaves the decisions to me and to whoever edits your site. It is one of the SEO automation pipelines I run for clients and maintain myself.
Key takeaways
- The crawl starts from a fresh sitemap, so it audits the site you have today, not a stale export.
- Every link is tagged by position - header, footer, sidebar, breadcrumb or body - because a menu link and an in-copy link do different jobs.
- Pages are classified by role (service, industry, location, hub, collection), which is what makes missing links detectable.
- Tasks are ranked P1 to P3, and template-level fixes are grouped so one change is not listed hundreds of times.
- The output is a working spreadsheet for whoever edits the site, not a score or a chart.
What's included in my internal linking automation
The work splits into three stages: crawl and tag, classify, then find the gaps. Each tab below is one component of the pipeline or one family of task it produces.
A full crawl seeded from a freshly fetched sitemap
The crawl begins by fetching your XML sitemap on the day, then visits every page it lists and records each link it finds. Starting from the live sitemap means new pages are included and removed pages are not, so the audit matches the site visitors see on the day it runs.
- Sitemap fetched at the start of each run
- Every listed page visited and parsed
- Every link on each page recorded
- Page HTML kept so the analysis can be rerun without refetching
Every internal link tagged by where it sits on the page
A link in the main menu, a link in the footer and a link in the middle of a paragraph are not equal. The crawler tags each one as header, footer, sidebar, breadcrumb or body. That lets the analysis tell apart a page that is only reachable through the footer from one that is recommended in context from related pages.
- Header and menu links tagged separately
- Footer and sidebar links tagged
- Breadcrumb trails recorded
- In-body contextual links isolated for analysis
Page roles classified so missing connections become visible
A crawler alone sees URLs. To say a link is missing, the analysis has to know what each page is for. Pages are classified into roles such as service, industry, location, hub and collection, and I check the classification before the gap analysis runs, since a wrong role produces wrong tasks.
- Service, industry and location pages identified
- Hub and pillar pages marked
- Collections and category pages recognised on shops
- Role list reviewed by me before tasks are generated
Missing links between services, industries and locations
The most valuable gaps are usually between money pages. A service page that never mentions the industries it serves, an industry page that does not link back to the services, or location pages that nothing points to are common. The analysis checks these pairings in both directions and lists every one that is missing.
- Service to industry links checked both ways
- Location pages checked for incoming links
- Orphaned money pages flagged
- Each task names the source page and the target
Pillar-to-cluster structure checked for broken chains
Where a site groups content under hub or pillar pages, each pillar should link to its cluster pages and each cluster page back to the pillar. The analysis maps those groups, lists the links missing in either direction, and notes clusters that have no pillar at all.
- Pillar pages linked to every page in their cluster
- Cluster pages checked for links back up
- Clusters without a pillar noted
- Suggested parent page for loose content
Unlinked mentions where the words are already on the page
Often the easiest link to add is one where the page already uses the right words and simply does not link them. The analysis finds mentions of a service, industry or location in body copy that are not linked to the page about it, and shows the surrounding text so an editor can add the link without rewriting anything.
- Mentions of other page topics found in body copy
- Surrounding sentence shown for context
- Target page suggested for each mention
- Quick wins an editor can make in minutes
Template-level fixes and broken links collapsed into single tasks
When the same gap appears on every page built from one template, listing it hundreds of times helps nobody. Those are grouped into one template-level task, such as adding a locations block to the footer. Broken internal links are listed separately with the page they appear on.
- Repeated gaps grouped by template
- One task per template change, not per page
- Broken internal links listed with their source
- Both kinds of fix ranked in the same spreadsheet
A P1 to P3 spreadsheet a designer or developer can work through
Every task lands in a spreadsheet, ranked P1, P2 or P3 so the most useful changes are done first. Each row says which page to edit, what to link to, and why. It is written for the person making the change, whether that is a designer working in the page builder or a developer editing templates.
- Tasks ranked P1, P2 or P3
- Source page, target page and reason on each row
- Template tasks separated from single-page edits
- Filterable by task type and priority
Experience
Internal Linking Automation from someone who has done it many times
You work with me from the first call to the last report. No account managers, no hand-offs to a junior team.
Book a free call- years in SEO
- 6+
- client websites worked on
- 60+
- internal-linking fixes raised across 44 sites
- 3,800+
- documented case studies
- 35
Why link position changes the diagnosis
Most crawlers count links. Counting alone hides the most common internal linking problems, because a page can have plenty of links and still be poorly connected.
Header links
Main navigation reaches every page on the site, which makes it powerful and blunt. It says a page matters, but not how it relates to the page a visitor is reading.
Footer and sidebar links
Repeated on every page and usually skimmed past. Useful for locations or service lists, but a page reachable only from the footer is weakly supported.
Breadcrumbs
They show hierarchy and help both visitors and crawlers understand where a page sits. Missing or flat breadcrumbs often point to a structure problem underneath.
Body links
Links placed within the copy, with descriptive anchor text, carry the clearest signal of relevance. These are the ones the task list most often asks you to add.
From sitemap to finished task list
A typical site goes from first crawl to delivered spreadsheet in about a week, mostly spent on checking roles and reviewing the tasks.
- 1
Fetch the sitemap and crawl
I fetch the current sitemap and crawl every page, tagging each link by position. Sites with bot protection may need a slower crawl or a browser-based fetch, which I arrange before starting.
- 2
Classify page roles
Each page is assigned a role. I review the list, correct mistakes and confirm the main services, since every later task depends on the roles being right.
- 3
Run the gap analysis
The analysis checks expected links between roles, pillar and cluster connections, unlinked mentions, template patterns and broken links, and generates a task for each gap.
- 4
Rank and group the tasks
Tasks are ranked P1 to P3, and repeated template gaps are collapsed into single changes so the list stays workable.
- 5
Review the list by hand
I read through the tasks, remove anything that would read awkwardly or does not make sense for your visitors, and add notes where a task needs explanation.
- 6
Hand over and recheck
You receive the spreadsheet. Once changes are made, a second crawl confirms what was done and catches anything new.
Free review
Get a free Internal Linking Automation review
Send me your site. Before the call I look at your rankings and pages, then tell you what I would fix first and what it is likely worth.
The task types the analysis produces
Each row in the spreadsheet belongs to one of these families. Who makes the change depends on whether it is a template edit or a single-page edit.
| Task type | What the crawl looks for | Usually handled by |
|---|---|---|
| Service and industry links | Money pages that should reference each other but do not | Content editor or designer |
| Location links | Area pages with few or no links pointing to them | Designer, often via a menu or footer block |
| Pillar to cluster | Hub pages and their cluster pages not linked in both directions | Content editor |
| Unlinked mentions | Body copy that names a topic without linking its page | Content editor |
| Template-level fixes | The same gap repeated on every page of one template | Developer or designer |
| Broken links | Internal links pointing to missing or redirected pages | Content editor or developer |
Priority is set per task, not per family. A missing link to your main service page can be P1 while a missing link between two blog posts is P3.
Automatic link plugins compared with a reviewed task list
Plugins that insert links automatically by keyword are tempting. They save time and they also make decisions nobody reviews.
Keyword-matching link plugin
- Inserts links wherever a phrase matches
- No view of which pages are money pages
- Cannot see header, footer or template gaps
- Anchor text and placement often clumsy
- Changes appear without anyone reading them
My crawl and task list
- Links proposed from page roles and site structure
- Service, industry and location pages prioritised
- Template-level problems identified and grouped
- Every task reviewed before handover
- Changes made by a person who reads the page
After the fixes
A recrawl shows whether the structure actually changed
Once your team has worked through the spreadsheet, a second crawl compares the new link map with the first and confirms which tasks are done. If new pages arrive from content writing automation or a site rebuild, the same crawl catches them before they go live unlinked.
Sites where linking gaps cost the most
The analysis is most useful on sites with many pages that should be connected by design: service businesses with service, industry and location pages, shops with collections and product sets, and sites with a library of guides grouped by topic. These structures generate predictable linking rules, and predictable rules are what a script can check reliably.
Smaller brochure sites benefit less. With fifteen pages, a person can review every link in an afternoon, and I would rather tell you that than sell you a crawl you do not need.
How internal linking connects to the rest of your SEO
Internal linking overlaps with several other jobs. Broken links, redirect chains and crawl depth are also part of technical SEO, and a full SEO audit includes a lighter review of site structure. If those are already in hand, this pipeline goes deeper on links specifically.
Anchor text and in-copy links are part of on-page SEO, and the unlinked mention tasks are often the quickest on-page wins on a site. When new topics come from content gap analysis automation, the role map from this crawl shows where each new page should be linked from.
For an overview of the pipelines and how a project is set up, see SEO automation services, or contact me with your domain.
Growth from real projects
Learn more about internal linking automation
Internal Linking Automation FAQs
Does internal linking automation add links to my site automatically?
No. The pipeline crawls, classifies and analyses, then produces a spreadsheet of recommended changes. A person makes each change, whether that is your designer, developer or me. Automatic insertion skips the judgement about whether a link reads naturally and helps the visitor, which is the part that matters most.
What does the spreadsheet look like?
One row per task, ranked P1, P2 or P3. Each row names the page to edit, the page to link to, the type of task and the reason. Template-level changes are grouped separately from single-page edits. It is meant to be filtered and worked through, and to be marked off as tasks are done.
Can you crawl sites with bot protection or a CAPTCHA?
Usually. Some hosting setups challenge automated requests, and those sites need a slower crawl or a browser-based fetch. A site behind a hard CAPTCHA may need you to allow my crawler or provide access. I will tell you before starting if your site is likely to need that.
How are page roles decided?
The pipeline proposes a role for each page from its URL, title and content - service, industry, location, hub, collection and so on - and I review the list before any tasks are generated. If you know your site well, a quick check of that list from you makes the result better still.
Does this work on Shopify and other e-commerce platforms?
Yes. On shops, collections and categories take the place of service pages, and the analysis looks at how collections link to each other and how products sit within them. Very large catalogues may be sampled rather than crawled product by product, and I will say so in the handover.
How often should the crawl be repeated?
After the first round of fixes, to confirm what changed, and then whenever the site gains a significant batch of new pages. A site that rarely changes does not need it often, and a recheck after major edits is usually enough.
Find out what internal linking automation would flag on your site
Send me your domain. I will run the crawl and share a sample of the highest-priority tasks, so you can see the kind of fixes involved before deciding on the full spreadsheet.