SEO

Strip away the folklore and the mechanism is straightforward. A search engine has to find your pages, understand them, decide they're trustworthy, and judge that they answer a query better than the alternatives. Every legitimate SEO activity serves one of those four.

Crawling. Can a search engine reach the page? Blocked by robots.txt, buried behind a form, or reachable only via JavaScript that fails to execute means it may as well not exist.

Indexing. Having reached it, does the engine store it? Pages get excluded for being duplicates, thin, or judged not worth the storage. A large site with most pages unindexed has an indexing problem, not a ranking problem, and the two need completely different fixes.

Relevance. Does the page address the query? This is largely about topic coverage and matching how people actually phrase things.

Authority. Is this source credible? Links from other sites remain a significant signal, alongside brand recognition and topical consistency.

What has genuinely changed. Search now answers many questions directly, with AI-generated summaries above the results. This has reduced click-through on informational queries substantially — you can rank first and receive far fewer visits than the same position produced a few years ago. Queries that survive this well are ones where the user needs to go somewhere or do something: local search, comparison, transactional intent, and anything requiring a real database of options. Informational content that merely explains a concept is the most exposed category.

The practical implication: build for query types where a summary can't substitute for visiting your site. Directories, tools, comparison data, local listings, and anything requiring live inventory are structurally more defensible than explainer articles.

Technical SEO is unglamorous and it's where most sites lose traffic they never knew they could have had.

Get indexing under control. Register the site with Google Search Console and Bing Webmaster Tools on day one. The index coverage report tells you which pages are in, which are excluded, and why. Read it regularly — it's the closest thing to direct feedback the search engines provide.

Site architecture. Every important page should be reachable within about three clicks of the homepage. Orphaned pages with no internal links pointing to them tend not to be indexed and certainly don't rank. Internal linking is also the cheapest ranking lever available and the most neglected.

Canonical tags. Where the same content is reachable at multiple URLs — parameters, pagination, filters, trailing slashes — tell the engine which version is authoritative. Faceted navigation on directory and ecommerce sites generates enormous quantities of near-duplicate URLs and is a common cause of crawl budget being wasted on pages that will never rank.

Rendering. If content only appears after JavaScript runs, indexing becomes slower and less reliable. Server-side rendering or static generation removes the risk entirely. On a Next.js site this is largely a matter of not opting out of it accidentally.

Core Web Vitals. Loading, interactivity, and visual stability. These are a genuine ranking factor, though a modest one — a fast page won't outrank a substantially better answer. Fix them because slow sites convert worse, and treat the ranking benefit as secondary.

Structured data. Marking up your content in a machine-readable format so engines can classify it precisely and display rich results. For a directory site, correct LocalBusiness, Organization, Product, or Event markup materially affects how listings appear in results. Errors here are common and consequential — invalid or mismatched schema can disqualify you from rich results entirely, silently. Validate with the official testing tools and check Search Console's enhancement reports for warnings, because this is a place where a small mistake costs a lot of visibility.

The goal is not high-volume keywords. It's queries where you can realistically compete and where the traffic is worth having.

Classify by intent first:

• Informational — seeking knowledge. Highest volume, lowest commercial value, and most exposed to AI summaries.
• Navigational — looking for a specific site. Hard to win unless you're the destination.
• Commercial — comparing options before buying. High value.
• Transactional — ready to act. Highest value, lowest volume, most competitive.

Assess difficulty honestly. Look at who currently ranks. If the first page is national publishers and established brands, a new site will not displace them regardless of content quality. Look for queries where the current results are weak — outdated pages, forum threads, results that don't quite answer the question.

Long-tail is where new sites win. Specific multi-word queries have less volume individually and far less competition, and their intent is clearer. A hundred queries at thirty searches a month each is a better realistic target than one at three thousand.

Local and geographic modifiers are consistently undervalued. "Wedding venues in Hertfordshire" is winnable in a way that "wedding venues" is not, and the traffic converts better.

Build topic clusters, not isolated pages. Search engines assess topical authority. Ten interlinked pages covering a subject thoroughly outperform ten unrelated pages on ten subjects. Decide what your site is about and go deep.

Generating large numbers of pages from structured data. Powerful, and the fastest way to get a site penalised if done carelessly.

When it works. You have a genuine dataset where each combination produces a page a user would actually want. Venues by location and type. Products by specification. Services by area. The test is whether a real person searching that specific query would find the page useful.

When it fails. Generating permutations for their own sake, where most pages have almost no content and differ only in a swapped noun. Search engines classify this as doorway pages and the outcome ranges from non-indexing to a manual penalty.

Requirements for doing it properly:

• Real data on every page. Not a template with a variable substituted. Actual listings, actual details, actual differences.
• A minimum content threshold. Suppress pages that fall below it rather than publishing empty shells. It is better to have four hundred good pages than four thousand pages of which most are thin.
• Unique titles and descriptions. Generated, but genuinely distinct.
• Sensible internal linking between related pages, so crawlers can traverse the set.
• A managed sitemap, segmented if large, so you can monitor indexing rates per section.

Watch indexing ratio as your primary metric. If you publish ten thousand pages and two thousand get indexed, the engine is telling you the other eight thousand aren't worth storing. Adding more pages will not fix this. Improving the ones you have might.

The minimum every page needs, and the shape to check for:

export const metadata = {
  title: 'Roof repairs in Hitchin | Fixed prices from £180',
  description: 'What a roof repair costs in Hitchin...',
  alternates: { canonical: '/roof-repairs-hitchin' },
  openGraph: { images: [{ url: '/og.jpg', width: 1200, height: 630 }] },
};

Titles are the highest-leverage text on the site and the most commonly wasted. Home | Company Name tells a searcher nothing. Lead with the thing they typed, and where a price exists, put it in — it is the strongest click signal you have.

Add something the data alone doesn't provide. Aggregation, comparison, filtering, reviews, or genuinely useful editorial context. Pages that merely restate a public dataset compete with everyone else restating the same dataset — and increasingly with an AI summary of it.

Content that earns rankings answers the query more completely than what currently ranks. Read the top results and identify what they've missed — the follow-up question they don't address, the practical detail they skip, the data they don't have.

Match the format to the query. Some queries want a short direct answer, some want a comparison table, some want a step-by-step process. Look at what's ranking; the format is a signal about what satisfies the intent.

Update rather than accumulate. Refreshing an existing page that ranks on page two frequently produces more traffic than publishing something new. Most sites have more upside in their existing content than they realise, and auditing it is cheaper than writing more.

On AI-generated content. Search engines' stated position is that they reward helpful content regardless of production method, and penalise content produced primarily to manipulate rankings. In practice, publishing large volumes of generated content that adds nothing is a well-documented route to losing visibility. Use the tools for research, structure, and drafting — supply the specifics, data, and judgement yourself.

Links. Still a major authority signal. What works, in rough order of return:

• Being genuinely useful. Tools, datasets, and original research attract links without asking.
• Digital PR. Producing something newsworthy from data you hold.
• Industry and local directories. Modest value individually, cheap, and legitimate.
• Guest contributions to relevant publications, where the audience is real.

What to avoid: buying links, link exchange schemes, and mass low-quality directory submission. These violate the guidelines, and the risk is a manual penalty that removes you entirely.

Relevance beats volume. A handful of links from genuinely related sites outperforms hundreds from unrelated ones.

The checks worth running by hand. Before any tool, before any subscription:

# What does a crawler actually receive? Not what the browser renders.
curl -s https://yoursite.com/ | grep -o '<title>[^<]*</title>'

# Every page serving the same title is the most common serious defect.
curl -s https://yoursite.com/sitemap.xml | grep -o '<loc>[^<]*</loc>' \
  | sed -e 's|<loc>||' -e 's|</loc>||' \
  | while read u; do curl -s "$u" | grep -o '<title>[^<]*</title>'; done \
  | sort | uniq -c | sort -rn

# Does the canonical point at the host you actually use?
curl -s https://yoursite.com/page | grep -o '<link rel="canonical"[^>]*>'

That middle command has found more real problems on more sites than any paid tool I have used. If the count next to any title is greater than one, that is your priority.

Structured data, tested rather than assumed. Paste a page into Google's Rich Results Test. If it reports nothing, your JSON-LD is either absent from the served HTML or malformed — and "absent from the served HTML" is what happens when a JavaScript framework injects it on the client. Check with curl, not DevTools. DevTools shows you the page after JavaScript; a crawler frequently sees it before.

Track outcomes, not vanity metrics. Rankings fluctuate and vary by location and device. What matters is impressions and clicks in Search Console, and what those visitors do once they arrive. A page ranking third with a poor click-through rate has a title and description problem, not a ranking problem — and that's a same-day fix.

Segment your reporting. Branded versus non-branded queries, by page type, by intent. Aggregate traffic numbers hide the changes you need to see. A site can grow overall while its commercially valuable pages decline.

Expect volatility. Algorithm updates happen regularly and cause real movement. The correct response to a drop is to wait for the update to finish rolling out, then assess whether the decline is site-wide or concentrated in particular sections. Panic changes made mid-update make diagnosis impossible.

Understand your dependency. A business whose revenue depends entirely on organic search has one point of failure that it does not control. Build an email list, direct traffic, and other channels alongside. This is the same lesson as platform risk anywhere else, and people relearn it painfully after every major update.

Timeframes. Meaningful organic growth takes months. A new site typically sees little for the first few months regardless of quality. Anyone promising rapid results is describing either a tiny niche or something that won't last.

On the current transition. Search is changing faster than at any point in the last decade, and informational content is losing ground to direct answers. The response isn't to abandon organic search — it's to build assets that a generated summary can't replace. Live data, real inventory, tools that do something, local specificity, and genuine community. Those are the properties that keep earning clicks when the answer to a question no longer requires a visit.