Deciding Which Facet Combinations to Index

The allowlist is the single most consequential SEO decision in a catalogue, and it is usually made implicitly by whoever wrote the filter component.

When to Use This Approach

Work through these tests when:

  • You are building the indexable allowlist for the first time and need a defensible basis for it.
  • Facet pages exist but are reported as “Crawled - currently not indexed”, which usually means they failed the distinctness test.
  • Merchandising wants a new filter combination promoted and you need criteria rather than an opinion.

Three Tests, All Required

A combination qualifies only if it passes all three. Passing two is the profile of a page that gets crawled, indexed briefly, and then dropped.

The three tests a facet combination must pass A combination passing through demand, inventory, and distinctness gates in sequence, with each failure branch showing the specific outcome that results. demand searched as a phrase inventory a full page of results distinctness differs from parent promote to a path URL indexed, in the sitemap no demand a view, not a page too few results thin, gets dropped too similar canonical dispute Each failure has a different symptom — which is how you diagnose a bad allowlist

Implementation Steps

Step 1: Apply the demand test

Demand means people search for the combination as a phrase, not that the filter is popular in your UI. Those are different things and only the first predicts search traffic.

# Existing impressions are the strongest evidence — the page already competes
curl -s -X POST \
  "https://searchconsole.googleapis.com/webmasters/v3/sites/$SITE/searchAnalytics/query" \
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -d '{"startDate":"2026-01-01","endDate":"2026-06-30",
       "dimensions":["query"],"rowLimit":25000}' \
  | jq -r '.rows[] | select(.impressions > 50) | [.keys[0], .impressions] | @tsv' \
  > queries.tsv

# Match queries against facet value vocabulary
python3 scripts/match-facet-phrases.py queries.tsv > candidate-combinations.tsv

Validation: A combination with impressions already accruing has passed empirically. One with only keyword-tool volume is a hypothesis — worth testing on a handful of combinations before promoting hundreds.

Step 2: Apply the inventory test

// scripts/inventory-check.mjs
const MIN_RESULTS = 24;         // one full page
const MIN_TROUGH  = 16;         // must hold through a seasonal dip

for (const combo of candidates) {
  const counts = await catalog.countByMonth(combo, { months: 12 });
  const min = Math.min(...counts);
  const now = counts.at(-1);
  const verdict = now >= MIN_RESULTS && min >= MIN_TROUGH ? 'pass' : 'fail';
  console.log([combo.key, now, min, verdict].join('\t'));
}

SEO impact: Testing the twelve-month trough rather than today’s count prevents promoting a combination that will be a three-product page for half the year.

Validation: Anything failing on min but passing on now is a seasonal candidate — worth revisiting, not worth promoting.

Step 3: Apply the distinctness test

A combination whose results are almost the same as its parent’s produces a near-duplicate page, which is what “Crawled - currently not indexed” usually means on a catalogue.

// scripts/distinctness.mjs — Jaccard overlap between result sets
export async function distinctness(combo) {
  const child = new Set(await catalog.ids(combo));
  const parent = new Set(await catalog.ids(combo.parent));
  const shared = [...child].filter((id) => parent.has(id)).length;
  const union = new Set([...child, ...parent]).size;
  return 1 - shared / union;      // 0 = identical, 1 = disjoint
}

Validation: Require a distinctness above roughly 0.3 in practice. A brand filter on a large category usually clears it easily; a price-range filter usually does not, which is why price bands are rarely worth indexing.

Step 4: Promote survivors properly

Promotion is more than adding a key to a set — the page needs the attributes of a real page.

// lib/indexable-facets.js — the allowlist, loaded from the CMS
export async function loadAllowlist() {
  const entries = await cms.facetPages.list({ where: { status: 'published' } });
  return entries.map((e) => ({
    key: e.combination,               // e.g. 'category+brand'
    path: e.path,                     // e.g. '/shoes/running/nike/'
    title: e.title,
    intro: e.introCopy,               // distinct copy — not the category's
    minResults: e.minResults ?? 24,
  }));
}
// The promoted page self-canonicalises and carries its own copy
export async function generateMetadata({ params }) {
  const entry = await allowlistEntry(params);
  return {
    title: entry.title,
    alternates: { canonical: `https://example.com${entry.path}` },
    robots: { index: true, follow: true },
  };
}

SEO impact: Distinct copy is what converts a filtered view into a page worth indexing. Without it, the combination passes the distinctness test on inventory but fails it on content.

Validation: Confirm the page appears in the sitemap, self-canonicalises, and has introductory copy that does not appear on the parent category.

Step 5: Re-run the tests on a schedule

Inventory moves. A combination that qualified in March can be a thin page by November.

# Weekly: demote anything that has fallen below the floor
node scripts/inventory-check.mjs --allowlist-only \
  | awk -F'\t' '$4 == "fail" { print $1 }' \
  > demotion-candidates.txt

Validation: Demotion means removing the sitemap entry and canonicalising to the parent — not deleting the URL, which would strand any links pointing at it.

The Shape of a Healthy Allowlist

Across catalogues the distribution is remarkably consistent, and a list that does not look like this is usually wrong in a predictable direction.

Typical composition of a healthy facet allowlist Bars showing single-facet pages making up most of the allowlist, a much smaller set of two-facet pairs, and effectively nothing at three facets, with annotations on what a deviation means. Allowlist entries by combination depth one facet ~85% of entries two facets ~15%, all demand-proven three facets rare — usually a mistake A list heavy at depth three means the tests were skipped, not that your catalogue is unusual

SEO Impact Summary

The allowlist is not a fixed artefact: entries enter it when they pass all three tests and leave it when inventory falls, so it needs a review cycle rather than a launch decision.

Lifecycle of a facet combination A combination moving from candidate to allowlisted when it passes the tests, and to demoted when inventory falls, with a path back to allowlisted when it recovers. candidate allowlisted demoted passes all three inventory falls recovers on the next weekly review
Signal What improves What breaks if misconfigured
Coverage quality Indexed facet pages earn impressions instead of being crawled and discarded Promoting thin combinations dilutes site-level quality signals
Canonical clarity Each promoted page is distinct enough to hold its own canonical Near-duplicate facet pages create disputes with their parent category
Merchandising velocity A reviewable list lets merchandisers propose pages without a deploy An unvalidated CMS field lets anyone publish a two-product page
Crawl efficiency A small deliberate set of facet URLs consumes proportionate budget A large allowlist recreates the explosion the allowlist exists to prevent

Measurable signals to watch:

  • Impressions per allowlisted facet page, with zero-impression pages reviewed for demotion.
  • Coverage state distribution across the allowlist, where “Crawled - currently not indexed” points at a distinctness failure.
  • Allowlist size over time, which should grow slowly and deliberately.

Edge Cases and Gotchas

Existing rankings override the demand test A combination already earning impressions has proved demand more convincingly than any keyword tool. Add those to the allowlist explicitly rather than re-deriving them, and never demote one because it failed a theoretical test.

Price and rating filters almost never qualify They rarely produce a distinct result set and are rarely searched as phrases. They are also unstable — the set changes whenever prices do. Treat them as views by default.

Multi-select facets are a different shape “Nike or Adidas” is not a combination in the sense used here; it is an ad-hoc set with no search demand behind it. Restrict the allowlist to single-value selections per facet.

Promoted pages need internal links A facet page in the sitemap with no internal link is discoverable but weakly signalled. Link the allowlist from the category page as a real anchor — those are exactly the links you kept when controlling facet URL explosion.

Frequently Asked Questions

How much inventory does a facet page need? Enough to fill a page of results and survive normal fluctuation — at minimum a full first page, with a buffer so a seasonal dip does not leave three products behind. A combination that only clears the bar at peak is a thin page for most of the year.

Can a facet page outrank the category it filters? Yes, and that is often correct: a specific query deserves a specific page. It becomes a problem only when the facet page starts ranking for the generic query the category should own, which is a sign the two are too similar.

Should the allowlist live in code or in the CMS? In the CMS, with validation. Merchandisers know which combinations have commercial value, and requiring a deploy for each change means the list stops being maintained. Validate every entry against the inventory and distinctness thresholds before it takes effect.

What happens to a demoted combination’s rankings? They transfer to the canonical target if the demotion is done with a canonical rather than a removal. Expect a transitional period where both are in flux; this is why demotion should be a scheduled review rather than a reaction to one bad week.

Part of: Faceted Navigation & URL Parameters

Related