You can keep useful duplicate pages accessible while using canonical signals to identify the preferred URL. Redirect an old address when that version is genuinely retired. Use noindex when exclusion from search is the goal, and crawl controls when unnecessary URL combinations are consuming resources. These are different decisions, not four settings to apply together to every duplicate.
Check whether the pages are actually duplicates
Compare the main content and the task a visitor can complete. Shared navigation, a common brand description or the same page template does not by itself make two pages interchangeable. Google's canonicalization overview explains that it selects a representative URL from duplicate or very similar pages; Google can also do this without an explicit canonical declaration.
For example, /guide/ and /guide/?utm_source=email may show the same article. In contrast, /products/?page=2 can contain the next set of products, and /product/?id=42 can identify a specific product. Read the delivered content before deciding that the question mark should disappear.
| Goal | Possible method | Condition to verify |
|---|---|---|
| Keep useful duplicate versions accessible | rel=canonical | The preferred page represents the same or very similar content |
| Retire an old URL permanently | Permanent redirect, such as 301 or 308 | There is a real corresponding destination and the old view is no longer needed |
| Keep a page available but out of search | Page-level noindex | The crawler can fetch the page and read the rule |
| Reduce crawling of unwanted filter combinations | Control URL generation and use scoped robots rules | Useful products, pagination and intended landing pages remain discoverable |
For retained versions, make one clear canonical choice
Google's canonical guidance treats redirects and rel=canonical as strong signals and sitemap inclusion as a weaker one. They are signals, not a promise that Google will adopt your preferred URL. Align internal links and sitemap entries with the choice rather than sending contradictory messages.
If the two guide URLs above show the same article, a teaching implementation would place this in the valid HTML head of both the duplicate and the preferred page:
<link rel="canonical" href="https://example.com/guide/">
This does not redirect the visitor. Verify the final HTML and rendered head, not just an SEO plugin field. There is no canonical requirement to place it within the first 15 lines, immediately after the title or within an invented byte threshold. Absolute URLs are recommended; relative URLs are supported but easier to misconfigure. See the canonical implementation and verification guide for HTML and HTTP-header examples.
Non-HTML resources, including PDFs, can use a canonical Link response header when their content supports that relationship. A detailed manual and a short product overview are not automatically duplicates. Server configuration also depends on the software: Apache's .htaccess is not a universal Nginx instruction.
Preserve pages with different jobs
Google's pagination guidance says to give each page its own URL and not canonicalize every page in a sequence to page one. Keep usable, crawlable links to later pages. A different set of items needs a different assessment from an identical tracking alias.
For color and size options, first choose the single-page or multi-page product-variant model. For real branches, assess location information separately from duplicate aliases. A complete translated page likewise should not automatically point its canonical to another language just because it is a translation.
Use noindex for exclusion, not as a canonical substitute
Google's noindex documentation requires crawler access so that the rule can be read. Blocking the URL in robots.txt can prevent that. Google also advises against using noindex to choose the canonical within one site. Decide whether you want a preferred representative or exclusion before adding a directive.
Large spaces of sorting and filtering combinations are a crawl-management problem as well. The faceted-navigation documentation describes controls for combinations that do not need search visibility. Scope those controls carefully. A canonical may reduce duplicate crawling over time, but it does not instantly reserve a fixed crawl budget or remove every noncanonical URL from results.
Review parameter handling by purpose and combination before deploying a rewrite rule. Preserve purchase options, pagination, authentication and attribution; removing every query string is not a safe normalization policy.
Verify the change at three different layers
- Application behavior: keep a before-and-after sample of URL, content, product selection and expected destination. Test direct entry in a fresh session as well as clicks through the site.
- Delivered signals: inspect statuses, redirects, response headers, initial HTML and rendered canonicals after relevant caches are refreshed. Check that templates and plugins do not produce conflicting declarations. Use a scoped Screaming Frog crawl to extend coverage.
- Google's processing: compare declared and selected canonicals in the indexed data of URL Inspection. A live test can check the current response but cannot predict Google's canonical choice or guarantee indexing.
When Google selects a different page, investigate the signals and content using its canonical troubleshooting guidance. Repeatedly changing a date is not a solution. Record deployment, recrawling, canonical selection and search performance separately; there is no universal deadline or guaranteed percentage improvement. 中文版:不删除页面时怎样处理重复内容。