Duplicate content

Duplicate content is when the same or nearly the same text can be found on several URLs, within your own website or spread across different sites. It happens much more often than you'd think, and usually not on purpose. An online store that shows the same product in three categories, a site that can be reached both with and without "www", or a series of local pages where only the town name changes: they're all forms of duplicate content.

The biggest misconception is that you automatically get a "penalty" for it. Google says duplicate content in itself isn't grounds for a penalty, unless it's clearly intended to mislead search engines. The real problem is more practical: the search engine has to choose which version to show, and that choice doesn't always go the way you want.

Why is duplicate content a problem?

• Confusion about the right version: Google chooses a main version itself, possibly a URL you'd rather not see in the results.
• Diluted signals: links and other signals are split across several URLs instead of coming together on one page.
• Wasted crawling: on large sites, crawlers spend time on copies instead of new pages, which puts pressure on your crawl budget.
• Little added value: pages that barely differ from each other give visitors and AI systems little reason to use them separately.

Common causes

1. Variants of the same URL: http and https, with and without www, with and without a trailing slash.
2. URL parameters for filters, sorting or tracking.
3. Product descriptions copied word for word from the supplier.
4. Print versions or separate mobile URLs.
5. Template pages per region with near-identical text.

How to fix it

The right solution depends on the situation. If a copy exists only for technical reasons, redirect it with a 301 redirect. If several variants need to stay accessible, such as a product in different categories, use a canonical tag that points to the preferred version. Pages you don't want in the search results at all can be excluded with noindex. And for pages with similar content, rewriting is often the best choice: give each page its own added value.

An example: local pages for a painting and decorating firm

A painting and decorating firm from Leeds creates pages for twelve towns in the region. Each page contains the same text; only the place name changes. Search engines will probably see those pages as near-identical and may only show a few of them. By adding local projects, photos of completed work, specific travel times and relevant customer reviews to each page, every page gets its own reason to exist.

Duplicate content and AI answers

AI tools such as ChatGPT and Perplexity look for sources that answer a question clearly and uniquely. If the same text is in ten places, there's little reason to quote your version in particular. Original content with your own experience and concrete details sets you apart, both in classic search results and in AI answers. A technical SEO analysis helps you quickly track down duplicate URLs and missing canonicals.