Whether you are a business owner, an SEO professional, or a web developer, a big problem you have to deal with is duplicate content on your or your client’s website. This happens when there are slight differences in the URL structure, like www.example.com vs. example.com. Though similar, each URL would be its own page in the eyes of a search engine algorithm. Duplicate content also arises frequently on e-commerce sites. This happens when you add tracking code to the end of the URL, list items in ascending order vs. descending order, etc.

 

So, what can you do about all the duplicate content? You have three options you can implement, which should be followed in this order:

 

  1. Use best practices, including normalizing the URLs from the start, constancy with linking to the desired URL, and submitting a sitemap that only lists the preferred URLs. This helps you avoid the issue of duplicate content.
  2. 301 Redirect – Tells the search engine that the page has permanently moved to a new location. Users do not see the old page; they are automatically redirected to the correct page.
  3. Rel=”canonical” – This element is added to the head of the page, and suggests to the search engine, “ignore this ugly URL, and instead only look at this pretty URL.”  It looks like this: <link rel=”canonical” href=”http://www.example.com” />

 

Best Practices to Minimize the Use of the 301 Redirect and Rel=“Canonical”

Ideally, the use of both a rel=canonical element and a 301 redirect would not be necessary. Through your content management system (CMS), you should make sure that there is standard procedure for the generation of new URLs so that you minimize the amount of duplicate content. If you already have multiple URLs for the same content, suggest to the search engines which URL you prefer. Do this through consistency in your linking and by submitting a sitemap that only contains the preferred URLs.

 

301 Redirects

Though the best practices are a good place to start, there is almost no way to get around the use of a 301 redirect in some parts of your site. Normalizing URLs and consistency with your linking just won’t be enough. For instance, if your site goes by example.com, people may link to you by manually typing in www.example.com. This will result in a split of traffic, which would make it impossible for you to accurately monitor traffic to single pages on your site. The only correct way to remedy this is through a 301 permanent redirect. When a user clicks the inaccurately entered link, they will be automatically redirected to the correct url, resulting in accurate analytics data. A 301 redirect is the best fix for existing duplicate content, because it tells search engines that the old URL has permanently moved to another location. This effectively “eliminates” any old page or inaccurate link, making it easier for search engines and users to find the most relevant data available on your site, and assuring that your analytics data is as accurate as possible.

 

Rel= “Canonical”

A 301 redirect is the best option for duplicate content, because it is a permanent fix, but adding the redirect isn’t always possible. This is often true for e-commerce sites. Landing pages created for PPC ads, URLs with session IDs or tracking codes, products listed by ascending vs. descending order, and URLs that follow breadcrumb navigation may all contain the same information, but have slightly different URLs. Redirecting and effectively eliminating any of those pages is not an option, so how do we get search engines to overlook the duplicate content? Through the rel=“canonical” tag! This tag is implemented in the code in the HTML <head> of the web pages with the “ugly” URLs – i.e. those with the session IDs or tracking codes. The tag “hints” to search engines (yes, they can choose to ignore the hint) that they should not look at the content of the page with the ugly URL, but instead direct their attention to the chosen, “pretty” URL. This means that most search engines will ignore the extra URLs and be able to crawl your website faster without getting bogged down by those pages that contain duplicate content.

 

A Word of Caution in Using the Rel= “Canonical” Tag

The most important thing to remember about the canonical tag is that unlike the 301 redirect, a canonical tag is only a hint to the search engines, not a direction. Google or Bing can choose to ignore your rel=“canonical” tag if they think that one of the pages with the “ugly URLs” contains the most relevant information for the user. Don’t think of the canonical tag as a “fix-all” for your website. Try to solve any issues you can through your CMS or by using 301 redirects first. The canonical tag can only be applied within a domain or sub-domain. If you want to direct search engines from one domain to another, you will have to use a 301 redirect.

 

The bottom line is that the canonical tag should only be used when necessary. Applying best practices in URL creation and linking and using a 301 redirect should be used whenever possible, but in those instances when duplicate content can’t be avoided, a rel=“canonical” tag is a fantastic way to assure that search engines accurately index your site.

 

Things to Remember:

 When to Use a 301 Redirect:

  • For old pages
  • Changing non-friendly URLs to friendly URLs
  • “www” vs. non “www” – redirect one to the other
  • If you need to cross domains

 

 When to Use a Rel=“Canonical” tag:

  • When URLs contain session IDs or tracking codes
  • For PPC landing pages
  • Ascending vs. descending product list
  • URLs that follow breadcrumb navigation for tracking purposes

 

 

Have you ever used the Rel=Canonical tag?  Have any questions for our search team?  Let us know in the comments section below!