Avoid Duplicate Content Issue while using Content Delivery Network (CDN) – SEO Consideration

Updated on Jan 12, 2023by App 10

At Crunchify we use MaxCDN as our Content Delivery Network partner. MaxCDN is one of the best Content Delivery Network (CDN) provider so far I came across. Last week I noticed some weird search result which shows cdn.crunchify.com in search result page (SERP) and immediately it catches my attention because this may cause my SEO index go down.

In this tutorial will show quick tip on how to avoid this situation and fix it completely. If you have similar questions like this then you are at right place.

How do I avoid creating duplicate content using a CDN
CDNs and Duplicate Content
Prevent CDN from caching webpages to avoid duplicate contents
Duplicate content issues using CDN. How to fix?
CDN using CNAME causing duplicate content

Other must read: Step by Step Guide to Setup MaxCDN

Let’s get started

Often times a blog is only hosting or caching resources like CSS, image, javascript and video files on the content delivery network. How to fix this?

Method-1

Use rel="canonical". Check this out.

Before enabling Canonical Header option setting under MaxCDN panel

After enabling Canonical Header option

MaxCDN setting

This setting will tell Google or Yahoo or Bing the original version of the page and it should take note of this hint. However according this is only a hint and not 100% to be relied upon.

Method-2

Robots.txt is a file that restricts access to your site by web crawlers, stopping pages from being indexed on search engine results. Add a robots.txt file to your CDN's root directory to disallow page crawl by search engines like Google. In the case of a CDN, assets are mirrored, so content is duplicated and then listed by crawlers on search engines, which is bad as pages rank lower.

User-agent: *
Disallow: /

NOTE: do NOT put under your blog’s root directory

If you liked this article, then please share it on social media. Have a question or suggestion? Please leave a comment to start the discussion.

About App

I'm an Engineer by profession, Blogger by passion & Founder of Crunchify, LLC, the largest free blogging & technical resource site for beginners. Love SEO, SaaS, #webperf, WordPress, Java. With over 16 millions+ pageviews/month, Crunchify has changed the life of over thousands of individual around the globe teaching Java & Web Tech for FREE.

10 Comments...

Rohan Singh says

Aug 26, 2022 at 7:44 am

Thanks for great informative
post. This article is very useful for the solve Duplicate Content issue.

Reply
Priya Ganesh says

May 16, 2017 at 6:55 am

Thanks for sharing such a great post. Already using your methods to fix my website. Keep sharing your tips!

Reply
- App Shah says
  
  May 22, 2017 at 3:42 pm
  
  Thanks Priya.
  
  Reply
Prabha Yadav says

Apr 19, 2016 at 4:24 am

Thanks for great informative
post. This article is very useful for the solve Duplicate Content issue. But I
have a question. You suggest two methods, But which is the best method from 1 or 2
and why?

Reply
- App Shah says
  
  Apr 19, 2016 at 10:27 pm
  
  Thanks Prabha. I followed method-1 on Crunchify’s maxcdn account as it was very simple 🙂
  
  Reply
Emily Johns says

Apr 16, 2016 at 1:38 am

Great Post which help us to fix duplicate content on our websites, definitely this super awesome tip can help out bloggers to keep away from Google animals, and help us to grow the ranking of our websites instantly.

Thanks for fantastic share once again 🙂

Reply
- App Shah says
  
  Apr 17, 2016 at 1:05 pm
  
  Thanks for nice words Emily. Happy blogging and keep sharing great tutorials.
  
  Reply
Avalonica says

Jun 17, 2015 at 10:56 am

If you want your images to appear in Google Image Search, then surely you have to allow crawling and indexing of those images (and only of them) in the CDN robots.txt, and put a server-side redirect in your origin domain to redirect those origin domain images to the CDN domain ?

Reply
- App Shah says
  
  Apr 17, 2016 at 1:11 pm
  
  Hi Avalonica – valid point. As you know our sitemap already includes all image attachments, it should be ok and no need to worry about it.
  
  I was worried when I saw cdn image in Google SERP page. Definitely it wasn’t something you like 🙂
  
  Reply
Rich says

May 4, 2015 at 1:29 pm

Thanks for the info. MaxCDN is a good network indeed. And another cdn to use is Edgecast for sure. It’s one of the leaders of the industry. WordPress and Twitter use them also. But any mid-sized webmaster also can benefit of their services. Though it’s more suitable for them to buy Edgecast’s services from their resellers. There is a number of them. Like Jodihost, for example. Resellers don’t require long-term contracts and offer affordable prices. And any webmaster can be sure that his website performs at it’s best.

Reply