At Crunchify we use MaxCDN as our Content Delivery Network partner. MaxCDN is one of the best Content Delivery Network (CDN) provider so far I came across. Last week I noticed some weird search result which shows cdn.crunchify.com in search result page (SERP) and immediately it catches my attention because this may cause my SEO index go down.
In this tutorial will show quick tip on how to avoid this situation and fix it completely. If you have similar questions like this then you are at right place.
- How do I avoid creating duplicate content using a CDN
- CDNs and Duplicate Content
- Prevent CDN from caching webpages to avoid duplicate contents
- Duplicate content issues using CDN. How to fix?
- CDN using CNAME causing duplicate content
Other must read: Step by Step Guide to Setup MaxCDN
Let’s get started
Often times a blog is only hosting or caching resources like CSS, image, javascript and video files on the content delivery network. How to fix this?
Method-1
Use rel="canonical". Check this out.
Before enabling Canonical Header option setting under MaxCDN panel
After enabling Canonical Header option
MaxCDN setting
This setting will tell Google or Yahoo or Bing the original version of the page and it should take note of this hint. However according this is only a hint and not 100% to be relied upon.
Method-2
Robots.txt is a file that restricts access to your site by web crawlers, stopping pages from being indexed on search engine results. Add a robots.txt file to your CDN's root directory to disallow page crawl by search engines like Google. In the case of a CDN, assets are mirrored, so content is duplicated and then listed by crawlers on search engines, which is bad as pages rank lower.
User-agent: * Disallow: /
NOTE: do NOT put under your blog’s root directory
Thanks for great informative
post. This article is very useful for the solve Duplicate Content issue.
Thanks for sharing such a great post. Already using your methods to fix my website. Keep sharing your tips!
Thanks Priya.
Thanks for great informative
post. This article is very useful for the solve Duplicate Content issue. But I
have a question. You suggest two methods, But which is the best method from 1 or 2
and why?
Thanks Prabha. I followed method-1 on Crunchify’s maxcdn account as it was very simple 🙂
Great Post which help us to fix duplicate content on our websites, definitely this super awesome tip can help out bloggers to keep away from Google animals, and help us to grow the ranking of our websites instantly.
Thanks for fantastic share once again 🙂
Thanks for nice words Emily. Happy blogging and keep sharing great tutorials.
If you want your images to appear in Google Image Search, then surely you have to allow crawling and indexing of those images (and only of them) in the CDN robots.txt, and put a server-side redirect in your origin domain to redirect those origin domain images to the CDN domain ?
Hi Avalonica – valid point. As you know our sitemap already includes all image attachments, it should be ok and no need to worry about it.
I was worried when I saw cdn image in Google SERP page. Definitely it wasn’t something you like 🙂
Thanks for the info. MaxCDN is a good network indeed. And another cdn to use is Edgecast for sure. It’s one of the leaders of the industry. WordPress and Twitter use them also. But any mid-sized webmaster also can benefit of their services. Though it’s more suitable for them to buy Edgecast’s services from their resellers. There is a number of them. Like Jodihost, for example. Resellers don’t require long-term contracts and offer affordable prices. And any webmaster can be sure that his website performs at it’s best.