They say “With great power comes great responsibility”. While adding an XML sitemap to your business website gives it great power, you have to ensure that the sitemap is made responsibly. If you fail to pull out a proper XML sitemap, it might not make your website as “powerful” as you’d hoped.
A highly common misconception with XML sitemaps is that they trigger faster indexing of your webpages. This is not true. Google, or any search engine for that matter, does not index webpages based on the presence or absence of an XML sitemap. With the help of an XML sitemap, you are merely telling the search engine which pages are important.
What is an XML sitemap?
As the name suggests, an XML sitemap is a map of your website. It tells Google crawlers which pages are important and how to get there. They particularly come handy when not all of the internal pages of your website are interlinked. In such a case, you can add the URLs to the XML sitemap and crawlers will understand that these pages belong to your site.
Why do I need an XML sitemap?
XML sitemaps prove to be the most useful when you are launching a new website. In a case like that, you would not have many external links. The sitemap would include all of your webpage URLs with tags that tell the search engine more about your website.
Websites that have a complicated architecture also benefit through XML sitemaps. While search engines crawlers spend time to take in the content you upload, they can take a glance at your sitemap to understand what is important and what isn’t. It also gives them an idea of how often specific pages can get updated.
XML sitemaps are highly beneficial if your business website has many dynamic pages. This is mostly the case with E-Commerce businesses that keep updating webpages or keep on adding new ones. Search engines find XML sitemaps very useful for sites like these.
You can explain the priority of webpages to the search engine crawlers through an XML sitemap. Specific tags that are used in the sitemap tell Google which pages are more important than others. In a way, you give the search engine the bigger picture about your business.
XML sitemap and robots.txt
Every business creates mainly two kind of webpages. One that caters to the customers. The objective of such a page is to capture or interest the user by showing and giving them what they want. The other kind of webpage is created for search engine rankings.
This is where the need for a robots.txt file comes in. A robots.txt file is a text-based file that acts as a guide for search engine crawlers. This file contains instructions that tells the search engine bots how they should crawl the site. It is a part of the Robots Exclusion Protocol (REP). Most bits added to this file indicate whether search engine bots should or should not be crawling certain parts of the website.
In simpler terms, you might not want some of the utility pages you create to be crawlable by the search engine. But you will definitely want all of your search engine – friendly pages to get indexed. This file also plays a role in transferring link equity and credibility.
While you are listing the URLs on your XML sitemap, you can create the robots.txt file on the side and add instructions for each of the links.
How to create an XML Sitemap for your website?
Most content management systems for websites allow you to generate on automatically. There is a WordPress plugin available for this (as is for almost everything else). You can use other online tools that are available for free as well.
Once you are done creating the XML sitemap, you should make sure to check the following:
- The file opens and end with the right protocol.
- All URLs mentioned are complete and <url> tag is used to denote the beginning of each URL.
- If you have the same page in different languages, then the same should be indicated appropriately in your XML sitemap.
- Proper canonicalization methods are used to indicate domain redirection.
- The file should not contain alphanumeric or latin characters.
You can make your XML sitemap file available to Google by uploading it on Google Webmasters. Or, you can include the below piece of code in your robots.txt file.
Sitemap: http://example.com/sitemap-location.xml
You will have to include the appropriate file location in the above code.