Sitemaps are XML files for search engines to learn what pages to crawl and how frequently to check for changes on each page.
Sitemaps are an easy way for webmasters to inform search engines about pages on their sites that are available for crawling. In its simplest form, a Sitemap is an XML file that lists URLs for a site along with additional metadata about each URL (when it was last updated, how often it usually changes, and how important it is, relative to other URLs in the site) so that search engines can more intelligently crawl the site.
Web crawlers usually discover pages from links within the site and from other sites. Sitemaps supplement this data to allow crawlers that support Sitemaps to pick up all URLs in the Sitemap and learn about those URLs using the associated metadata. Using the Sitemap protocol does not guarantee that web pages are included in search engines, but provides hints for web crawlers to do a better job of crawling your site.
The sitemap protocol format consists of XML tags. The encoding of the file must be UTF-8 .
The sitemap must begin with <urlset> and end with </urlset>. The name space must be specified inside the <urlset>. An <url> entity for each URL must be included as a parent XML tag. <loc> (URL of the page)child element for each <url> parent tag must be included. Optionally, <lastmod> (The date of last modification of the file in YYYY-MM-DD format), <changefreq> (How frequently the page is likely to change), <priority> (The priority of this URL relative to other URLs on your site.) child elements may be included for each <url> parent tag.
A sample XML Sitemap that contains a single URL.
<?xml version="1.0" encoding="UTF-8"?>
For more information: http://www.sitemaps.org/protocol.php
In November 2006, Google, Yahoo and Microsoft announced that they all use the same sitemap protocol described in Sitemaps.org to index sites around the web. (For more information: http://www.techcrunch.com/2006/11/15/google-yahoo-and-microsoft-agree-to-standard-sitemaps-protocol/)
In April 2007, they announced to use robots.txt to allow webmasters to share their Sitemaps. To do this, simply add the following line to your robot.txt file. (for more information: http://www.ysearchblog.com/archives/000437.html)
All requests to IIS are handled through Internet Server Application Programming Interface (ISAPI) extensions. ASP.NET has its own filter to ensure pages are processed appropriately. By default, the ASP.NET ISAPI filter (aspnet_isapi.dll) only handles ASPX, ASMX, and all other non-display file formats used by .NET and Visual Studio. However, this filter can be registered with other extensions in order to handle requests to those file types, too, but that will be covered later.
Every request flows through a number of HTTP modules, which cover various areas of the application (i.e. authentication and session intofmation). After passing through each module, the request is assigned to a single HTTP handler, which determines how the system will respond to the request. Upon completion of the request handler, the response flows back through the HTTP modules to the user.
HTTP modules are executed before and after the handler and provide a method for interacting with the request. Custom modules must implement the System.Web.IHttpModule interface. Modules are typically synchronized with events of the System.Web.IHttpModule class (implemented within the Global.asax.cs or .vb file).
HTTP handlers process the request and are generally responsible for initiating necessary business logic tied to the request. Custom handlers must implement the System.Web.IHttpHandler interface. Additionally, a handler factory can be created which will analyze a request to determine what HTTP handler is appropriate. Custom handler factories implement the System.Web.IHttpHandlerFactory interface.
For more information:
Here is a brief overview of how SiteMap HttpHandler will work: A request for SiteMap, will be intercepted and passed to our SiteMap HttpHandler which will generate the SiteMap XML.
Step 1: Create HttpHandler
Inside the App_Code folder, create SiteMapHandler.cs.
Here is the code for the Asp.Net Sitemap Handler implementing the IHttpHandler interface.
I commented out the loop that adds pages. You may get URL of your pages from web.sitemap file, database or another sitemap provider.
Step 2: Modify Sitemap
Add the following section inside system.web
<add verb="*" path="sitemap.axd"
In order to test your sitemap; browse the sitemap.axd file.
Step 3: Use robots.txt to announce Sitemap to Search Engines
Create a text file in the root of your application and name it: robots.txt
Insert the following line changing bloggingdeveloper.com with your domain name:
For an online demo: http://www.bloggingdeveloper.com/sitemap.axd
Download Visual Studio Project demo
The ranking algorithms of search engines are kept secret and they change often. Moreover, there is no exact formula for perfectly optimized HTML. On the other hand, some HTML tags/elements have great effect on search engine rankings.
The <title> Tag
The title of a page is probably the most important factor in getting high search engine rankings. You must include most important keyword(s) and phrase(s) into the page title. Besides, you must write the title in a way that makes sense because it appears in the clickable link on the search engine results page therefore it must attract people’s attention.
Do not put the name of the site first, unless the name contains the essential keyword phrase.
<title>Blogging Developer - Search Engine Friendly Pages</title>
form title tags as:
<title>Engine Friendly Pages - Blogging Developer </title>
More information on forming effective page titles: Who else wants to build effective page titles and rank high in search engines?
The <meta> Tag
A META tag is an HTML tag that provides information about web page’s content, such as what HTML specifications a web page follows or a description of its content. META tags do not affect how a web page is displayed in a browser in other words these tags are mostly invisible to site’s visitors, but will be seen by search engines.
There are only two Meta tags that are important for search engine optimization:
<META NAME="KEYWORDS" CONTENT="tags seo search engine friendly optimization meta title HTML">
<META NAME="DESCRIPTION" CONTENT="The ranking algorithms of search engines are kept secret and they change often. Moreover, there is no exact formula for perfectly optimized HTML. On the other hand, some HTML tags/elements have great effect on search engine rankings”>
In the KEYWORDS meta tag you must enter keywords and phrases relevant to page’s content. Many choose to separate the keyword phrases with commas.
I usually do not use commas to separate keywords and phrases because they take up space. Moreover, forming keywords / phrases without commas makes it possible to include more combinations of keywords.
Some search engines may interpret
- repeating keywords/phrases
- entering keywords that are not relevant to the page’s content
Except Yahoo and Ask, search engines do not apparently take KEYWORD meta tag into consideration these days.
The DESCRIPTION meta tag is important as search engines use this text for the description given in the search result listings.
This text should be a normal sentence which gives exact information on what the page’s content offers.
Search engines do not display more than 250 characters including spaces, commas, and periods. Therefore you should enter the most important part into the beginning of the description.
More information on meta tags: Metatags from search engine rankings perspective - Are they dead?
The <body> Tag
Some search engines reward keyword repetitions. Therefore, include the selected keyword phrases several times throughout the content.
Here are some tips on forming the page content:
- Do not overdose repetition. The language must seem natural otherwise search engines may interpret repetition as spam.
- Using phrase variations may help.(e.g. “search engine ranking” and “search engine rankings”)
- In addition, include the keyword phrases in the first paragraph.
- Bold text and larger size fonts may help in some search engines.
- Do not use hidden texts.
The Headline Tags
Include the keyword phrases in headlines using the standard headline tags (<h1>,<h2>)
The Image ALT Attribute
Image ALT Tags are used to describe pictures to people using non-graphic browsers or that have images turned off in order to gain download speed. Although the alt attribute must include a description of the picture, most developers include logos and image-based titles with alt attribute to include keyword phrases.
<IMG src="pics/logo.jpg" WIDTH="60" HEIGHT="72" BORDER="0" ALT="Tips & Tricks on Search Engine Optimization">
The TITLE Attribute
Title (do not confuse with the “<TITLE>” tag) attribute lets you describe any tag. The text pops up when someone is moving his or her mouse pointer over the element.
<A href="/default.aspx" TITLE="Tips & Tricks on Search Engine Optimization</A>.
Keywords in Links and URLs
Including keyword phrases in links not only improves the ranking of the page that contains the link, but also boosts the relevancy of the page the link is pointing to.
<A href="/default.aspx">Click here!</a>
The link above is a wasted opportunity.
Naming files, directories with the keyword phrases may help as some engines take the URL into consideration.
<A href="aspnet/tips-on-aspnet.aspx">Tips on ASP.NET</a>.
Lastly, do not forget that web pages do not exist for search engines, it is easy to overlook the fact that what really matters is good content. If spending time on optimization is taking you away from improving user experience, it may be time to either refocus your efforts or to outsource your optimization work.
In 1990s, META tag content was known as a secret weapon for top search engine positions. A META tag is an HTML tag that provides information about web page’s content, such as what HTML specifications a web page follows or a description of its content. META tags do not affect how a web page is displayed in a browser in other words these tags are mostly invisible to site’s visitors, but will be seen by search engines.
A number of different META tags are available that you can use to include page related information, but only two of them are important from SEO perspective: the keyword tag and the description tag.
Some search engines take the Keyword and the Description tags into consideration while ranking the sites for that keyword or phrase. META tags only have a limited effect on resulting high search engine rankings. Since search engines uses META tags information in some part of their ranking formula and while displaying the search results, it’s worth adding them.
The META tags look like:
<meta name="keywords" content="meta tags SEO">
<meta name="description" content="Roles of META Tags in Search Engine Rankings">
How to use the META keywords tag
The META keywords tag used to be an important way to get your page listed under specific keywords under the search engines. However, major search engines ignore keywords tag and it no longer has the importance it used to have. The META keywords tag should be placed in the <head> portion of a web page.
<meta name="keywords" content="put your keywords here" />
Optimizing Keywords META Tag
Concentrate on some specific keywords. Avoid using more than 3 keywords. Use keywords which are relevant to your page and target audience.
Repeating keywords does not add any value. Unnecessary keyword repetition may be considered as spam by search engines
<meta name="keywords" content="keyword 1, keyword 2, keyword 3">
<meta name="keywords" content="keyword1 keyword2 keyword3">
- Use space-separated keywords instead of comma separated ones, since it allows search engine to make combination of keywords into possible phrases.
- Putting in common misspellings of keywords that you spelt correctly in your content might be a good practice. However, some search engines penalize web pages that contain keywords that are not in the page itself.
How to use the META description tag
Although major search engines ignore the description META tag, we have to use it since smaller search engines use the description tag data and takes only 2 minutes to write it. The information in this tag is listed directly below the title tag on search engine results. As a result, a good description make a website stand out from other sites.
The META description tag looks like:
<meta name="description" content="content summary of the page">
Optimizing Description META Tag
- Give a relevant description since some search engines display this description in the search results.
- Use only targeted keywords.
- Restrict the number of words in this tag to 15 words.
- Place the most important phrase at the start of the description because some search engines may omit some parts of description.
- Do not repeat the HTML title tag content in the description META tag. If the title-tag content is automatically displayed as a hypertext link in a search engine result page, exact repetition as a description would not encourage people to click the link. When you think about your META description tags, you should think of them in terms of getting click troughs, not ranking.
Today, some search engines still look at META tags, but increasingly they put much more emphasis on both visible text on the page and "off-page factors" (popularity, linking structure of the Internet, etc.) to measure page relevance.
As a conclusion, optimizing the META tags might not sound important. However, it takes about 15-20 minutes to make a big difference if your competitors haven't optimized the tags as well as you have.
For a couple of days, I’ve gotten many emails from people trying to sell me text links on sites with high Page Ranks (PR). Having links into your site from popular sites that also have lots of links coming into them is probably the most important factor for getting good search engine listings. Page Rank system from Google is a rough measure on a scale of 0 to 10 based on the relevance & content of the page and the number & quality of the link leading to it from other sites.
PageRank Explained: (Source: http://www.google.com/technology/)
PageRank relies on the uniquely democratic nature of the web by using its vast link structure as an indicator of an individual page’s value. In essence, Google interprets a link from page A to page B as a vote, by page A, for page B. But, Google looks at more than the sheer volume of votes, or links a page receives; it also analyzes the page that casts the vote. Votes cast by pages that are themselves “important” weigh more heavily and help to make other pages “important.”
Important, high-quality sites receive a higher PageRank, which Google remembers each time it conducts a search. Of course, important pages mean nothing to you if they don’t match your query. So, Google combines PageRank with sophisticated text-matching techniques to find pages that are both important and relevant to your search. Google goes far beyond the number of times a term appears on a page and examines all aspects of the page’s content (and the content of the pages linking to it) to determine if it’s a good match for your query.
Beside e-mails, there are many auctions at eBay selling domains with high Page Ranks. Fake Page Rank domains are becoming more common each day. Many inexperienced users purchase domain names that have fake Page Rank at eBay. If you haven’t seen the problem, you may check this search.
So how do they fake Page Rank?
It is simple to fake a page rank of a site. Faking a site’s page rank is achieved by implementing a script which detects if a user to the site is Googlebot (Google’s spider/crawler that finds and indexes web pages). If Googlebot is visiting the site, the script will redirect Googlebot to a site with high Page Rank. For example, if they direct Googlebot to google.com, the page rank will be 10.
Then how can we detect fake Page Rank?
Fortunately, in a few easy steps, you can detect fake page rank.
Check the domain with fake PR checkers like SEOlogs, CheckPageRank, or RankAlert. Do not use standard PR tools or the Google toolbar.
Type the domain into Google (domainname.com) and if another domain comes up in the first result, then the page is being redirected and domainname.com is fake.
Check backlinks manually by typing link:www.domainname.com. Go to each site listed, if there is any, and search for the link to www.domainname.com. If you find a site with a link on it, record its PageRank. This procedure is very important since “Fake PageRank Detection Tools” do not check for this.
If you’re using eBay, make sure to check out the person’s feedback and past auctions.
I have checked plenty of domains and almost every domain that is for sale with PR does not have any backlinks. According to above mentioned PageRank algorithm, it is impossible to have high PageRanks without quality backlinks.
Beside Page Rank validity, getting links from other web sites just because these web sites have high Page Ranks, does not make much sense if a web page is not related to your site. It is always better to get links from sites that are related to yours. These links will increase your search engine rankings and you’ll get targeted visitors from these sites. Focus on web users instead of search engines.
The title of a page is probably the most important factor in getting high search engine rankings. Web page title appears between <title> and </title> tags.
Effective Web Page Title Tags–Rank High in Search Engines
The WWW Consortium describes the title element as a required HTML element in an HTML document.
The title of a page appears in the clickable link on the search engine results page and appears on top of your browser when viewing a page. A page’s title is used by the search engines as the most important piece of information available to determine the ranking of your page in search results. In other words, search engines rank pages with relevant titles higher than web pages without relevant titles. The basic idea is that you are helping search engines understand your web page through the page title.
Darren Rowse from ProBlogger.net was moaning about poor rankings in Google although the site has thousands of inbound links and high page rank. Even for a term like “Blog Tips” that he writes about repeatedly, the site was ranking around 60th on Google.
One of his friends asked him a simple question: ‘What is in the title of your blog?’
The title of the front page was “ProBlogger: Helping Bloggers Earn Money”. It was a good descriptive title but it was not including key terms that he wanted hits on. Then he changed the title of the front page to “Problogger Blog Tips: Helping Bloggers Earn Money.”
After a few days later, his search ranking on the term “Blog Tips” went from 60th something to 10th for Google and from 40th to 1st on MSN.
Darren Rowse incorporates his blog’s name and main keywords in his front page. On individual pages, he uses title of his post with his blog’s name.
Titles not only help your web page to rank well, but also they attract people. Search users usually ignore a web page with unrelated page title. One of the best ways to attract repeat visits is to have someone bookmark your web page. Almost every web browser assigns the page title as the label for the bookmarked web page. If your page title is not descriptive or missing, most people will not remember why they bookmarked your site. Moreover, usually browsers use page titles to label pages in the browser’s history. Lastly, most people link to web pages using the page title for anchor text on which search engines place high importance so when someone links to a web page with the same anchor text as the page title, ranking for words found in the page title will get boosted.
Tips on Forming Effective Page Titles
- When forming a page’s title, make sure that the page’s search term is contained in page’s title, and place it to the beginning of the page title. If the target search term for a specific page is “Pagerank”, the effective page title may be: “PageRank - Google's PageRank and how to make the most of it.”
- If you want to include your company name or your website name, place it at the end of the Title tag. The name of your company or your website doesn’t make much difference to a user. They are more interested in knowing the content of your page. The title of a page is a valuable thing and adding the name of your company or web site might reduce the relevancy of your web page.
- All titles should be typed in Title Case. Details can be found at Writer's Block - Writing Tips - Capitalization in Titles.
- Try to avoid stop words in the title tag. Stop words (a, an, and, but, he, her, his, I, in, it, of, on, or, she, the, etc.) are common words and characters ignored by some search engines to enhance the speed and relevancy of search results.
- Every page must have a unique title. Putting the same title on every page of your website is not a good practice.
- Home page titles should clearly indicate what is available on the page, and for the purpose of web credibility they should indicate what organization is represented on the page. In other words, you definitely need to include the name of your organisation in the title tag of your home page.
- Do not use “Untitled” or “Homepage” as your page title. Avoid using only your company name as your title.
The WWW Consortium states that: “The title should ideally be less than 64 characters in length. That is, many applications will display document titles in window titles, menus, etc where there is only limited room." A 64 character limit is a good starting place, but oversimplifies things a bit.”
Google displays up to 66 characters of a title tag, cropping to complete words. On the other hand, Yahoo displays exactly up to 120 characters and titles are simply be cropped, whether the display title ends in a complete word or not. Therefore, the best strategy for building a long page title is to make sure that the title works for both Google and Yahoo.
Title tags are powerful tools and must be used to their fullest advantage. The Title tag helps the search engines decide the theme/content of the web page being crawled for indexing. When a search for keywords is conducted, the Title tag is given heavy consideration by all search engine algorithms. Also remember, each page in your website is unique and needs a different Title tag. Place the most important keyword phrase for that specific page in the Title tag, and the page will get a certain boost in the search engines.