Sometimes you wonder why you should invest into optimizing you website.   And certainly if you have lots of JavaScript on you page.

Let me give an example.  This is what I found in the top 10 results if I search on my name. 

You see, even the masters of search might struggle to get their SEO right.  Or is it the Google index failing this time?

Side note: you might wonder why this pages ranks so high.  I guess it’s the strong top domain.

A few weeks ago someone told me that Google is not capable to export lots of data to CSV.  Although you have a export to CSV option, it is limited to the “show 500 lines”.  To export you have of course the Data export API, but if you are not ready to use the API, you might be interested to use the following tip.

Today, I read a useful tip how to export more rows to CSV from SeoMoz

• Add "&limit=#" in the URL
• Then Select Export to CSV

A lot of blog posts, coming from Google have to do speed. It is all about a faster internet today.  And to help you, Google announced their public DNS services. 

I learned that not any statement made by major sources are true.  Before, configuring the DNS settings, I downloaded a free tool called Domain Name Server Benchmark, and started 2 times a benchmark rounds. These were launched from a corporate network up with an upstream connection to COLT. 

The Google public DNS servers (8.8.8.8 and 8.8.4.4) were not present in the fastest segment of the results produced by the 2 benchmarks I have launched.  The only advise, apart from any possible privacy concerns, is that you shouldn’t use (any) name server without first testing its speed.  There might be public facing DNS servers near you providing faster results.

De national Post service of Belgium, has changed it’s site and Bing managed to indexed the obsolete page…  Look at the URL: www.post.be/site/nl/obsolete.html.  I was curious and tried to find out why Bing gave me the wrong result.

Ok, i admit.  It was not the best search i have done.  Searching for post, would give the good result. 

Anyway, this is a screenshot of the web page:

See the text at the top?  It’s Dutch and French.  Translated into English it says: “This page is not available anymore, click a link below to visit our new site”.   I didn’t got it.  If the page does not exists anymore, was Bing wrong indexing it? 
No!  Bing got it right (sort of).  It was in fact the webmaster of Post.be who forgot something.

Explanation

The page you are entering is for a normal visitor a standard NOT FOUND page.  When visiting a page which doesn’t exists, the server should give a 404 status in its response.  The HTML can be customized like the screenshot above for an improved user experience.  But crawlers & indexers need the correct status.  In this case a 404 error.

When you investigate all requests send to the web server, every requests gets a 200 Status code: “Successful”.  I used Fiddler to verify the page request:

As anyone can find on the web, it tells the visitor that the request was successful.  Not a single 404 status code can be found in the second column.

A user centric solution

For a better user experience, it would have been nice of “De Post” to redirect me automatically to the new homepage.  The page I was visiting here was obviously an obsolete page.  This can be done in HTML using a Meta Refresh Tag.  Looking at the HTML source, no redirect is made.

It would have been more user friendly, but not the best solution.

What about 301

A 404 would have been a good start.  Bing, Google and other search engines would not mention the result in their SERPs for long. I guess de webmaster of “De Post” must have had some reason not to use a 301.  A 301 is a status code telling the visitor (human or not) that the page has moved permanently to a new place.  Together with this status code the new URL is supplied.

What exactly happens when a 301 is send from the web server back to the visitor?

A human visitor

The browser (Internet Explorer, Firefox, …) of the visitor will notice the 301 status code and load the new page supplied with the 301 status code.  The end user might notice a new page is loaded because the shown URL in the browser will be updated.

A Bot / Spider

A spider or a bot will notice the 301 also, and will remove the old URL from their index and update it with the new URL.  Using a 301 has major advantages when migrating to a new site because you can retain all page rank from any incoming links.