TITLE: Banned By Google And Back Again.

SUMMARY: How my website got de-indexed by the search engine Google, and how I got it back in.

Article Body Follows:

The date: 29th July 2005. The time: early morning. I got out of bed and fired up my PC. Opened my browser to check my site. Had a look at the Google toolbar plugin (http://toolbar.google.com/) on my browser. It showed grey.

Ice formed in my stomach.

I opened my bugged version of Internet Explorer: my PageRank was 0.

By now I was frantic. I went to google.com and typed in 'site:www.tigertom.com'.

No pages listed.

I did this for two other satellite sites of mine: ditto.

What had happened?

TigerTom.Com (http://www.tigertom.com) had been banned by Google.

I went to the WebmasterWorld forum (http://www.webmasterworld.com), and found out the awful truth. Google was doing one of its periodic updates of its algorithm, and had filtered out my sites completely.

Further research there revealed why. I had too many pseudo-directory pages with auto-generated external links. Snippets from search engine results were used as descriptions of said links. Said links were run though a redirect script.

These are hallmarks of pseudo-directories and 'AdSense scraper'* sites. Google is reportedly trying to filter these from its 'SERPs'**. I say reportedly, because Google doesn't announce these purges. They are inferred.

To compound my sins, these pages were also effectively doorway pages†.

The theory was that legitimate sites had been hit as 'collateral damage'. I say theory, in that Google rarely comments on individual cases. It won't tell you exactly why your site was banned. I guess this is for reasons of time, and to give no clues to spammers.

In my case the ban was justified for my two satellite sites; while not looking like spam, they were effectively doorway sites.

My main site was different. It had offending pages, but was mostly a diverse labour of seven years; a personal site on steroids.

Google bans sites algorithmically: a site that fits their 'spammer' profile gets dropped via software from their index automatically. Real spammers shrug their shoulders and move on; honest webmasters write emails begging for mercy.

Like me.

I did some searching to find out how to do a re-inclusion request. Here's how:

1. First, you check your site is truly gone, by going to http://www.google.com, typing 'site:www.yourdomain.com' without the apostrophes. If it returns no pages at all ...

2. You check Google's webmaster guidelines at http://www.google.com/webmasters/guidelines.html. These are not really guidelines; you should treat them as iron-clad rules.

3. You stop the offending content from being web-accessible, permanently.

If you're familiar with Apache web-server mod_rewrite you can:

- Send a 410 'Gone' response to requests for the offending pages, or
- CHMOD them to 400, which will return a 403 'Forbidden' response, or
- Move them to another directory if you need to keep them, or
- Just delete them.

Don't try to be clever. Just get rid of them.

4. You go to http://www.google.com/support/bin/request.py, tick the relevant boxes, and type 'Re-inclusion request' in the subject box of the form.

4a. You add the complete URL of your site i.e. http://www.naughtydomain.com,

4b. You state that you have read the webmaster guidelines above,

4c. You admit what you did wrong; simply, succinctly, with no carping or special pleading.

Don't try to be clever. Don't argue. Don't lie. Don't waffle.

Google has cached copies of your site. When an engineer checks your site, he'll look for the offending content, and compare it against their cache. He'll spend about two minutes on it; don't give him a reason to continue to exclude you.

5. You ask for re-inclusion.

6. You wait.

In my case, it took about a week; a long, unpleasant, fretful week. I sent follow up emails saying what I was doing, and a fax, and I was going to write letters if that didn't work. That was probably excessive. Once you have a ticket number, that's all that should be necessary.

They emailed a standard reply saying "the problem had been passed to their engineers". That was good. I understand they send no reply to spammers.

A week later my site was back in. Lesson learnt.

My inference from this?

Make sure you have all-unique text on your site. Don't use softwares to auto-generate it. Avoid popular CMS's even. Then Google's algorithms will leave your site in peace.

------

* A site specifically set up to host Google Adsense advertisements (http://www.google.com/adsense). Usually of low quality, consisting of pages of links to other web sites, text copied from free-print articles, and a big Google ad block 'above the fold'.

** Search Engine Result Pages

† Doorway pages are low-quality keyword-rich web pages whose sole purpose is to lead the viewer to the 'real' content, usually whatever the site is selling.

------

About the author: T. O' Donnell http://www.tigertom.com is an ecommerce consultant living in London, UK.