My website has disappeared from Google. What do I do now?

One day you note a fall off in the traffic Google sends your website. As Google is the main source of your traffic, as is the case for many websites, alarm bells naturally start ringing. Investigating, you realize that the site does not appear at all in Google or has poor visibility at best in search results. What is a poor site owner to do? Did someone say panic?

Understand why the site disappeared from Google

There are several reasons why a site no longer appears in typical Google search results.

a) The site is not accessible to Google

A website administrator can deny Google and other search engines access to the pages of the site. In such a situation, Google will have little information about the site and, consequently, indexing and retrieval of the site will suffer. Google still does its best to give some sort of answer to web searchers even without direct access to a site. Unfortunately, due to a lack of information available to Google, the site will most likely only appear in a few highly targeted queries. Generally the site entry will lack a summary; if there is a title, it won’t always be logical. The link to view a cached page will be missing. Here’s a real world example:

This example is considered in further detail in an earlier previous post on Google search results for sites that block crawling by Google (in Italian).

Generally the problem lies with the directives specified in the file robots.txt. Sometimes there has been an oversight; I have seen at least one case where the robots.txt file was tampered with by a supplier with whom the client no longer had a good relationship.

b) The site was banned or penalized by Google.

Google wants to provide the most relevant results as possible to all of the queries made world-wide every second. To ensure high quality, Google actively removes from its index any site that violates Google’s Webmaster guidelines.

There are certainly cases where violation of a Google webmaster guideline is due to a naive and most likely dated understanding of search engine optimization requirements. In other cases, an unprofessional or even dishonest “SEO agency” deploys underhanded and discredited techniques such as hiding text in the pages of the site. One of the methods to detect the presence of hidden text is to use a character based browser such as elinks or lynx.

In other cases, problems arise when not so nice people inject bad code into the pages of a site. This happens more often than you might think, especially to popular CMS platforms, such as blogs. Security is an element of SEO for a blog and other sites as well.

How does Google detect guideline violations?

Google can detect many issues automatically, particularly when the problem occurs in a similar way on multiple sites. As policy, Google leans to automated solutions wherever possible. Manual solutions are not very scalable – they’re not suitable to a problem the size of the web.

However, some violations can arrive to Google’s attention through direct reports, such as from a competitor impacted by someone’s use of questionable SEO techniques.

The letter from Google

Google does not want to eliminate quality sites from its index – this is not a move to push Google’s AdWords pay-per-click platform, despite what you might imagine. The evidence lies in that Google does make reasonable efforts to contact the owners of sites which become penalized. This is usually in the form of an e-mail sent to the website administrator, using the contact details for the domain registration when available (this is one reason to avoid using the privacy services which may be available when you register a domain). The same e-mail will be available in the Google Webmaster Tools dashboard – one more reason to incorporate Google Webmaster Tools in your SEO processes.

What if my site is banned by Google?

The initial reaction in most cases is going to panic. While understandable, panicking won’t help in solving the problem. Yelling and throwing furniture from a balcony isn’t very useful either. There is just one thing to do and that is to fix the problem and ask Google to reconsider your site for inclusion in their index.

Remove all traces of overzealous SEO

The first step is to do a methodical check of the site to see where your site is in violation of Google’s guidelines. The most common problems, as mentioned above, are hidden text and dubious code, often written and inserted by third parties, to add hidden links and the like on the site. This step requires great care: it is useless, if not harmful, to ask Google to reconsider your site if the problems known to Google persist. You won’t have much credibility with Google if you make repeated unsuccessful requests for the same site.

Dear Google: the letter to Google

Once you are sure that all the problems identified by Google (and perhaps others not yet discovered by Google!) have been addressed, it’s time to contact Google, asking for pardon. This entails sending a message to Google through an option available in the Google Webmaster Tools. In the message specify:

1. What happened from your point of view. Honesty and sincerity are virtues. If you have done something wrong, say so in black and white while keeping a professional tone. Should you have any doubts on this one, keep in mind that collectively Google knows more than you think and is definitely smarter than you are.
2. Why are you sure that what happened will not repeat itself. You are sure, right?
3. You can consider including a brief reference to the value of your site as an important web resource – but only if third parties would objectively concur.

Avoid irrelevant information such as:

1. The damage you are suffering. That is your problem as they say.
2. Any participation in AdSense and/or AdWords programs. Google’s organic visibility is independent, at least in principle, from economic considerations.

You can also review the official Google information on this topic.

Video: Matt Cutts, head engineer of Google’s anti-spam group (search quality), discusses the reconsideration process (4 August 2006).

Ask for help from a reputable SEO practitioner

In many cases the best solution is to engage a SEO professional to help you overcome visibility problems in Google. Rather than mastering Google’s guidelines and cross-checking them against your site, it might be more effective to ask a SEO consultant for help.

Time required for reconsideration

If all goes smoothly, your site will be back in Google in a short time – from days to a few weeks. But it could take more than a month, especially if you make a request for reconsideration before all issues affecting the website are resolved. An informal survey done by Search Engine Roundtable yielded rather varying answers (I’ve reworked their data into a bar graph chart; I find pie charts hard to read):

Chart: time required for the reinclusion in Google.

Source: Search Engine Roundtable, October 2008

What has your experience been?

Similar Posts:

• Microsoft / Yahoo Search Alliance: a closer look at Bing Webmaster Tools & a free data export tool
• The Google Webmaster Dashboard, a.k.a. Google Sitemaps
• Now there are 6 ways to keep website content out of search engines
• 8 reasons to avoid Flash (or Silverlight) like the plague when designing a website
• Domain & URL Strategies for Multilingual & Multinational Sites

Registration is now open for the next SEO Course (June 13 and 14) and Google Analytics Course (June 20 and 21) in Milan. Don’t miss the opportunity!