It is every business owners worst nightmare – you wake up one morning and your website has vanished from Google. This was the plight of one frantic website owner we encountered recently. The site owner was convinced that something his SEO company had done caused his website to receive a Google penalty. His site was gone from Google… completely. You could not find the site by searching for the brand name or url. Any pages that were still indexed were not performing for any keyword search, and the cached versions were old. Diving into the crawl stats confirmed that Google not crawling the website – flat-lined.
Our team at RightNow was not overly optimistic that there was much that could be done for the site quickly, but agreed to dig in and map out a plan for the future. The first step was to check the usual suspects – robots.txt files, noindex issues, conflicting files or redirects, IP blacklists, hacking, etc. Nada. The next step was to reverse-engineer the SEO tactics that had been applied. Most Google penalties are algorithmically generated, so you need to discover any likely culprits The most common reason for getting booted from Google is participating in link building that breaks the Webmaster Guidelines.
During our research we discovered two things; First, the previous SEO company had done a pretty good job; Second, there were no glaring violations that should get a site ejected from Google. This came to us just after a Google update, so maybe it was possible that the site had been caught in the update by accident? Not likely, but we sent a researcher down that rabbit hole to investigate.
The other direction we investigated was to find out if there was something at the host level causing Google to not crawl the website. The hosting company flatly denied that there was any reported issues. Then we noticed something strange in Google Webmaster Tools – you could not fetch the site as Googlebot. Back to the hosting company, and another denial that it was their issue. As a last resort, we pulled a reverse IP list of all the sites on the shared IP address and manually checked them. Bingo – other sites had been dropped from Google! Armed with this info, we contacted the host again and they agreed to flush the server firewall as a “courtesy.” The problem was that the Googlebot IP address had been banned by the server firewall. We had the website reindexed 24 hours later, problem solved, end of story. Or is it?
SEO requires more than a keyword list, some links, a few articles, and some magic dust. The entire process has become highly technical, and there are a number of things that can go wrong. Most sites will never disappear from Google, but many suffer from missed opportunities that could be solved with correct website optimization.
Looking for an SEO partner? Give us a call!