How to get your website content indexed within hours instead of days…
Usually when you publish a new page on your website, you wait a few days for it to appear in search engine results.
What if you want people to find it sooner?
You might have a new service you want to promote. A blog reviewing a brand-new product. A landing page with an offer expiring soon.
You can give Google an “excuse me, I’m here” nudge, so it appears in search rankings quicker.
How Google works
Google uses “bots” to constantly explore (“crawl”) the internet.
Bots are automated pieces of software that send signals back to Google HQ. “I’ve found a new website”, or “I’ve found a new page on an existing website”. Google then decides how to and where to rank it.
It’s a big job – 571 websites are created every minute.
The problem is, these bots aren’t like Father Christmas, capable of visiting everyone within one night.
Websites get prioritised. Because the BBC and Guardian are updated each minute, bots will pop over and crawl these websites all the time.
Your website might have to wait longer.
With my website, I know Google visits every 2-4 days.
How do you know when Google has visited?
Google leaves no footprints or ” We visited but you were out” cards through your door.
But you can run an easy test to find out.
You or your developer will need to go to the meta description of a page on your website. This is the information Google displays under its search results.
Here’s the meta description for my homepage:
Now, change something in the meta description. It can be something really minor if you want, a word or punctuation mark.
The next time a bot visits and sees the change, Google will update the displayed meta description.
You just need to Google your page at least once a day to see when it happens for you.
Bots like to vary their visits, so do this 3 times and work out the average.
A case study
Do you watch The Apprentice? The most recent winner was Mark Wright, a digital marketer. His idea was an SEO agency: Climb Online.
With average viewing figures of 6 million, imagine how many people Googled “Climb Online” after Lord Sugar made his decision.
And imagine how many of those were businesses who thought “That looks like what I need, I’ll Google them.”
Climb Online wasn’t launched until a few weeks after. So initially, an outdoor company called Climb Online saw a flood of visits to their site.
Any Apprentice fans interested in outdoor activities would probably have been happy.
But for Mark it was a missed opportunity to reach new customers.
And for his competitors?
Within hours of the result, SEO agencies published blogs on Climb Online.
As you’d expect from professional agencies, the blogs were keyword-optimised. They met all the criteria to rank highly for the phrase “Climb online”. And they all invited businesses to contact them to discuss SEO services.
But publishing quickly was only part of the job.
They also needed Google to index quickly.
Waiting hours or days for a bot to come crawling was no good.
Because by then most people would have forgotten about Climb Online.
They needed another way…
They needed to Fetch as Google.
“Hey Google, over here!”
Fetch as Google is the secret weapon to getting your pages indexed quickly.
You’re cordially inviting Google to send over a bot to crawl and index it. As soon as possible.
How to Fetch
This bit assumes you have Webmaster Tools set up on your website.
Not sure if you have Webmaster Tools enabled? Send me your website URL and I’ll tell you.
Open up Webmaster Tools and go to your website.
Click Fetch as Google.
Enter the URL of the page you want indexed, and click Fetch (the red box underneath).
(Fetch and Render shows you how the page looks to a bot, if you’re into that sort of thing)
You’ll get a bot visiting within hours. Assuming you’ve done everything right, you can expect to see your new page in the rankings soon after.
How often does Google crawl your site? Have you noticed any changes over time? Please let me know, so we can identify any trends.