In a blog post on June 8, Google announced, “Our new search index: Caffeine”

And as Google works to improve their search engine, on a day-to-day basis, they have introduced new ways to speed up the process of searching. With the completion of Google Caffeine, they have improved the functionality, and speed of search engine results and indexing.

The Basics of Google Caffeine

• 50% fresher results for Web searches, better and faster content retrieval, quicker updates…
• A search index capable of retrieving results in parallel; able to process hundreds and thousands of pages in a matter of seconds…
• With parallel processing, Google Caffeine is able to index and retrieve Web pages on an enormous scale.
• Temporal relevancy determines newer, fresher and more updated content. Think of temporal relevancy like breaking news; when something important happens, it needs to be quickly found online.

If you are wondering, or a bit confused, as to what we are talking about, do not worry–Google has a video on how search and caffeine work. When you search Google, for say the keyword “cute puppy” you are in fact retrieving Google’s cache of its search index; you are not retrieving a real time result, but a copy of Google’s cache of that Web page.

“We’ve moved from a batched indexing system to an incremental system that enables us to more quickly and efficiently refresh pages.” – Google spokesperson Jake Hubert

When you search for any keyword, Google’s API will query its massive index for all matching terms, and then it will display the results. You may ask, “How did Google collect the index?” Well, Google employs the use of bots that crawl the Web continuously; they download new or existing pages, cache them, and then update the index. Bots work by crawling the links on a page–that is how they got their name.

The effort to maintain and keep this index up-to-date is a massive one. Google datacenters are the heart of operations, with super clusters of servers that network their entire infrastructure and indices together. Therefore, when you search for the keyword “cute puppy” all of their technology comes together in unison to bring you the results.

Google realizes that search engine users want the freshest content they can find. With the older indexing system, results came up relatively fast, but as more content was added to the Web, the pace of new information would have been troublesome to maintain.

“In fact, every second Caffeine processes hundreds of thousands of pages in parallel. If this were a pile of paper it would grow three miles taller every second.” – Carrie Grimes, Software Engineer at Google

The Scale of Google Caffeine

Google processes Web pages at astronomical speeds and lengths:

• One index can take up nearly 100 million gigabytes of storage.
• Google adds new information to the index at an alarming and rapid rate; hundreds and thousands of gigabytes per day.

Estimated size of Google’s index – Image courtesy of http://www.worldwidewebsize.com

However, with Google Caffeine, and possible future speed improvements, the company is able to keep up with an ever-increasing amount of Web content–social media for example has been one large data mine. Google does not index certain elements of many social media websites, for privacy reasons. And some social media websites do employ the “content=noindex” or “rel=nofollow” HTML tags. Noindex tells Google’s bots to ignore the index, i.e. not to cache the Web page. Nofollow tells the bots not to assign any importance on the link, i.e. Page Rank/authority; it makes a link transparent.

What Google has also done is index certain public elements of social media, for example, the public profile section of your Facebook account–if you set it to public. They have also employed real time results via social media services like Twitter. If you want to see real time results in action, search for anything that is popular in the news, say “congress.” Then click the “latest” link on the left side. You should get a Twitter feed in the results.

Essentially, Google fully realizes the need to scale their search engine with the growth of information online.

Cirrus ABS – Experts at Web development, design and technology…

Since its founding in 1995, Cirrus ABS has worked to deliver award-winning Website design and Web development on a national scale. From our headquarters in Fort Wayne, IN to other regional offices like Indianapolis, IN we have delivered astonishingly crafted websites to businesses who want professional Web design, development marketing, and more–using our unique Web technology, and solutions that we call the NetCentered Business Strategy and the Cirrus eBusiness Suite.

We are a full-service Web developer, offering many services that can achieve success and measurable results for your business online. We like to say that, “If you can dream it, we can build it.” And we put heavy importance on the ability of a business to get found online with search engine visibility.

For search engine optimization (SEO) services and localization like “Fort Wayne seo” or “Indianapolis seo” we have it covered. We understand that your business needs to be found for the localities that you serve–on a local, regional or national scale.

Contact Cirrus ABS today, ask about our award-wining Web design, or NetCentered Strategy, we are here to help serve your businesses needs online. You can call 1-877-817-4442 to speak with a solutions consultant. You can also request a free needs assessment to tell us what you may want for your website, or you can request an online demo with us that will highlight our advanced Web technologies.

We also have expert Web design and development that is focused on small business. View our Small Business Suite for an overview–it offers everything that a small business needs to succeed online.