How Search Engines Work

In our previous Back to Basics blog, we walked through how the internet works. Now knowing the internet is essentially a vast collection of computers and servers, it’s time to move on to how search engines work.

 

Search engines are a powerful everyday tool used to answer questions and locate information on the internet. Who’s in the search engine game? Think big names like Google, Yahoo, and Bing. With the familiar phrase “Google it”, it goes without saying that Google dominates the space (we’ll use them as our main reference below).

 

Search engines are made up of a few different things. Crawlers, databases, and algorithms. They work together to provide the most relevant search results to a user.

 

 Crawlers

 

Crawlers can be referred to as spiders. They are going to crawl each page of a website, grabbing content, html markups, and links. 

 

If you’ve ever heard the term “a search engine friendly website,” it means that the navigation is set up so the crawlers can easily sort through content on the various pages.

 

Indexing

 

Once the spider has crawled the website page, it’s going to put all the information it finds into a database, which can also be called an index. 

 

Step one for making a website is making sure you’re actually in the index, because if not, you’re never going to be shown for anything. 

 

Algorithms

 

Search engine algorithms look at the website as a whole, and determine for a specific search term, which websites should be ranking above others. Nowadays, this happens in real-time. As crawers add information to the indices, the corresponding search results appear on your screen.

 

Here is the basic structure of an algorithm:

All of this is important because ranking on the first page of Google values your website (therefore business) as higher, gaining credibility and visibility which can result in more leads.

As Ricky Bobby once wisely said, “if you’re not first, you’re last!”

 

Ranking - Common Mistakes

 

The mistake we see most companies make is that they only focus on small pieces of the algorithm or implementing the “newest trick”, such as adding rich snippet codes to their website. 

 

It’s important to take a step back and think about the ultimate goal of the search engine. Who is their customer? Who are they serving? Google’s customer is the user (that’s me, you, anyone using their tool to find information on the web). Thus, Google’s goal is to provide the best resource for any given search term. 

 

New tricks and loopholes don’t always work. Companies still have to put in good old fashioned work to rank at top of a search page.

 

Ranking - Popularity Metrics

 

We can’t forget about all the information the crawlers have gathered when simplifying the algorithm to an input/output model. They pay close attention to the amount of links that are found within a website. The more relevant links, the more power is given to the website, as the search engine will now recognize it as a popular reference. Google values this and in return will rank a site higher in search results.

 

Popularity metrics used to be based primarily on the amount of credible links on your website pages. But over the years, search engines have expanded their algorithms to factor in other characteristics in order to provide their users with better results. These include positive Google ratings and reviews, social media influence, and cookies (tracking user behavior).

 

It’s also important to understand what search engines, like Google, devalue.

 

Google devalues:

 

1. Sites that are not mobile optimized. 
2. Sites that have a higher ad to content ratio. 
3. Sites with a long page load time. 
4. Sites that do not have accurate and reflective title tag and meta description.

 

A majority of these scenarios contribute to a poor user experience. They are extremely important aspects of your site to consider, especially before working on search engine optimization. 

 

Knowledge is Power

 

Knowing how crawlers, indices and algorithms work will set you up for success when working with an agency to build your website or implement SEO strategies. Even though you may not be doing the nitty gritty work yourself, knowledge is power. Understanding the process and why an agency is recommending a specific strategy will smooth over and speed up the collaboration process.

 

We hope you enjoyed our second Back to Basics article. You can look forward to our next series topic on Pay Per Click (PPC) advertising.

 

Have questions about the “basics”? We’d love to hear them. Send us your topic suggestions here.

RECENT POSTS