Web Optimization & Analytics: A Brief History

This is a new series dedicated to defining and learning about web optimization – from its inception all the way through to what the future may hold for the technology. To kick us off, we’re taking you on a whistle stop tour of the past.

But first, I wanted us to understand what it means today. Our friends over at Optimizely have done a lovely job of defining web optimization:

“Website optimization is the process of using controlled experimentation to improve a website’s ability to drive business goals. To improve the performance of their website, website owners implement A/B testing to experiment with variations on pages of their website to determine which changes will ultimately result in more conversions (eg. demo requests, increases in organic search results, more purchases, reduced customer service time, etc.).”

Now, that’s a pretty clear description. Yet before website owners started experimenting with their user experience (UX), web optimization had more modest ambitions: Attracting visitors, often through search engines.

Web Optimization Starts Here

In our always-connected world, it’s hard to believe that search engines as we know them today weren’t even a thing until the early 1990s.

Once the Internet was accessible to everybody, it became necessary to index its content so people could find what they were looking for. Early attempts, such as Archie (circa 1993, which later became Excite), sorted results based on keywords in a site’s content as well as optimization on the backend.

In the latter part of 1993, the first bot-driven search engine came to be, and in 1994, we came to know Yahoo, Lycos, Alta Vista, and Infoseek. Google founders entered the scene in 1996 with a product that was initially called BackRub, and in 1997, Google.com was officially registered as a domain.

This is where we can begin talking about SEO in earnest.

Keywords and Meta Tags

SEO (Search engine Optimization) was developed as a remedy, to more efficiently deliver relevant results in response to search queries. This ability opened the doors for e-commerce and online marketing. Along with the potential for sales, companies found value in promoting their websites to drive better search engine results.

Before Google, most search engines were based on either a human-powered directory of “approved” websites or crawler-driven sites like AltaVista, Lycos, and Yahoo.

In those times, the only method of achieving SEO was through on-page techniques, like keywords and meta tags. The trouble with this was that it gave rise to a lot of spammy and low-quality websites. All you really had to do to rank higher was to include more keywords than your competitor.

At this point, site owners still had to submit their sites to search engines in order to be indexed. SEO became an industry unto itself, with technicians working hard to drive their clients’ sites higher in the SERPs (Search Engine Results Pages). Ranking factors included keywords, indexing, domain names, and basic site structure.

Link Building Becomes a Ranking Factor

Google’s PageRank and web crawler algorithms were a revolutionary new way to approach search. Instead of only considering keywords and meta tags, they looked at off-page factors, such as the quantity and quality of backlinks pointing to your site. Essentially, the idea was that the more people who were talking about your site, the more important you must be.

Link-building became a sub-industry. Much like keywords, it became a highly abused tactic that future algorithms were created to combat.

Page rank was a race to the finish, competing with paid results through AdWords and other players. However, Google’s monthly index updates often resulted in significant rank fluctuations, which imbued SEO technicians with fresh challenges on a regular basis.

AdSense  

Google itself was responsible for the next wave of spammy tactics in the form of AdSense, a system that monetized websites with AdWords advertising, effectually spawning a blogger revolution.

The problem was, people were creating spammy sites. Often based on stolen content, the aim was simply to get clicks, rank higher in the SERPs, and make money from ad revenue.

Local SEO and personalization 

Around 2004, Google and other search engines started ranking results for search queries that included a geographic element – such as searching for a plumber in your town. The Maps Plus box also rolled out around this time and personalization became a significant ranking factor.

Personalization meant that your search results would take into account your individual search history and interests.

 

Google Analytics and YouTube 

Google acquired YouTube in 2006, which went on to become one of the most vital search tools on the web. With the growing popularity of video, brands began to jump on the bandwagon, and video SEO was born.

Google Analytics also launched around this time, giving webmasters access to critical insights about their site’s performance. Google Webmaster Tools was a companion product, allowing visibility into crawling errors and delivering information on what search queries your site was appearing in.

Other notable SEO advances include:

• XML Sitemaps: An XML sitemap gave webmasters the ability to index every page on their site, not just the homepage.

• Universal Search: Search engines include news, video, images, and various other types of results in searches, in addition to organic HTML and paid results.

• Brands and Trust: To weed out the digital riffraff, brands were given greater weight in the SERPs as trust became an overarching ranking factor.

• Site Speed: Site speed was announced as a ranking factor in 2010.

• Social Signals: As social media began a factor in driving traffic, it became an indirect ranking factor in SEO.

• Prioritizing Quality: Google’s Panda update targeted low-quality sites and sites that contained plagiarized, auto-generated, or meager content, while Penguin sought out sites that used aggressive spam tactics.

• Mobile First: In 2015, mobile search outclassed desktop. Google answered this statistic by prioritizing mobile-friendly sites.

Web Optimization Grows Up

As you can see website optimization has a rich, varied history, spanning close to 30 years. With deep roots in SEO, its evolution has been a sort of bellwether for how we as humans understand and use the web.

From education to commerce to play, the Internet truly underpins our lives – no wonder there are so many companies fighting to improve how we experience it.