Search engine optimization (SEO) is the process of improving the volume and quality of traffic to a web site from search engines via “natural” (“organic” or “algorithmic”) search results. Usually, the earlier a site is presented in the search results, or the higher it “ranks,” the more searchers will visit that site. SEO can also target different kinds of search, including image search, local search, and industry-specific vertical search engines.
From the early 1990’s, website owners woke up to the idea that having highly visible sites in the search engines brought valuable business results. In tandem, the idea of gaining higher rank, which brought higher visibility, would drive increased traffic. Initially, all a webmaster needed to do was submit a page, or URL, to the various engines which would send a spider to “crawl” that page, extract links to other pages from it, and return information found on the page to be indexed. The process involves a search engine spider downloading a page and storing it on the search engine’s own server, where a second program, known as an indexer, extracts various information about the page, such as the words it contains and where these are located, as well as any weight for specific words, as well as any and all links the page contains, which are then placed into a scheduler for crawling at a later date. 
Here is the brief History of Search Engine Optimization
1995 - The early days of Yahoo.
Optimization was born out of the roots of AAA, A#1, and Acme style yellow pages/white pages alphabetical optimizations.
1996 – Blind luck and keyword seasoning to taste.
The early days were stabs in the dark using simple keyword seasoning. Poke it here, and look for a reaction there. The first concepts of density and location started to be used.
You could still get a site listed in Yahoo by merely submitting it. As long as it wasn’t too gaudy, you were in within 72 hours.
Late 96 – The first papers begin to appear on the web about text matching, data mining, and interviews with se programmers.
Light bulbs of understanding begin going off around the early seo community. People began to realize just how databases work to match text and how they would be applied to the greater database of the web.
1997 – The first algo crackers appear.
The first algo crackers were quite rudimentary by simply studying the make up of pages in the results many of the major clues to the algo’s could be understood.
More specifically, several seo’s decoded all 35 parameters to Excite and were able to build pages precisely to the algo; thus, generating #1 pages at will.
Se’s begin waking up to the fact that their sites are “portals” (in one door and out the other). Se’s begin their first attempts at keeping people on the site in various ways. Some were intentional algo manipulations designed to keep people around the se and searching longer than they should have. (There are some big time stories here if any se techs would like to talk)
Spam becomes a very serious problem for the SE’s as unscrupulous spam sites began to understand algos and how to manipulate them. Hotbot and Altavista were next to useless in late 97 due to spam (last half of 97 and most of 98 were the dark ages for se’s).
More page jacking incidents happen regularly. Hardly any top seo doesn’t have top ranked pages stolen and copied. often copied into foreign domains out of jurisdiction.
Algo crackers begin to talk about the first cloaked pages appearing in the insurance and auto sectors.
1998 – Let’s get serious.
After several papers were delivered at the WWW conferences, it became clear se’s were going to move to off-The-Page criteria. Prerequisites such as link pop, directory listings, and listings age were going to be main parts of the new algo’s.
Decoding algo’s became very sophisticated in mid 98 and 99. Several optimization firms hired programmers to write efficient algo crackers.
Se’s begin to modernize with multi-languages, word lists (term vectors), and other language expertise – the era of the word guru is born.
Google hits the scene in earnest. Page and Brin founded Google in 1998. Google attracted a loyal following among the growing number of Internet users, who liked its simple design. Off-page factors (such as PageRank and hyperlink analysis) were considered as well as on-page factors (such as keyword frequency, meta tags, headings, links and site structure) to enable Google to avoid the kind of manipulation seen in search engines that only considered on-page factors for their rankings. Although PageRank was more difficult to game, webmasters had already developed link building tools and schemes to influence the Inktomi search engine, and these methods proved similarly applicable to gaining PageRank. Many sites focused on exchanging, buying, and selling links, often on a massive scaleTheir first build of 25million urls makes it clear they have a future. I review it and am the first (beep beep) to propose link programs. People begin thinking in earnest about link pop and how to effect it.
Spam page/doorway page auto generators show up on the web every where and some are very good.
Referrals hold steady for those that know the game and stay off the radar. Using quality seo – that doesn’t look like seo – rules the day.
Late 98-early 99 – Altavista fights back with “too many urls” and bans huge segments of sites and sites with auto doorway page generators. Other engines begin out-and-out wars against seo. If a site said “we optimize” or “we promote” anywhere on it, they were banned in massive quantities. Much of that same mentality still exists today in many search engine offices.
Although the algo crackers are at their peak of performance, their utility falls as off-the-page factors such as link popularity become main stream in the se’s. Decoding what makes a page top ten has never been more difficult. Those that know, now spend 10 times (literally) as much time to acheive half the rankings they did in 98. Algo crackers are not much more than statistic generators now.
Google’s PageRank begins to bear fruit while the other se’s self destruct under management chaos and mountains of red ink.
Cloaking becomes almost mandatory on many se’s to protect rankings and code. It is unfortunately used by those not so interested in those factors and more interested in spamming for the sake of instant successes.
Late 99 the effects of the end of seo begin to sink in.
Goto begins to make it’s major push. seo’s begin ppc’izing their billing with store-front redirect sites showing up every where.
Link pop schemes explode.
Other se’s cut huge swaths out of their db’s for unknown reasons. Part of it was size, some of it was spam, and some what just because they could.
seo and traditional algo decoding techniques as we knew it, are all but relegated to the ash heap of history.
Referrals begin to plummet as competition sky rockets and the web matures. I secretly think 99 was when people “settled in” to a daily routine and began using search engines less and less. It was no longer this huge mystery that needed to be explored – they now used it to do productive things. eg: sites such as news take off in record numbers.
2000 – A fairly deep shudder goes through the remaining industry as the end of what was left of Infoseek is gone.
The paid for play schemes and ppc schemes crank up in rapid succession in 2000. From Ink, to Alta, to even buying banners based on keywords – ppc and pfp is every where.
Meanwhile back in the real search industry, surfers look for an engine that actually works at finding them info – Google solidifies its position as the new defacto se.
The link pop craze of 99 begins to fade as it becomes very clear they are risky items – too easily tracked.
The last gasp for link pop programs is the building of fake awords programs, fake guestbooks, fake directories, and fake forum systems just to build fake link pop.
2001-Bought and Paid For listings are everywhere. Goto is on all the major hubs from Yahoo, AOL, to even MSN. People abandon other se’s such as Hotbot, Altavista, and Excite in record numbers. It’s an exodus.
seo is we knew it, is all but over. We are down to talking about the few remaining free specific engines and their systems. There is now a major difference in how se’s work and how to “work them”.
Welcome to the era of “All Google All The Time”.
2002 – Underneath all this was the growing realization that linking was driving our listings, our process of linking new clients to old in an effort to get them spidered was originally designed to avoid the use of automated submission software and also cut down on the lead time for getting indexed. We now realized that ‘inadvertently’ link popularity had driven our clients to the top. In addition to this, working in vertical markets had ensured that the links were on topic and themed to each business type.
This was also the year that Google started to get tough with its linking algorithm; websites that were found to be linking with each other from the same server or IP address were penalized.
2003 – The momentum continued with the anti spam band wagon, and finally companies realized that their websites were going to have to change if they wanted to get listed in search engines. Website design changed radically, flash was now a code to be used carefully, content was now a growing part of any presentation by smaller web agencies
This was the year that saw the complete demise of search engines such as Alta Vista, sold then resold for paltry sums of money, Inktomi once led search technology but was now in Google’s wake. Google maintained its strangle hold and ever tightened it.
2004 – In many ways 2004 was the year the industry will never forget, it benchmarked the sheer power that Google had.
The Florida Update changed the industry forever, ethical website improvement is the only way forward and there is now no room for smart ‘black hat’ tactics. PPC is now firmly embedded in the marketing strategy and SEO is a part of SEM and each few months there will be further shake ups as the technology employed by search facilities just gets better and better
2005 – This was a settling year, the industry was adjusting.
Websites were being designed with functionality, the .com boom and bust was just a memory and many businesses realized that spending money on their Internet facility was money well spent. Google’s database was doubling in size, its indexing rate was slow and the ‘Google Dance’ was a thing of the past. Forums became less busy and dull and the industry became calm.
This year saw the rise of the stealth cull, many websites vanishing from page1 positions with little publicity. Much of this was due to Google testing new algorithms, working on its plan to eradicate artificial link generation. It’s incredible to think that all those years ago Google was created with linking as its core doctrine and now as 2006 arrives Google launches its anti link-spam filters, this update was implemented in three phases, Jagger 1, 2 and 3.
2006- So here we are, full circle, from linking to linking, but once again compliance dictates that a website’s linking must conform to strict guidelines. We are now at a point where Google has publicly stated, “get compliant as we are now able to tell if you are not”. If you don’t think that’s true then just ask the marketing director at BMW and how he felt when Google publicly threw them out for being non compliant with regulations.
By 2007, search engines had incorporated a wide range of undisclosed factors in their ranking algorithms to reduce the impact of link manipulation. Google says it ranks sites using more than 200 different signals. The three leading search engines, Google, Yahoo and Microsoft’s Live Search, do not disclose the algorithms they use to rank pages.
Sources – http://www.webmasterworld.com/
http://www.highposition.net/
http://en.wikipedia.org/wiki/Search_engine_optimization
