Search Isn’t Broken
ZDNet blogger Tom Foremski poses the question, “Is Search Broken?” Foremski’s thesis is that Google, Yahoo, MSN et al should be able to find Web content and figure out what it’s about without so much human intervention:
– There are many publishers that try to make sure their headlines catch the attention of the search engines rather than catch the attention of readers. The same is true for content, editors increasingly optimize it for the search engines rather than the readers.
– Why should I have to tag my content, and tag it according to the specific formats that Technorati, and other search engines recommend? Aren’t they supposed to do that? …
– The search engines ask web site owners to mask-off parts of their sites that are not relevant, such as the comment sections, with no-follow and no-index tags…
– Every time I publish something, I send out notification “pings” to dozens of search engines and aggregators. Again, they don’t have to send out their robots to check if there is new content.
My first reaction was, “Welcome to MY world, Tom.” Perhaps Foremski’s rant is merely linkbait… but it raises an important issue. People DO have to design their sites and write their content with search engines in mind… but only if they hope to boost their search traffic. Is that a bad thing? I’d argue that today, specific search engine optimization steps in site design and content writing are less important than they have ever been… the reason that the requirements for content creators seem to be increasing is that big companies are finally aware that search engine traffic counts.
Since the early days of search engines, the quality of results has improved dramatically, largely because search engines have become LESS demanding of the content writers and site designers. In the early days of search, the only way to rank might have been to repeat the keyword a dozen times in a short article. Even major corporate sites might not be found for a search as simple as the company name, perhaps because the designers included the firm name only in an image. Innovations like Google’s PageRank and analysis of link text on other sites suddenly took much of the onus off site developers and content writers – no matter how badly GM.com was designed, it would now pop up at the top of searches for “general motors”.
One challenge that search engines face is an ongoing war with search engine optimizers. In a perfect world, Google and its brethren would have been delivering better results years ago with a lot less work. The fact that search rankings drive traffic, and traffic makes money, mean that whatever improvements are made to the algorithm, creative SEO practitioners will find a way to shortcut the process. Keyword density works? Fine, we’ll stuff pages full of keywords. Density is out, and link text works? OK, we’ll set up link networks. The search engines can find the networks? OK, we’ll buy links on other sites by the cartload. It truly is an arms race – I can’t think of another field of endeavor in business (except, perhaps, various security and military fields) where continuous product improvement is required because well-financed opponents are working to defeat your technology.
Today, search results are better than ever, despite (and perhaps partially because of) the attempts to game them. The fact that content creators like Foremski have to spend a bit of time tweaking their content for search engines is less an indication that search is broken – that content would be indexed, even without a keyword title, Google sitemap, and other things Foremski complains about – and more of an indication that the firm’s management wants to rank better than the competition and drive more traffic to their sites.