Real-time search is really expensive - CEO Technorati


	Real-time search is really expensive - CEO Technorati

arifsali

6-7-2009 11:59 AM

172 views

tags:

internet, search, technology

Add a Comment

Login to Comment. Not a member yet? Sign up

Today's Top Clips

visit the Top Clips page

View the Top Clips from June 7, 2009

Embed This Clip In Your Site...

<div style="margin: 12px 0px; font-family: arial; color: #333333; background: #ffffff; border: solid 4px #e5e5e5; width: 100%; clear: left;"><div class="CM_CTB_Content_Wrap" style="margin: 0px; padding: 0px;background-color: #ffffff;"><div style="border-bottom: solid 1px #dcdcdc; white-space: nowrap; margin-bottom: 8px; background-color: #eeeeee ;background-image: url(http://www.clipmarks.com/images/source-bg.gif); background-repeat: repeat-x; height: 24px; line-height: 24px; vertical-align: middle; padding-bottom: 4px; color: #666666; font-size: 10px;" ><a href="http://clipmarks.com/clip-to-blog/" title="see clips that are hot right now"><img src="http://content.clipmarks.com/blog_embed/3d9bdfb6-4a96-41ed-8334-156664caa018/A5A878CD-8CAE-4653-8FF2-E824DD97FB86/" alt="" width="19" height="19" border="0" style="vertical-align: middle; margin: 0px 4px; display: inline; border: none; float:none;" /></a>clipped from <a title="http://www.jalichandra.com/2009/05/new-real-time-search-buzz-isnt-that-new.html" href="http://www.jalichandra.com/2009/05/new-real-time-search-buzz-isnt-that-new.html" style="font-size: 11px;">www.jalichandra.com</a></div><blockquote style="text-align: left; padding: 0px 8px; margin: 4px 0px 8px 0px; background: transparent; border: none;" cite="http://www.jalichandra.com/2009/05/new-real-time-search-buzz-isnt-that-new.html">Making real-time work is very complex at scale, and there's a reason there are only two real-time blog search indexes (Google Blog Search and Technorati). The volume of data presents multiple challenges: the data becomes nearly irrelevant shortly after it appears (often within days and certainly within weeks -- over 90% of all searches on Technorati are looking for something less than a month old); it’s much easier to spam* (Twitter is just beginning to experience this -- just wait...); it’s hard to balance recency and relevancy together; and lastly, it's expensive -- spinning large quantities of data so it's readily available to query is really expensive, and the entire live web is a really large place (Technorati only focuses on the blogosphere).</blockquote></div><div style="margin: 0px 6px 6px 4px;"><table style="font-size: 11px;border-spacing: 0px;padding: 0px;" cellpadding="0" cellspacing="0" width="100%"><tr><td style="background:transparent;border-width:0px;padding:0px;">&nbsp;</td><td align="right" style="background:transparent;border-width:0px;padding:0px;width:107px" width="107"><a href="http://clipmarks.com/share/A5A878CD-8CAE-4653-8FF2-E824DD97FB86/blog/" title="blog or email this clip"><img src="http://content8.clipmarks.com/images/c2b-foot.png" border="0" alt="blog it" width="107" height="17" style="border-width:0px;padding:0px;margin:0px;" /></a></td></tr></table></div></div>

Clipmarks: Home; New Clips; Top Clips; Dashboard

Popular Topics: News; Life; Science; Technology; Entertainment

Get Started: Sign Up; Install Clipping Tool; How Clipping Works; Clip-to-Blog™; ClipSearch

Tools and Resources: FAQ; ClipWeek; Top Clippers; Top Tags; Site Map

About Clipmarks: About Us; Contact; Blog; Copyright; Privacy; EULA

Search Options

Today's Top Clips