IAR: Hungry Bot Behind Microsoft's New Search

While it remains hidden behind closed doors in Redmond, WA, Microsoft's developing new search engine is proving ravenous. Site managers across the Internet are reporting frequent and deep visits by the indexing engine, which returns nightly to Microsoft with its haul of data. Some of the site managers complained that the frequent searches caused bandwidth problems, but Microsoft allayed those concerns for many by indicating how sites could be configured to reduce the frequency with which the crawler visited. Traffic from MSNbot grew rapidly in May, and grew again in June. Changes made to Microsoft's search engine yesterday do not yet incorporate results from the new back-end system.