What happens if we search CPAN for a module to help parse robots.txt?
The second result is something from 1997, which is part of some kind of obsolete bundle, and the most useful result, the module which everyone uses, which should be the top, is languishing at number ten, underneath a bunch of modules with abstracts written in Japanese, and parts of some web framework or other. What's even worse is that the second result, from 1997 is actually a very old copy of the 2011 module at number ten, with an "s" added to its name:
# $Id: RobotRules.pm,v 1.14 1997/01/26 14:35:12 aas Exp $ package WWW::RobotRules; =head1 NAME WWW::RobotsRules - Parse robots.txt files =head1 SYNOPSIS
I've been using search.cpan.org for several years, and it's always been like this. I don't know what logic is used to order the results, but in my opinion, a completely random ordering of search results would be an improvement, since then there would be a small chance of relevant modules coming top in the search results.
lcwa is the bundle of modules which contains the WWW::RobotsRules module mentioned above.