[reposted from something I wrote on JB Searchblog]

Yes, it can be explained, but it's complicated. It has to do with all the maneuvering over a legal argument called "least restrictive means", and testing censorware vs. the law at issue. The idea here was to get a random sample of sites that people might find by searching, which is why they went to the search engines. I *think* they simply didn't do it themselves because the statistics professor expert wanted a sampling procedure that was more complex than the obvious crawler.

Remember, nobody knew this was going to be such a tempest, and the other search engines didn't make an issue over it.