As I continue to play with the AOL search data one conclusion has become obvious: The tools for wandering through large datasets are dreadful. There are more than 20-million searches in the data, which means, for example, that Excel (with its 65,536 row limit) can only hold about 3% of the data. I have also been using SPSS, which is better. It can hold something like 2-billion cases, which is nice, but even with a few hundred thousand cases loaded it gets very, very slow.
Playing with the AOL Search Data
By August 7, 2006 · ··