Our search engines tried to impose structure and find relationships using mainly unintentional clues. You therefore couldn’t rely on them to find everything that would be of help, and not because the information space was too large. Rather, it was because the space was created by us slovenly humans.
Interesting article on how search algorithms have changed as the web has grown in scale. In the beginning, we got results that were determined by precision and recall (although optimising for one meant reducing the importance of the other). Then relevance became necessary to include as the number of possible results became too large i.e. when you have 100 000 articles that match the topic, the search engine must decide how to rank them for you. Over time, interestingness was another concept that was built into the algorithm; it’s not just that the results should be accurate and relevant, but they should be interesting too.
Currently, there’s interest in serendipity, where search engines return results that are slightly different to what you’re looking for and may serve to provide an alternative point of view (but not so different that you ignore it) and so avoid the filter bubble. As we move forward, we may also begin seeing calls for an increase in the truthfulness of results (which may reasonably be called quality). As I said, it’s an interesting article that covers a lot with respect to how search engines work, and