Search has become one of the primary way to access information online. There are six major drivers that have caused this departure from “orientation & navigation” tools as main design challenge:
Having good search means you need to have a very good index. Indexing vast amounts of data means having to dig through all that data and create a search index from that. In case of private local searches (like Spotlight on OS X) this requires the own computer to work through all. In case of search engine it meant that the part of the Internet that needs to be indexed needs to be downloaded time and time again. Bandwidth is still a limitation — but dealing with this has become a completely different problem today it is more a question of optimization and not so much a hard limit.
2. Search speed
IT systems that enable search (crawlers, indexing, storage, etc) have become much faster. An average Google search delivers a result page in less than a second — sometimes less than half a second. Searching and browsing through search results have become instantaneous. Also the local searches — like the search inside an E-Mail client — have become much faster recently due to the adoption of SSD as fast storage medium in consumer laptops and desktops.
3. Users have more experience with search
Users have become use to search as a means to access information on large website. It is sometimes far more efficient to find stuff by using the site search than to try to guess the right location of information within a sometimes arbitrary hierarchy of pages and overviews.
Mobile has been trying to make users be able to use websites almost as convenient an accessible as their desktop versions. Designers have tried to make mobile websites feel a lot like native apps — not always for the good. But generally mobile designs are less optimal for browsing large hierarchies — so search is simply and often a more effective way to get around.
5. Natural language processing
The adoption of natural language has improved hands free access to information. It has also improved the way search engines parse what users are entering as query.
6. Artificial Intelligence and Big Data
Beside of the natural language processing, search engines have been busy to improve the quality of results by observing how users interact with the search results. Often clicked items appear to be of higher relevance to users — so they might get a boost in the ranking. By statistically analyzing billions of interactions this approach has become feasible. All in all it has become obvious that users prefer search over navigation to find and access information on the web.