Wiki Search Engine

This project helped me appreciate how much thought goes into creating a index for search (Google is truly remarkable in this regard, let’s hope Bing equipped with ChatGPT rises to the challenge). Dealing with large text dumps from wikipedia, figuring out how to make python work at high speeds (without using Cython or any other compiled python of course) and trying to figure out why a search for “Harry Potter” was returning “A New Hope”, “The Empire Strikes Back” and “Return of the Jedi” (I still believe that these search results are more relevant).

The README for this project is pretty comprehensive, do check it out here!