There is so much amazing work being done globally by researchers, but it can be difficult to grasp the overwhelming volume of their output. I wanted to build a map to help navigate this vast scientific landscape through spatial exploration and discovery, rather than just parsing traditional search lists.
How it works:
* Data & Embeddings: Sourced the latest 10M papers from OpenAlex and generated embeddings using SPECTER 2 on titles and abstracts.
* Layout & Clustering: Reduced dimensionality with UMAP, then applied Voronoi partitioning on density peaks to create distinct semantic neighborhoods.
* Labelling: The floating topic labels are generated via custom labelling algorithms (definitely still a work in progress!).
* Frontend: Rendering 10M points directly would be impractical, so I implemented geospatial tiling with dynamic level-of-detail loading on a web Mercator projection.
The search supports both keyword and semantic queries, and there's an analytics layer for ranking institutions, authors, and topics across the corpus.
It's live and free to use. Would really appreciate any feedback or suggestions!
Hi HN,
There is so much amazing work being done globally by researchers, but it can be difficult to grasp the overwhelming volume of their output. I wanted to build a map to help navigate this vast scientific landscape through spatial exploration and discovery, rather than just parsing traditional search lists.
How it works:
* Data & Embeddings: Sourced the latest 10M papers from OpenAlex and generated embeddings using SPECTER 2 on titles and abstracts. * Layout & Clustering: Reduced dimensionality with UMAP, then applied Voronoi partitioning on density peaks to create distinct semantic neighborhoods. * Labelling: The floating topic labels are generated via custom labelling algorithms (definitely still a work in progress!). * Frontend: Rendering 10M points directly would be impractical, so I implemented geospatial tiling with dynamic level-of-detail loading on a web Mercator projection.
The search supports both keyword and semantic queries, and there's an analytics layer for ranking institutions, authors, and topics across the corpus.
It's live and free to use. Would really appreciate any feedback or suggestions!