TL;DR: A new interactive visualization lets you explore GPT-2-small's token embedding space via t-SNE and minimum spanning tree, with mobile support and search.
Summary: An interactive map of GPT-2-small's 32,070 alphabetic token embeddings (from the WTE layer) was released. It uses t-SNE for layout and a minimum spanning tree to show nearest-kin connections. Users can pinch-to-zoom, tap tokens to see nearest neighbors, walk the graph, and search for any token directly — all without a forward pass or context.
Why it matters: This tool provides an intuitive way for AI builders to explore token relationships and embedding geometry, aiding interpretability and debugging of language models. Try it to quickly locate and visualize how GPT-2 groups semantically or syntactically similar tokens.
Source: reddit