The gist

PixelRAG is an open-source retrieval pipeline from Berkeley researchers that searches documents based on how they look, not just their text. It solves information loss in traditional RAG systems by rendering pages as screenshots. This preserves the visual context of tables, charts, and layouts, allowing language models to accurately read and interpret complex visual information that text-parsing would otherwise destroy.

What it does

Renders web pages and PDFs into a series of screenshot tiles.
Builds a searchable FAISS index from document screenshots using a fine-tuned vision model.
Performs retrieval over visual content using either text or image queries.
Integrates with the Claude AI assistant as a plugin to visually browse and read pages.
Provides a pre-built, queryable index of 8.28 million Wikipedia articles via a public API.

How it works

PixelRAG is an open-source Python tool installed via pip. A user provides a document source, which the pixelshot command renders into image tiles. The pipeline then uses a fine-tuned vision-language model to create embeddings and builds a searchable FAISS index. The system can be run as a local CLI pipeline or a self-hosted API server. A public API is also available for querying a pre-built Wikipedia index.

Best for

This tool is ideal for developers building RAG systems that must process documents where critical information is embedded in visual formats like tables, charts, or complex layouts.

Watch out for

Downloading and self-hosting the pre-built Wikipedia index requires significant disk space (~217 GB). Building a new index from your own documents is a multi-stage, computationally intensive process.