Infinity Embedding Server

Infinity serves embedding and reranking models behind a high-throughput, low-latency REST API. Created by Michael Feil and MIT licensed, it deploys suitable models straight from the Hugging Face Hub: text embedding families such as BGE and mixedbread, rerankers for semantic relevance scoring, classification models for sentiment and emotion, and multimodal encoders including CLIP for image-text, CLAP for audio-text, and ColPali for document images. Requests flow through dynamic batching with dedicated tokenization workers, and inference runs on PyTorch, Optimum with ONNX or TensorRT, and CTranslate2, with FlashAttention acceleration across NVIDIA CUDA, AMD ROCm, CPU, AWS Inferentia2, and Apple MPS hardware. The API follows OpenAI's embedding specification so existing clients work unchanged, and a CLI can orchestrate multiple models in one server. Docker images per accelerator plus a pip install path make it a frequent choice for self-hosted retrieval stacks.