README

<!-- Project Info -->
<a href="https://github.com/kreuzberg-dev/liter-llm/blob/main/LICENSE">
	<img src="https://img.shields.io/badge/License-MIT-007ec6" alt="License" />
</a>
<a href="https://docs.liter-llm.kreuzberg.dev">
	<img src="https://img.shields.io/badge/Docs-liter--llm-007ec6" alt="Docs" />
</a>

A lighter, faster, safer universal LLM API client — one Rust core, 14 native language bindings, 143 providers.

What and Why?

liter-llm is a universal LLM API client compiled from the ground up in Rust: one core, 14 native language bindings, and 143 providers. No interpreter, no transitive dependency tree, no supply-chain surface area — and a drop-in OpenAI-compatible proxy plus an MCP server in a single 35 MB binary.

• Compiled Rust core — no pip install supply chain, no .pth auto-execution hooks, no runtime dependency tree to compromise.
• Secrets stay secret — API keys are wrapped in secrecy::SecretString: zeroed on drop, redacted in logs, never serialized.
• Polyglot from day one — Python, TypeScript, Go, Java, Kotlin, C#, Ruby, PHP, Elixir, Dart, Swift, Zig, and WebAssembly, all thin wrappers over the same Rust core, plus a C/FFI surface for everything else.
• Observability built in — production-grade OpenTelemetry with GenAI semantic conventions, not an afterthought callback system.
• Composable middleware — rate limiting, caching, cost tracking, health checks, and fallback as Tower layers you stack like building blocks.

We credit litellm for proving the category; our provider registry was bootstrapped from theirs. See ATTRIBUTIONS.md.

Features

⭐ Star this repo to show your support — it helps others discover liter-llm.

Quick Start

Language Packages

pip install liter-llm

pnpm add @kreuzberg/liter-llm

cargo add liter-llm

go get github.com/kreuzberg-dev/liter-llm/packages/go

gem install liter_llm

composer require kreuzberg-dev/liter-llm

dotnet add package LiterLlm

dart pub add liter_llm

pnpm add @kreuzberg/liter-llm-wasm

brew install kreuzberg-dev/tap/liter-llm

docker run -p 4000:4000 -e LITER_LLM_MASTER_KEY=sk-your-key ghcr.io/kreuzberg-dev/liter-llm

liter-llm api --config liter-llm.toml   # OpenAI-compatible proxy (22 endpoints)
liter-llm mcp --transport stdio         # MCP tool server

AI Coding Assistants

Install the liter-llm plugin from the kreuzberg-dev/plugins marketplace. It ships the liter-llm agent skills (chat, streaming, tools, embeddings across 143 providers) and works with every major coding agent — expand your harness below.

/plugin marketplace add kreuzberg-dev/plugins
/plugin install liter-llm@kreuzberg

/plugins add https://github.com/kreuzberg-dev/plugins

gemini extensions install https://github.com/kreuzberg-dev/plugins

droid plugin marketplace add https://github.com/kreuzberg-dev/plugins
droid plugin install liter-llm@kreuzberg

copilot plugin marketplace add https://github.com/kreuzberg-dev/plugins
copilot plugin install liter-llm@kreuzberg

Documentation

Full guides, the unified chat() API for every binding, multimodal I/O, the proxy/gateway, and the complete provider list live at docs.liter-llm.kreuzberg.dev.

Contributing

Contributions are welcome! See CONTRIBUTING.md for guidelines.

Join our Discord community for questions and discussion.

Part of Kreuzberg.dev

• Kreuzberg — document intelligence: text, tables, metadata from 91+ formats with optional OCR.
• Kreuzberg Cloud — managed extraction API with SDKs, dashboards, and observability.
• kreuzcrawl — web crawling and scraping with HTML→Markdown and headless-Chrome fallback.
• html-to-markdown — fast, lossless HTML→Markdown engine.
• liter-llm — universal LLM API client with native bindings for 14 languages and 143 providers.
• tree-sitter-language-pack — tree-sitter grammars and code-intelligence primitives.
• alef — the polyglot binding generator that produces every per-language binding across the 5 polyglot repos.

License

MIT — see LICENSE for details.