Open Source Ollama Alternatives

The best open source alternative to Ollama is Jan.ai. If that doesn't suit you, our users have ranked more than 100 alternatives to Ollama and loads of them are open source so hopefully you can find a suitable replacement. Other interesting open source alternatives to Ollama are Ensu, AnythingLLM, Alpaca - Ollama Client and Ollama App.

filter to find the best alternatives

Most Ollama alternatives are Large Language Model (LLM) Tools, but you can also narrow the list to AI Chatbots or AI Writing Tools. Another popular filter is Open Source. You can also filter for EU-based alternatives if you prefer software from the European Union.
Copy a direct link to this comment to your clipboard
Alternatives
Ollama alternatives page was last updated

Alternatives list

  1. Off Grid AI icon
     6 likes

    The Swiss Army Knife of offline AI. Chat, speak, and generate images. Privacy first, zero internet. Download an LLM and use it on your mobile device. No data ever leaves your phone.

    87 Off Grid AI alternatives

    Cost / License

    • Freemium (Pay once or Subscription)
    • Open Source (MIT)

    Since

    • Since 2026

    Platforms

    • Windows
    • Mac
    • Linux
    • Android
    • iPhone
    Good alternative?
     
  2. ChatMCP icon
     2 likes

    ChatMCP is a cross-platform AI chat client built with Flutter that integrates with Model Context Protocol (MCP) servers to provide enhanced AI interactions. This document provides a high-level overview of the system architecture, core components, and technology stack that powers...

    Cost / License

    Since

    • Since 2026

    Platforms

    • Windows
    • Mac
    • Linux
    • Online
    • Android
    • iPhone
    Good alternative?
     
  3. SuperAGI icon
     5 likes

    A dev-first open source autonomous AI agent framework. Enabling developers to build, manage & run useful autonomous agents quickly and reliably.

    Cost / License

    • Free
    • Open Source (MIT)

    Application type

    Since

    • Since 2023

    Platforms

    • Online
    Good alternative?
     
  4. MLC LLM icon
     4 likes

    MLC LLM is a machine learning compiler and high-performance deployment engine for large language models. The mission of this project is to enable everyone to develop, optimize, and deploy AI models natively on everyone’s platforms.

    Cost / License

    Application type

    Since

    • Since 2025

    Platforms

    • Windows
    • Mac
    • Linux
    • Online
    • Android
    • iPhone
    • iPad
    • Python
    Good alternative?
     
  5. oMLX icon
     5 likes

    LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar.

    56 oMLX alternatives

    Cost / License

    Since

    • Since 2026

    Platforms

    • Mac
    Good alternative?
     
  6. GPT Engineer icon
     5 likes

    GPT Engineer is a ChatGPT agent designed to code based on how you want it to. It generates an entire codebase based on a prompt along with some feedback.

    Cost / License

    • Free
    • Open Source (MIT)

    Since

    • Since 2023

    Platforms

    • Windows
    • Linux
    Good alternative?
     
  7. RWKV Chat icon
     4 likes

    Experience the power of RWKV models directly on your device. Completely offline, privacy-first, and efficient. No internet required.

    Cost / License

    Since

    • Since 2026

    Platforms

    • Windows
    • Mac
    • Linux
    • Android
    • iPhone
    • iPad
    • Android Tablet
    Good alternative?
     
  8. txtai icon
     2 likes

    Embeddings databases are a union of vector indexes (sparse and dense), graph networks and relational databases. This enables vector search with SQL, topic modeling, retrieval augmented generation and more.

    Cost / License

    Since

    • Since 2021

    Platforms

    • Self-Hosted
    • Docker
    Good alternative?
     
  9. Warden icon
     4 likes

    Warden is a minimalist, simple and beautiful macOS AI chat app, that supports most AI providers: ChatGPT, Anthropic (Claude), xAI (Grok), Google Gemini, Perplexity, Groq, Local LLMs through Ollama, OpenRouter, and almost any OpenAI-compatible APIs.

    Cost / License

    Application type

    Since

    • Since 2025

    Platforms

    • Mac
    Good alternative?
     
  10. Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema on the model output on the generation level.

    Cost / License

    • Free
    • Open Source (MIT)

    Since

    • Since 2024

    Platforms

    • Windows
    • Mac
    • Linux
    • Android
    Good alternative?
     
  11. LanceDB icon
     2 likes

    LanceDB is an open-source database for vector-search built with persistent storage, which greatly simplifies retrieval, filtering and management of embeddings.

    Cost / License

    Application type

    Since

    • Since 2023

    Platforms

    • Self-Hosted
    Good alternative?
     
You are at page 3 of Ollama alternatives