llama.cpp is described as 'The main goal of llama.cpp is to enable LLM inference with minimal setup and state-of-the-art performance on a wide range of hardware - locally and in the cloud' and is a large language model (llm) tool in the ai tools & services category. There are more than 10 alternatives to llama.cpp for a variety of platforms, including Mac, Linux, Windows, Android and Self-Hosted apps. The best llama.cpp alternative is Ollama, which is both free and Open Source. Other great apps like llama.cpp are GPT4ALL, Jan.ai, LM Studio and PocketPal.
Supports local deployment of Llama 3, Code Llama, and other language models, enabling users to customize and create personalized models. Ideal for AI development, it offers flexibility for offline AI needs and integrates AI writing and chatbot tools in local setups.
An ecosystem of open-source chatbots trained on a massive collections of clean assistant data including code, stories and dialogue.
Jan is an open source alternative to ChatGPT that runs 100% offline on your computer.
Maid is a cross-platform Flutter app for interfacing with GGUF / llama.cpp models locally, and with Ollama and OpenAI models remotely.
Drop-In OpenAI replacement, On-device, local-first, Generate text/image/speech/music/etc... Backend Agnostic: (llama.cpp, diffusers, bark.cpp, etc...), Optional Distributed Inference(P2P/Federated).
A Gradio web UI for Large Language Models. Supports transformers, GPTQ, llama.cpp (GGUF), Llama models.
KoboldCpp is an easy-to-use AI text-generation software for GGML models. It's a single self contained distributable from Concedo, that builds off llama.cpp, and adds a versatile Kobold API endpoint, additional format support, backward compatibility, as well as a fancy UI...
Run LLMs on device or connect to various commercial or open source APIs. ChatterUI aims to provide a mobile-friendly interface with fine-grained control over chat structuring.
A modern web interface for managing and interacting with vLLM servers (www.github.com/vllm-project/vllm). Supports both GPU and CPU modes, with special optimizations for macOS Apple Silicon and enterprise deployment on OpenShift/Kubernetes.
Run LLMs on AMD Ryzen™ AI NPUs in minutes. Just like Ollama - but purpose-built and deeply optimized for the AMD NPUs.
United States
Singapore
Australia