Apps tagged with 'llm-inference'

Private offline on-device local LLM app by Ente, open source code, no accounts or tracking, and works cross-platform on mobile and desktop devices.

Run large language models offline with a built-in chat UI, OpenAI-compatible server, HuggingFace integration, local discovery, private model management.

Minimal, clean full-stack LLM chatbot, running tokenization, pretraining, finetuning, evaluation, inference, and web UI on a single 8xH100 node.

Custom Swift and Metal runtime enabling 26B instruction-tuned model inference in around 2 GB RAM by streaming weights from SSD, optimized for Apple Silicon Macs.

Hundreds of models & providers. One command to find what runs on your hardware.

Build and deploy generative AI on the fastest and most efficient inference engine, fine-tuning and switching between models without extra costs.

Local AI app and inference engine for agents. Run open-weight LLMs locally — private, 100% offline on your computer.

LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar.

Run LLMs on AMD Ryzen™ AI NPUs in minutes. Just like Ollama - but purpose-built and deeply optimized for the AMD NPUs.

Run Llama, Gemma, Qwen, DeepSeek, and more locally on your iPhone, iPad, and Mac. Offline. Private. No login. Optimized for Apple Silicon.

This application provides a full suite of generative AI features for chat, code assistance, document search, image analysis, image and video generation. All features run offline and are powered by your PC’s Intel® Core™ Ultra with built-in Intel Arc GPU or Intel Arc™ dGPU...

AI00 RWKV Server is an inference API server for the RWKV language model based upon the web-rwkv inference engine.

Nexa Studio: Offline, Private AI with Unlimited Use.

Learn how to add AI with local models and APIs to Windows apps. Discover AI scenarios and models such as Phi, Mistral, Stable Diffusion, Whisper, and many more to delight your users. The AI Dev Gallery is an open-source app designed to help Windows developers integrate AI...

Multimodal Offline LLM Chat App with MNN.

Lemonade helps users discover and run local AI apps by serving optimized LLMs right from their own GPUs and NPUs.

Experience the power of RWKV models directly on your device. Completely offline, privacy-first, and efficient. No internet required.

Inferencer lets you run, host and deeply control the latest SOTA AI models (OSS, DeepSeek, Qwen, Kimi, GLM, MiniMax and more) from your own computer.

MiniMax Platform is a versatile AI ecosystem offering advanced models for text, speech, video, and music generation, optimized for coding, creative expression, and immersive interaction.

SmolChat allows you to download and run popular LLMs on your Android device, locally, without needing an internet connection. Customize the model used for each chat, tune settings like temperature and min-p, and pin your favourite chats on the home-screen with shortcuts.

LLM Hub is an open-source Android app for on-device LLM chat and image generation. It's optimized for mobile usage (CPU/GPU/NPU acceleration) and supports multiple model formats so you can run powerful models locally and privately.

An AI-powered chat interface designed for exploring advanced topics like quantum computing, programming, and emerging AI trends through interactive conversation.

RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all through the familiar language of containers.

A unified, OpenAI-compatible gateway to every major LLM provider. One endpoint, one key, one bill - with automatic routing and failover.

NVIDIA NIM is a set of accelerated inference microservices that allow organizations to run AI models on NVIDIA GPUs anywhere.