Inferencer

Inferencer lets you run, host and deeply control the latest SOTA AI models (OSS, DeepSeek, Qwen, Kimi, GLM and more) from your own computer. No data is sent to the cloud for processing - maintaining your complete privacy. Advanced inferencing controls give you complete control on their accuracy and outputs.

Models Start in the models section where you can select the location of existing models or download new ones directly from Hugging Face. Use the model streaming feature to inference larger models partially from storage - for low memory devices.

Chats Select the model to interact with on the top menu bar and write a prompt to begin. At any point you can switch between models and continue the chat to see what else they can uncover. You can also selectively delete past messages to keep the model focused and less scatterbrain.

Chat Controls Control the inferencing parameters including intensity of processing and model streaming to allows you to multi-task with other applications better.

Token Entropy and Inspection Select the inspectors to peek into the inner-workings of each word outputted and see the model's confidence levels and alternative choices.

Prompt Framing Expanding the prompt section to utilise the framing feature which allows you to control the output the model generates.

Server If enabled, the server feature allows you to serve and connect to your own or trusted devices. No data is sent elsewhere. Also includes compatible APIs for application development.

Xcode Intelligence Use the server feature with Compatibility APIs enabled and SSL disabled to allow Xcode to use Inferencer as a service provider.

Shortcuts Use the Shortcuts app to automate inferencing workflows (e.g., copy text from clipboard > inference > speak result).

Settings Includes parental controls, an automatic deletion policy and more.

Privacy For maximum privacy, all AI processing happens offline and on your device, by default.