LLM Connect

dev.io

About this app

LLM Connect is a high-performance, developer-focused local LLM interface that lets you run artificial intelligence models entirely offline on your personal device. With total privacy and zero internet dependency, LLM Connect puts complete control of on-device machine learning in your hands.

Key Features

100% Offline & Private: Your prompts, context, and chat histories stay strictly on your local device. No cloud servers, no data tracking, and no external API keys required.

Multi-Format Model Execution: Natively load and run popular model weights, including .gguf and .litertlm formats (supports Llama, Qwen, Gemma, Phi, and more).

Hardware Acceleration Control: Toggle seamlessly between CPU, GPU, and NPU execution modes to maximize performance and efficiency based on your device hardware.

Granular Inference Parameters: Fine-tune generation settings on the fly:

Temperature, Top-P, and Top-K sliders

Max Tokens and custom/dynamic seed management

One-tap prompt templates tailored for Code/Math, General Conversation, and Creative generation

Live System & Generation Telemetry: Monitor real-time performance metrics directly from the terminal header, including RAM/VRAM consumption, device temperature, generation duration, and token counts.

Local Storage Integration: Effortlessly scan and select model files stored in your internal storage directories.

Power-User Terminal Interface: Designed with a sleek, minimalist CLI aesthetic for smooth readability and fast interaction.

Important Information

Bring Your Own Model (BYOM): LLM Connect is an execution environment and does not come bundled with pre-packaged AI models. Users must supply their own lawfully acquired model files.

System Requirements: Running local language models requires adequate available device memory (RAM) and processor capacity depending on the size of the loaded model.

What’s new

New release | Updates on the way to fix bugs.