AlexsJones/llmfit

This page introduces an external open-source repository. It is not an HDATF product.

What it does

llmfit checks a machine's CPU, RAM, GPU and VRAM and recommends which open-source LLMs and quantizations it can run. It offers a terminal UI, a web dashboard, a REST API, and local benchmarks of real tokens per second.

How it helps ATF

Could be a reference in AX consulting when checking which open-source LLMs a customer's on-site hardware can run.

License

MIT Permissive. Commercial use and changes are allowed if the copyright notice is kept.

More in this category

• maderix/ANE
  A research proof of concept that trains neural networks directly on Apple's Neural Engine through reverse-engineered private APIs, without CoreML training APIs, Metal or the GPU. The author states it is not a maintained framework.
• p-e-w/heretic
  A command-line tool that removes refusal behavior, called safety alignment, from transformer language models using directional ablation. An Optuna-based optimizer picks parameters that reduce refusals while keeping the model close to the original.
• karpathy/nanochat
  A minimal codebase for training and chatting with your own LLM on a single GPU node, covering tokenization, pretraining, finetuning, evaluation and inference. One depth setting determines the other hyperparameters.
• RunanywhereAI/runanywhere-sdks
  SDKs built on one shared C++ core that run LLMs, vision, speech, voice agents, embeddings, RAG and image generation locally on phones, browsers, desktops and servers. A capability registry routes each call to an engine on the device.
• jundot/omlx
  A local LLM inference server for Macs with Apple Silicon, managed from the macOS menu bar. It uses continuous batching and keeps KV cache in memory and on SSD so past context can be reused across requests.

Only repositories in the ranked Trendshift lists are included, and the lists are used only to find candidates. We do not copy their ranks. Descriptions, licenses and star counts come from each GitHub repository. The notes are our own reading. We have not tested these projects, and a place on a trending list does not prove quality.