microsoft/BitNet

This page introduces an external open-source repository. It is not an HDATF product.

What it does

bitnet.cpp is the official inference framework for 1-bit LLMs, with optimized kernels for CPUs and GPUs. The project has also released 1-bit embedding models and a guide for converting and running them.

How it helps ATF

Worth comparing for AX consulting when a customer site needs language or embedding models to run on CPU servers it already has. The 1-bit embedding models could also be a reference for Company Brain search on modest hardware.

License

MIT Permissive. Commercial use and changes are allowed if the copyright notice is kept.

More in this category

• RunanywhereAI/runanywhere-sdks
  SDKs built on one shared C++ core that run LLMs, vision, speech, voice agents, embeddings, RAG and image generation locally on phones, browsers, desktops and servers. A capability registry routes each call to an engine on the device.
• Lordog/dive-into-llms
  A free series of Chinese hands-on LLM programming tutorials from Shanghai Jiao Tong University courses, with slides, guides and notebooks on fine-tuning and deployment, prompting and chain of thought, knowledge editing, math reasoning, GUI agents and alignment.
• p-e-w/heretic
  A command-line tool that removes refusal behavior, called safety alignment, from transformer language models using directional ablation. An Optuna-based optimizer picks parameters that reduce refusals while keeping the model close to the original.
• deepseek-ai/Engram
  Engram is the official code for a paper on conditional memory in large language models. Its module modernizes N-gram embeddings for constant-time lookup and fuses the retrieved static memory with hidden states, complementing Mixture-of-Experts.
• AlexsJones/llmfit
  llmfit checks a machine's CPU, RAM, GPU and VRAM and recommends which open-source LLMs and quantizations it can run. It offers a terminal UI, a web dashboard, a REST API, and local benchmarks of real tokens per second.

Only repositories in the ranked Trendshift lists are included, and the lists are used only to find candidates. We do not copy their ranks. Descriptions, licenses and star counts come from each GitHub repository. The notes are our own reading. We have not tested these projects, and a place on a trending list does not prove quality.