sgl-project/sglang

This page introduces an external open-source repository. It is not an HDATF product.

What it does

A serving framework for large language models and multimodal models. It serves open models such as DeepSeek, Qwen, Llama and GLM, and also covers diffusion models for image and video generation.

How it helps ATF

Could be a reference when an AX consulting site wants open models served on its own servers. Worth comparing with other serving engines for a self-hosted model endpoint that Harness executors could call.

License

Apache-2.0 Permissive, with a patent grant. Commercial use is allowed; keep the notices and state your changes.

More in this category

• jingyaogong/minimind
  An educational project that implements a small language model of about 64M parameters from scratch in native PyTorch. Its code covers the full training chain, such as pretraining, SFT, LoRA, RLHF, tool use and distillation.
• FlashML-org/FreeToken
  A serving engine for running large open-weight Mixture-of-Experts models on personal hardware by splitting work across GPUs, CPUs and memory. It exposes Anthropic- and OpenAI-compatible APIs so coding agents can connect.
• experientiallabs/experiential
  An open source gateway that puts hosted, BYOK and local models behind one OpenAI-compatible API. It sets per-user and per-agent model access and spending limits, and can turn production traffic into a custom router or model.
• cactus-compute/needle
  Needle 2 is an open 45M-parameter model for tool calling, device use and structured extraction, packed into a single 14MB engine. The Python package covers inference, LoRA fine-tuning and export, returning JSON tool calls with a confidence score.
• bojieli/ai-infra-book
  An open-source book, written in Chinese, that derives LLM inference and training system design from hardware constraints and model architecture using quantitative estimates. The repository includes the full text, a PDF and companion calculation tools and experiments.

Only repositories in the ranked Trendshift lists are included, and the lists are used only to find candidates. We do not copy their ranks. Descriptions, licenses and star counts come from each GitHub repository. The notes are our own reading. We have not tested these projects, and a place on a trending list does not prove quality.