snchimata/tokenfold

This page introduces an external open-source repository. It is not an HDATF product.

What it does

Tokenfold provides SDKs and tools for fitting material into an LLM token budget. Its users are developers supplying JSON, tool definitions and grouped context in requests. Text, context groups or provider requests produce fitted content and a receipt describing transforms, token accounting and warnings. The Rust-based Core reduces structural redundancy and checks reversible transforms by restoring the original values. Data defaults to compact ordinary JSON, while columnar representation is explicit opt-in. Select ranks whole groups for a query but is experimental and needs separately provisioned model files, runtime configuration and per-request consent. Python requires version 3.12 or later and Node requires 22 or later; base fitting needs no model or GPU. MCP and proxy interfaces are separate entry points, so installation alone does not intercept existing agent requests. Checked result access raises an error when the budget cannot be satisfied, and counts do not guarantee exact provider billing. Exact data recovery does not guarantee unchanged model answer accuracy.

License

Apache-2.0 Permissive, with a patent grant. Commercial use is allowed; keep the notices and state your changes.

More in this category

• Tencent-Hunyuan/Hy-Memory
  Hy Memory is a Python memory SDK and engine for agents that need to retain conversations and work traces across tasks. It accepts text, messages and tool-call traces, and returns memories organized for chat, productivity work or coding. Chat uses hierarchical memory, working separates session facts from reusable experiences, and coding adds version-change context to that two-layer structure. The documentation describes storing real diffs from Git or recognizable replacement actions, but the chain is best effort and its implementation was not independently inspected here. Starting requires configuring model and embedding endpoints and explicitly wiring memory writes and searches into the caller; installation alone does not capture application activity. Working and coding behavior requires pro or ultra mode and tool-call evidence, while lite ignores the scene and stores raw memory. The same repository also contains MemHub, a separate local-first desktop and CLI product with independent permissions for reading sources, model processing and host access; its documented release targets are Linux and macOS, not Windows. SDK installation, MemHub runtime installation and connecting authorized applications are different steps, and neither local-first operation nor publishing model weights guarantees that all processing remains offline.
• qf-studio/navigator
  A Claude Code plugin that loads documentation as needed rather than all at once. It also describes features for remembering user preferences across sessions and detecting repeated corrections in collaboration.
• facebookresearch/context-language-models
  The official repository for the Context Language Models paper, which describes language models that manage their own context by treating the context as a file the model may update without restriction. The authors report that building such models zero shot beats existing context management strategies across several tasks, including higher accuracy with fewer FLOPs on BrowseComp-Plus and on EdgeBench, and that the approach extends to multi agent settings where several agent contexts coexist as files.
• tigerless-labs/agent-memory
  A long-term memory runtime for AI agents. Plain Markdown files are the source of truth, a SQLite index is a deletable cache, local ranked retrieval returns file paths, and Claude Code and Codex share one store.
• kunchenguid/backpass
  A tool that reads local agent session transcripts from seven harnesses and proposes evidence-backed edits to AGENTS.md, CLAUDE.md and project skills under a token budget. Analysis never writes, and a person accepts or rejects each edit.

Only repositories in the ranked Trendshift lists are included, and the lists are used only to find candidates. We do not copy their ranks. Descriptions, licenses and star counts come from each GitHub repository. The notes are our own reading. We have not tested these projects, and a place on a trending list does not prove quality.