pathwaycom/pathway

This page introduces an external open-source repository. It is not an HDATF product.

What it does

A Python ETL framework for batch and stream processing, real-time analytics, LLM pipelines and RAG. Python code runs on a Rust engine that computes incrementally, and the same code serves local development, batch jobs and streaming.

How it helps ATF

Could be a reference for AX consulting when records from systems such as MES or ERP need to flow into analysis or search as they change. Its incremental processing is also worth comparing for keeping Company Brain current.

License

Custom license Custom or non-standard license. Read the license file before any reuse.

More in this category

• koala73/worldmonitor
  A real-time dashboard that gathers curated news feeds into AI-written briefs and tracks geopolitical, infrastructure and market signals, with 3D globe and flat map views. Its AI can run locally with Ollama.
• D4Vinci/Scrapling
  Scrapling is a Python web scraping framework whose parser relocates elements when pages change. It offers fetchers that bypass anti-bot systems and a spider framework for concurrent crawls with pause, resume and proxy rotation.
• pinchtab/pinchtab
  A standalone HTTP server in Go that gives AI agents control over Chrome, with headless browser instances, a dashboard, an HTTP API and an MCP server. It is designed first for local, single-user use.
• lightpanda-io/browser
  Lightpanda is a headless browser written from scratch in Zig for AI agents and automation. It is not a fork of Chromium or WebKit.
• calesthio/Crucix
  A locally run intelligence dashboard that pulls 27 open-source intelligence feeds, such as flight tracking, radiation monitoring, conflict data and market prices, every 15 minutes. With an LLM connected, it sends alerts to Telegram and Discord and answers commands.

Only repositories in the ranked Trendshift lists are included, and the lists are used only to find candidates. We do not copy their ranks. Descriptions, licenses and star counts come from each GitHub repository. The notes are our own reading. We have not tested these projects, and a place on a trending list does not prove quality.