Data pipelines & ETL
1,206 agents · most starred first
Data pipelines & ETL, page 4 of 13
Chicago Divvy bike-share availability on Google Cloud: a Cloud Run pipeline collects live data every 15 minutes into BigQuery, dbt models and checks it hourly, and a hosted AI agent (Strands + MCP) answers plain-English questions from read-only data tools. Complete code private, only shareable part published in this repo
Agentic Dataset Engineering Workstation
streamlit region:us
docker region:us
docker region:us
DataQuality&Anomaly Detection Agent review: compare features, pricing, use cases, access model, and alternatives for this Data Science agent in 2026. Ensure Data Quality, Spot Anomalies, and Validate Readiness — All in One Click
Magick review: compare features, pricing, use cases, access model, and alternatives for this Coding Assistant agent in 2026. Visual AI development environment for no-code data pipelines.
TensorStax review: compare features, pricing, use cases, access model, and alternatives for this Data Science agent in 2026. Autonomous AI Agents for Data Engineering
Automate critical data workflows with AI agents, no coding needed.
Automate complex industrial tasks using AI that truly understands your factory.
Get a single, clear view of all your customer data in real time.
Animated data-flow diagrams your AI agent can write. CLI.
AI Agent templates - Skills, Agents, and Workflows for Data Teams
IP2Location geolocation component
High-performance synthetic data generator for AI/ML training, RAG systems, and agentic workflows with DSPy.ts, Gemini, OpenRouter, and vector databases
Complete data engineering plugin with Airflow, Kedro, LangGraph, Kafka, dbt, pandas experts, data quality rules, and example scripts for ClaudeAutoPM
Read and write files on Google Cloud Storage and Amazon S3 as if they are on local computer.
Makes sure, that there are never more files in one directory, than your LIMIT.
This code provides a new storage engine for DURUS, an excellent persistence system for Python
An optimized tinyDB storage extension
PyPI Package for Circles Storage functions
Coherence - DLNA/UPnP framework for the digital living
AI-powered synthetic data generation pipeline with web search, topic extraction, and persistent state management
Awesome and easy to use database
A library used to fetch data from deltalake tables locally.
A ZODB Storage which uses one file per revision of an object
User-Space write-once Network Filesystem
Api unofficial de dragon ball inspirado de la fandom de dragon ball
Flask extension that provides integration with Azure Storage
Google Cloud Storage for Flask
Qiniu Storage for Flask
Qiniu object storage for flask
SAE Storage for Flask
Various file storage backends for Flask apps.
A Python wrapper for Fragstats.
full matrix coordinates
A python package to find the best features in a dataset using Geometric Particle Swarm Optimisation
--/--
Biblioteca para gerenciamento de diretórios e arquivos
A simple storage interface with multiple backends for use in a Keg app.
This Python package creates a file locally in the user's device more easily.
A Mouritech package
A simple LAN storage.
Akamai's NetStorage API communication kit
wrapper for the OpenAI to label survey results
Widgets to load and save tables to/from Azure Data Lake Storage (ADLS) Gen 2
Unleash the Potential of Datasets with Intelligent Tasks, Tutorials, and Algorithm Recommendations.
An add-on Plone product which provides an extra storage for Archetypes.
TIDStorage Product provides a way to have consistent backups when running a multi-storage instance (only ZEO is supported at the moment).
A Python client for Active Storage
This repository contains Python functions for extracting gene features from a GFF3 file and performing keyword analysis using TF-IDF (Term Frequency-Inverse Document Frequency).
A backend for ZODB that stores pickles in a relational database.
Synthetic data generator for LLM evaluation
Use sqlalchemy to commit/rollback changes to files on the filesystem or any other external storage.
A scientific computation library for performance analysis of erasure coding and/or replication in a distributed data storage environment.
MetricDB is a database for storing and querying time based metrics data.
UpStorage Api Client.
Data handler with txt,and jsons
xflow xcom storage
This service implements storage of abc project
Core utilities for managing blob storage operations with support for various cloud providers and local storage systems
ADRI Storage Python SDK
ae namespace module portion cloud_storage: distribute files to and retrieve them from cloud storage hosts
Agent-as-Annotators: Structured Distillation of Web Agent Capabilities
Unified multi-source content CLI for discovery, sync, local query, and explicit interaction.
Connect agents to data
Agent 训练数据的校验、转换与批处理工具。
Config-driven database component for agent pipelines (PostgreSQL + SQLAlchemy + Pydantic + fsspec).
Universal data comprehension for AI agents
YAML-first, domain-driven data governance for AI agents
Agentic Data Engineering tools and protocols (MCP, stdio, sse, FastAPI, etc)
HDF5 tools and semantic metadata for agentic workflows
Unified CLI for reproducible and auditable Agentic SWMM workflows.
Intelligent ML preprocessing pipeline tool with AI agents
Airbyte Airtable Connector for AI platforms
Airbyte Amplitude Connector for AI platforms
Airbyte Github Connector for AI platforms
Airbyte Gitlab Connector for AI platforms
Airbyte Google-Drive Connector for AI platforms
Source implementation for Azure Blob Storage.
Source implementation for Zapier Supported Storage.
Inject connections into the airflow database from configuration
Airless package to work with google cloud storage
Storage based off of sqlalchemy_marshmallow
CLI to check with alembic migrations are in sync with your SQLAlchemy models.
Alembic hook that allows enums values to be upgraded and downgraded in migrations automatically
Alembic autogenerate extension for PostgreSQL-specific objects
Tool for accessing blob storage
A trait to implement a standard mongodb server which logs events and also caches some data related to them
A lightweight server to work as a simple storage for games, done with MongoDB and Flask
A generator of production-ready HTTP storage stacks for WindRose/NetRose games
Alliance Platform Storage
Automatic topic discovery with zero LLMs, minimal input
secure, decentralized, fault-tolerant filesystem
A non-persistent in-memory data storage backend for Django.
amsdal_storages plugin for AMSDAL Framework
A custom Apache Airflow plugin to integrate a React UI
Apiqa user storage backend for django projects