Data, search & knowledge

Data pipelines & ETL

1,206 agents · most starred first

Data pipelines & ETL, page 11 of 13

tinydb-encrypted-jsonstorage

A TinyDB storage implementation that uses JSON and encryption.

tinydbstorage

Redis and S3 storages for TinyDB that refuse to overwrite each other.

tle-storage-service

Small aiohttp server application for TLE storage

topicgpt

A package for integrating LLMs like GPT-3.5 and GPT-4 into topic modelling

topicgpt-python

Official implementation of TopicGPT: A Prompt-based Topic Modeling Framework (NAACL'24)

torm-storage

TinyDB storage with orjson and msgpack

tracy-blob-storage-client

A unified async interface to upload and download blobs to/from Azure or AWS S3, switchable via env var.

tree-storage

Tree hash-storage files

trollmoves

Pytroll file utilities

true-storage

A boilerplate utility package

ts-agents

Time-series skills, tool contracts, and sandboxes for agentic workflows.

ts-storagetools

SQLite schema, Python classes and data management tools for working with time series from engineering test beds

tsagentkit

Minimalist time-series forecasting toolkit for coding agents

tvb-storage

A package which handles the storage of TVB data

tvrage

Python interface to the TVRage television information database.

ubuntuone-storageprotocol

The protocol implementation for the Magicicada filesync server (open source fork of the Ubuntu One filesync).

uitl-dragonfly-sdk

Health check execution SDK for 'Dragonfly, The Data Quality Monitoring Tool' from Upshot Ideas Technology Lab.

universal-storage

An util for python backend storage packaging.

use-storage

A package for storing key-value data with backup history.

user-analytics-gae-ml-llm-service
user-analytics-ml-llm-service
userstorage

Create storage for tests

ustorage

Unified Storage Interface for Python.

utf-storage-client-python

Storage Service

valer.storage

Sample storage add-on for VALER LIMS

vdk-storage

Library for access to different managed storages

vention-storage

A framework for storing and managing component and application data for machine apps.

vercel_storage

A wrapper around the vercel storage blob api

verse-storage

Verse Storage Components

vexstorage

Database storage for vexbot

video-fragment-ingest

Kafka-based ingest and decoding classes for VideoIngestFragments

views-storage

Storage driver used throughout views 3.

virtual-storage-manager

Virtual Storage Manager

vtr_storages

Storage collection, filesystem forked from django

w3storage

A Python client for https://web3.storage

wagtail-storages

Use AWS S3 with private documents in Wagtail.

watchmen-storage
watchmen-storage-adls
watchmen-storage-engine
watchmen-storage-mongodb
watchmen-storage-mssql
watchmen-storage-mysql
watchmen-storage-oracle
watchmen-storage-oss
watchmen-storage-postgresql
watchmen-storage-rds
watchmen-storage-s3
waylay-sdk-storage

Waylay Storage

waylay-sdk-storage-types

Waylay Storage Types

wcraas-storage

WCraaS Storage Service

wdg-file-storage

Support for s3 storage backends in Django

weighted-average

Smooth data across multiple dimensions using weighted averages

wellcome-storage-service

A client for the Wellcome Storage Service

wellmap

File format for 96-well plate layouts.

wix-protos-answers-streams-cdc-agent-cdc-agent

wix-protos-answers-streams-cdc-agent-cdc-agent

wix-protos-answers-streams-prod-agg-agent-agg-agent

wix-protos-answers-streams-prod-agg-agent-agg-agent

wix-protos-answers-streams-prod-cdc-agent-cdc-agent

wix-protos-answers-streams-prod-cdc-agent-cdc-agent

wix-protos-answers-streams-prod-dwh-agent-dwh-agent

wix-protos-answers-streams-prod-dwh-agent-dwh-agent

wix-protos-answers-streams-prod-view-agent-view-agent

wix-protos-answers-streams-prod-view-agent-view-agent

wix-protos-answers-streams-services-agent-aggregate-service

wix-protos-answers-streams-services-agent-aggregate-service

wm-topicgpt

This is a package to generate topics for the text corpus.

ws-storage

pytohn object storage

xl-storage

xilong storage library

yeonjae-universal-data-storage

Universal data storage module for persisting development data

ykenan-fragments

Forming fragments files through three files (barcodes.txt.gz, .mtx.gz and .peaks.txt)

zstorage

zen storage library

plottie

Plottie review: compare features, pricing, use cases, access model, and alternatives for this Data Science agent in 2026. Generate professional scientific figures for manuscripts, grants, and presentations. The accurate, n

TensorStax

AI that builds, fixes, and manages your data pipelines automatically.

@cuser/proto

typing, schemas and services based in protocol buffers

arrow-orm

API Builder ORM

objection-model-autogen

to generate bare bones models for objection

@axway/api-builder-orm

API Builder ORM

@terabits/grapi

Grapi Schema Generator For GraphQL Server

@danroshko/influxdb-node

Minimal InfluxDB client library for Node.js

@5to3/influx-firehose-forwarder

A simple forwarder that acts as an InfluxDB subscriber, forwarding each write to an AWS Firehose stream.

Adala

Adala lets AI agents learn to label and process data on their own.

agents-schema

Populate an in-warehouse agents.* schema from dbt manifests, OSI YAML, LookML, and markdown skills.

pystac-ext-storage

Storage extension for PySTAC

etld-sdk

ETL-D SDK: The Agentic Data & Finance Middleware

@ruvector/agentic-synth-examples

Production-ready examples for @ruvector/agentic-synth - DSPy training, multi-model benchmarking, and advanced synthetic data generation patterns

agentic-dataset-builder

Pure TypeScript agentic dataset builder for Pi, Codex, and Claude Code history

aillmcleaner

An AI-powered Python library for context-aware data cleaning using local LLMs

aind-dynamic-foraging-database

Query (and build) the AIND dynamic-foraging behavioral parquet database.

wattnet-storage

Extensible plugin-based client for unified wattnet time-series metric storage across multiple backends.

odoo-addon-fs-storage-backup-environment

Allows to use server environment with fs storage attachment

odoo-addon-fs-storage-environment

Allows to use server environment with fs storage

odoo-addon-storage-file-swap-backend-queue

Delegate storage file backend swap to queue jobs

etiket-sync-agent

Sync agent for managing eTiKeT synchronization from various data sources

etiket-sync-agent-folderbase

Folder-based connector for eTiKeT sync agent

etiket-sync-agent-native

Native filesystem connector for eTiKeT sync agent

etiket-sync-agent-qcodes

QCoDeS connector for eTiKeT sync agent

etiket-sync-agent-qh-converters

File converters for eTiKeT sync agent

qualitative-response-llm-analyst

LLM-assisted qualitative survey coding with auditable Excel workbooks

datagallery-dataagent

DataAgent - A powerful multi-modal Data Agent workflow template framework

fetchkit-agents

A YAML-configured data fetching library for agentic applications.

langchain-sibfly

An integration package connecting SibFly (measured ground motion per US address, from NASA InSAR) and LangChain

s-storagekit

ПЕРЕХОДНИК на s-persistkit[object] (волна 4). Объектное хранилище (ObjectStorage protocol + MinIO adapter + image normalization for Skillery)

crewlyze

Autonomous Multi-Agent Business Intelligence and Data Engineering Platform

pookie-storage

A Django storage field that works everywhere. One import. Any backend. Zero plumbing.

@inflexa-ai/inflexa

Local-first AI agent for reproducible biological data analysis

More in Data, search & knowledge