Data, search & knowledge

Data pipelines & ETL

1,206 agents · most starred first

Data pipelines & ETL, page 6 of 13

dataops-llm

LLM-powered data operations engine for Excel/CSV files

datarobot-storage

Reusable storage access for DataRobot

dataset-preprocessing-agent

LLM-based automated dataset standardization and evaluation framework

datasetGPT

Generate textual and conversational datasets with LLMs.

datasette-enrichments-llm

Enrich data by prompting LLMs

datastorage

Dict-like object that can be saved in hdf5 or numpy format

datateer-upload-agent

An agent that can be installed inside a firewall or VPN and used to push data to Datateer

db-crawl-agents

Agentic core for DB crawling.

dbrhino-agent

Agent for dbrhino

dbt-coverage

One-stop-shop for docs and test coverage of dbt projects

devpro-s3-storages

Django Project to handle Static and Upload Files on s3 the right way

distributions-mariofaragalla

Gaussian and Binomial distributions

dj-inmemorystorage

A non-persistent in-memory data storage backend for Django.

dj-kaos-storages

A set of django media storage classes for different use cases such as private file storage on S3

dj-storage

Nice file storage support for Django

dj-storage-url

Configure Django storage from a URL

dj-webdav-storage

This application allows you easily save media and static files into webdav storage

dj3-cloudinary-storage

Django package that provides Cloudinary storages for both media and static files as well as management commands for removing unnecessary files.

django-akamai-storage

Django storage engine that connects to Akamai NetStorage.

django-aliyun-oss2-storage

A simple Django storage backend for aliyun oss2.

django-aliyun-oss2-storage-noproblem

A simple Django storage backend for aliyun oss2.

django-aliyun-storage

Django storage for Aliyun OSS Storage

django-almastorage

A simple Django app to use SwiftStack Storage

django-atc-profile-storage

ATC Profile storage app

django-aws-s3-storage

AWS S3 Bucket file storage system for Django applications

django-azure-storage

Django storage backends for Windows Azure blob storage.

django-azure-storage2

Django storage backends for Windows Azure blob storage.

django-backblazeb2-storage

Django BackBlaze B2 Storage

django-blob-storage
django-dropbox-storage-fork

A Dropbox Storage for your Django apps

dnastorage

dnastorage provides basic support for encoding and decoding files into and out of DNA with several convenient interfaces.

dobby-storage

A package for handling various storage options.

dolmen.storage

Dolmen Zope3 Grok Storage Annotation

dq-agent

Data quality CLI demo with deterministic checks, anomaly detection, and replayable artifacts

dq-summary-llm
dragen-results-utils

Collection of Python scripts for post-processing Illumina DRAGEN results output files.

dragenfly

Utilities for Dragen

dragoman-tool

An Optimized, RML-engine-agnostic Interpreter for Functional Mappings. It planns the optimized execution of FnO functions integrated in RML mapping rules, interprets and transforms the rules into function-free ones efficiently. Since Dragoman is engine-agnostic it can be adopted by any RML-compliant Knowledge Graph creation framework.

dragon-ml-toolbox

Complete pipelines and helper tools for data science and machine learning projects.

dragon-prep

Preprocessing scripts for the DRAGON benchmark

dragonfly-sync-lib

A library that facilitates transporting data around.

dsPyLib

A common python library for DS.

dsagent

Data service agent

dsw-storage

Library for managing DSW S3 storage

dynafed-storagestats

Dynafed Storage Stats Module

e2llm-medsynth

Synthetic medical record generator with realistic schema variance across locales

easyJsonStorage

........

ec-storage

ec-storage,it is used to replace the three copy storage strategy of HDFS, so as to save storage space

echoss-storage

echoss AI Bigdata Solution - Object Storage like S3 handler

edeposit.amqp.storage

Binary storage for the Edeposit project.

errbot-rethinkdb-storage

RethinkDB storage plugin for ErrBot

ert-storage

Storage service for ERT

eve-s3storage

python-eve S3 MediaStorage extension

ez-storage

mostly it is a dependency

ezllm

Enhance your data pipelines with the power of Large Language Models. EzLLM is a developer tool that opens the door to the LLM ecosystem. Automatic webscrapers, natural language data extraction, semantic search and as little as 3 lines of code

ezstorages

Easy storage handler

fairagro-middleware-api-client

The FAIRagro advanced middleware API client

fairagro-middleware-shared

The FAIRagro advanced middleware shared components

fake-data-agents

Python library for generating synthetic data with LLMs

fake-py-django-storage

Django storage for fake.py

fake-py-pathy-storage

Pathy storage for fake.py

fastuseragent

User-agent parsing library with Go backend

feature-suggestion-agent

An intelligent agent for suggesting features for machine learning datasets

featureforge-llm

Automated Feature Engineering Toolkit Based on Large Language Models

ff-storage

Fenixflow storage package with Pydantic ORM, temporal data management, and state-based schema sync

fieldspy

Well Logs Tool for Oil & Gas

file-storage

Universal tool for file storages

fileStorage3

storage handler for multiple services

filestorage

Save uploaded files in configurable stores

filestostorage

package for file storage

fillme

A lightweight library to generate dummy data for database using OpenAI

fillmydb

Fill your database with mocked instances.

fiuai-object-storage

一个支持阿里云OSS和MinIO的对象存储抽象包

flare-storage-adapters

FLARE storage adapter package

flash-storage

Multi-cloud storage manager with automatic fallback for Git LFS (Cloudinary + Google Drive)

flask-storage

Simple and easy file storages for Flask

floe-storage-aws-s3

AWS S3 object storage plugin for the floe data platform

floe-storage-minio

MinIO object storage plugin for the floe data platform

forager-server

A web-based data exploration system for rapid task definition.

fragua

Framework for modeling, orchestrating,and executing ETL pipelines in a controlled environment.

fragua-sets

Package with fragua sets with reutilizable functions for ETL process.

frogml-storag
fsd-app-ranking-storage
fsd-tools-storage
fstorage

Fstorage, Secure file storage, SYNC/ASYNC clients, easy to learn, fast to code.

ftw.dictstorage

Provides a layer for storing key / value paires.

fustor-agent

realtime data sync agent for various data sources

fw-storage

Unified storage interface.

ga-storage-manager

This is a (POC) simple json storage manager.

gallerist-azurestorage

Azure Storage file store for Gallerist

garage61api

Garage-61-Api is a simple Python wrapper around the Garage61 API.

gauragos-sample-pypi

desc

gc-data-storage

for managing data storage between local environments and Google Cloud Storage (GCS) bucketss.

gcloud-aio-storage

Python Client for Google Cloud Storage

gcloud-rest-storage

Python Client for Google Cloud Storage

gcloud-storage-emulator

A stub emulator for the Google Cloud Storage API

gcloud-storage-manager

A package to manage Google Cloud Storage

gcloud_storage_api

gCloud - The Cloud made in Italy Python API

gcmaverager

A lightweight parallel post process package for GCM output.

gcp-storage-client

Package for accessing into Cloud Storage in Google Cloud Platform.

More in Data, search & knowledge