home/categories/data-engineering
category focus

Data Eng.

ETL pipelines and big data infrastructure.

1541 skillsall categories
sorting
stars
current ordering strategy
query
all entries
refine the visible subset
data-engineering
634

data-engineering-data-pipeline

You are a data pipeline architecture expert specializing in scalable, reliable, and cost-effective data pipelines for batch and streaming data processing.

rmyndharis
rmyndharis
data-ai
open
data-engineering
634

spark-optimization

Optimize Apache Spark jobs with partitioning, caching, shuffle optimization, and memory tuning. Use when improving Spark performance, debugging slow jobs, or scaling data processing pipelines.

rmyndharis
rmyndharis
data-ai
open
data-engineering
634

data-quality-frameworks

Implement data quality validation with Great Expectations, dbt tests, and data contracts. Use when building data quality pipelines, implementing validation rules, or establishing data contracts.

rmyndharis
rmyndharis
data-ai
open
data-engineering
634

database-architect

Expert database architect specializing in data layer design from scratch, technology selection, schema modeling, and scalable database architectures. Masters SQL/NoSQL/TimeSeries database selection, normalization strategies, migration planning, and performance-first design. Handles both greenfield architectures and re-architecture of existing systems. Use PROACTIVELY for database architecture, technology selection, or data modeling decisions.

rmyndharis
rmyndharis
data-ai
open
data-engineering
634

data-engineer

Build scalable data pipelines, modern data warehouses, and real-time streaming architectures. Implements Apache Spark, dbt, Airflow, and cloud-native data platforms. Use PROACTIVELY for data pipeline design, analytics infrastructure, or modern data stack implementation.

rmyndharis
rmyndharis
data-ai
open
data-engineering
634

airflow-dag-patterns

Build production Apache Airflow DAGs with best practices for operators, sensors, testing, and deployment. Use when creating data pipelines, orchestrating workflows, or scheduling batch jobs.

rmyndharis
rmyndharis
data-ai
open
data-engineering
634

temporal-python-pro

Master Temporal workflow orchestration with Python SDK. Implements durable workflows, saga patterns, and distributed transactions. Covers async/await, testing strategies, and production deployment. Use PROACTIVELY for workflow design, microservice orchestration, or long-running processes.

rmyndharis
rmyndharis
data-ai
open
data-engineering
634

scala-pro

Master enterprise-grade Scala development with functional programming, distributed systems, and big data processing. Expert in Apache Pekko, Akka, Spark, ZIO/Cats Effect, and reactive architectures. Use PROACTIVELY for Scala system design, performance optimization, or enterprise integration.

rmyndharis
rmyndharis
data-ai
open
data-engineering
634

tdd-orchestrator

Master TDD orchestrator specializing in red-green-refactor discipline, multi-agent workflow coordination, and comprehensive test-driven development practices. Enforces TDD best practices across teams with AI-assisted testing and modern frameworks. Use PROACTIVELY for TDD implementation and governance.

rmyndharis
rmyndharis
data-ai
open
data-engineering
634

database-admin

Expert database administrator specializing in modern cloud databases, automation, and reliability engineering. Masters AWS/Azure/GCP database services, Infrastructure as Code, high availability, disaster recovery, performance optimization, and compliance. Handles multi-cloud strategies, container databases, and cost optimization. Use PROACTIVELY for database architecture, operations, or reliability engineering.

rmyndharis
rmyndharis
data-ai
open
data-engineering
634

error-debugging-error-analysis

You are an expert error analysis specialist with deep expertise in debugging distributed systems, analyzing production incidents, and implementing comprehensive observability solutions.

rmyndharis
rmyndharis
data-ai
open
data-engineering
634

error-diagnostics-error-analysis

You are an expert error analysis specialist with deep expertise in debugging distributed systems, analyzing production incidents, and implementing comprehensive observability solutions.

rmyndharis
rmyndharis
data-ai
open
data-engineering
629

db-status

Show database schema migration status (alembic current, alembic heads, pending migrations) for Backend.AI components (manager, accountmgr, appproxy)

lablup
lablup
data-ai
open
data-engineering
607

migration-helper

Plan and execute Convex schema migrations safely, including adding fields, creating tables, and data transformations. Use when schema changes affect existing data.

waynesutton
waynesutton
data-ai
open
data-engineering
604

v3-deep-integration

Deep agentic-flow@alpha integration implementing ADR-001. Eliminates 10,000+ duplicate lines by building claude-flow as specialized extension rather than parallel implementation.

ruvnet
ruvnet
data-ai
open
data-engineering
604

v3-memory-unification

Unify 6+ memory systems into AgentDB with HNSW indexing for 150x-12,500x search improvements. Implements ADR-006 (Unified Memory Service) and ADR-009 (Hybrid Memory Backend).

ruvnet
ruvnet
data-ai
open
data-engineering
573

aws-dynamodb

AWS DynamoDB single-table design, GSI patterns, SDK v3 TypeScript/Python

alinaqi
alinaqi
data-ai
open
data-engineering
573

azure-cosmosdb

Azure Cosmos DB partition keys, consistency levels, change feed, SDK patterns

alinaqi
alinaqi
data-ai
open
data-engineering
571

systematic-review

Orchestrates a systematic review and meta-analysis workflow following PRISMA 2020 guidelines, from protocol development through multi-database search, screening, data extraction, and evidence synthesis. Use when conducting evidence-based reviews, meta-analyses, or scoping reviews. NOT for single-study analysis or narrative literature surveys.

beita6969
beita6969
data-ai
open
data-engineering
568

dataframely

Best practices for polars data processing with dataframely. Covers definitions of Schema and Collection, usage of .validate() and .filter(), type hints, and testing. Use when writing or modifying code involving dataframely or polars data frames.

Quantco
Quantco
data-ai
open
data-engineering
565

excel-weekly-dashboard

Designs refreshable Excel dashboards (Power Query + structured tables + validation + pivot reporting). Use when you need a repeatable weekly KPI workbook that updates from files with minimal manual work.

sundial-org
sundial-org
data-ai
open
Previous
Page 18 / 65
Next