home/categories/data-engineering
category focus

Data Eng.

ETL pipelines and big data infrastructure.

1541টি স্কিলall categories
sorting
stars
current ordering strategy
query
all entries
refine the visible subset
data-engineering
0

oni-calculator

Oxygen Not Included production chain calculator with SQLite database extracted from decompiled game source

lawless-m
lawless-m
data-ai
open
data-engineering
0

airbyte-connection-setup

Эксперт Airbyte. Используй для настройки ETL/ELT пайплайнов, коннекторов, синхронизации данных и data pipelines.

dengineproblem
dengineproblem
data-ai
open
data-engineering
0

dbt-transformations

ALWAYS USE when working with dbt models, SQL transformations, tests, snapshots, or macros. Use IMMEDIATELY when editing dbt_project.yml, profiles.yml, or creating SQL models. MUST be loaded before any transform-layer work. Enforces dbt owns SQL principle - never parse, validate, or transform SQL in Python.

Obsidian-Owl
Obsidian-Owl
data-ai
open
data-engineering
0

duckdb-lakehouse

ALWAYS USE when building data lakehouse with DuckDB compute, configuring dbt-duckdb with Polaris plugin, or designing catalog-first architecture in floe-platform. Use IMMEDIATELY when reading/writing Iceberg tables via Polaris catalog, creating Dagster assets with DuckDB, or connecting to REST catalogs with inline credentials. Provides research steps for DuckDB + Dagster + Iceberg/Polaris integration patterns.

Obsidian-Owl
Obsidian-Owl
data-ai
open
data-engineering
0

debug-parse

Isolate and test parser behavior on specific text snippets to debug pattern matching, validate regex patterns against edge cases, understand which extraction rules triggered, and test parser changes before full deployment without running the complete pipeline or database commit. Use this skill when: (1) Debugging why parser misinterpreted a specific line or exercise description, (2) Testing new regex patterns against edge cases before adding to parser, (3) Validating parser changes on isolated examples without full workflow, (4) Understanding which parsing rule triggered for specific input text, or (5) Developing and testing new extraction patterns in isolation

zohar-ui
zohar-ui
data-ai
open
data-engineering
0

senior-data-engineer

World-class data engineering skill for building scalable data pipelines, ETL/ELT systems, and data infrastructure. Expertise in Python, SQL, Spark, Airflow, dbt, Kafka, and modern data stack. Includes data modeling, pipeline orchestration, data quality, and DataOps. Use when designing data architectures, building data pipelines, optimizing data workflows, or implementing data governance.

nimeshgurung
nimeshgurung
data-ai
open
data-engineering
0

data-expert

Performs data analysis and engineering tasks with a senior-level perspective, focusing on data quality, migration pipelines, SQL optimization, and business insights. Triggers when tasks involve database migrations, ETL, data validation, or analytical queries.

cesaramirez
cesaramirez
data-ai
open
data-engineering
0

pm-07-conformance

Evaluate conformance of the event log to discovered models and generate deviation artefacts.

Wattysaid
Wattysaid
data-ai
open
data-engineering
0

elt-modeling

Comprehensive guide to ELT (Extract, Load, Transform) modeling patterns, dimensional modeling, fact and dimension tables, and data warehouse design

AmnadTaowsoam
AmnadTaowsoam
data-ai
open
data-engineering
0

fof-preflight

Diff-aware guardrail checker for Fear-of-Falling (FOF) changes; fails closed on raw data edits, Kxx intro/req_cols mismatches, and output discipline risks.

Tupatuko2023
Tupatuko2023
data-ai
open
data-engineering
0

database-changes

Make database schema changes in IdeaForge. Triggers: create migration, add table/column, modify column type, add index, use JSONB, use pgvector. File-based migrations with raw SQL, no ORM.

Holo00
Holo00
data-ai
open
data-engineering
0

atft-pipeline

Manage J-Quants ingestion, feature graph generation, and cache hygiene for the ATFT-GAT-FAN dataset pipeline.

wer-inc
wer-inc
data-ai
open
data-engineering
0

dc-cube-definition

Create and configure Drizzle Cube semantic layer cube definitions with proper security context, measures, dimensions, and joins.

cliftonc
cliftonc
data-ai
open
data-engineering
0

sync-check

원본과 생성 파일의 동기화 검증이 필요할 때

younwony
younwony
data-ai
open
data-engineering
0

reading-plan-designer

Design and implement Bible reading plans for the KR92 Bible Voice project. Use when: - Creating new reading plans (7-day, 30-day, yearly) - Adding daily readings to existing plans - Generating reading plan SQL migrations - Understanding the reading plan data model - Designing reading sequences (chronological, topical, book-based) - Validating reading reference formats Triggers: "reading plan", "lukusuunnitelma", "daily readings", "create plan", "add readings"

Spectaculous-Code
Spectaculous-Code
data-ai
open
data-engineering
0

kafka-streaming

Comprehensive guide to Apache Kafka for real-time data streaming including topics, producers, consumers, stream processing, and production best practices

AmnadTaowsoam
AmnadTaowsoam
data-ai
open
data-engineering
0

database-manager

Comprehensive database management workflow that orchestrates database architecture, schema design, performance optimization, and data governance. Handles everything from database design and implementation to performance tuning, backup strategies, and data migration.

ajianaz
ajianaz
data-ai
open
data-engineering
0

dagster-orchestration

ALWAYS USE when working with Dagster assets, resources, IO managers, schedules, sensors, or dbt integration. CRITICAL for: @asset decorators, @dbt_assets, DbtCliResource, ConfigurableResource, IO managers, partitions. Enforces CATALOG-AS-CONTROL-PLANE architecture - ALL Iceberg writes via catalog (Polaris/Glue). Provides pluggable orchestration patterns abstractable to Airflow/Prefect. Compute abstraction: DuckDB (default), Spark, Snowflake - all via dbt.

Obsidian-Owl
Obsidian-Owl
data-ai
open
data-engineering
0

dbt-model-builder

Create dbt models following FF Analytics Kimball patterns and 2×2 stat model. This skill should be used when creating staging models, core facts/dimensions, or analytical marts. Guides through model creation with proper grain, tests, External Parquet configuration, and per-model YAML documentation using dbt 1.10+ syntax.

zazu-22
zazu-22
data-ai
open
data-engineering
0

supabase-seeding

Database seeding toolkit for Supabase projects. Use when: (1) Creating seed data files, (2) Populating lookup/reference tables, (3) Generating test data, (4) Bulk loading data with COPY, (5) Running seed files against database, (6) Managing large seed files with DVC

ninyawee
ninyawee
data-ai
open
data-engineering
0

data-pipeline-patterns

Follow these patterns when implementing data pipelines, ETL, data ingestion, or data validation in OptAIC. Use for point-in-time (PIT) correctness, Arrow schemas, quality checks, and Prefect orchestration.

colingwuyu
colingwuyu
data-ai
open
data-engineering
0

matrix-filtering-and-deduplication

Reduce matrix builds from 47 jobs to 3 with path filtering, deduplication, and dynamic generation. Run only what changed and eliminate redundant combinations.

adaptive-enforcement-lab
adaptive-enforcement-lab
data-ai
open
Previous
Page 61 / 65
Next