Data Engineering

Use to run structured opportunity inspections that align pipeline data with buyer reality.

Unity Editor control and automation, WebSocket-based real-time communication. 유니티에디터제어및자동화, WebSocket기반실시간통신. Features/기능: GameObject control 게임오브젝트제어, Transform manipulation 트랜스폼조작, Component management 컴포넌트관리, Scene management 씬관리, SQLite database integration SQLite데이터베이스통합, GUID-based persistence GUID기반영구식별, Multi-scene synchronization 멀티씬동기화, Command Pattern with Undo/Redo 명령패턴실행취소재실행, Menu execution 메뉴실행, ScriptableObject management 스크립터블오브젝트관리, Array/List manipulation 배열리스트조작, All field types support 모든필드타입지원, Material/Rendering 머티리얼/렌더링, Prefab system 프리팹시스템, Asset Database 애셋데이터베이스, Animation 애니메이션, Physics 물리, Console logging 콘솔로깅, EditorPrefs management 에디터프리퍼런스관리, Editor automation 에디터자동화, Build pipeline 빌드파이프라인, Lighting 라이팅, Camera 카메라, Audio 오디오, Navigation 네비게이션, Particles 파티클, Timeline 타임라인, UI Toolkit, Profiler 프로파일러, Test Runner 테스트러너. Protocol 프로토콜: JSON-RPC 2.0 over WebSocket (port 9500-9600). 500+ commands 명령어, 25 categories 카테고리. Real-time bidirectional communication 실시간양방향통신. Security 보안: Defense-in-depth 심층방어 (path traversal protection 경로순회방지, command injection defense 명령어인젝션방어, JSON injection prevention JSON인젝션방지, SQL injection prevention SQL인젝션방지, transaction safety 트랜잭션안전성). Localhost-only connections 로컬호스트전용. Cross-platform 크로스플랫폼 (Windows, macOS, Linux).

Build end-to-end ML pipelines with automated data processing, training, validation, and deployment using Airflow, Kubeflow, and Jenkins

Build Jenkins declarative and scripted pipelines with stages, agents, parameters, and plugins. Implement multi-branch pipelines and deployment automation.

Expert guidance for Dagster data orchestration including assets, resources, schedules, sensors, partitions, testing, and ETL patterns. Use when building or extending Dagster projects, writing assets, configuring automation, or integrating with dbt/dlt/Sling.

Build robust processes for data cleaning, missing value imputation, outlier handling, and data transformation for data preprocessing, data quality, and data pipeline automation

Build integrations between multiple CRM systems with data mapping and sync logic. Use when syncing contacts between Salesforce and HubSpot, building bidirectional data flows, implementing CRM migration pipelines, or managing multi-CRM environments.

GPU memory management for CuPy. Use when optimizing CPU/GPU data transfers, avoiding out-of-memory errors, implementing memory-efficient pipelines, or managing GPU memory pools.

Fix common mesh defects including holes, non-manifold edges, degenerate triangles, and self-intersections. Use for preparing models for 3D printing, simulation, and rendering. Essential for mesh cleanup pipelines.

Work with message queues for async data processing. Use when implementing pub/sub patterns, processing data asynchronously, or building distributed data pipelines. NOT needed for simple batch file ETL.

Connect to and query SQL databases, execute queries, and handle database transactions. Use when reading from or writing to databases, performing batch inserts, managing connections, or executing complex SQL queries in data pipelines.

Process real-time data streams and implement streaming pipelines. Use when handling continuous data feeds, implementing event-driven processing, or building real-time analytics pipelines. NOT needed for batch file processing.

Use BWA for aligning sequencing reads to a reference genome for variant calling pipelines. Use when mapping short-read Illumina data to reference genomes for variant calling pipelines.

Handle errors and implement logging in data pipelines. Use when implementing retry logic, logging sync operations, tracking failed records, or building fault-tolerant data processing workflows.

Clean, normalize, and validate scraped data for ETL pipelines. Use this skill when removing duplicates, handling missing values, normalizing text and dates, validating data formats, or preparing raw scraped data for storage or analysis.

Deploying and managing Snowflake database objects using version control with schemachange. Use this skill when you need to manage database migrations for objects not handled by dbt, implement CI/CD pipelines for schema changes, or coordinate deployments across multiple environments.

Configuring Snowflake connections using connections.toml (for Snowflake CLI, Streamlit, Snowpark) or profiles.yml (for dbt) with multiple authentication methods (SSO, key pair, username/password, OAuth), managing multiple environments, and overriding settings with environment variables. Use this skill when setting up Snowflake CLI, Streamlit apps, dbt, or any tool requiring Snowflake authentication and connection management.

dbt testing strategies using dbt_constraints for database-level enforcement, generic tests, and singular tests. Use this skill when implementing data quality checks, adding primary/foreign key constraints, creating custom tests, or establishing comprehensive testing frameworks across bronze/silver/gold layers.

Writing dbt models with proper CTE patterns, SQL structure, and layer-specific templates. Use this skill when writing or refactoring dbt models, implementing CTE patterns, creating staging/intermediate/mart models, or ensuring proper SQL structure and dependencies.

Monitor dbt execution using the dbt Artifacts package. Use this skill when you need to track test and model execution history, analyze run patterns over time, monitor data quality metrics, or enable programmatic access to dbt execution metadata across any dbt version or platform.