fake-driven-testing
This skill should be used when writing tests, fixing bugs, adding features, or modifying the gateway layer. Use when you need guidance on testing architecture, working with fakes, implementing ABC gateway interfaces, or understanding the defense-in-depth testing strategy. Essential for maintaining test quality and understanding where different types of tests belong.
$ Instalar
git clone https://github.com/dagster-io/erk /tmp/erk && cp -r /tmp/erk/.claude/skills/fake-driven-testing ~/.claude/skills/erk// tip: Run this command in your terminal to install the skill
name: fake-driven-testing description: This skill should be used when writing tests, fixing bugs, adding features, or modifying the gateway layer. Use when you need guidance on testing architecture, working with fakes, implementing ABC gateway interfaces, or understanding the defense-in-depth testing strategy. Essential for maintaining test quality and understanding where different types of tests belong.
Fake-Driven Testing Architecture for Python
Use this skill when: Writing tests, fixing bugs, adding features, or modifying gateway layers in Python projects.
Prerequisites: For Python code standards, load the dignified-python-313 skill first. This skill focuses on testing architecture, not Python syntax.
Overview
This skill provides a defense-in-depth testing strategy with five layers for Python applications:
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Layer 5: Business Logic Integration Tests (5%) โ โ Smoke tests over real system
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ Layer 4: Business Logic Tests (70%) โ โ Tests over fakes (MOST TESTS)
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ Layer 3: Pure Unit Tests (10%) โ โ Zero dependencies, isolated testing
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ Layer 2: Integration Sanity Tests (10%)โ โ Fast validation with mocking
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ Layer 1: Fake Infrastructure Tests (5%)โ โ Verify test doubles work
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
Philosophy: Test business logic extensively over fast in-memory fakes. Use real implementations sparingly for integration validation.
Terminology note: The "gateway layer" (also called adapters/providers) refers to thin wrappers around heavyweight external APIs (databases, filesystems, HTTP APIs, message queues, etc.). The pattern matters more than the name.
Quick Decision: What Should I Read?
Adding a feature or fixing a bug?
โ Read quick-reference.md first, then workflows.md#adding-a-new-feature
Need to understand where to put a test?
โ Read testing-strategy.md
Working with Python-specific patterns?
โ Read python-specific.md
Adding/changing a gateway interface?
โ Read gateway-architecture.md, then workflows.md#adding-a-gateway-method
Creating a backend (higher-level abstraction over gateways)?
โ Read gateway-architecture.md#gateways-vs-backends - backends compose gateways and do NOT have fakes
Need to implement a specific pattern (CliRunner, builders, etc.)?
โ Read patterns.md
Not sure if I'm doing it right?
โ Read anti-patterns.md
Just need a quick lookup?
โ Read quick-reference.md
When to Read Each Reference Document
๐ gateway-architecture.md
Read when:
- Adding or changing gateway/ABC interfaces
- Understanding the ABC/Real/Fake/DryRun pattern
- Need examples of gateway implementations
- Want to understand what gateways are (and why they're thin)
- Creating a backend (higher-level abstraction that composes gateways)
Contents:
- What are gateway classes? (naming: gateways/adapters/providers)
- The four implementations (ABC, Real, Fake, DryRun)
- Code examples for each
- When to add/change gateway methods
- Design principles (keep gateways thin)
- Common gateway types (Database, API, FileSystem, MessageQueue)
- Gateways vs Backends - critical distinction for DI boundaries
๐ testing-strategy.md
Read when:
- Deciding where to put a test
- Understanding the five testing layers
- Need test distribution guidance (5/70/10/10/5 rule)
- Want to know which layer tests what
Contents:
- Layer 1: Unit tests of fakes (verify test infrastructure)
- Layer 2: Integration sanity tests with mocking (quick validation)
- Layer 3: Pure unit tests (zero dependencies, isolated testing)
- Layer 4: Business logic over fakes (majority of tests)
- Layer 5: Business logic integration tests (smoke tests over real systems)
- Decision tree: where should my test go?
- Test distribution examples
๐ python-specific.md
Read when:
- Working with pytest fixtures
- Need Python mocking patterns
- Testing Flask/FastAPI/Django applications
- Understanding Python testing tools
- Need Python-specific commands
Contents:
- pytest fixtures and parametrization
- Mocking with unittest.mock and pytest-mock
- Testing web frameworks (Flask, FastAPI, Django)
- Python testing commands
- Type hints in tests
- Python packaging for test utilities
๐ workflows.md
Read when:
- Adding a new feature (step-by-step)
- Fixing a bug (step-by-step)
- Adding a gateway method (complete checklist)
- Changing an interface (what to update)
- Managing dry-run features
Contents:
- Adding a new feature (TDD workflow)
- Fixing a bug (reproduce โ fix โ regression test)
- Adding a gateway method (8-step checklist with examples)
- Changing an interface (update all layers)
- Managing dry-run features (wrapping pattern)
- Testing with builder patterns
๐ patterns.md
Read when:
- Implementing constructor injection for fakes
- Adding mutation tracking to fakes
- Using CliRunner for CLI tests
- Building complex test scenarios with builders
- Testing dry-run behavior
- Need code examples of specific patterns
Contents:
- Constructor injection (how and why)
- Mutation tracking properties (read-only access)
- Using CliRunner (not subprocess)
- Builder patterns for complex scenarios
- Simulated environment pattern
- Error injection pattern
- Dry-run testing pattern
๐ anti-patterns.md
Read when:
- Unsure if your approach is correct
- Want to avoid common mistakes
- Reviewing code for bad patterns
- Debugging why tests are slow/brittle
Contents:
- โ Testing speculative features
- โ Hardcoded paths in tests (catastrophic)
- โ Not updating all layers
- โ Using subprocess in unit tests
- โ Complex logic in gateway classes
- โ Fakes with I/O operations
- โ Testing implementation details
- โ Incomplete test coverage for gateways
๐ quick-reference.md
Read when:
- Quick lookup for file locations
- Finding example tests to reference
- Looking up common fixtures
- Need command reference
- Want test distribution guidelines
Contents:
- Decision tree (where to add test)
- File location map (source + tests)
- Common fixtures (tmp_path, CliRunner, etc.)
- Common test patterns (code snippets)
- Example tests to reference
- Useful commands (pytest, ty, etc.)
- Quick checklist for adding gateway methods
Quick Navigation by Task
I'm adding a new feature
- Quick start:
quick-reference.mdโ Decision tree - Step-by-step:
workflows.md#adding-a-new-feature - Patterns:
patterns.md(CliRunner, builders) - Avoid:
anti-patterns.md(speculative tests, hardcoded paths)
I'm fixing a bug
- Step-by-step:
workflows.md#fixing-a-bug - Patterns:
patterns.md#constructor-injection-for-fakes - Examples:
quick-reference.md#example-tests-to-reference
I'm adding/changing a gateway method
- Understanding:
gateway-architecture.md - Step-by-step:
workflows.md#adding-a-gateway-method - Checklist:
quick-reference.md#quick-checklist-adding-a-new-gateway-method - Avoid:
anti-patterns.md#not-updating-all-layers
I don't know where my test should go
- Decision tree:
quick-reference.md#decision-tree - Detailed guide:
testing-strategy.md - Examples:
quick-reference.md#example-tests-to-reference
I need to implement a pattern
- All patterns:
patterns.md - Examples:
quick-reference.md#common-test-patterns
I think I'm doing something wrong
- Anti-patterns:
anti-patterns.md - Correct approach:
workflows.md
Visual Layer Guide
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Layer 5: Business Logic Integration Tests (5%) โ
โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ
โ โ Real database, filesystem, APIs, actual subprocess โ โ
โ โ Purpose: Smoke tests, catch integration issues โ โ
โ โ When: Sparingly, for critical workflows โ โ
โ โ Speed: Seconds per test โ โ
โ โ Location: tests/e2e/ or tests/integration/ โ โ
โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Layer 4: Business Logic Tests (70%) โ MOST TESTS HERE โ
โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ
โ โ FakeDatabase, FakeApiClient, FakeFileSystem โ โ
โ โ Purpose: Test features and business logic extensively โ โ
โ โ When: For EVERY feature and bug fix โ โ
โ โ Speed: Milliseconds per test โ โ
โ โ Location: tests/unit/, tests/services/, tests/commands/ โ โ
โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Layer 3: Pure Unit Tests (10%) โ
โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ
โ โ Zero dependencies, no fakes, no mocks โ โ
โ โ Purpose: Test isolated utilities and helpers โ โ
โ โ When: For pure functions, data structures, parsers โ โ
โ โ Speed: Milliseconds per test โ โ
โ โ Location: tests/unit/ โ โ
โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Layer 2: Integration Sanity Tests (10%) โ
โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ
โ โ RealDatabase with mocked connections โ โ
โ โ Purpose: Quick validation, catch syntax errors โ โ
โ โ When: When adding/changing real implementation โ โ
โ โ Speed: Fast (mocked) โ โ
โ โ Location: tests/integration/test_real_*.py โ โ
โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Layer 1: Fake Infrastructure Tests (5%) โ
โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ
โ โ Test FakeDatabase itself โ โ
โ โ Purpose: Verify test infrastructure is reliable โ โ
โ โ When: When adding/changing fake implementation โ โ
โ โ Speed: Milliseconds per test โ โ
โ โ Location: tests/unit/fakes/test_fake_*.py โ โ
โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
Key Principles
- Thin gateway layer: Wrap external state, push complexity to business logic
- Fast tests over fakes: 70% of tests should use in-memory fakes
- Defense in depth: Fakes โ sanity tests โ pure unit โ business logic โ integration
- Test what you're building: No speculative tests, only active work
- Update all layers: When changing interfaces, update ABC/real/fake/dry-run
- Gateways vs Backends: Gateways have fakes; backends compose gateways and do NOT have fakes
Layer Selection Guide
Distinguishing Layer 3 (Pure Unit) from Layer 4 (Business Logic):
-
Layer 3 (Pure Unit Tests): ZERO dependencies - no fakes, no mocks, no external state
- Testing string utilities:
sanitize_branch_name("feat/FOO")โ"feat-foo" - Testing parsers:
parse_git_status("## main")โ{"branch": "main"} - Testing data structures:
LinkedList.append()without any external dependencies
- Testing string utilities:
-
Layer 4 (Business Logic Tests): Uses fakes for external dependencies
- Testing commands:
create_worktree(fake_git, name="feature") - Testing workflows:
submit_pr(fake_gh, fake_git, ...) - Testing business logic that coordinates multiple integrations
- Testing commands:
If your test imports a Fake*, it belongs in Layer 4, not Layer 3.
Default Testing Strategy
When in doubt:
- Write test over fakes (Layer 4) for business logic
- Write pure unit test (Layer 3) for utilities/helpers with no dependencies
- Use
pytestwith fixtures - Use
tmp_pathfixture (not hardcoded paths) - Follow examples in
quick-reference.md
Summary
For quick tasks: Start with quick-reference.md
For understanding: Start with testing-strategy.md or gateway-architecture.md
For step-by-step guidance: Use workflows.md
For implementation details: Use patterns.md
For validation: Check anti-patterns.md
For Python specifics: Check python-specific.md
Repository
