testing-strategy

name: testing-strategy description: Comprehensive testing strategies and coverage standards. Use when testing strategy guidance is required.

Purpose

Provide comprehensive testing strategies and coverage standards that can be applied across services, including thresholds, critical path tests, and environment setup.

IO Semantics

Input: Test suites, coverage reports, and service architectures that require structured testing guidance.

Output: Concrete coverage targets, configuration examples, and critical path testing patterns that can be enforced in CI.

Side Effects: Raising coverage thresholds or enforcing new critical paths may require additional tests and refactoring.

Deterministic Steps

1. Coverage Requirements Enforcement

Apply mandatory coverage thresholds:

Overall code coverage: ≥ 80%
Critical business logic coverage: ≥ 95%
Security-related code coverage: ≥ 90%
New feature coverage: ≥ 85% before merge

Apply coverage configuration examples:

# .pytest.ini for Python
[tool:pytest]
addopts =
    --cov=src
    --cov-report=term-missing
    --cov-report=html:htmlcov
    --cov-fail-under=80
    --cov-branch
    --cov-context=test

# Go coverage configuration
# Makefile
.PHONY: test test-coverage
test:
	go test -v ./...

test-coverage:
	go test -v -coverprofile=coverage.out ./...
	go tool cover -html=coverage.out -o coverage.html
	go tool cover -func=coverage.out | grep "total:" | awk '{print $3}' | sed 's/%//' | \
		awk '{if ($1 < 80) {print "Coverage below 80%: " $1 "%"; exit 1} else {print "Coverage: " $1 "%"}}'

2. Critical Path Testing

Identify and prioritize critical paths:

# test_critical_paths.py
import pytest
from unittest.mock import Mock, patch
from app.payment import PaymentProcessor
from app.user_management import UserService

class TestCriticalPaths:
    def test_payment_processing_complete_flow(self):
        """Test complete payment flow with real dependencies"""
        processor = PaymentProcessor()

        # Test successful payment
        result = processor.process_payment(
            user_id=123,
            amount=100.00,
            payment_method="credit_card"
        )

        assert result.success is True
        assert result.transaction_id is not None
        assert result.amount == 100.00

    def test_user_registration_with_validation(self):
        """Test user registration with all validation rules"""
        user_service = UserService()

        # Test valid registration
        user = user_service.register_user(
            email="test@example.com",
            password="SecurePass123!",
            name="Test User"
        )

        assert user.email == "test@example.com"
        assert user.is_active is True
        assert user.id is not None

    @pytest.mark.parametrize("status_code,expected_result", [
        (200, {"status": "success"}),
        (400, {"status": "error", "message": "Invalid request"}),
        (500, {"status": "error", "message": "Internal server error"})
    ])
    def test_api_endpoint_error_handling(self, status_code, expected_result):
        """Test API error handling scenarios"""
        with patch('requests.post') as mock_post:
            mock_post.return_value.status_code = status_code
            mock_post.return_value.json.return_value = expected_result

            response = self.client.call_external_api({"data": "test"})

            assert response == expected_result

Testing Framework Configuration

Multi-Language Testing Setup

Python Testing Configuration:

# pyproject.toml
[tool.pytest.ini_options]
minversion = "7.0"
addopts = [
    "--strict-markers",
    "--strict-config",
    "--cov=src",
    "--cov-report=term-missing",
    "--cov-report=html",
    "--cov-report=xml",
    "--cov-fail-under=80"
]
testpaths = ["tests"]
python_files = ["test_*.py", "*_test.py"]
python_classes = ["Test*"]
python_functions = ["test_*"]

markers = [
    "unit: Unit tests",
    "integration: Integration tests",
    "e2e: End-to-end tests",
    "slow: Slow running tests",
    "network: Tests requiring network access",
    "database: Tests requiring database"
]

Go Testing Configuration:

// testing_setup.go
package testsetup

import (
	"testing"
	"time"
)

// Test configuration
type TestConfig struct {
	DatabaseURL string
	RedisURL    string
	Timeout     time.Duration
}

// Global test configuration
var Config TestConfig

func TestMain(m *testing.M) {
	// Setup test environment
	setupTestEnvironment()

	// Run tests
	code := m.Run()

	// Cleanup
	cleanupTestEnvironment()

	os.Exit(code)
}

func setupTestEnvironment() {
	Config = TestConfig{
		DatabaseURL: "postgres://test:test@localhost:5432/testdb?sslmode=disable",
		RedisURL:    "redis://localhost:6379/1",
		Timeout:     30 * time.Second,
	}

	// Wait for services to be ready
	waitForServices()
}

Test Quality Assurance

AAA Pattern Implementation

Apply Arrange-Act-Assert consistently:

import pytest
from calculator import Calculator

class TestCalculator:
    def test_addition_positive_numbers(self):
        # Arrange
        calculator = Calculator()
        operand_a = 5
        operand_b = 3

        # Act
        result = calculator.add(operand_a, operand_b)

        # Assert
        assert result == 8
        assert isinstance(result, (int, float))

    def test_division_by_zero_raises_error(self):
        # Arrange
        calculator = Calculator()
        dividend = 10
        divisor = 0

        # Act & Assert
        with pytest.raises(ZeroDivisionError, match="Cannot divide by zero"):
            calculator.divide(dividend, divisor)

    def test_complex_calculation_chain(self):
        # Arrange
        calculator = Calculator()
        initial_value = 10

        # Act
        result = (calculator
                  .add(initial_value, 5)
                  .multiply(2)
                  .subtract(3)
                  .divide(4))

        # Assert
        assert result == 5.5

Test Isolation and Independence

Ensure tests run independently:

import pytest
import tempfile
import shutil
from pathlib import Path

class TestFileOperations:
    @pytest.fixture
    def temp_directory(self):
        """Create isolated temporary directory for each test"""
        temp_dir = tempfile.mkdtemp()
        yield Path(temp_dir)
        shutil.rmtree(temp_dir)

    def test_file_creation_and_read(self, temp_directory):
        """Test file operations in isolated environment"""
        test_file = temp_directory / "test.txt"
        test_content = "Hello, World!"

        # Act
        test_file.write_text(test_content)
        read_content = test_file.read_text()

        # Assert
        assert read_content == test_content
        assert test_file.exists()

    def test_file_operations_persistence(self, temp_directory):
        """This test uses different temp_directory, ensuring isolation"""
        # This test cannot interfere with previous test due to separate temp_directory
        another_file = temp_directory / "another.txt"
        another_file.write_text("Different content")

        assert another_file.read_text() == "Different content"

Performance and Load Testing

Load Testing Implementation

Automated performance validation:

import pytest
import time
import threading
from concurrent.futures import ThreadPoolExecutor
from app.api_server import app

class TestPerformance:
    @pytest.mark.slow
    def test_api_response_time_under_load(self):
        """Test API response time under concurrent load"""
        url = "http://localhost:8000/api/test"
        concurrent_requests = 50
        max_response_time = 1.0  # seconds

        def make_request():
            start_time = time.time()
            response = requests.get(url)
            end_time = time.time()
            return end_time - start_time, response.status_code

        with ThreadPoolExecutor(max_workers=concurrent_requests) as executor:
            futures = [executor.submit(make_request) for _ in range(concurrent_requests)]
            results = [future.result() for future in futures]

        # Assert all requests succeeded
        status_codes = [result[1] for result in results]
        assert all(code == 200 for code in status_codes)

        # Assert response times are within limits
        response_times = [result[0] for result in results]
        assert max(response_times) < max_response_time
        assert sum(response_times) / len(response_times) < max_response_time * 0.8

    @pytest.mark.performance
    def test_memory_usage_stability(self):
        """Test memory usage remains stable during extended operation"""
        import psutil
        import os

        process = psutil.Process(os.getpid())
        initial_memory = process.memory_info().rss

        # Perform many operations
        for _ in range(1000):
            self.heavy_operation()

        final_memory = process.memory_info().rss
        memory_increase = final_memory - initial_memory

        # Memory increase is minimal (< 10MB)
        assert memory_increase < 10 * 1024 * 1024, f"Memory increased by {memory_increase} bytes"

    def heavy_operation(self):
        """Simulate memory-intensive operation"""
        data = list(range(1000))
        processed_data = [x * 2 for x in data]
        return sum(processed_data)

Integration and E2E Testing

Integration Test Architecture

Comprehensive integration testing:

import pytest
from testcontainers.postgres import PostgresContainer
from testcontainers.redis import RedisContainer
from app.database import DatabaseConnection
from app.cache import RedisCache

@pytest.mark.integration
class TestDatabaseIntegration:
    @pytest.fixture(scope="class")
    def postgres_container(self):
        """Start PostgreSQL container for integration tests"""
        with PostgresContainer("postgres:15-alpine") as postgres:
            yield postgres

    @pytest.fixture(scope="class")
    def redis_container(self):
        """Start Redis container for integration tests"""
        with RedisContainer("redis:7-alpine") as redis:
            yield redis

    @pytest.fixture(scope="class")
    def database(self, postgres_container):
        """Configure database connection with test container"""
        db = DatabaseConnection(
            host=postgres_container.get_container_host_ip(),
            port=postgres_container.get_exposed_port(5432),
            database="test",
            user="test",
            password="test"
        )
        db.create_tables()
        yield db
        db.close()

    def test_user_creation_and_retrieval(self, database):
        """Test complete user creation and retrieval flow"""
        user_data = {
            "email": "test@example.com",
            "name": "Test User"
        }

        # Create user
        user_id = database.create_user(user_data)
        assert user_id is not None

        # Retrieve user
        retrieved_user = database.get_user(user_id)
        assert retrieved_user["email"] == user_data["email"]
        assert retrieved_user["name"] == user_data["name"]

    def test_transaction_rollback(self, database):
        """Test transaction rollback on error"""
        initial_count = database.count_users()

        with pytest.raises(ValueError):
            with database.transaction():
                database.create_user({"email": "good@example.com", "name": "Good User"})
                raise ValueError("Simulated error")

        # Ensure no user was created due to rollback
        final_count = database.count_users()
        assert final_count == initial_count

End-to-End Testing

Full application workflow testing:

import pytest
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

@pytest.mark.e2e
class TestUserWorkflows:
    @pytest.fixture
    def browser(self):
        """Setup browser for E2E tests"""
        options = webdriver.ChromeOptions()
        options.add_argument('--headless')
        options.add_argument('--no-sandbox')
        options.add_argument('--disable-dev-shm-usage')

        driver = webdriver.Chrome(options=options)
        driver.implicitly_wait(10)

        yield driver

        driver.quit()

    def test_complete_user_registration_flow(self, browser):
        """Test complete user registration from UI"""
        # Navigate to registration page
        browser.get("http://localhost:3000/register")

        # Fill registration form
        browser.find_element(By.ID, "email").send_keys("test@example.com")
        browser.find_element(By.ID, "password").send_keys("SecurePass123!")
        browser.find_element(By.ID, "confirm_password").send_keys("SecurePass123!")
        browser.find_element(By.ID, "name").send_keys("Test User")

        # Submit form
        browser.find_element(By.ID, "register-button").click()

        # Verify successful registration
        WebDriverWait(browser, 10).until(
            EC.presence_of_element_located((By.CLASS_NAME, "success-message"))
        )

        # Verify redirect to dashboard
        assert "dashboard" in browser.current_url

    def test_login_with_registered_user(self, browser):
        """Test login with previously registered user"""
        # First register a user
        self.register_test_user(browser)

        # Navigate to login page
        browser.get("http://localhost:3000/login")

        # Fill login form
        browser.find_element(By.ID, "email").send_keys("test@example.com")
        browser.find_element(By.ID, "password").send_keys("SecurePass123!")
        browser.find_element(By.ID, "login-button").click()

        # Verify successful login
        WebDriverWait(browser, 10).until(
            EC.presence_of_element_located((By.CLASS_NAME, "user-dashboard"))
        )

    def register_test_user(self, browser):
        """Helper method to register test user"""
        browser.get("http://localhost:3000/register")
        browser.find_element(By.ID, "email").send_keys("test@example.com")
        browser.find_element(By.ID, "password").send_keys("SecurePass123!")
        browser.find_element(By.ID, "confirm_password").send_keys("SecurePass123!")
        browser.find_element(By.ID, "name").send_keys("Test User")
        browser.find_element(By.ID, "register-button").click()
        WebDriverWait(browser, 10).until(
            EC.presence_of_element_located((By.CLASS_NAME, "success-message"))
        )

Continuous Integration Testing

Automated Test Pipeline

Comprehensive CI/CD testing workflow:

# .github/workflows/test.yml
name: Test Suite

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest

    services:
      postgres:
        image: postgres:15
        env:
          POSTGRES_PASSWORD: test
          POSTGRES_DB: testdb
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5

      redis:
        image: redis:7
        options: >-
          --health-cmd "redis-cli ping"
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5

    steps:
      - uses: actions/checkout@v3

      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.11'

      - name: Set up Go
        uses: actions/setup-go@v4
        with:
          go-version: '1.21'

      - name: Install dependencies
        run: |
          pip install -r requirements.txt
          pip install -r requirements-dev.txt

      - name: Run Python unit tests
        run: |
          pytest tests/unit/ -v \
            --cov=src \
            --cov-report=xml \
            --cov-fail-under=80

      - name: Run Python integration tests
        run: |
          pytest tests/integration/ -v \
            --cov=src \
            --cov-append \
            --cov-report=xml

      - name: Run Go tests
        run: |
          go test -v -race -coverprofile=coverage.out ./...
          go tool cover -html=coverage.out -o coverage.html

      - name: Run E2E tests
        run: |
          docker-compose -f docker-compose.test.yml up -d
          sleep 30
          pytest tests/e2e/ -v
          docker-compose -f docker-compose.test.yml down

      - name: Upload coverage reports
        uses: codecov/codecov-action@v3
        with:
          files: ./coverage.xml,coverage.out

  security-scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Run security scan
        run: |
          pip install bandit safety
          bandit -r src/ -f json -o bandit-report.json
          safety check --json --output safety-report.json

      - name: Upload security reports
        uses: actions/upload-artifact@v3
        with:
          name: security-reports
          path: |
            bandit-report.json
            safety-report.json

$ インストール