Pytest Parametrization & GitHub Actions CI Pipeline

Purpose

The Problem

Most developers write tests like this:

def test_validate_pressure_empty():
    result, valid = validate_pressure("")
    assert valid == True

def test_validate_pressure_invalid_string():
    result, valid = validate_pressure("ABC")
    assert valid == False

def test_validate_pressure_too_low():
    result, valid = validate_pressure("3")
    assert valid == False

# ... 15 more nearly identical functions

This approach has three major problems:

1. Massive code duplication - Same test logic repeated 20+ times
2. Hard to maintain - Bug fix requires changing every function
3. Poor test reporting - “1 test failed” tells you nothing about which input case broke

Professional test suites use parametrized tests that run one test function with dozens of input combinations, and continuous integration pipelines that automatically run tests on every code change.

The Solution

We’re studying the test suite in test_server.py and test_patient_GUI.py, which uses pytest’s @pytest.mark.parametrize decorator to test 40+ scenarios with 5 test functions. We’ll also examine the GitHub Actions workflow that automatically runs these tests with code style checking on every push.

Why It Matters

This testing approach demonstrates production-level practices:

• Data-driven testing with parametrization
• Pycodestyle enforcement alongside unit tests
• CI/CD pipeline with automated quality gates
• Test fixtures for database setup/teardown

Prerequisites & Tooling

Knowledge Base:

• Python basics (functions, decorators)
• Pytest fundamentals (assert, test discovery)
• Git basics (commits, branches, push)
• YAML syntax (for GitHub Actions)

Environment:

pip install pytest pytest-pycodestyle testfixtures
python --version  # 3.10+

GitHub Setup:

• Repository hosted on GitHub
• Actions enabled (enabled by default for public repos)

High-Level Architecture

graph TD
    A[Developer Writes Code] --> B[Git Commit]
    B --> C[Git Push to GitHub]
    C --> D[GitHub Actions Triggered]
    D --> E[Spin up Ubuntu VM]
    E --> F[Checkout Code]
    F --> G[Setup Python 3.10]
    G --> H[Install Dependencies]
    H --> I[Run pytest with pycodestyle]
    I --> J{All Tests Pass?}
    J -->|Yes| K[✅ Green Check Mark]
    J -->|No| L[❌ Red X - Build Failed]
    K --> M[Safe to Merge PR]
    L --> N[Fix Issues, Push Again]
    N --> D

    style K fill:#90EE90
    style L fill:#FFB6C6

Local Testing Flow:

graph LR
    A[Test Function with @parametrize] --> B[Pytest Generates Test Cases]
    B --> C[Test Case 1: empty string]
    B --> D[Test Case 2: valid int]
    B --> E[Test Case 3: invalid string]
    B --> F[Test Case N...]
    C --> G[Run & Report]
    D --> G
    E --> G
    F --> G

Analogy: Think of parametrized testing like a production assembly line with quality control:

• Test function = Quality inspection station (pressure gauge, scales, etc.)
• Parameters = Items moving down the conveyor belt
• Parametrize decorator = Conveyor belt that feeds items automatically
• CI pipeline = Automated quality control that runs before shipping

Implementation

Step 1: Basic Parametrized Test - Single Parameter

Logic: Instead of writing 10 separate test functions for different CPAP pressure values, write ONE function that pytest runs 10 times with different inputs.

import pytest

@pytest.mark.parametrize("pressure, msg, met",
                         [("", "Information Uploaded", True),
                          ("A", "CPAP pressure is not an integer", False),
                          ("A1", "CPAP pressure is not an integer", False),
                          ("@1", "CPAP pressure is not an integer", False),
                          ("0", "CPAP Pressure is not between 4 and 25", False),
                          ("-1", "CPAP Pressure is not between 4 and 25", False),
                          ("3", "CPAP Pressure is not between 4 and 25", False),
                          ("26", "CPAP Pressure is not between 4 and 25", False),
                          ("99999", "CPAP Pressure is not between 4 and 25", False),
                          ("4", "Information Uploaded", True),
                          ("25", "Information Uploaded", True)
                          ])
def test_validate_pressure(pressure, msg, met):
    """
    Tests validate_pressure function with multiple input scenarios

    Parametrization allows testing:
    - Empty string (should pass as "not provided")
    - Non-numeric strings (should fail)
    - Out-of-range integers (should fail)
    - Valid boundary values 4 and 25 (should pass)

    Each tuple in the list becomes one test case
    """
    from patient_GUI import validate_pressure
    answer0, answer1 = validate_pressure(pressure)
    assert answer0 == msg  # Check error message matches expected
    assert answer1 == met  # Check pass/fail boolean matches expected

🔵 Deep Dive: How parametrize works under the hood

What pytest does:

1. Parse decorator: Extract parameter names "pressure, msg, met"
2. Generate test cases: Create 11 separate test executions
3. Inject arguments: Pass each tuple as function arguments
4. Run independently: Each test case is isolated (one failure doesn’t stop others)
5. Report individually: Output shows which specific case failed

Test execution:

$ pytest test_patient_GUI.py::test_validate_pressure -v

test_patient_GUI.py::test_validate_pressure["-"Information Uploaded"-True] PASSED    [  9%]
test_patient_GUI.py::test_validate_pressure[A-CPAP pressure is not an integer-False] PASSED    [ 18%]
test_patient_GUI.py::test_validate_pressure[A1-CPAP pressure is not an integer-False] PASSED    [ 27%]
...
test_patient_GUI.py::test_validate_pressure[25-Information Uploaded-True] PASSED    [100%]

========== 11 passed in 0.23s ==========

If one case fails:

test_patient_GUI.py::test_validate_pressure[26-...] FAILED    [ 72%]

FAILED test_patient_GUI.py::test_validate_pressure[26-...] - AssertionError
Expected: "CPAP Pressure is not between 4 and 25"
Got: "Information Uploaded"

🔴 Danger: Parametrize lists must match parameter count!

# WRONG - 3 parameters, but tuples have 2 values
@pytest.mark.parametrize("pressure, msg, met",
                         [("4", True),  # Missing msg!
                          ("25", "Info", True)])  # Correct
def test_validate_pressure(pressure, msg, met):
    pass

# Error: TypeError: test_validate_pressure() missing 1 required positional argument: 'met'

Step 2: Complex Parametrization - Nested Dictionaries

Logic: When testing API functions, you often need complex input structures like JSON dictionaries. Parametrization handles this elegantly.

@pytest.mark.parametrize("in_data, expected_keys, expected_types, Error",
                         [({"patient_mrn": "123",
                            "patient_name": "Jason",
                            "pressure": "55"},
                           ["patient_mrn", "patient_name", "pressure"],
                           [int, str, int],
                           True),
                          ({"patient_mrn": 123,
                            "patient_name": "Jason",
                            "pressure": 55},
                           ["patient_mrn", "patient_name", "pressure"],
                           [int, str, int],
                           True),
                          ({"patient_mrn": "a123",
                            "patient_name": "Jason",
                            "pressure": "55"},
                           ["patient_mrn", "patient_name", "pressure"],
                           [int, str, int],
                           "Key patient_mrn is not an int or numeric string"),
                          ({"patient_mrn": "123",
                            "patient_name": "Jason",
                            "pressure": "a55"},
                           ["patient_mrn", "patient_name", "pressure"],
                           [int, str, int],
                           "Key pressure is not an int or numeric string"),
                          (["patient_mrn", "a123",
                            "patient_name", "Jason",
                            "pressure", "55"],
                           ["patient_mrn", "patient_name", "pressure"],
                           [int, str, int],
                           "Input is not a dictionary"),
                          ({"patient_mrn": "123",
                            "pressure": "55"},
                           ["patient_mrn", "patient_name", "pressure"],
                           [int, str, int],
                           "Key patient_name is missing from input"),
                          ({"patient_mrn": "123",
                            "patient_name": 3,
                            "pressure": "55"},
                           ["patient_mrn", "patient_name", "pressure"],
                           [int, str, int],
                           "Key patient_name has the incorrect value type")
                          ])
def test_validate_input_data_generic(in_data, expected_keys,
                                     expected_types, Error):
    """
    Tests generic validation function with various failure modes

    Test cases cover:
    1. Numeric string conversion (pass)
    2. Proper types (pass)
    3. Invalid numeric string (fail with specific message)
    4. Invalid pressure string (fail with specific message)
    5. Wrong input type - list instead of dict (fail)
    6. Missing required key (fail)
    7. Wrong value type (fail)
    """
    from server import validate_input_data_generic
    answer = validate_input_data_generic(in_data, expected_keys,
                                         expected_types)
    assert answer == Error

Test organization strategy:

# Group test cases by category for readability
VALID_INPUTS = [
    ({"mrn": 123, "name": "John"}, [...], [...], True),
    ({"mrn": "123", "name": "Jane"}, [...], [...], True),
]

INVALID_TYPE_INPUTS = [
    ({"mrn": "abc", "name": "John"}, [...], [...], "Key mrn..."),
    ({"mrn": 123, "name": 456}, [...], [...], "Key name..."),
]

MISSING_KEY_INPUTS = [
    ({"mrn": 123}, [...], [...], "Key name is missing"),
    ({"name": "John"}, [...], [...], "Key mrn is missing"),
]

@pytest.mark.parametrize("in_data, keys, types, expected",
                         VALID_INPUTS + INVALID_TYPE_INPUTS + MISSING_KEY_INPUTS)
def test_validate_input_data_generic(in_data, keys, types, expected):
    # Test implementation

Step 3: Test Fixtures for Setup/Teardown

Logic: Some tests need database setup or cleanup. Pytest fixtures provide reusable setup code.

import pytest
from PatientModel import Patient, Session

@pytest.fixture(scope="function")
def clean_database():
    """
    Fixture that cleans database before each test

    Scope options:
    - function: Run before each test function (default)
    - class: Run once per test class
    - module: Run once per test file
    - session: Run once for entire test session
    """
    # Setup: Clear all patients
    session = Session()
    try:
        patients = session.query(Patient).all()
        for patient in patients:
            session.delete(patient)
        session.commit()
    finally:
        session.close()

    # Yield control to test
    yield

    # Teardown: Clean up after test (if needed)
    # This code runs after the test completes


# Use fixture by adding it as parameter
def test_add_patient_to_db(clean_database):
    from server import new_patient_to_db

    in_data = {
        "patient_name": "Jason",
        "patient_mrn": 100,
        "room_number": 100,
        "CPAP_pressure": 10,
        "breath_rate": 10.0,
        "apnea_count": 10,
        "flow_image": ""
    }

    # Database is clean at this point
    result = new_patient_to_db(in_data, "2023-04-26 00:00:00")

    # Verify patient was created
    session = Session()
    try:
        patient = session.query(Patient).filter_by(room_number=100).first()
        assert patient is not None
        assert patient.patient_name == "Jason"
    finally:
        session.close()

🔵 Deep Dive: Fixture scopes and performance

Problem: Running database cleanup before every test is slow:

# Slow: Cleans DB 50 times for 50 tests
@pytest.fixture(scope="function")
def clean_database():
    clear_all_data()  # Takes 200ms
    yield
# Total time: 50 tests × 200ms = 10 seconds

Solution: Use module or session scope:

# Fast: Cleans DB once, all tests share clean state
@pytest.fixture(scope="module")
def clean_database():
    clear_all_data()  # Takes 200ms
    yield
# Total time: 1 × 200ms + test time = 0.5 seconds

🔴 Danger: Module-scoped fixtures can cause test interdependence:

@pytest.fixture(scope="module")
def database():
    clear_data()
    yield

def test_create_patient(database):
    create_patient(id=1)
    # Patient with id=1 exists

def test_create_duplicate(database):
    # Test assumes clean DB, but patient id=1 still exists from previous test!
    create_patient(id=1)  # Fails with "Duplicate key error"

Fix: Either use scope="function" or explicitly clean up in each test.

Step 4: GitHub Actions Workflow - The CI Pipeline

Logic: Every push to GitHub should automatically run tests to catch bugs before merging. GitHub Actions provides free CI/CD for open-source projects.

File: .github/workflows/pytest_runner.yml

name: Pytest with Pycodestyle

# Trigger conditions
on: [push, pull_request]

jobs:
  build:
    # Use Ubuntu virtual machine (free on GitHub)
    runs-on: ubuntu-latest

    steps:
    # Step 1: Download repository code
    - uses: actions/checkout@v2

    # Step 2: Install Python
    - name: Set up Python 3.10
      uses: actions/setup-python@v2
      with:
        python-version: '3.10'

    # Step 3: Install dependencies
    - name: Install dependencies
      run: pip install -r requirements.txt

    # Step 4: Run tests
    - name: Test with pytest
      run: pytest -v --pycodestyle

🔵 Deep Dive: Understanding the workflow syntax

on: [push, pull_request]

• push: Runs when code is pushed to any branch
• pull_request: Runs when a PR is opened or updated
• Alternative: on: push: branches: [main] - Only main branch

runs-on: ubuntu-latest

• GitHub provides free VMs: ubuntu-latest, windows-latest, macos-latest
• Each workflow run gets a fresh VM (isolated environment)
• VM is destroyed after workflow completes

uses: actions/checkout@v2

• Pre-built action that clones your repository
• Equivalent to: git clone <your-repo> && cd <your-repo>

run: pytest -v --pycodestyle

• -v: Verbose output (shows each test name)
• --pycodestyle: Runs PEP 8 style checker on all files

Step 5: Pycodestyle Integration - Enforcing Code Style

Logic: Code style consistency matters in team projects. Pycodestyle (formerly pep8) checks Python code against PEP 8 guidelines.

Configuration in pytest.ini or setup.cfg:

[pycodestyle]
max-line-length = 120
ignore = E402,W503
exclude = .venv,venv,env,.git,__pycache__

What pycodestyle checks:

# ❌ FAILS pycodestyle

# E501: Line too long (>120 characters)
def some_function_with_a_very_long_name(parameter1, parameter2, parameter3, parameter4, parameter5, parameter6, parameter7, parameter8):
    pass

# E302: Expected 2 blank lines, found 1
class MyClass:
    pass
def my_function():  # Need 2 blank lines before function
    pass

# E231: Missing whitespace after ','
my_list = [1,2,3,4]

# W291: Trailing whitespace
x = 5

# ✅ PASSES pycodestyle

def some_function_with_a_very_long_name(
    parameter1, parameter2, parameter3, parameter4,
    parameter5, parameter6, parameter7, parameter8
):
    pass


class MyClass:
    pass


def my_function():
    pass


my_list = [1, 2, 3, 4]

x = 5

Running locally:

# Run tests with style checking
pytest -v --pycodestyle

# Run style check only (no unit tests)
pycodestyle .

# Auto-fix some issues (requires autopep8)
autopep8 --in-place --aggressive --aggressive <filename>

Step 6: Viewing CI Results on GitHub

Logic: After pushing code, GitHub Actions runs your tests automatically. You can view results in the GitHub UI.

Workflow:

1. Push code:

   git add test_server.py
   git commit -m "Add validation tests"
   git push origin main

2. GitHub automatically:

   • Detects the push
   • Reads .github/workflows/pytest_runner.yml
   • Spins up Ubuntu VM
   • Runs all steps

3. View results:

   • Go to your repo on GitHub
   • Click “Actions” tab
   • See list of workflow runs
   • Green ✅ = All tests passed
   • Red ❌ = Tests failed

4. Debugging failures:

   • Click on failed run
   • Expand “Test with pytest” step
   • See full pytest output with failures

Example failure output:

Run pytest -v --pycodestyle
============================= test session starts ==============================
test_server.py::test_validate_input_data_generic[in_data0-expected_keys0-expected_types0-Error0] PASSED [ 14%]
test_server.py::test_validate_input_data_generic[in_data1-expected_keys1-expected_types1-Error1] FAILED [ 28%]

=================================== FAILURES ===================================
______ test_validate_input_data_generic[in_data1-expected_keys1-expected_types1-Error1] ______

    def test_validate_input_data_generic(in_data, expected_keys, expected_types, Error):
        from server import validate_input_data_generic
        answer = validate_input_data_generic(in_data, expected_keys, expected_types)
>       assert answer == Error
E       AssertionError: assert "Key patient_mrn has the incorrect value type" == True
E        +  where True = Error

test_server.py:66: AssertionError
=========================== 1 failed, 6 passed in 2.34s ===========================
Error: Process completed with exit code 1.

Under the Hood

How Pytest Discovers and Collects Tests

Discovery rules:

# ✅ Discovered as tests
test_validate_pressure()          # Starts with "test_"
def test_something():             # Function starts with "test_"
class TestValidation:             # Class starts with "Test"
    def test_method(self):        # Method starts with "test_"

# ❌ NOT discovered
def validate_pressure_test():     # Doesn't start with "test_"
class ValidationTest:             # Doesn't start with "Test"
def helper_function():            # Not a test

Pytest collection process:

1. Scan directories: Find all test_*.py or *_test.py files
2. Import modules: Load Python files as modules
3. Inspect functions: Look for test_* functions
4. Inspect classes: Look for Test* classes with test_* methods
5. Check parametrize: Expand parametrized tests into multiple cases
6. Build test tree: Create hierarchical structure

Test IDs:

test_file.py::test_function                           # Basic test
test_file.py::test_function[param1]                   # Parametrized test case 1
test_file.py::test_function[param2]                   # Parametrized test case 2
test_file.py::TestClass::test_method                  # Class-based test
test_file.py::TestClass::test_method[param1]          # Parametrized class method

Parametrize Implementation Details

Under the hood, parametrize is a decorator that:

1. Stores metadata:

   def parametrize(argnames, argvalues):
       def decorator(func):
           # Store parameters in function metadata
           func.parametrize = (argnames, argvalues)
           return func
       return decorator

2. Pytest reads metadata during collection:

   # Simplified pytest collection
   for test_func in discover_test_functions():
       if hasattr(test_func, 'parametrize'):
           argnames, argvalues = test_func.parametrize
           for values in argvalues:
               # Create separate test case
               test_cases.append((test_func, values))
       else:
           test_cases.append((test_func, None))

3. Generates unique test IDs:

   # For readability, pytest creates IDs from parameter values
   test_validate_pressure[""-"Information Uploaded"-True]
   test_validate_pressure[A-CPAP pressure is not an integer-False]

Memory usage:

• Parametrize does NOT duplicate test function in memory
• Only stores parameter tuples
• For 100 test cases: 1 function + 100 small tuples ≈ 10KB

GitHub Actions Pricing and Limits

Free tier (public repos):

• Unlimited minutes for public repositories
• 2,000 minutes/month for private repos
• 500MB artifact storage

Resource limits per workflow:

• 6 hours max runtime per workflow
• 72 hours max queued time
• 20 concurrent jobs per free organization

VM specifications:

ubuntu-latest:
- 2-core CPU
- 7 GB RAM
- 14 GB SSD storage

Cost optimization:

# Slow: Runs 3 workflows sequentially (9 minutes)
on: [push]
jobs:
  test-python-310:
    runs-on: ubuntu-latest
    # ... 3 minutes
  test-python-311:
    runs-on: ubuntu-latest
    # ... 3 minutes
  test-python-312:
    runs-on: ubuntu-latest
    # ... 3 minutes

# Fast: Runs 3 workflows in parallel (3 minutes)
on: [push]
jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ['3.10', '3.11', '3.12']
    steps:
      - uses: actions/setup-python@v2
        with:
          python-version: ${{ matrix.python-version }}

Edge Cases & Pitfalls

Edge Case 1: Parametrize with IDs for Readability

Problem: Default test IDs can be unreadable for complex objects:

@pytest.mark.parametrize("data", [
    {"patient_mrn": 123, "room": 100, "name": "John"},
    {"patient_mrn": 456, "room": 200, "name": "Jane"},
])
def test_patient_data(data):
    assert data["patient_mrn"] > 0

# Output:
# test_file.py::test_patient_data[data0] PASSED
# test_file.py::test_patient_data[data1] PASSED
# "data0" and "data1" are not helpful!

Solution: Custom IDs

@pytest.mark.parametrize("data", [
    {"patient_mrn": 123, "room": 100, "name": "John"},
    {"patient_mrn": 456, "room": 200, "name": "Jane"},
], ids=["john-room-100", "jane-room-200"])
def test_patient_data(data):
    assert data["patient_mrn"] > 0

# Output:
# test_file.py::test_patient_data[john-room-100] PASSED
# test_file.py::test_patient_data[jane-room-200] PASSED

Edge Case 2: Fixtures with Parametrize

Combining fixtures and parametrize:

@pytest.fixture
def database():
    db = setup_database()
    yield db
    teardown_database(db)

@pytest.mark.parametrize("patient_id, expected", [
    (100, True),
    (999, False),
])
def test_patient_exists(database, patient_id, expected):
    # 'database' comes from fixture
    # 'patient_id' and 'expected' come from parametrize
    result = database.check_patient(patient_id)
    assert result == expected

Execution order:

1. Setup fixture (database)
2. Run test with first parameter set
3. Teardown fixture
4. Setup fixture again
5. Run test with second parameter set
6. Teardown fixture

Performance issue: Fixture runs N times for N parameter sets!

Solution: Use module scope:

@pytest.fixture(scope="module")
def database():
    db = setup_database()
    yield db
    teardown_database(db)
# Now fixture runs once for all parametrized tests

Edge Case 3: CI Failures with Secrets

Problem: Tests that need API keys or database passwords:

# This works locally but fails in CI
def test_database_connection():
    conn = connect(username=os.environ["DB_USER"],
                   password=os.environ["DB_PASS"])
    # KeyError: DB_USER not set in CI environment

Solution: GitHub Secrets

1. Add secrets to GitHub:

   • Go to repo → Settings → Secrets and variables → Actions
   • Click “New repository secret”
   • Name: DB_USER, Value: admin

2. Access in workflow:

   - name: Test with pytest
     env:
       DB_USER: ${{ secrets.DB_USER }}
       DB_PASS: ${{ secrets.DB_PASS }}
     run: pytest -v

3. Handle missing secrets:

   import os
   import pytest
   
   @pytest.mark.skipif(
       "DB_USER" not in os.environ,
       reason="Database credentials not available"
   )
   def test_database_connection():
       conn = connect(username=os.environ["DB_USER"],
                      password=os.environ["DB_PASS"])

Security: Never Commit Secrets

🔴 Danger: Accidentally committing secrets to Git:

# ❌ NEVER DO THIS
DB_PASSWORD = "super_secret_password_123"

# ❌ NEVER DO THIS
import requests
r = requests.get("https://api.example.com",
                 headers={"Authorization": "Bearer sk_live_12345abcdef"})

GitHub scans for secrets and will alert you!

Safe approaches:

# ✅ Use environment variables
import os
DB_PASSWORD = os.environ.get("DB_PASSWORD")

# ✅ Use config files (add to .gitignore)
import json
with open("config.json") as f:  # Not committed to Git
    config = json.load(f)
    DB_PASSWORD = config["db_password"]

# ✅ Use secret management
from keyring import get_password
DB_PASSWORD = get_password("myapp", "database")

Conclusion

What You Learned:

1. Parametrized Testing: Eliminate test duplication by running one function with multiple input sets
2. Pytest Decorators: Use @pytest.mark.parametrize for data-driven tests
3. Test Fixtures: Set up reusable test infrastructure with proper teardown
4. CI/CD with GitHub Actions: Automatically run tests on every push using YAML workflows
5. Code Style Enforcement: Integrate pycodestyle checks into test pipeline
6. Professional Test Organization: Group test cases, use descriptive IDs, handle secrets securely

Advanced Concepts Demonstrated:

• Test discovery and collection internals
• Fixture scope optimization for performance
• GitHub Actions VM architecture and limits
• Secret management in CI environments
• Parametrize implementation under the hood

Skill Transfer: These patterns apply to:

• Any Python project with pytest (Flask, Django, FastAPI, ML pipelines)
• Other CI platforms: GitLab CI, CircleCI, Travis CI (similar YAML syntax)
• Other languages: JUnit (Java), NUnit (C#), Jest (JavaScript) have parametrization
• Test-driven development (TDD): Write parametrized tests before implementation

Next Steps:

1. Add test coverage reporting with pytest-cov and Codecov
2. Implement property-based testing with hypothesis
3. Add performance benchmarks with pytest-benchmark
4. Create test matrices for multiple Python versions and OS platforms
5. Integrate security scanning (Bandit, Safety) into CI pipeline