Mohammad Shaker
← Back

Skip Optional Dependencies in CI: Flexible Test Environment

·code-quality

Skip Optional Dependencies in CI: Flexible Test Environment

CI pipeline failures blocked pull requests due to missing optional dependencies. A developer added tests for a new analytics script that required pandas and matplotlib—libraries not used in the core application. The tests passed locally where these packages were installed. In CI, the pipeline failed with ModuleNotFoundError: No module named 'pandas'.

Adding pandas to production dependencies would bloat the Lambda deployment package by 50MB for a feature used only in offline analysis scripts. The options seemed binary: install unnecessary dependencies in production or skip those tests entirely. We implemented conditional test skipping based on dependency availability.

The Problem: All-or-Nothing Dependencies

Python projects traditionally separate dependencies into production requirements (requirements.txt) and development requirements (requirements-dev.txt). Production requirements deploy to Lambda and must stay minimal to reduce cold start time and package size. Development requirements include testing tools, linters, and utilities that never reach production.

The problem appears when tests require libraries that are optional—useful for specific features but not core functionality. Examples include:

  • Data analysis scripts: pandas, numpy, matplotlib
  • Report generation: openpyxl, reportlab
  • Advanced text processing: spaCy, nltk
  • Performance profiling: memory_profiler, py-spy

These libraries are heavyweight (pandas alone is 50MB) and serve narrow use cases. Installing them in production wastes resources. Not installing them breaks tests in CI.

Before: Rigid Dependency Model

CI Pipeline
┌──────────────────────────────────────┐
│ pytest src/tests/                    │
│   ├─ Core tests ✓                    │
│   ├─ API tests ✓                     │
│   ├─ Script tests ✗ (deps missing)   │
│   └─ PIPELINE FAILED                 │
│                                      │
│ Options:                             │
│ 1. Add pandas to requirements.txt    │
│    → Bloat production (bad)          │
│ 2. Skip script tests entirely        │
│    → No test coverage (bad)          │
└──────────────────────────────────────┘

Both options were unsatisfactory. Option 1 added 50-100MB to the Lambda package, increasing cold start time by 200-400ms and deployment time by 30 seconds. Option 2 left important code untested.

Production Impact:

We identified 23 tests requiring optional dependencies:

  • 12 tests for analytics scripts (pandas, matplotlib)
  • 6 tests for report generation (openpyxl)
  • 3 tests for text processing (spaCy)
  • 2 tests for performance utilities (memory_profiler)

Skipping these tests meant 23 functions with zero automated coverage—a significant gap in quality assurance.

The Solution: Conditional Test Skipping

We implemented pytest markers and import guards to skip tests when optional dependencies are unavailable. Tests attempt to import required libraries. If the import succeeds, the test runs. If it fails, pytest skips the test with a clear message explaining why.

After: Flexible Test Skipping

CI Pipeline
┌──────────────────────────────────────┐
│ pytest src/tests/                    │
│   ├─ Core tests ✓                    │
│   ├─ API tests ✓                     │
│   ├─ Script tests ⊘ (skipped)        │
│   └─ PIPELINE PASSED                 │
│                                      │
│ Summary:                             │
│ - 301 passed                         │
│ - 23 skipped (optional deps)         │
│ - 0 failed                           │
└──────────────────────────────────────┘

Local Development (with optional deps):
┌──────────────────────────────────────┐
│ pytest src/tests/                    │
│   ├─ Core tests ✓                    │
│   ├─ API tests ✓                     │
│   ├─ Script tests ✓                  │
│   └─ ALL TESTS PASSED                │
│                                      │
│ Summary:                             │
│ - 324 passed                         │
│ - 0 skipped                          │
└──────────────────────────────────────┘

CI passes without optional dependencies. Developers with full development environments run all tests. Both scenarios work without forcing production dependencies.

Implementation Details

We created a utility module for dependency checking:

src/tests/utils/dependency_check.py:

import importlib.util
from typing import List

def has_optional_deps(modules: List[str]) -> bool:
    """
    Check if optional dependencies are available.

    Args:
        modules: List of module names to check (e.g., ['pandas', 'matplotlib'])

    Returns:
        True if all modules are importable, False otherwise
    """
    for module_name in modules:
        spec = importlib.util.find_spec(module_name)
        if spec is None:
            return False
    return True

# Common dependency groups
ANALYTICS_DEPS = ['pandas', 'matplotlib', 'numpy']
REPORTING_DEPS = ['openpyxl', 'reportlab']
NLP_DEPS = ['spacy']
PROFILING_DEPS = ['memory_profiler']

This utility provides a clean API for checking multiple dependencies at once.

Pattern 1: pytest.mark.skipif Decorator

The primary pattern uses pytest's skipif marker:

import pytest
from src.tests.utils.dependency_check import has_optional_deps, ANALYTICS_DEPS

@pytest.mark.skipif(
    not has_optional_deps(ANALYTICS_DEPS),
    reason="Analytics dependencies (pandas, matplotlib) not installed"
)
def test_generate_analytics_report():
    """Generate user engagement analytics report."""
    import pandas as pd
    import matplotlib.pyplot as plt

    # Test implementation
    data = pd.DataFrame({'users': [100, 150, 200]})
    report = generate_analytics_report(data)

    assert report is not None
    assert 'summary' in report

When pandas or matplotlib is missing, pytest skips the test and displays the reason:

SKIPPED [1] src/tests/scripts/test_analytics.py:45: Analytics dependencies (pandas, matplotlib) not installed

This provides clear feedback about why the test was skipped and what's needed to run it.

Pattern 2: Module-Level Skip

For test files that entirely depend on optional packages, we skip at the module level:

# test_advanced_reporting.py
import pytest
from src.tests.utils.dependency_check import has_optional_deps, REPORTING_DEPS

# Skip entire module if reporting dependencies unavailable
pytestmark = pytest.mark.skipif(
    not has_optional_deps(REPORTING_DEPS),
    reason="Reporting dependencies (openpyxl, reportlab) not installed"
)

# All tests in this file are skipped if dependencies missing
def test_generate_excel_report():
    import openpyxl
    # ... test implementation

def test_generate_pdf_report():
    import reportlab
    # ... test implementation

This pattern is cleaner when an entire test file requires the same optional dependencies.

Pattern 3: Fixture-Based Skipping

For complex setup, we use fixtures that skip tests if dependencies are unavailable:

@pytest.fixture
def analytics_environment():
    """
    Set up analytics environment with required dependencies.
    Skips test if dependencies unavailable.
    """
    if not has_optional_deps(ANALYTICS_DEPS):
        pytest.skip("Analytics dependencies not installed")

    import pandas as pd
    import matplotlib
    matplotlib.use('Agg')  # Non-interactive backend for CI

    # Set up test environment
    yield {
        'pd': pd,
        'matplotlib': matplotlib
    }

def test_visualize_user_growth(analytics_environment):
    """Test user growth visualization."""
    pd = analytics_environment['pd']

    data = pd.DataFrame({'month': [1, 2, 3], 'users': [100, 150, 200]})
    chart = visualize_user_growth(data)

    assert chart is not None

Fixtures provide reusable skip logic and environment setup.

Dependency Groups

We documented dependency groups in requirements-optional.txt:

# Analytics and Data Science
pandas>=2.0.0
numpy>=1.24.0
matplotlib>=3.7.0

# Report Generation
openpyxl>=3.1.0
reportlab>=4.0.0

# Natural Language Processing
spacy>=3.5.0

# Performance Profiling
memory-profiler>=0.61.0
py-spy>=0.3.14

Developers install these locally for full test coverage:

# Install core dependencies (required)
pip install -r requirements.txt

# Install development dependencies (required for running tests)
pip install -r requirements-dev.txt

# Install optional dependencies (optional, for full test coverage)
pip install -r requirements-optional.txt

CI installs only core and dev dependencies, skipping optional packages.

CI Configuration

We updated CircleCI to show skip summaries:

# .circleci/config.yml
jobs:
  test:
    docker:
      - image: python:3.11
    steps:
      - checkout

      # Install core dependencies only
      - run:
          name: Install dependencies
          command: |
            pip install -r requirements.txt
            pip install -r requirements-dev.txt
            # Note: requirements-optional.txt NOT installed in CI

      # Run tests with skip summary
      - run:
          name: Run tests
          command: |
            pytest src/tests/ -v --tb=short

      # Generate and upload coverage (excluding skipped tests)
      - run:
          name: Generate coverage
          command: |
            pytest --cov=src --cov-report=html --cov-report=term src/tests/
            # Coverage calculated only for tests that ran

Test output shows skip summary:

======================== test session starts ========================
platform linux -- Python 3.11.4
collected 324 items

src/tests/unit/test_auth.py ................. [  5%]
src/tests/unit/test_content.py .............. [ 12%]
...
src/tests/scripts/test_analytics.py sssssssss [ 95%]
src/tests/scripts/test_reporting.py ssssss   [100%]

=============== 301 passed, 23 skipped in 12.34s ================

The summary clearly shows 23 tests were skipped (not failed), and CI passes.

Local Development Workflow

Developers working on analytics features install optional dependencies:

# Create dedicated virtual environment
python -m venv venv-analytics
source venv-analytics/bin/activate

# Install all dependencies
pip install -r requirements.txt
pip install -r requirements-dev.txt
pip install -r requirements-optional.txt

# Run all tests (including analytics)
pytest src/tests/
# =============== 324 passed in 15.67s ================

Developers not working on analytics features skip optional dependencies:

# Standard virtual environment
python -m venv venv
source venv/bin/activate

# Install core and dev dependencies only
pip install -r requirements.txt
pip install -r requirements-dev.txt

# Run tests (analytics tests skipped)
pytest src/tests/
# =============== 301 passed, 23 skipped in 12.34s ================

Both workflows succeed—one runs all tests, the other skips optional tests. Neither blocks development.

Documentation

We added clear documentation to the README:

README.md:

## Testing

### Running Tests

```bash
# Run all tests (skips tests requiring optional dependencies)
pytest src/tests/

# Run with coverage
pytest --cov=src src/tests/

Optional Dependencies

Some tests require optional dependencies (pandas, matplotlib, etc.). Install them for full coverage:

pip install -r requirements-optional.txt

Tests are automatically skipped if optional dependencies are unavailable. This is normal in CI and local development if you're not working on those features.

Skipped tests indicate missing optional dependencies, not failures.


Clear documentation prevented confusion when developers saw skipped tests.

## Real-World Impact

**Before Implementation:**
- CI failed when optional dependencies missing
- Developers added unnecessary production dependencies (50MB+ bloat)
- 23 tests had no automated coverage
- Pull requests blocked by CI failures
- Confusion about which dependencies are required

**After Implementation:**
- CI passes without optional dependencies
- Production package stays minimal (no bloat)
- 23 tests run when dependencies available, skip gracefully otherwise
- Pull requests merge smoothly
- Clear documentation explains skip behavior

**CI Performance:**

Before (with optional deps installed in CI):

  • Install time: 3m 45s (pandas, matplotlib, etc.)
  • Package size: 250MB
  • Test run time: 15s

After (skipping optional tests):

  • Install time: 1m 20s (2m 25s faster)
  • Package size: 180MB (70MB smaller)
  • Test run time: 12s (3s faster, fewer tests)

Faster CI meant faster feedback cycles and reduced CircleCI costs.

**Lambda Deployment Package:**

Before (with optional deps):

  • Package size: 95MB
  • Cold start: 2.8s
  • Deployment time: 45s

After (without optional deps):

  • Package size: 48MB (51% reduction)
  • Cold start: 1.2s (57% improvement)
  • Deployment time: 22s (51% faster)

Removing optional dependencies from production had significant performance benefits.

## Edge Cases and Solutions

**Edge Case 1: Version Conflicts**

Optional dependencies sometimes conflicted with core dependencies. Pandas 2.0 required numpy>=1.24, but our application used numpy 1.22.

**Solution:** We documented version requirements and tested compatibility:

```txt
# requirements-optional.txt
# Note: Requires numpy>=1.24 (conflicts with core requirement)
# Only install in isolated environments
pandas>=2.0.0  # requires numpy>=1.24

Developers use separate virtual environments to avoid conflicts.

Edge Case 2: Partial Feature Testing

Some features had both required and optional components. Core functionality ran in production; analytics features required optional dependencies.

Solution: We split tests into core and optional:

def test_user_engagement_core():
    """Test core engagement calculation (no optional deps)."""
    engagement = calculate_user_engagement(user_id=1)
    assert engagement >= 0

@pytest.mark.skipif(not has_optional_deps(ANALYTICS_DEPS))
def test_user_engagement_visualization():
    """Test engagement visualization (requires pandas/matplotlib)."""
    import pandas as pd
    import matplotlib.pyplot as plt

    engagement_data = get_engagement_data(user_id=1)
    df = pd.DataFrame(engagement_data)
    chart = plot_engagement(df)

    assert chart is not None

Core tests run always; visualization tests run only with optional dependencies.

Key Takeaways

  1. Optional dependencies should stay optional - Don't force them into production
  2. Tests should adapt to environment - Skip gracefully when dependencies unavailable
  3. Clear skip messages - Explain why tests were skipped and how to run them
  4. Document dependency groups - Separate requirements files for different use cases
  5. Fast CI is valuable - Removing unnecessary dependencies speeds up pipelines

Implementing conditional test skipping solved a common dilemma: how to test code requiring heavyweight dependencies without bloating production deployments. The solution—pytest skip markers with dependency checking—was simple, clear, and effective.

Tests run when possible, skip when necessary, and never block CI unnecessarily. Developers with full environments get full coverage. CI runs lean and fast. Production deployments stay minimal. Everyone wins.