Skip to content

Conversation

@jasonouellet
Copy link

Description

This PR implements a warning system that alerts users when they attempt to use severity-based filtering parameters without a Bridgecrew/Prisma Cloud API key.

The Problem:
Users attempting to use --check, --skip-check, --hard-fail-on, or --soft-fail-on with severity codes (CRITICAL, HIGH, MEDIUM, MODERATE, LOW, INFO, NONE) without an API key experience silent failures - these parameters are completely ignored with no indication of why. This leads to confusion, wasted debugging time, and potentially false security confidence in CI/CD pipelines.

The Solution:
This PR adds clear, actionable warnings when severity codes are detected in filtering parameters without an API key configured. The warnings:

  • Identify which parameters contain severity codes
  • List the specific severity codes detected
  • Explain that parameters will be ignored during the scan
  • Provide guidance on how to enable the feature (configure API key)

Impact:

  • Improves user experience by eliminating silent failures
  • Reduces support burden by making API key requirements explicit
  • Helps users understand platform dependencies
  • Maintains full backward compatibility (warnings only, no behavior changes)

Fixes #7379

New/Edited policies (Delete if not relevant)

Not applicable - This is a UX improvement, not a policy change.

Implementation Details

Files Added:

  • checkov/common/util/api_key_warnings.py - Core warning system module with three main functions:
    • check_for_severity_filtering_without_api_key(): Detects severity codes in CLI parameters (supports CSV format)
    • check_for_api_key_usage_warnings(): Main warning coordinator
    • warn_about_missing_metadata_without_api_key(): General informational message about API-dependent features

Files Modified:

  • checkov/main.py (lines 310-315): Integration point - calls warning functions after API key initialization
  • tests/common/util/test_api_key_warnings.py: Comprehensive test suite with 13 tests covering all scenarios

Key Features:

  1. Severity Detection: Identifies all 7 severity codes (CRITICAL, HIGH, MEDIUM, MODERATE, LOW, INFO, NONE)
  2. Multi-Parameter Support: Monitors --check, --skip-check, --hard-fail-on, --soft-fail-on
  3. CSV Format Support: Properly handles comma-separated values: --check CKV_1,HIGH,CKV_2
  4. Smart Filtering: Only warns about severity codes, not regular check IDs
  5. Clear Messaging: Emphasizes functional impact ("parameters will be ignored")
  6. Type Safety: Uses proper type hints with Namespace and from __future__ import annotations

Example Output:

Without API key (warning displayed):

$ checkov -d . --hard-fail-on HIGH,CRITICAL

⚠️  WARNING: Severity codes cannot be used without an API key.
Parameters --hard-fail-on contain severity codes: CRITICAL, HIGH
These parameters will be ignored during this scan.
Configure an API key with --bc-api-key to use severity-based filtering.

# Scan continues normally...

With API key (no warning):

$ checkov -d . --hard-fail-on HIGH,CRITICAL --bc-api-key <key>
# No warning, severity filtering works as expected

Mixed parameters:

$ checkov -d . --check CKV_AWS_1,HIGH --hard-fail-on CRITICAL

⚠️  WARNING: Severity codes cannot be used without an API key.
Parameters --check, --hard-fail-on contain severity codes: CRITICAL, HIGH
These parameters will be ignored during this scan.
Configure an API key with --bc-api-key to use severity-based filtering.

# CKV_AWS_1 is still processed correctly

Test Coverage

All scenarios are covered by 13 unit tests:

  • test_no_severity_filtering - No warning when only check IDs used
  • test_severity_in_check_parameter - Detects severities in --check
  • test_severity_in_skip_check_parameter - Detects severities in --skip-check
  • test_severity_in_hard_fail_on_parameter - Detects severities in --hard-fail-on
  • test_severity_in_soft_fail_on_parameter - Detects severities in --soft-fail-on
  • test_multiple_severities_same_parameter - Multiple severities in one parameter
  • test_severity_filtering_multiple_parameters - Severities across multiple parameters
  • test_mixed_check_ids_and_severities - Mix of check IDs and severities
  • test_with_api_key_no_warning - No warning when API key is present
  • test_severity_filtering_with_csv_string - CSV format: 'HIGH,CKV_AWS_1,MEDIUM'
  • test_severity_filtering_with_mixed_csv - CSV with checks and severities
  • test_complex_csv_combination - Complex CSV across all parameters
  • test_no_warning_with_only_check_ids_in_csv - No false positives for CSV check IDs

Test Results:

$ pytest tests/common/util/test_api_key_warnings.py -v
==================== 13 passed in 0.010s ====================

Technical Implementation Notes

Python Best Practices:

  • Uses from __future__ import annotations for modern type hints
  • Proper type safety with Namespace instead of Any
  • Conditional imports with TYPE_CHECKING to avoid circular dependencies
  • Follows existing Checkov code patterns and style

CSV Format Handling:

Leverages existing convert_csv_string_arg_to_list() utility from checkov.common.util.type_forcers:

  • Handles argparse lists: ['CKV_1,HIGH,CKV_2']['CKV_1', 'HIGH', 'CKV_2']
  • Works with all documented parameter formats
  • No new dependencies required

Logging:

  • Uses Python's standard logging framework at WARNING level
  • Includes emoji indicator (⚠️) for visual clarity
  • Non-intrusive: warnings don't block execution

Integration:

  • Minimal changes to existing code (5 lines in main.py)
  • No breaking changes to existing functionality
  • Fully backward compatible

Real-World Use Case

This change directly addresses the issue described in #7379 where users building CI/CD pipelines encounter silent failures when attempting to use severity-based filtering without understanding the API key requirement.

Before this PR:

# CI/CD pipeline - user expects security controls
- run: checkov -d . --hard-fail-on HIGH,CRITICAL
  # Silent failure - parameters ignored, pipeline always succeeds

After this PR:

# CI/CD pipeline - user gets explicit feedback
- run: checkov -d . --hard-fail-on HIGH,CRITICAL
  # Clear warning displayed:
  # ⚠️  WARNING: Severity codes cannot be used without an API key.
  # Parameters --hard-fail-on contain severity codes: CRITICAL, HIGH
  # These parameters will be ignored during this scan.

User now understands they need to either:

  1. Configure an API key: --bc-api-key ${{ secrets.PRISMA_API_KEY }}
  2. Use check IDs instead: --hard-fail-on CKV_AWS_1,CKV_AWS_2

Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have added tests that prove my feature, policy, or fix is effective and works
  • New and existing tests pass locally with my changes

Additional Notes for Reviewers

  • This is a quality-of-life improvement focused on user experience
  • No performance impact (warnings only shown during initialization)
  • The severity code set is kept in sync with existing severity definitions in the codebase
  • Warning messages are user-friendly and actionable
  • Implementation follows the DRY principle by reusing existing CSV parsing utilities

Thank you for considering this contribution! This change will help prevent the frustration I experienced when trying to implement severity-based filtering in my CI/CD pipelines.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

- Focus on behavior differences rather than warnings
- Explain that severity filtering uses estimated defaults without API key
- Clarify which parameters require API key for full functionality
- Update CLI Command Reference with concise API key requirement notes
- Update Hard and soft fail doc to emphasize functional impact

delete: second doc
@jasonouellet jasonouellet marked this pull request as draft January 7, 2026 15:36
@jasonouellet jasonouellet marked this pull request as ready for review January 7, 2026 15:36
Copy link
Author

@jasonouellet jasonouellet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yuvalmich, @omriyoffe-panw, can you review this ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(general): Add warnings when API-dependent parameters are used without API key

1 participant