Slow down the Claude code review

[caufieldjh]

The Claude code review can get overzealous and read old versions of documents following new commits.
This leads to reviews erroneously identifying solved issues as remaining issues.
Example:
Claude finished @claude[bot]'s task —— View job

I don't know if this is due to the order of events in the workflow, some kind of race condition, cached files, or something else.

It may help to have the Claude review run less often and/or in response to a different trigger (like moving a PR from draft to ready).
Or, this may be addressed by explicitly having Claude only review changed files.

[github-actions]

Issue Summary

This issue describes a timing problem with the automated Claude code review workflow where the review occasionally reads stale file versions after new commits are pushed to a PR, resulting in false positive feedback about already-resolved issues.

Root Cause Analysis

The workflow is configured in .github/workflows/claude-code-review.yml with these triggers:

on:
  pull_request:
    types: [opened, synchronize]

The synchronize event fires when new commits are pushed to a PR. The workflow uses:

- uses: actions/checkout@v6
  with:
    fetch-depth: 1

Potential causes:

1. Checkout timing: The actions/checkout@v6 may be fetching the repository state before the new commit is fully available to the Actions runner
2. No explicit ref specified: Without explicitly checking out the PR head ref, there could be ambiguity about which commit is checked out
3. Shallow clone (fetch-depth: 1): May occasionally miss the latest commit in race conditions
4. Multiple rapid commits: If commits are pushed in quick succession, reviews may queue up and run against different states

Recommended Solutions

Option 1: Explicit PR Head Checkout (Recommended)

- name: Checkout repository
  uses: actions/checkout@v6
  with:
    ref: ${{ github.event.pull_request.head.sha }}
    fetch-depth: 0  # Or increase depth for better reliability

This ensures the workflow always checks out the exact commit that triggered the synchronize event.

Option 2: Change Trigger to Manual/Label-Based

on:
  pull_request:
    types: [opened, ready_for_review]  # Remove 'synchronize'
  # Or use label-based trigger
  pull_request:
    types: [labeled]

Then only trigger reviews when:

• PR moves from draft to ready
• A specific label like "needs-review" is added

Option 3: Review Only Changed Files

Add to the direct_prompt:

direct_prompt: |
  Please review ONLY the files that were changed in this PR commit.
  Focus your review on the diff, not the entire file state.
  
  Use: git diff ${{ github.event.pull_request.base.sha }} ${{ github.event.pull_request.head.sha }}

Option 4: Use Sticky Comments + Debouncing

use_sticky_comment: true  # Already commented out in config

Combined with a debounce action to prevent multiple rapid reviews.

Option 5: Add Concurrency Control

concurrency:
  group: claude-review-${{ github.event.pull_request.number }}
  cancel-in-progress: true

This cancels in-progress reviews when new commits arrive.

Related Patterns in the Codebase

The repo uses several other Claude workflows:

• claude-ontology-review.yml
• claude-issue-triage.yml
• claude-issue-summarize.yml
• post-review-agent.yml

Consider applying similar fixes to these if they experience timing issues.

Context from Example

The referenced workflow run (#21074706453) was triggered by PR #77 (weekly compliance update for Clostridioides difficile infection). This is an automated weekly PR, so multiple rapid commits during the compliance update process may be triggering the race condition.

Implementation Priority

Given this affects automated weekly compliance PRs that likely have multiple sequential commits, I'd recommend:

1. Short term: Add explicit PR head ref checkout (Option 1) + concurrency control (Option 5)
2. Medium term: Consider whether synchronize events are needed at all for automated bot PRs - maybe filter by author
3. Long term: Implement sticky comments to update the same review comment rather than creating new ones

Additional Considerations

The workflow currently has commented-out filtering options for:

• Specific file paths (lines 7-11)
• PR author filtering (lines 16-19)
• Skip conditions (lines 75-77)

These could be combined with the above solutions for more granular control over when reviews run.

[cmungall]

It is a bit of a mad max pile on at times, I will admit