[prompt-analysis] Copilot PR Prompt Pattern Analysis - 2026-01-21

[github-actions[bot]]

Summary

Analysis Period: Last 30 days
Total PRs: 1000 | Merged: 646 (64%) | Closed: 350 (35%)

Prompt Categories and Success Rates

Category	Total	Merged	Success Rate
Refactor	94	70	75%
Testing	400	281	71%
Feature	704	480	68%
Documentation	246	162	66%
Bug Fix	532	337	64%
Other	52	27	52%

Prompt Analysis

✅ Successful Prompt Patterns

Common characteristics in merged PRs:

• Average prompt length: 224 words
• Most common keywords: workflow, changes, github, added, updated, test
• Action verbs used: added, updated, implemented, fixed

Top distinctive keywords in merged PRs:

• "added" (3.8x more frequent than closed PRs)
• "updated" (3.4x more frequent)
• "changes" (2.8x more frequent)
• "test" (2.8x more frequent)

Example Successful Prompts

1. PR #10955: Campaign discovery returned 0 items because worker workflows had no mechanism to record their outputs for campaign tracking. Implemented a cache-memory based approach where campaigns read from existin... → ✅ Merged

2. PR #10948: ## Analysis and Improvement of Agentic-Campaign Template References in sync-templates ### Completed - [x] Analyzed sync-templates target in Makefile - [x] Identified 6 agentic-campaign templates being... → ✅ Merged

3. PR #10943: JavaScript module requires were failing with Cannot find module '/tmp/gh-aw/scripts/update_activation_comment.cjs' because ScriptsBasePath constant pointed to /opt/gh-aw/scripts while the setup ... → ✅ Merged

4. PR #10940: The safeoutputs MCP server running at the gateway lacked access to GitHub Actions context (workflow run, actor, event, ref information). This prevented context-aware operations that depend on the curr... → ✅ Merged

5. PR #10935: The safeoutputs MCP server receives GITHUB_WORKSPACE as an environment variable but cannot access workspace files because the directory isn't mounted in the container. ## Changes - **Added workspace... → ✅ Merged

❌ Unsuccessful Prompt Patterns

Common characteristics in closed PRs:

• Average prompt length: 188 words (shorter than merged)
• Most common keywords: copilot, coding, make
• "copilot" appears 1.8x more in closed PRs
• "coding" appears 6.8x more in closed PRs

Example Unsuccessful Prompts

1. PR #10952: Analysis of three security workflows (code-scanning-fixer, security-fix-pr, security-review) reveals Code Scanning Fixer experiencing 40% failure rate while other workflows operate nominally. ## Deliv... → ❌ Closed

2. PR #10947: Analyzed campaign run 21188591703 to verify the Security Alert Burndown orchestrator is functioning correctly. ## Analysis Results **Sta... → ❌ Closed

3. PR #10938: Thanks for assigning this issue to me. I'm starting to work on it and will keep this PR's description up to date as I form a plan and make progress. - Fixes githubnext/gh-aw#10888 --- 💡 You can make C... → ❌ Closed

4. PR #10937: Issue Triage Agent workflow failed with Error: missing finish_reason for choice 0 - a transient Copilot API error occurring during response finalization after successful task completion (agent ran 3... → ❌ Closed

5. PR #10936: Workflow run #21186843566 failed with Error: missing finish_reason for choice 0 from Claude Sonnet 4.5. ## Investigation Findings Root Cause: Transient API error in Claude completion response ... → ❌ Closed

Key Insights

• 📊 Refactoring has the highest success rate (75%), followed by testing (71%) and feature additions (68%)
• 📝 Longer prompts correlate with success: Merged PRs average 224 words vs 188 words for closed PRs
• 🔑 Action verbs matter: PRs using "added", "updated", "implemented" have higher merge rates
• ⚠️ Meta-discussions fail: PRs mentioning "copilot" or "coding" (self-referential) have lower success rates

Recommendations

Based on today's analysis:

1. DO: Write detailed prompts (200+ words) with specific context about what changed and why
2. DO: Use concrete action verbs ("added X", "fixed Y", "implemented Z") rather than vague descriptions
3. DO: Include test changes when applicable - testing-related PRs have 71% success rate
4. AVOID: Self-referential language ("this PR", "copilot", "coding agent") - focus on the technical changes
5. AVOID: Analysis-only PRs without concrete changes - these tend to get closed

Historical Trends

Date	PRs	Success Rate	Top Category
2026-01-21	1000	65%	Feature
2026-01-20	1000	65%	N/A
2026-01-19	1000	67%	N/A
2026-01-18	1000	68%	Testing
2025-12-26	1000	74%	Testing
2025-12-25	1000	74%	N/A

Trend: ➡️ Stable: Success rate holding steady around 65%

---

References:

• Workflow run: §21208716151

AI generated by Copilot PR Prompt Pattern Analysis

• expires on Jan 28, 2026, 12:04 PM UTC