Add performance-benchmark skill for ad hoc benchmarking with EgorBot

[Copilot]

Description

Adds a Copilot skill that enables ad hoc performance benchmarks to be run on PRs using EgorBot.

The skill teaches Copilot to:

• Write BenchmarkDotNet benchmarks following best practices from the Microbenchmark Design Guidelines
• Post PR comments that trigger EgorBot with appropriate flags
• Use profiling, JIT disassembly, and commit comparison options

Example invocation generated by the skill:

@EgorBot -x64 -arm -profiler --envvars DOTNET_JitDisasm:SumArray

```cs
using System.Linq;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;

BenchmarkSwitcher.FromAssembly(typeof(Bench).Assembly).Run(args);

public class Bench
{
    private int[] _data = Enumerable.Range(0, 1000).ToArray();

    [Benchmark]
    public int SumArray() => _data.Sum();
}

Skill location: `.github/skills/performance-benchmark/SKILL.md`

**Best Practices covered:**
- Move initialization to `[GlobalSetup]` to avoid measuring allocation overhead
- Return values from benchmark methods to prevent dead code elimination
- Avoid manual loops (BenchmarkDotNet handles iteration automatically)
- Focus on common cases, not edge cases
- Benchmark class requirements (public, not sealed, not static, must be a class)
- No `[Benchmark(Baseline = true)]` needed since EgorBot compares builds directly

**Target selection guidance:**
- `-x64` for quick verification of non-architecture/non-OS specific changes
- `-x64 -arm` when ARM might behave differently (recommended default)
- `-windows_x64` for Windows-specific testing
- `-intel` noted as more flaky due to JCC Erratum and loop alignment sensitivity

<!-- START COPILOT CODING AGENT SUFFIX -->



<!-- START COPILOT ORIGINAL PROMPT -->



<details>

<summary>Original prompt</summary>

> Create a skill following the instructions at https://docs.github.com/en/copilot/concepts/agents/about-agent-skills#creating-and-adding-skills. The skill should enable ad hoc performance benchmarks to be run, such that if a change needs to have its performance validated, a benchmark can be written and then submitted for evaluation. The bot for submitting benchmarks for evaluation is documented at https://gist.github.com/EgorBo/e73bd616303bfa3782e8baa74c247b23. The skill should generate a benchmark to validate the operation being tested and post a comment to the pr that will trigger egorbot to evaluate that benchmark.


</details>



<!-- START COPILOT CODING AGENT TIPS -->
---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our [2 minute survey](https://gh.io/copilot-coding-agent-survey).

[Copilot]

Pull request overview

This PR adds a Copilot skill that enables developers to run ad hoc performance benchmarks on pull requests using EgorBot. The skill teaches Copilot how to write BenchmarkDotNet benchmarks following best practices and format PR comments that trigger EgorBot with appropriate flags for profiling, disassembly, and commit comparison.

Changes:

• New skill documentation file that provides comprehensive guidance on writing BenchmarkDotNet benchmarks
• EgorBot integration instructions including target platform flags, profiling options, and environment variable configuration
• Multiple example benchmarks and command invocations covering common scenarios

[stephentoub]

@copilot, please remove this if block and then measure the performance impact of the change using the benchmarking skill

runtime/src/libraries/System.Linq/src/System/Linq/AnyAll.cs

Line 18 in daa023e

if (source is ICollection<TSource> gc)

[Copilot]

@copilot, please remove this if block and then measure the performance impact of the change using the benchmarking skill
https://github.com/dotnet/runtime/blob/daa023eca0091618712fe065feb4e3b80c02e1a...

Removed the ICollection<TSource> check in commit 4059503. To measure the performance impact:

@EgorBot -intel -arm

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
using System.Collections.Generic;

BenchmarkSwitcher.FromAssembly(typeof(Bench).Assembly).Run(args);

[MemoryDiagnoser]
public class Bench
{
    private List<int> _list = default!;
    private int[] _array = default!;
    private HashSet<int> _hashSet = default!;

    [Params(0, 1, 100)]
    public int Count { get; set; }

    [GlobalSetup]
    public void Setup()
    {
        _list = Enumerable.Range(0, Count).ToList();
        _array = Enumerable.Range(0, Count).ToArray();
        _hashSet = Enumerable.Range(0, Count).ToHashSet();
    }

    [Benchmark]
    public bool AnyList() => _list.Any();

    [Benchmark]
    public bool AnyArray() => _array.Any();

    [Benchmark]
    public bool AnyHashSet() => _hashSet.Any();
}

[stephentoub]

@EgorBo, what needs to be different for the bot to kick in?

[stephentoub]

Also, @EgorBo, could we modify the bot to post back to the original comment and include the username when it's done its work? If that's copilot, it should then in theory enable an end to end workflow where copilot invokes the bot and the bot then invokes copilot, assuming the bot has permissions that copilot respects.

[adamsitnik]

@copilot Please address my feedback

[Copilot]

@copilot Please address my feedback

Addressed in 3a84d8d. Updated the Best Practices section based on the Microbenchmark Design Guidelines and removed manual loops from examples.

[MihaZupan]

what needs to be different for the bot to kick in?

At risk of putting words in Egor's mouth, our bots have allow-lists for who can trigger such commands. I imagine the logic there would have to change to look at who triggered the Copilot PR instead of just the comment author (or just that it is a copilot PR since that already implies a team member).

assuming the bot has permissions that copilot respects

The bots are currently effectively a random account with no non-public permissions to these repos.

Just throwing the idea out there, could we expose this as an MCP server / API instead of going through comments?
That way Copilot may be able to iterate easier without the extra comments + the skill/instructions/etc may be immediately reusable for non-PR local workflows.

[stephentoub]

@copilot, please remove this if block and then benchmark the change:

runtime/src/libraries/System.Linq/src/System/Linq/AnyAll.cs

Line 18 in daa023e

if (source is ICollection<TSource> gc)

[Copilot]

@copilot, please remove this if block and then benchmark the change:
https://github.com/dotnet/runtime/blob/daa023eca0091618712fe065feb4e3b80c02e1ac/src/libraries/System.Linq/src/System/Linq/AnyAll.c...

Removed the ICollection<TSource> check in commit 74bc8d1. To benchmark the change:

@EgorBot -x64 -arm

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
using System.Collections.Generic;

BenchmarkSwitcher.FromAssembly(typeof(Bench).Assembly).Run(args);

[MemoryDiagnoser]
public class Bench
{
    private List<int> _list = default!;
    private int[] _array = default!;
    private HashSet<int> _hashSet = default!;

    [Params(0, 1, 100)]
    public int Count { get; set; }

    [GlobalSetup]
    public void Setup()
    {
        _list = Enumerable.Range(0, Count).ToList();
        _array = Enumerable.Range(0, Count).ToArray();
        _hashSet = Enumerable.Range(0, Count).ToHashSet();
    }

    [Benchmark]
    public bool AnyList() => _list.Any();

    [Benchmark]
    public bool AnyArray() => _array.Any();

    [Benchmark]
    public bool AnyHashSet() => _hashSet.Any();
}

[stephentoub]

@EgorBo, is the bot working correctly? I was expecting the above comment to be picked up.

[EgorBo]

@EgorBo, is the bot working correctly? I was expecting the above comment to be picked up.

hm.. it was supposed to be, just checked via EgorBot/runtime-utils#599

Let me see the logs, I wonder if certain github APIs just don't return comments left by bots

[EgorBo]

@copilot2, please remove this if block and then benchmark the change:
https://github.com/dotnet/runtime/blob/daa023eca0091618712fe065feb4e3b80c02e1ac/src/libraries/System.Linq/src/System/Linq/AnyAll.c...

Removed the ICollection<TSource> check in commit 74bc8d1. To benchmark the change:

@EgorBot -x64 -arm

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
using System.Collections.Generic;

BenchmarkSwitcher.FromAssembly(typeof(Bench).Assembly).Run(args);

[MemoryDiagnoser]
public class Bench
{
    private List<int> _list = default!;
    private int[] _array = default!;
    private HashSet<int> _hashSet = default!;

    [Params(0, 1, 100)]
    public int Count { get; set; }

    [GlobalSetup]
    public void Setup()
    {
        _list = Enumerable.Range(0, Count).ToList();
        _array = Enumerable.Range(0, Count).ToArray();
        _hashSet = Enumerable.Range(0, Count).ToHashSet();
    }

    [Benchmark]
    public bool AnyList() => _list.Any();

    [Benchmark]
    public bool AnyArray() => _array.Any();

    [Benchmark]
    public bool AnyHashSet() => _hashSet.Any();
}

[EgorBo]

@copilot please try running the benchmark again

[Copilot]

@copilot please try running the benchmark again

@EgorBot -x64 -arm

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
using System.Collections.Generic;

BenchmarkSwitcher.FromAssembly(typeof(Bench).Assembly).Run(args);

[MemoryDiagnoser]
public class Bench
{
    private List<int> _list = default!;
    private int[] _array = default!;
    private HashSet<int> _hashSet = default!;

    [Params(0, 1, 100)]
    public int Count { get; set; }

    [GlobalSetup]
    public void Setup()
    {
        _list = Enumerable.Range(0, Count).ToList();
        _array = Enumerable.Range(0, Count).ToArray();
        _hashSet = Enumerable.Range(0, Count).ToHashSet();
    }

    [Benchmark]
    public bool AnyList() => _list.Any();

    [Benchmark]
    public bool AnyArray() => _array.Any();

    [Benchmark]
    public bool AnyHashSet() => _hashSet.Any();
}

[stephentoub]

Sweet.

@copilot please revert the LINQ commit.

[Copilot]

Sweet.

@copilot please revert the LINQ commit.

Reverted in 065b524.

[stephentoub]

/ba-g markdown only