Skip to content

Modes of Operation

Jump to bottom
Steve Gilham edited this page Dec 4, 2021 · 18 revisions

AltCover can be used in a variety of ways, from being almost like the original NCover or current OpenCover, where everything is done all in one, to a complete separation of the stages of instrumentation, testing and then reporting.

Contents

This is a non-exhaustive list of combinations

In the following, when I say altcover, that will mean either <path to>/AltCover.exe or dotnet <path to>/AltCover.dll unless explicitly specified otherwise.

NOTE tl;dr -- "Classic" mode is really only useful for the old .net framework or Mono.

On old-fashioned .net framework, the ProcessExit event handling window of ~2s is sufficient for processing significant bodies of code under test (several 10s of kloc, as observed in production back in the '10-'11 timeframe); under dotnet test the vstest.console process imposes a 100ms guillotine, even though .net Core imposes no time-limit of its own. This is about enough time to fill in an NCover report for a program of no more than 1kloc, hence the development of a "write it all promptly to file and post-process" Runner mode. With version 5.3 and above, the dotnet test integration now hooks the VSTest in-process data collection, allowing an indefinite window to write collected data from memory, thus removing the file I/O bottleneck.

Instrument and test now, classic mode

  1. Run altcover to instrument your binaries, using any of the options up to and including --opencover, plus any of [--single] [--linecover|branchcover], with the test command line being specified after a -- to execute the tests on the assemblies in the new location, and rely on the ProcessExit handler to write from memory into the XML file.

This is closest in style to running with OpenCover or the old NCover, and is sufficient for moderately large coverage sets in the full .net framework, where the handler has a couple of seconds to work.

Disadvantages

  • requires more work for dotnet test because the instrumented assemblies are somewhere else (use --inplace for instrumentation or redefine the build output directory /p:OutputPath= to point at the instrumented location for the test run). It will be necessary to use --no-build, of course, and --settings to pick up a collector runsettings file to mitigate the short shutdown window allowed
  • only a subset of OpenCover format (sufficient for basic use with coveralls.io and ReportGenerator) is produced

RunSettings template

If you want to use the AltCover data collector for classic-mode with dotnet test (e.g. after using the global-tool AltCover version for instrumentation) use this template, filling in the major & minor version numbers and the path

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
  <InProcDataCollectionRunSettings>
    <InProcDataCollectors>
      <InProcDataCollector friendlyName="AltCover" uri="InProcDataCollector://AltCover/Recorder/#.#.0.0" assemblyQualifiedName="AltCover.DataCollector, AltCover.DataCollector, Version=#.#.0.0, Culture=neutral, PublicKeyToken=c02b1a9f5b7cade8" codebase="fully qualified path to AltCover.DataCollector.dll">
        <Configuration>
          <Offload>true</Offload>
        </Configuration>
      </InProcDataCollector>
    </InProcDataCollectors>
  </InProcDataCollectionRunSettings>
</RunSettings>

though just using the dotnet test integration via the standard package, which automates this for you, is simplest.

For a worked example, see self-test build target.

Instrument now, test later, classic mode

  1. Run altcover to instrument your binaries, using any of the options up to and including --opencover, plus any of [--single] [--linecover|branchcover]
  2. Execute the tests on the assemblies in the new location (/outputDirectory), and rely on the ProcessExit handler to write from memory into the XML file.

This is sufficient for moderately large coverage sets in the full .net framework, where the handler has a couple of seconds to work

Disadvantages

  • sort-of works for dotnet test, with the same limitations as above
  • only a subset of OpenCover format (sufficient for basic use with coveralls.io and ReportGenerator) is produced

Instrument and test now, runner mode

  1. Run altcover to instrument your binaries, using any of the options up to and including --opencover, plus any of [--single] [--linecover|branchcover]; then specify altcover runner after a -- with the test command line being specified after yet another -- to execute the tests on the assemblies in the new location (/outputDirectory).

This is close in style to running with OpenCover or the old NCover, and will produce all the supported OpenCover format output

Disadvantages

  • sort-of works for dotnet test, with the same limitations as above
  • cumbersome command line

Instrument in place and test now, runner mode

  1. Run altcover to instrument your binaries, using any of the options up to and including --opencover, plus any of [--single] [--linecover|branchcover], also specifying --inplace; then specify altcover runner after a -- with the test command line being specified after yet another -- to execute the tests on the assemblies in the built location.

This is close in style to running with OpenCover or the old NCover, and will produce all the supported OpenCover format output

Disadvantages

  • your build artifacts are saved off, so many not be where your build pipeline wants them
  • sort-of works for dotnet test, with the same limitations as above
  • cumbersome command line

Instrument now, test later, runner mode

  1. Run altcover to instrument your binaries, using any of the options up to and including --opencover, plus any of [--single] [--linecover|branchcover]
  2. Run altcover runner to fill in the XML report with the test command arguments being specified after a -- to execute the tests on the assemblies /outputDirectory unless --inplace was used.

With --inplace as part of step 1, this will work with dotnet test

Disadvantages

  • With --inplace your build artifacts are saved off, so many not be where your build pipeline wants them
  • dotnet test -- still needs --settings

Instrument now, test later, collect coverage after that

  1. Run altcover to instrument your binaries, using any of the options up to and including --opencover, plus any of [--single] [--linecover|branchcover], also specifying --save
  2. Execute the tests on the instrumented assemblies (in /outputDirectory unless --inplace was used)
  3. Run altcover runner --collect -r <path to instrumented folder> to fill in the XML report

This is the style of operation of coverlet; and with --inplace as part of step 1, this will work with dotnet test in exactly the same way.

Disadvantages

  • multiple steps
  • With --inplace your build artifacts are saved off, so many not be where your build pipeline wants them
  • dotnet test -- still needs --settings

Clone this wiki locally