GeezerVisonThe World Wide Web, restored to 800 pixels of glory. |
| Home|About|Gaming cabinet|Archives|How it works|Starter sites |
GeezerVison is showing docs.coveralls.io — Coveralls Builds | Coveralls Docs Get StartedIntegrating with CoverallsCoveralls BuildsThe Web AppThe APIGet HelpCoveralls BuildsIt’s important to know a little bit about Coveralls builds and build types in order to understand how to configure your CI service to work best with Coveralls: IntroductionCoveralls plugs into into your CI/CD toolchain. Once configured to use Coveralls:
Coveralls Builds ≈ CI BuildsBecause of this close relationship to your CI builds, Coveralls uses the same metaphor of a “build” to represent its primary unit of coverage reporting. However, a Coveralls build is not identical to a CI build. Instead, it’s a highly detailed coverage report for a CI build. Over time, Coveralls stores a history of coverage reports matching the history of your CI builds. Which Builds = ConfigurableIt’s possible to send coverage reports for some CI builds and not others, but Coveralls works best when you send it coverage reports for all your CI builds.* We make some recommendations below regarding the ideal CI configuration for reporting coverage to Coveralls; but first, it helps to know about the different types of Coveralls builds in order to understand those recommendations. * At least for all your
Coveralls Build TypesIf you’re familiar with CI services, you’ll be familiar with the two types of builds they generate:
Coveralls mirrors these same two build types and we also refer to them as push builds and pull request builds. But since Coveralls builds are coverage reports, let’s discuss how coverage reports can differ for these two types of builds, and what to consider when configuring your CI service to send coverage reports for them. Push BuildsIn Coveralls, push builds are simply coverage reports for a As with push builds in CI, Coveralls push builds are the most straightforward type of build because they represent coverage for a single, historical commit on a branch of your repo’s git tree. Tracking coverage changes for push buildsOne of the key value propositions of Coveralls is tracking how coverage changes on your project over time. In the case of push builds, Coveralls does this by comparing each push build to the previous push build on the same branch: In fact, Coveralls can compare coverage between any two push builds on your repo, which makes sending coverage reports for all your push builds the most flexible way to configure Coveralls. Pull Request BuildsIn Coveralls, pull request builds are coverage reports for a In contrast to push builds, pull request builds are more complicated because they are based on a temporary commit on an ephemeral branch of your repo called the merge branch, making these builds prospective builds, rather than actual, historical ones, like push builds. Tracking coverage changes for pull request buildsGenerating accurate coverage reports for pull request builds can be tricky due to the way most CI services generate these prospective builds for most use cases. For now, just know that Coveralls calculates the coverage changes for a pull request build by comparing the report for its current head commit (aka. the pull request build) to the report for its base commit, otherwise known as its base build:
COMMON ISSUE: “DRIFTED” BUILDS & TWO-DOT VS. THREE-DOT DIFF COMPARISONS IN COVERAGE REPORTSAs mentioned above, accurately calculating coverage reports for pull request builds is challenging, and the reason is the reliance on two-dot diff comparisons in generating pull requests, whereas three-dot diffs are necessary to calculate precise PR coverage. Key Concept:
This second scenario leads to what we call “drifted” builds:
Solutions for Drifted Builds:
With this configuration, sending Coveralls reports for Further Insights: The following section delves into the nuances of two-dot and three-dot diffs, helping you understand and address drifted build alerts in Coveralls. Two-Dot-Diff Comparisons (Default PR Build)In This is useful for understanding what the impact of a PR will be when it’s merged into the target branch, but it’s not as helpful for understanding the coverage changes introduced by a PR. Three-Dot-Diff Comparisons (Accurate PR Coverage Report)In In this way, a three-dot diff comparison You’ll recall, this is the same kind of diff Coveralls wants to make to calculate the change in coverage for a pull request build: Configuring your CI service for accurate PR coverage reportsWe recommend configuring your CI service in a manner that allows Coveralls to generate three-dot diff comparisons for all of your pull request builds, so we can always guarantee accurate coverage reports for them. That’s because looking at a coverage report for a two-dot diff comparison can be misleading, since it shows the coverage changes introduced by the PR, plus all changes that have occurred on the target branch since the PR was opened. Since those latter changes aren’t relevant to the PR’s changes, they can make it difficult to understand the coverage changes introduced by the PR. For details on how to configure your CI service to generate accurate coverage reports for Parallel BuildsParallel builds are a crucial build type in Coveralls, complementing the primary push and pull request builds. These builds are not mutually exclusive with push or pull request builds; any push or pull request build can also function as a parallel build. Definition: A parallel build aggregates multiple coverage reports into a single report for a specific commit. These individual reports are typically generated by various jobs or steps within a CI process, running concurrently. Each job covers a different segment of the project, and their reports are independently sent to the Coveralls API. Advantages: Parallel builds are particularly beneficial for large projects that can be segmented into smaller, independent parts. By analyzing coverage reports from these segments separately and then combining them, a comprehensive and unified coverage report for each build is produced. This approach enhances efficiency and can provide more detailed insights into the coverage of different parts of the project. Setting Up Parallel BuildsTo leverage parallel builds in Coveralls, you need to configure them within your CI setup, specifically in the workflow steps that interact with Coveralls. Detailed Configuration Guide: The complete setup process is outlined here. Integration with Core Build Types: Configuring a parallel build essentially involves expanding a core build type (push or pull request) into multiple steps. If your project requires parallel builds, this configuration applies to all builds, regardless of whether they are push or pull request builds. Considerations Based on Build Types:
Recommended CI ConfigurationsGiven that it can be difficult to guarantee accurate coverage reports for pull request builds—since they may be based on two-dot diff comparisons instead of three-dot diff comparisons—below are our recommended CI configurations to reliably receive accurate coverage reports at Coveralls. Beyond stating our ideal and minimum recommendations, for clarity we’ve also listed these recommendations by build type. K.I.S.S. RecommendationFollowing the K.I.S.S. principle, here’s our current recommendation in the simplest possible terms:
Ideal CI Configuration
Minimum CI Configuration
Sub-Minimum CI Configuration
By Build Type
Any problems, questions or comments about this doc? Let us know. |
| You are visitor 01402639 — GeezerVison does not store the pages you look up. Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required. |