Code Coverage: From Failing the Build To Publishing The Report With Gitlab Pages

By Sudhanshu Singh

Before going into how gitlab can help us in publishing code coverage reports and making build fail if the coverage is not upto the mark, let’s first go over some of the basic questions.

What is Code coverage?

Code coverage is a technique which tells us that how much of application code has been covered and exercised if we run our test cases. There are many different tools for different languages to measure the code coverage.I will be mainly focusing on Java in this blog.

Why Code Coverage Is Important?

Code coverage gives us insight on how effective our test cases are, what parts of our source code are thoroughly executed, coverage report helps in finding untested part in your codebase. It helps us in increasing and maintaining the quality of code. But having a 100% test coverage doesn’t mean 100% tested because code coverage executed by the two test cases can be the same but input data of 1st test case can find the defect while input of 2nd test case cannot. There are also other metrics like cyclomatic complexity, lines of code, maintainability index and depth of inheritance etc which can also help you in making your code better.I will cover these topics in future blogs.

How to generate Code Coverage Report?

Now that’s just basic understanding of code coverage. Now let’s see how we can generate code coverage reports in java using gradle.

For generating coverage report in java we will use jacoco , it’s very easy to integrate with your codebase. If you are using gradle, you need to apply the Jacoco plugin in your build.gradle.

build.gradle
apply plugin: “jacoco”

After applying plugin a new task named jacocoTestReport is created which is dependent on test task. You just need to run gradle test jacocoTestReport and your coverage report is available at $buildDir/reports/jacoco/test. By default, HTML report is generated. But what about we want running test task to also generate the coverage report. For that you can use finalisedBy property provided by gradle. This will finalize test task after jacocoTestReport.

test {
    finalizedBy jacocoTestReport
}

You can also add JacocoReport task in your build.gradle, it can be used to generate code coverage reports in different formats.

build.gradle

jacocoTestReport {
    reports {
        xml.enabled false
        csv.enabled false
        html.destination "${buildDir}/jacocoHtml"
    }
}

Now what about we make our build fail if code coverage metrics are not upto the mark. This will help us in preventing untested code to go into production. You can do this by adding jacocoTestCoverageVerification task.

jacocoTestCoverageVerification {
    violationRules {
        rule {
            limit {
                minimum = 0.7
            }
        }
    }
}

0.7 states that your tests should cover minimum 70% of your codebase, violating that will fail the build. Now this requirement can be specified for the project as a whole, for individual files, and for particular JaCoCo-specific types of coverage, e.g., lines covered or branches covered. You can read more about it on gradle docs.

Since now we have code coverage report , how about we publish it on gitlab pages.

How to publish Report to Gitlab Pages

The configuration of gitlab CI is defined in .gitlab-ci.yml file. Let’s see how to write configuration for publishing report to gitlab pages.

Before publishing we need to first execute test and generates coverage report within a single job in the CI pipeline.

test:
  stage: test
  script:
    - ./gradlew test
  artifacts:
    paths:
      - build/reports/jacoco/

Everything inside build/reports/jacoco/ will be stored as artifact.We will use this artifact in our next job.

Let’s publish the coverage report we generated in our last job on gitlab pages. You need to add pages job in .gitlab-ci.yml.

pages:
  stage: deploy
  dependencies:
    - test
  script:
   - mkdir public
   - mv build/reports/jacoco/test/html/* public
  artifacts:
    paths:
      - public
  only:
  - master

This job will publish your test coverage report to gitlab pages. Not that you need to use pages as job name. Using dependencies keyword will tell gitlab to download artifacts stored as a part of test job. You also need to move coverage report to public directory because this is the directory that GitLab Pages expects to find static website in.Using only keyword will publish new coverage report only when CI pipeline runs on master branch.

Once you push your changes in .gitlab-ci.yml to Gitlab for the first time you will see new job in CI pipeline. When pages:deploy job is successfull you can access your coverage report using URL http://group-path.gitlab.io/project-path/index.html. The page should look something like:

If you reach till this point of the blog , then now you will be able to publish your code coverage report to gitlab pages.