Scalable R Package Qualification Using a CI/CD Approach
{qcthat} generates qualification reports from your test suite
R package qualification is necessary but challenging
R packages might do what you need, but you’ll need proof
- Many R packages are developed on GitHub
- Lots of information here, but not clear how it connects
Qualification is often a manual, retrospective burden
- Qualification documents often created after development
- Development team manually maps specifications to functions and evidence
- Repeated for every new release
“Validation debt” delays releases and increases risk
- Deferred qualification: harder to reconstruct what was tested and why
- Manual qualification docs drift out of sync with code
- Teams avoid using updates to dodge re-qualification
The standard R package lifecycle generates evidence but lacks traceability
Modern R packages use {testthat} for robust automated testing
- Industry-standard testing framework
- Confirms functions behave as expected and keep working after changes
- Activate with
usethis::use_testthat()
CI/CD pipelines ensure code integrity on every commit
- GitHub Actions run tests on every push (
usethis::use_github_action())
- Failures caught long before code reaches production
- Standard practice for well-maintained R packages
Standard {testthat} scripts provide the evidence of implementation
- Every test run produces machine-readable results: pass, fail, skip, or warning
- Proves that code works
But: A passing test suite doesn’t prove which requirement was met
{testthat} shows what was tested, not why- No built-in link between GitHub issue and its tests
- Reviewers must manually reconstruct the mapping
{qcthat} connects issues to tests for continuous qualification
Connecting tests to issues is straightforward
- Add
(#123) or (#84, #132) (issue numbers) to test_that() description to tag test to issue
![]()
{qcthat} Reports are easy to generate
QCPackage() generates a full report linking issues to tagged tests- Organized by milestone; tests nested under linked issues
- Filter to subsets:
QCCompletedIssues(), QCPR(), QCMilestones()
The “Issue-Test Matrix” shows the connections between issues and tests
![]()
User Acceptance Testing (UAT) is integrated directly into tests
ExpectUserAccepts() creates GitHub issues requiring manual sign-off
![]()
User Acceptance Testing (UAT) is integrated directly into tests
- UAT issues = children of originating requirement issue
- UAT issue closed → test passes
![]()
Automated bots bring qualification reports directly to the Pull Request
- GitHub Action comments on every PR with reports:
- PR-Associated Issues
- Completed Issues
- Milestone
- UAT
- Reviewers see qualification status before merging
- Reports update automatically as the PR changes
Immutable artifacts are automatically attached to GitHub Releases
- Completed Issues and Milestone reports embedded in GitHub Release description
- Each release carries its own qualification evidence
- Version-controlled and tamper-evident by design
Packages are qualified continuously with every change
- Qualification as a byproduct of development, not a separate phase
- Developers tag tests to issues as they’re written
- Reports identify untested issues and unlinked tests
You can add {qcthat} to your package today
Setup takes minutes, not hours
pak::pak("Gilead-BioStats/qcthat")qcthat::use_qcthat()- Tag your tests:
test_that("Thing works (#123)", ...)
- Experimental:
qcthat::Skill_TagTestsWithIssues()
- Tell an agent “tag tests with issues”
