Calculating Site Risk Scores

Overview

This vignette describes how site-level risk scores are calculated using Key Risk Indicators (KRIs), their associated flag severities, and predefined weights. The risk scoring framework provides a standardized, quantitative way to highlight sites with elevated risk.

Risk Indicators and Flags

Each KRI (e.g., Adverse Events, Serious Adverse Events, Protocol Deviations) can generate flags of different severity:

• Low Amber
• Low Red
• High Amber
• High Red

Each flag type is assigned a weight that reflects its contribution to site risk.

Flag Weights for Key Risk Indicators

KRI	Low_Red	Low_Amber	High_Amber	High_Red
Adverse Event	32	16	1	2
Serious Adverse Event	8	0	4	8
Protocol Deviations	8	4	8	16
Important Protocol Deviations	0	0	16	32
Labs	0	0	1	2
Query Rate	0	0	1	2
Delayed Query Rate	0	0	1	2
Delayed Data Entry Rate	0	0	1	2
Data Change Rate	0	0	1	2
Screen Failure	0	0	8	16
Treatment Discontinuation	0	0	16	32
Study Discontinuation	0	0	16	32

Example Site Calculation

For each site, the observed flags are multiplied by the corresponding weights and summed to produce a site risk score.

Below is a worked example for one site:

Example Site Contributions to Risk Score

KRI	Contribution
Adverse Event	32
Serious Adverse Event	0
Protocol Deviations	4
Important Protocol Deviations	8
Labs	0
Query Rate	1
Delayed Query Rate	0
Delayed Data Entry Rate	0
Data Change Rate	0
Screen Failure	2
Treatment Discontinuation	0
Study Discontinuation	0

These weights are added up at the site level and divided by the maximum possible score to create a normalized risk score between 0 and 100. This normalized site risk score is what is reported in the KRI report, and if desired, any sites above a certain threshold can be flagged for further review.

normalized_score <- (sum(site_example$Contribution)/178) * 100
normalized_score
#> [1] 26.40449

Benchmarking Across Sites

Site-level scores can be contextualized in two ways:

• Relative ranking: comparing each site’s score against other sites (e.g., percentile, rank order).

• Temporal comparison: computing scores across multiple months to understand site-level fluctuations.

  Conclusion

This scoring framework provides an interpretable, quantitative measure of risk at each site. By combining information across KRIs and weighting severity appropriately, it allows data monitoring teams to identify high-risk sites and prioritize monitoring resources effectively.