The Bias Score is a new Zindi feature built in collaboration with Humane Intelligence and Radiant Earth, supported by Heising-Simons Foundation and launched in August 2026 with the Bias Bounty Mapping Equity Challenge. Standard data science challenges score submissions on accuracy alone. The Bias Score asks a different question: accurate for whom? When you make a submission to a challenge that uses it, you get two things back: your accuracy score (RMSE in the Bias Bounty Mapping Equity Challenge), which determines your leaderboard position, and your bias scorecard. It requires no work on your part: every submission is run through it automatically. The Bias Score does not affect your ranking in any way. This post explains what it is, how it works, and why it is there.

How it works

Your submission contains a coverage gap score for every tract in the challenge. The Bias Score joins those scores against a preloaded tract-level lookup of demographic, vulnerability, and hazard attributes, then computes disparity metrics across predefined strata: urban vs. rural, high-SVI vs. low-SVI, high-CVI vs. low-CVI, tribal vs. non-tribal, and high vs. low exposure to heat, wildfire, and drought.

The core idea is a disparity ratio: the mean coverage gap of the disadvantaged group divided by the mean coverage gap of the reference group. A ratio above 1 means the disadvantaged group has worse coverage. For example, a ratio of 2 would mean coverage gaps in rural tracts are, on average, twice as large as in urban tracts.

What the scorecard shows

Five equity metrics, each broken down by stratum:

• Coverage Disparity Ratio: ratio of mean coverage gap between the disadvantaged group and the reference group; above 1 means the disadvantaged group has worse coverage.
• POI Desert Index: share of tracts whose points-of-interest are almost entirely unmapped (poi_gap at or above the threshold, currently 0.8); reported overall and split rural vs. urban, so a higher rural share means essential places are missing from the map disproportionately in rural communities.
• Emergency Access Gap: rural vs. urban ratio of the mean emergency-facility gap; above 1 means critical facilities in rural tracts are less completely mapped.
• Road Network Equity Ratio: ratio of mean road network gap between groups; above 1 means the disadvantaged group's road network is less completely mapped.
• Climate-Justice Composite: the mean coverage gap of the tracts that are both climate-exposed and socially vulnerable (the intersectional group); higher means the most-vulnerable communities are more poorly mapped.

You are not judged on it

The Bias Score does not affect your leaderboard ranking. The leaderboard is decided by the accuracy metric alone. The Bias Score exists for educational, research, and applied remediation purposes: it shows not only how well your model performs, but where and for whom it performs better or worse, it supports research on mapping equity, and it points to where future mapping effort should go. You are strongly encouraged to reflect on it in your methodology writeup.

The same strata tables used to compute the Bias Score are included in the data package, so you can explore these dimensions yourself. Until now, this kind of equity analysis has required custom scoring rubrics or fairness tooling applied by hand in a research notebook. The Bias Score turns it into platform infrastructure that runs automatically alongside accuracy, and after this pilot it will be available for future Zindi challenges in similar domains.