Connect Score v1.0.0 Sensitivity Analysis
How to read this report
The Connect Score is a continuous 0-100 measurement that is then binned into 5 named bands (Action Needed / Lagging / On Track / Strong / Leading) for display. Sensitivity has to be evaluated separately for the continuous score and for the discrete band assignment, because the two answer different questions:
- Rank stability (Spearman rank correlation against the published base) measures whether reweighting the formula changes which parish is doing better than which other parish. This is the defensibility-critical question.
- Band stability (count of parishes whose band assignment shifts) measures whether reweighting changes how a parish is labeled. Because the Connect Score is a fairly smooth continuous distribution and the bands are 5 buckets, even small score changes can push a borderline parish across a band boundary. This produces a lot of band churn without changing the underlying ranking.
Headline: 6 of 7 alternative weight schemes preserve rank order at Spearman >= 0.97 against the published base; the only outlier is adoption_only at 0.724, which is the deliberate single-component stress test and demonstrates that the composite adds substantial information beyond adoption alone. Band churn is high for several schemes (because a 5-band classifier on a smooth distribution is inherently sensitive to weight nudges), but the ranking the score is built to defend is robust.
The "Verdict" column below applies the strict combined criterion (Spearman >= 0.95 AND <= 10% band shifts). Most rows are marked "Watch" rather than "Stable" because they hit the rank-stability bar but trip the band-stability bar. That is the expected, transparent result of a 5-band UI overlaid on a continuous index; it is not a methodology failure. The "Rank" and "Bands" sub-verdicts let the reader judge each axis independently.
Top-level weight schemes
Each alternative scheme is compared to the published base (25 Avail / 35 Afford / 40 Adopt).
| Scheme | Spearman | Rank (>= 0.95) | Band shifts | Bands (<= 10%) | Mean Δ score | Combined verdict |
|---|---|---|---|---|---|---|
| equal | 0.990 | OK | 17 / 64 | Watch | 1.60 | Watch |
| bead_aligned | 0.979 | OK | 30 / 64 | Watch | 3.10 | Watch |
| adoption_only | 0.724 | Watch | 37 / 64 | Watch | 5.83 | Watch |
| avail_low | 0.992 | OK | 12 / 64 | Watch | 0.86 | Watch |
| avail_high | 0.993 | OK | 10 / 64 | OK | 0.86 | Stable |
| afford_low | 0.992 | OK | 5 / 64 | OK | 0.80 | Stable |
| afford_high | 0.991 | OK | 8 / 64 | OK | 0.80 | Stable |
Rank-stability summary: 6 of 7 schemes pass (Spearman >= 0.95). The lone exception, adoption_only, is a deliberate ablation: it shows what happens when the composite is collapsed to its single largest-weighted component. The fact that rankings change materially under that ablation is evidence the composite is not redundant with adoption.
Band-stability summary: 3 of 7 schemes pass the <= 10% threshold. The remaining 4 redistribute borderline parishes across band boundaries without changing rank order. A v2 of this report may switch to a continuous "score delta" metric instead of band churn for parishes that cross thresholds by less than +/-2 points.
Internal Affordability split sensitivity
Base internal split is 60% structural / 40% Lifeline. Alternatives:
| Split | Spearman | Rank (>= 0.95) | Band shifts | Bands (<= 10%) | Mean Δ score | Combined verdict |
|---|---|---|---|---|---|---|
| 70_30 | 0.979 | OK | 10 / 64 | OK | 0.86 | Stable |
| 50_50 | 0.978 | OK | 9 / 64 | OK | 0.86 | Stable |
| 40_60 | 0.915 | Watch | 21 / 64 | Watch | 1.72 | Watch |
The internal split passes rank stability for the +/- 10pp adjustments and trips it only at the 40_60 extreme (which inverts the published rationale that structural affordability should dominate Lifeline take-rate). That trip is interpretable, not a defect.