Exercise IV-A. The Uncertainty Cascade Assessment
Part IV. Systemic Risk and Uncertainty: Feedback Loops
Part IV. Systemic Risk and Uncertainty: Feedback Loops
Chapters 10, 11 and 12 trace a progression from propagation to absence to recursion: uncertainty that moves through a decision system, risk that never enters it, and decisions that change the evidence the system later learns from. Chapter 10 specifies an Uncertainty Cascade Assessment in section 10.10 and sensitivity, ensemble and traceback testing in 10.11. Chapter 11 specifies an Unmapped Risk Assessment in 11.10 and testing the unknown in 11.11. Chapter 12 specifies the GeoAI Feedback Reflex Assessment in 12.10 and intervention aware validation in 12.11.
Fundamentals in play. These exercises use Monte Carlo propagation of published margins of error, random seeds and reproducibility, rank stability under simulated sampling error, and iterated simulation. Each fails in a known way. A Monte Carlo result is only as good as the distribution assumed for each input, an unstated seed makes a result irreproducible while looking rigorous, rank stability measured under one error model says nothing about the errors that model leaves out, and an iterated simulation whose sampling depends on its current estimate chases whatever its early observations happened to favor, concentrating effort far above the correct level without settling.
| At a glance | |
|---|---|
| Textbook sections | sections 10.2, 10.3, 10.4, 10.5, 10.10, 10.11, 10.12, 10.13, 15.7.2 |
| Technical demand | Tier 1 and Tier 2 required. Python with pandas, numpy and matplotlib |
| Effort | three hours including the write up |
| Prerequisites | Exercises II-A and III-B |
Overview
Section 10.3 describes the anatomy of an uncertainty cascade, in which uncertainty entering at one of six linked stages passes downstream, where later stages inherit, transform, amplify or mask it. Section 10.4 names spatial overconfidence, in which precision outruns evidence. This exercise runs a cascade end to end on published margins of error and reports what arrives at the far end.
The chain has three links. Published estimates carrying published margins of error feed a composite index, the index feeds a threshold that selects qualifying tracts, and the selection feeds a population count of the kind that appears in a funding request. Students draw a thousand realizations, sampling each of the three numerators from its own published error distribution while holding the denominators and the tract populations at their published estimates, and report the fifth to ninety fifth percentile range of that final count alongside the single number the point estimate produced. The single number masks the interval, which section 10.3.4 calls masking, and the tract by tract selection examined in Step 3 claims a geographic precision that section 10.4 calls spatial overconfidence.
Section 10.10 organizes the Uncertainty Cascade Assessment around six dimensions, being Source, Transformation, Dependency, Sensitivity, Visibility and Consequence. The steps below exercise five of them. The sampling block identifies the Source, the index and the threshold carry the Transformation, Steps 2 and 3 measure Sensitivity, the comparison between the point estimate and the interval measures Visibility, and the funding count in Step 6 carries the Consequence. The program draws each numerator independently, which is the treatment the Dependency dimension exists to question, and the handoff asks what correlated errors would do to the interval.
Step by step
- Establish the point estimate. Tier 1. Change to the toolkit folder inside GeoAI_Exercises and type python exercise_4a.py. Record the qualifying tract count and the population inside them under the published estimates with no simulation. This is the number that would appear in a report.
- Run the cascade. Tier 1. Record the fifth, fiftieth and ninety fifth percentiles of the population count across a thousand realizations with the stated seed. State the width of that interval as a percentage of the point estimate.
Figure. Histogram of the final population count across a thousand realizations, with the point estimate marked and the fifth to ninety fifth percentile band shaded.
- Test rank retention. Tier 1. Record the probability that the tract ranked first under the point estimate ranks first in a realization, and record how many distinct tracts appear in the selected set at least once. Section 10.4 defines spatial overconfidence as apparent geographic precision that exceeds the precision, coverage or stability of the supporting evidence, and these two numbers measure the stability of the tract selection.
- Demonstrate reproducibility. Tier 1. Block 3 reruns one realization twice with the stated seed and once with seed 1. Confirm the first two match exactly, record the change the third produces, and state why an analysis that omits its seed is irreproducible even when its code is published.
- Build the ensemble. Tier 2. Section 10.11 asks for ensembles. Block 4 recomputes the qualifying set from the point estimates under the three index weightings listed in
WEIGHTINGSat the top of exercise_4a.py and reports the union and intersection of those three sets. Record both, then add a fourth weighting of your own choosing toWEIGHTINGS, rerun, and report how the union and intersection change. State which tracts qualify under every configuration. Section 10.11 treats agreement across an ensemble as a sign of a stable conclusion, and section 10.5 warns that configurations built on the same inputs supply much less independent evidence, so these tracts have survived the weighting choice and nothing more.
- Communicate it. Tier 1. Section 10.12 warns against communicating uncertainty in ways that paralyze decisions. Write the two sentences you would put beneath the number in a funding request, conveying the interval without inviting the reader to discard the analysis.
- Read the code and complete the handoff. Tier 1. Annotate the sampling block, including the division that converts a margin of error to a standard deviation, and answer the decisions file.