Exercise IV-B. The GeoAI Feedback Reflex Assessment

Part IV. Systemic Risk and Uncertainty: Feedback Loops

Overview

Section 12.3 describes the anatomy of a self-fulfilling spatial error and section 12.4 names feedback laundering, in which intervention gets mistaken for validation. This exercise builds the loop on Mississippi tract geography with an event rate set identical in every tract, so any concentration the model produces is an artifact and nothing else.

Each iteration allocates a fixed inspection budget in proportion to the current risk score, observes events only where it inspected, and refits the score on observations accumulated so far. The effort concentrates within the first few iterations in whichever tracts happened to record early events, and it decays only partway by iteration 20. Students then add a reserved fraction of random allocation, sweep that fraction, and report the smallest value that holds the top decile share of effort within the tolerance the program states. That fraction is a design parameter with a defensible value, which converts a general warning about feedback into something an engineer can specify.

Section 12.10 organizes the GeoAI Feedback Reflex Assessment around five dimensions, being Intervention, Observation, Independence, Adaptation and Persistence. The simulation exercises the first three, because the budget is the intervention, events are observed only where effort goes, and the refit consumes observations that are no longer independent of the model. Adaptation and Persistence fall outside it, since nothing in the simulated tracts responds to inspection and the loop runs for twenty iterations only.

Step by step

1. Run the uncorrected loop. Tier 1. Change to the toolkit folder inside GeoAI_Exercises and type python exercise_4b.py. Record the share of the inspection budget falling in the top decile of tracts at iterations 1, 5, 10 and 20. The true rate is uniform, so any lasting concentration comes from the loop itself.

2. Map the outcome. Tier 1. Record the map of cumulative inspection effort at iteration 20 and describe the pattern, then state what a reviewer shown only that map, with no knowledge of the simulation, would conclude about the geography of the phenomenon.

3. Demonstrate the laundering. Tier 1. The program reports the correlation between each tract's cumulative inspection effort and its observed event rate, the indicator Table 12.10.5.1 lists as risk scores correlating with inspection intensity. Record it, note that it is high, and explain in one paragraph why agreement with observations the model chose to collect cannot separate genuine predictive validity from self confirmation. The true rate here is identical everywhere, so in this simulation the agreement reflects self confirmation alone. Section 12.5 explains why conventional validation can miss the loop.

4. Add exploration. Tier 2. Record the top decile share at iteration 20 under the reserved random fractions listed in EXPLORE at the top of exercise_4b.py, being none, ten, twenty and thirty percent. Identify the smallest fraction holding the share below the program's tolerance of 20 percent, set in TOLERANCE, then add a fraction of your own to EXPLORE, rerun, and refine the estimate. State whether you would defend that tolerance, and why.

Figure. Top decile share of effort against iteration, one line per exploration fraction, with the ten percent correct level drawn as a reference.

5. Validate the intervention aware way. Tier 3, optional. Section 12.11 asks for validation that accounts for the intervention. Block 4 pools every inspection into one rate and computes no held out statistic, so this step asks for an extension. Have an assistant modify the loop so the random share of each iteration's budget is logged separately and serves as the only validation set, report the agreement between the model's score and the rate observed in that random sample, and compare it with the self generated agreement from Step 3.

6. Read the code and complete the handoff. Tier 1. Annotate the score update line, which is where the loop closes, and answer the decisions file.