Exercise V-B. Adversarial Stress Testing

Part V. Adversarial and Emerging Risks

Overview

Every action in this exercise stays inside the student's own model and frozen data, and no third party system is involved at any point. Section 14.8 describes the anatomy of a GeoAI vulnerability and section 14.9 specifies the Adversarial GeoAI Exposure Assessment. The injection this exercise performs is a controlled poisoning test of the kind section 14.10.3 describes, run against a sandboxed copy of the student's own ranking.

Students define a threat model explicitly, being an actor who can add fabricated households to a tract's household count without deleting or altering existing records, whose objective is to move a chosen tract into the published top ten of the need ranking. They inject fabricated records in increments, rerun the model at each increment, and report the smallest fraction of the record count that achieves the objective. They then run one detection control and report whether it fires before the attack succeeds. The control's false positive rate on clean data is reported alongside, because a control nobody can afford to run is not a mitigation.

Step by step

1. State the threat model. Tier 1. Change to the toolkit folder inside GeoAI_Exercises and type python exercise_5b.py. Record the four elements the program states, being the actor's access, their objective, their budget and their constraint, and relate them to the three elements section 14.2 names, being capability, knowledge and consequence. Then rate the ranking on the six dimensions of the Adversarial GeoAI Exposure Assessment in section 14.9, being Incentive, Access, Observability, Manipulability, Consequence and Recoverability. An assessment without a stated threat model tests nothing in particular.

2. Find the minimum injection. Tier 1. The program attacks five tracts, the first just outside the top ten and each later one further down the ranking. Record the smallest number of injected records that moves each one into the top ten, and that number as a percentage of its households, and name the tracts the budget cannot move.

Figure. Fabricated records needed to move a tract into the top ten, plotted against the tract's starting rank, for the tracts the attack moved within budget.

3. Measure the blast radius. Tier 1. Record how many other tracts changed rank as a side effect of the cheapest injection, how many left the top ten, and the Jaccard overlap of the top ten before and after. State whether the attack is detectable from its side effects alone.

4. Tune the control. Tier 2. Section 14.11 asks for resilient design. The program runs one detection control, a robust z score on the no-vehicle share with CONTROL_SIGMA set to 3.0. Record its false positive rate on clean data and the number of records at which it fires on the cheapest attack. Set CONTROL_SIGMA to 2.0 and then 4.0, rerun after each change, and report how both numbers move. State the operational cost of running that control continuously.

5. Read the code and complete the handoff. Tier 1. Annotate the injection loop and answer the decisions file.