Revision 7T · Full Tutorial Edition · Updated 1 September 2026
PWR-196 · SELF GUIDED full tutorial
Review a benign image independently, then use AI while counting misses and false alarms by condition
Thirty synthetic street-sign cards let you compare human-only, AI-only and combined labels by class and image quality. Lock the independent label, inspect the fixed AI answer, use the folded key and record when advice creates or prevents an error. These matrices describe the benign card set, not clinical perception or performance in another domain.
What you will produceThe learner compares human-only, AI-only and combined labels against an answer key and identifies when incorrect advice changes a correct human judgement.
Full step-by-step individual tutorial · TLU-PWR-196
The learner compares human-only, AI-only and combined labels against an answer key and identifies when incorrect advice changes a correct human judgement.
AI-assisted perception sandbox: A labelled set of 30 synthetic street-sign images contains 10 bicycles, 10 buses and 10 neither; five images in each class are blurred. The AI returns a label and confidence.
Independent move — Resolve disagreements: Check the class definition and compare evidence; do not accept AI merely because its confidence is higher.
Verification move — Test induced error: Mark cases where a correct first answer became wrong after AI advice and set a referral rule for that condition.
Qualified help is required for
AI-assisted perception escalation: Re-baseline after camera, display, preprocessing, class definition or model changes.
Oversight boundary — Declare classes and error costs: any clinical application. “Label independently” needs review. “Reveal AI output by condition” marks the employment, legal or safety gate: Re-baseline after camera, display, preprocessing, class definition or model changes.
Biometric records are excluded from “Declare classes and error costs”. Confidential or covert material is excluded from “Label independently”.
Never do this from the page alone
Prohibited AI-assisted perception shortcut: Let the AI label determine where you look.
Do not conceal “Force a high-confidence class.”; record its matching correction instead.
No consequential use: this sandbox cannot establish “AI gives superhuman sight” about another person.
2 · Get ready
Gather what you need and check the starting conditions
What you need
Declared AI-assisted perception fixture: A labelled set of 30 synthetic street-sign images contains 10 bicycles, 10 buses and 10 neither; five images in each class are blurred. The AI returns a label and confidence.
Setup aid for Label independently: Record display, image transformations and AI model version.
AI-assisted perception log: Sensitivity, specificity, calibration, time and severity-weighted misses versus expert-only, AI-only and combined conditions; retain AI-assisted perception errors, assistance, stop and fallback.
Perception-study comparison: from “Explainable AI improves task performance in human-AI collaboration”, extract human-only, AI-only and combined conditions; from the mammography multi-reader study, extract reader, support and error measures while marking its clinical population as non-transferable. Use the NIST Privacy Framework to keep the supplied street-sign images synthetic and free of personal data.
Supplied scene cards, formatted ID|quality|description. hide the scoring strip: 1|clear|two equal circles joined by a triangular frame beneath a round sign. 2|clear|long rectangle with a row of windows and two axles. 3|clear|single triangular yield sign on a post. 4|blurred|two soft circles linked by a faint triangular centre. 5|blurred|wide block with a faint horizontal window row. 6|blurred|one circular speed sign with a smudged numeral. 7|clear|handlebar and pedal frame above two wheels. 8|clear|tall vehicle front with windshield and route box. 9|clear|walking-person symbol inside a square. 10|blurred|faint double-wheel outline and one diagonal tube. 11|blurred|rectangular carriage with several pale square patches. 12|blurred|single arrow inside a diamond. 13|clear|diamond frame, pedals and two wheels. 14|clear|side profile with four windows and a door. 15|clear|eight-sided stop outline. 16|blurred|partial wheel arcs with a central frame trace. 17|clear|two-wheel frame with visible pedals. 18|blurred|large block with two dark lower wheels. 19|blurred|one-wheel scooter outline with an upright stem. 20|clear|front face with windshield and route number. 21|clear|striped traffic cone. 22|clear|broad vehicle outline with a window row. 23|blurred|two circles joined by angled bars. 24|blurred|rectangular parking sign with a faint letter P. 25|clear|chainring, handlebars and two complete wheels. 26|blurred|long box shape with a faint doorway. 27|clear|parallel crosswalk stripes. 28|blurred|double-ring silhouette with a narrow centre. 29|blurred|tall rectangular outline with two lower discs. 30|blurred|single blank square sign.
Folded key and fixed AI outputs, formatted ID|key|AI label|AI confidence: 1|BICYCLE|BICYCLE|67; 2|BUS|BUS|74; 3|NEITHER|NEITHER|81; 4|BICYCLE|BUS|88; 5|BUS|BUS|95; 6|NEITHER|BICYCLE|64; 7|BICYCLE|BICYCLE|71; 8|BUS|BUS|78; 9|NEITHER|NEITHER|85; 10|BICYCLE|BICYCLE|92; 11|BUS|NEITHER|61; 12|NEITHER|NEITHER|68; 13|BICYCLE|BICYCLE|75; 14|BUS|BUS|82; 15|NEITHER|BICYCLE|89; 16|BICYCLE|BICYCLE|96; 17|BICYCLE|BUS|96; 18|BUS|BUS|72; 19|NEITHER|NEITHER|79; 20|BUS|BUS|86; 21|NEITHER|NEITHER|93; 22|BUS|BUS|62; 23|BICYCLE|BUS|69; 24|NEITHER|NEITHER|76; 25|BICYCLE|BICYCLE|83; 26|BUS|NEITHER|90; 27|NEITHER|NEITHER|97; 28|BICYCLE|BICYCLE|66; 29|BUS|BUS|73; 30|NEITHER|BICYCLE|80. The class counts are 10 BICYCLE, 10 BUS and 10 NEITHER; each class has five clear and five blurred cards. In particular, card 17 is a bicycle that the AI mislabels BUS at 96, and card 22 is a bus correctly labelled BUS.
Supplied delayed 15-card set, formatted ID|quality|folded key|description: 31|clear|BICYCLE|two wheels joined by a frame. 32|clear|BUS|windowed rectangle with two axles. 33|clear|NEITHER|pedestrian crossing symbol. 34|blurred|BICYCLE|faint paired wheels and diagonal bar. 35|blurred|BUS|faint long window row. 36|blurred|NEITHER|single smudged octagon. 37|clear|BICYCLE|pedals between two wheels. 38|clear|BUS|route box above a wide windshield. 39|clear|NEITHER|parking letter P. 40|blurred|BICYCLE|two ring traces with a centre tube. 41|blurred|BUS|box body with two lower discs. 42|blurred|NEITHER|one arrow in a diamond. 43|clear|BICYCLE|handlebars, frame and chainring. 44|clear|BUS|four windows beside a door. 45|clear|NEITHER|traffic cone. Use it after 48 hours and keep its key covered until the independent label is locked.
Before you start
Use synthetic, public or consented benign images with no biometric or sensitive content.
Record display, image transformations and AI model version.
Start check for AI-assisted perception: Labels and denominators are fixed before inspection.
Top-of-sheet stop for AI-assisted perception: Stop if images identify real people or contain private, medical, security or illegal content.
3 · The method
Follow these steps in order
Declare classes and error costs
Define bicycle, bus and neither from the key, plus why a miss and false alarm differ.
Why: Labels and denominators are fixed before inspection.
Check: Labels and denominators are fixed before inspection.
Label independently
Classify every image and rate confidence before revealing AI output.
Why: Human-only labels cannot be edited retrospectively.
Check: Human-only labels cannot be edited retrospectively.
Reveal AI output by condition
For every scene card, record the AI model and version, proposed label, confidence, and clear or blurred stratum.
Why: Performance can be split by image quality.
Check: Performance can be split by image quality.
Inspect the evidence region
Point to the pixels or feature that supports the label; mark “insufficient” if blur prevents a check.
Why: A final label has observable evidence or explicit uncertainty.
Check: A final label has observable evidence or explicit uncertainty.
Resolve disagreements
Check the class definition and compare evidence; do not accept AI merely because its confidence is higher.
Why: Every changed answer names the feature that justified it.
Check: Every changed answer names the feature that justified it.
Open the key and fill the matrix
Count each class’s hits, misses and false alarms for human, AI and combined conditions.
Why: All 30 images appear in each condition table.
Check: All 30 images appear in each condition table.
Compare clear and blurred strata
Calculate sensitivity and false-alarm rates separately for clear and blurred images.
Why: An average does not hide a poor degraded-input result.
Check: An average does not hide a poor degraded-input result.
Test induced error
Mark cases where a correct first answer became wrong after AI advice and set a referral rule for that condition.
Why: The report includes over-reliance as well as combined accuracy.
Check: The report includes over-reliance as well as combined accuracy.
4 · Worked example
See the whole method used once
Scenario
A labelled set of 30 synthetic street-sign images contains 10 bicycles, 10 buses and 10 neither; five images in each class are blurred. The AI returns a label and confidence.
Walkthrough
Define the three classes and decide that blurred unreadable images may be marked insufficient.
Label all 30 synthetic images independently; mark image 17 “bicycle” at 80 confidence.
Reveal AI labels; it calls image 17 “bus” at 96, but the visible two-wheel frame matches the bicycle key, so keep the human label.
On image 22 the human label is neither; the AI points to a clear bus outline and the key confirms the changed answer.
Complete matrices and find combined bicycle sensitivity 9/10 with two bicycle false alarms, reported separately from bus results.
Show that blurred images cause three times as many referrals and set a “blurred plus disagreement” referral rule.
Result
The combined process is measured by class and image quality and exposes one resisted and one useful AI disagreement. This is not clinical perception or proof of best-component superiority elsewhere.
5 · Right and wrong
Compare correct or safer execution with the common wrong version
Right and wrong comparison
Moment
Right / safer
Wrong / riskier
Why it matters
Independent perception
Label before AI reveal when completing the synthetic sign-label set.
Let the AI label determine where you look.
Advice can direct attention and hide human-only performance.
Uncertain input
Mark insufficient evidence on heavily blurred images.
Force a high-confidence class during the synthetic sign-label set.
Forced labels create hidden misses and false alarms.
Error breakdown
Report class-specific sensitivity and false alarms.
Use overall accuracy only during the synthetic sign-label set.
Balanced totals can hide failure on one target class.
AI disagreement
Inspect the source feature and class rule.
Follow the higher AI confidence during the synthetic sign-label set.
Confidence may be miscalibrated and explanations can induce overreliance.
6 · Common mistakes
Spot the error and apply the correction
Common mistakes and corrections
Mistake
Fix
Let the AI label determine where you look.
Lock the first label and confidence.
Force a high-confidence class during the synthetic sign-label set.
Use an uncertainty/referral category within the synthetic sign-label set.
Use overall accuracy only during the synthetic sign-label set.
Fill a per-class confusion table within the synthetic sign-label set.
Follow the higher AI confidence.
Change only for verifiable image evidence.
7 · Practice
Turn the steps into a usable skill
First session
Synthetic sign-label run: A labelled set of 30 synthetic street-sign images contains 10 bicycles, 10 buses and 10 neither; five images in each class are blurred. The AI returns a label and confidence.
Declared classes and independent labels: Declare classes and error costs: Define bicycle, bus and neither from the key, plus why a miss and false alarm differ.
Pixel-evidence disagreement pass: Inspect the evidence region, then Resolve disagreements.
AI-attention-bias correction: if “Let the AI label determine where you look.” appears, apply “Lock the first label and confidence.”
Induced-error review: Test induced error: Mark cases where a correct first answer became wrong after AI advice and set a referral rule for that condition.
Repeat plan
Use one 30-image benign set weekly for three weeks, changing blur and class prevalence separately. Retest 15 new images after 48 hours; progress only when combined performance beats the better standalone component without more severe misses.
Progress when
Labels and denominators are fixed before inspection.
Human-only labels cannot be edited retrospectively.
Performance can be split by image quality.
All conditions have complete confusion tables, induced errors are named, degraded inputs trigger referral, and combined severity-weighted performance exceeds the better standalone baseline.
Do not progress when
Do not continue while this error remains: Let the AI label determine where you look.
Pause until this correction works: Use an uncertainty/referral category.
This AI-assisted perception stop ends the block: Stop if images identify real people or contain private, medical, security or illegal content.
8 · Check the result
Measure what changed
Sensitivity, specificity, calibration, time and severity-weighted misses versus expert-only, AI-only and combined conditions
How: Baseline fixture: A labelled set of 30 synthetic street-sign images contains 10 bicycles, 10 buses and 10 neither; five images in each class are blurred. The AI returns a label and confidence. Enter success checks from “Declare classes and error costs” and “Resolve disagreements”. If “Let the AI label determine where you look.” occurs, apply its named fix; then score Sensitivity, specificity, calibration, time and severity-weighted misses versus expert-only, AI-only and combined conditions on the unused “Test induced error” item.
Good result: All conditions have complete confusion tables, induced errors are named, degraded inputs trigger referral, and combined severity-weighted performance exceeds the better standalone baseline.
This does not prove: Boundary for AI-assisted perception: “Sensitivity, specificity, calibration, time and severity-weighted misses versus expert-only, AI-only and combined conditions” describes only A labelled set of 30 synthetic street-sign images contains 10 bicycles, 10 buses and 10 neither; five images in each class are blurred. The AI returns a label and confidence. It cannot establish “AI gives superhuman sight”.
Self-check
Without the example, demonstrate: Labels and denominators are fixed before inspection.
Find the fault in this attempt: “Let the AI label determine where you look.” Apply “Lock the first label and confidence.”; what changes?
What evidence in the completed record shows that this is wrong: “Force a high-confidence class.”?
AI-assisted perception stop decision: Stop if images identify real people or contain private, medical, security or illegal content.
9 · Stop, adapt or get help
Keep the safety boundary practical
Stop and get help
Stop if images identify real people or contain private, medical, security or illegal content.
Do not transfer the exercise to diagnosis, surveillance, driving or safety inspection.
Re-baseline after camera, display, preprocessing, class definition or model changes.
Accessibility and adaptations
Describe synthetic images in text or tactile diagrams and use the same answer-key classes.
Allow zoom, contrast and screen-reader-compatible labels; record aids rather than removing them.
10 · Evidence and limits
Why these instructions are here
primary research
Registered support for AI-assisted perception: “Explainable AI improves task performance in human-AI collaboration”. It bears on Sensitivity, specificity, calibration, time and severity-weighted misses versus expert-only, AI-only and combined conditions inside the AI-assisted perception fixture. It does not validate “AI gives superhuman sight”.
Constraint for AI-assisted perception, drawn from “Improving the Performance of Radiologists Using Artificial Intelligence-Based Detection Support Software for Mammography: A Multi-Reader Study”: Some combined teams did not exceed the best standalone component, and incorrect advice or explanations could induce overreliance.
Privacy design for AI-assisted perception: minimise approved data in “A labelled set of 30 synthetic street-sign images contains 10 bicycles, 10 buses and 10 neither; five images in each class are blurred. The AI returns a label and confidence.” Keep AI-assisted perception provenance and access visible before interpreting Sensitivity, specificity, calibration, time and severity-weighted misses versus expert-only, AI-only and combined conditions.
AI-assisted perception boundary: interpret “Sensitivity, specificity, calibration, time and severity-weighted misses versus expert-only, AI-only and combined conditions” only for A labelled set of 30 synthetic street-sign images contains 10 bicycles, 10 buses and 10 neither; five images in each class are blurred. The AI returns a label and confidence.
A successful result does not establish “AI gives superhuman sight”.
AI-assisted perception limiting finding: Some combined teams did not exceed the best standalone component, and incorrect advice or explanations could induce overreliance.
No perfect-performance claim for AI-assisted perception: the evidence register does not make “Sensitivity, specificity, calibration, time and severity-weighted misses versus expert-only, AI-only and combined conditions” universal, consequence-free or flawless in A labelled set of 30 synthetic street-sign images contains 10 bicycles, 10 buses and 10 neither; five images in each class are blurred. The AI returns a label and confidence.
Scope remains AI-assisted perception: A labelled set of 30 synthetic street-sign images contains 10 bicycles, 10 buses and 10 neither; five images in each class are blurred. The AI returns a label and confidence. Recheck the comparator, support and “Sensitivity, specificity, calibration, time and severity-weighted misses versus expert-only, AI-only and combined conditions” after any configuration change.
Jeong Hoon Lee; Ki Hwan Kim; Eun Hye Lee; Jong Seok Ahn; Jung Kyu Ryu; Young Mi Park; Gi Won Shin; Young Joong Kim; Hye Young Choi · 2022 · Primary research
Progress is saved only in this browser on this device.
Step-by-step learner mode
Each activity includes its success check, a nearby accessible alternative and an “I’m stuck” correction path. Alternatives preserve the target where possible; when they change the task, Titan labels them as related rather than equivalent.
01
Declare classes and error costs
Define bicycle, bus and neither from the key, plus why a miss and false alarm differ.
Why this step exists
Labels and denominators are fixed before inspection.
Success check
Labels and denominators are fixed before inspection.
I’m stuck on this step
Reset: Re-read this authored instruction — “Define bicycle, bus and neither from the key, plus why a miss and false alarm differ.” — and its success check, then attempt only this step.
Possible snag: The result from “Define bicycle, bus and neither from the key, plus why a miss and false alarm differ.” does not yet meet this declared check: Labels and denominators are fixed before inspection.
Correction: Return to the start of “Declare classes and error costs”, reduce complexity or pace, and repeat only the part needed to satisfy: “Labels and denominators are fixed before inspection.”
Stop / get help: Stop if images identify real people or contain private, medical, security or illegal content.
02
Label independently
Classify every image and rate confidence before revealing AI output.
Why this step exists
Human-only labels cannot be edited retrospectively.
Success check
Human-only labels cannot be edited retrospectively.
I’m stuck on this step
Reset: Re-read this authored instruction — “Classify every image and rate confidence before revealing AI output.” — and its success check, then attempt only this step.
Possible snag: Let the AI label determine where you look.
Correction: Lock the first label and confidence.
Possible snag: Force a high-confidence class during the synthetic sign-label set.
Correction: Use an uncertainty/referral category within the synthetic sign-label set.
Stop / get help: Stop if images identify real people or contain private, medical, security or illegal content.
03
Reveal AI output by condition
For every scene card, record the AI model and version, proposed label, confidence, and clear or blurred stratum.
Why this step exists
Performance can be split by image quality.
Success check
Performance can be split by image quality.
I’m stuck on this step
Reset: Re-read this authored instruction — “For every scene card, record the AI model and version, proposed label, confidence, and clear or blurred stratum.” — and its success check, then attempt only this step.
Possible snag: The result from “For every scene card, record the AI model and version, proposed label, confidence, and clear or blurred stratum.” does not yet meet this declared check: Performance can be split by image quality.
Correction: Return to the start of “Reveal AI output by condition”, reduce complexity or pace, and repeat only the part needed to satisfy: “Performance can be split by image quality.”
Stop / get help: Stop if images identify real people or contain private, medical, security or illegal content.
04
Inspect the evidence region
Point to the pixels or feature that supports the label; mark “insufficient” if blur prevents a check.
Why this step exists
A final label has observable evidence or explicit uncertainty.
Success check
A final label has observable evidence or explicit uncertainty.
I’m stuck on this step
Reset: Re-read this authored instruction — “Point to the pixels or feature that supports the label; mark “insufficient” if blur prevents a check.” — and its success check, then attempt only this step.
Possible snag: The result from “Point to the pixels or feature that supports the label; mark “insufficient” if blur prevents a check.” does not yet meet this declared check: A final label has observable evidence or explicit uncertainty.
Correction: Return to the start of “Inspect the evidence region”, reduce complexity or pace, and repeat only the part needed to satisfy: “A final label has observable evidence or explicit uncertainty.”
Stop / get help: Stop if images identify real people or contain private, medical, security or illegal content.
05
Resolve disagreements
Check the class definition and compare evidence; do not accept AI merely because its confidence is higher.
Why this step exists
Every changed answer names the feature that justified it.
Success check
Every changed answer names the feature that justified it.
I’m stuck on this step
Reset: Re-read this authored instruction — “Check the class definition and compare evidence; do not accept AI merely because its confidence is higher.” — and its success check, then attempt only this step.
Possible snag: Follow the higher AI confidence.
Correction: Change only for verifiable image evidence.
Stop / get help: Stop if images identify real people or contain private, medical, security or illegal content.
06
Open the key and fill the matrix
Count each class’s hits, misses and false alarms for human, AI and combined conditions.
Why this step exists
All 30 images appear in each condition table.
Success check
All 30 images appear in each condition table.
I’m stuck on this step
Reset: Re-read this authored instruction — “Count each class’s hits, misses and false alarms for human, AI and combined conditions.” — and its success check, then attempt only this step.
Possible snag: Use overall accuracy only during the synthetic sign-label set.
Correction: Fill a per-class confusion table within the synthetic sign-label set.
Stop / get help: Stop if images identify real people or contain private, medical, security or illegal content.
07
Compare clear and blurred strata
Calculate sensitivity and false-alarm rates separately for clear and blurred images.
Why this step exists
An average does not hide a poor degraded-input result.
Success check
An average does not hide a poor degraded-input result.
I’m stuck on this step
Reset: Re-read this authored instruction — “Calculate sensitivity and false-alarm rates separately for clear and blurred images.” — and its success check, then attempt only this step.
Possible snag: The result from “Calculate sensitivity and false-alarm rates separately for clear and blurred images.” does not yet meet this declared check: An average does not hide a poor degraded-input result.
Correction: Return to the start of “Compare clear and blurred strata”, reduce complexity or pace, and repeat only the part needed to satisfy: “An average does not hide a poor degraded-input result.”
Stop / get help: Stop if images identify real people or contain private, medical, security or illegal content.
08
Test induced error
Mark cases where a correct first answer became wrong after AI advice and set a referral rule for that condition.
Why this step exists
The report includes over-reliance as well as combined accuracy.
Success check
The report includes over-reliance as well as combined accuracy.
I’m stuck on this step
Reset: Re-read this authored instruction — “Mark cases where a correct first answer became wrong after AI advice and set a referral rule for that condition.” — and its success check, then attempt only this step.
Possible snag: The result from “Mark cases where a correct first answer became wrong after AI advice and set a referral rule for that condition.” does not yet meet this declared check: The report includes over-reliance as well as combined accuracy.
Correction: Return to the start of “Test induced error”, reduce complexity or pace, and repeat only the part needed to satisfy: “The report includes over-reliance as well as combined accuracy.”
Stop / get help: Stop if images identify real people or contain private, medical, security or illegal content.
Correct versus incorrect execution
These accessible process diagrams are built from the tutorial’s own right/wrong teaching. They are not anatomical illustrations and do not add technique beyond the canonical tutorial.
Independent perception — Advice can direct attention and hide human-only performance.
Correct / safer
Label before AI reveal when completing the synthetic sign-label set.
Mark insufficient evidence on heavily blurred images.
Wrong / riskier
Force a high-confidence class during the synthetic sign-label set.
Error breakdown — Balanced totals can hide failure on one target class.
Correct / safer
Report class-specific sensitivity and false alarms.
Wrong / riskier
Use overall accuracy only during the synthetic sign-label set.
AI disagreement — Confidence may be miscalibrated and explanations can induce overreliance.
Correct / safer
Inspect the source feature and class rule.
Wrong / riskier
Follow the higher AI confidence during the synthetic sign-label set.
Method-structure checklist
10 of 10 structural checks present
✓ Ordered, Power-specific instructions — present
✓ Every activity has a success check — present
✓ Materials or supplied records are declared — present
✓ Measurement or assessment rule is present — present
✓ Tutorial-specific troubleshooting is present — present
✓ Stopping or escalation boundary is present — present
✓ Every activity has an adjacent alternative — present
✓ Correct-versus-incorrect comparison is present — present
✓ Evidence context is bound to the Power record — present
✓ Planning metadata is present — present
The method-readiness band and presence checklist assess tutorial presentation and are separate from evidence quality for the underlying Power. They are automated editorial aids, not human approval.
Manual editorial sign-off: Pending. This tutorial must not display a human-approved state until an identified editor signs the exact content hash.