Quantifying Surprise Slides Quantifying Surprise
The Rescorla-Wagner Model
Advanced Associative Learning Series
Prediction Error is Key
Learning occurs only when there is a discrepancy between what is expected and what actually occurs.
Contiguity (mere pairing) is insufficient; informativeness drives the change in associative strength.
Essential Concept
"Surprise" = Learning
The Mathematical Engine
\[ \Delta V_i = \alpha_i \beta (\lambda - \Sigma V) \]
\(\Delta V_i\): Change in associative strength of stimulus \(i\)
\(\alpha_i\): Saliency of the CS
\(\beta\): Saliency/Learning rate of the US
\(\lambda\): Maximum associative strength (asymptote)
\(\Sigma V\): Total associative strength of all present stimuli
\((\lambda - \Sigma V)\): The Prediction Error
Why Does Blocking Happen?
Phase 1
Stimulus A is paired with US until it reaches asymptote.
\(V_A \approx \lambda\)
Phase 2
Stimulus A + Stimulus B are paired with US.
\[ \Sigma V = V_A + V_B \]
\[ \Delta V_B \propto (\lambda - (V_A + V_B)) \]
Outcome
Since \(V_A \approx \lambda\), the error term is ZERO.
No surprise = No learning for Stimulus B.
Overshadowing
When two CSs are presented together, the more salient one (\(\alpha\)) captures more of the limited associative strength available.
"The squeaky wheel gets the oil (and the associative value)."
Competition for \(\lambda\)
A zero-sum game in the brain
Surprise Simulator Worksheet Modeling Prediction Error
LABORATORY SIMULATION WORKBOOK | PSYC-802
Date: ________________
Investigator: ________________
Objective: To manually simulate the acquisition of associative strength using the Rescorla-Wagner equation: \(\Delta V = \alpha \beta (\lambda - \Sigma V)\). Students will analyze how prediction error decreases over time and how stimulus saliency influences the rate of learning.
Phase 1: Simple Acquisition Parameters
CS Saliency (\(\alpha\))
0.5
US Strength (\(\beta\))
1.0
Max Value (\(\lambda\))
1.0
Trial-by-Trial Calculation
Trial Current \(V\) Prediction Error \((\lambda - V)\) \(\Delta V\) \((0.5 \times (\text{Error}))\) New \(V\) 1 0.00 1.00 0.50 0.50 2 0.50 3 4 5
Critical Analysis
1. The Asymptote Paradox
Observe the value of \(\Delta V\) as trials progress. Explain why the rate of learning decreases even though the stimulus pairings remain constant.
2. Simulating Blocking
If we introduce Stimulus B (\(\alpha_B = 0.5\)) alongside Stimulus A in Trial 6, and Stimulus A has already reached \(V_A = 0.98\), calculate \(\Delta V_B\). Show your work.
3. Competitive Saliency
Hypothesize: How would a decrease in \(\alpha_i\) (CS Saliency) to 0.1 affect the number of trials required to reach \(\lambda = 0.95\)? Explain using the error term.
Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement.
Prediction Error Teacher Guide Predicting Learning Dynamics
FACILITATION GUIDE | INSTRUCTOR RESOURCE
Lesson 1.1
The "Aha" Moment: Theoretical Essence
The fundamental shift of the Rescorla-Wagner model is moving the "locus of control" from the stimulus to the expectancy of the organism. Learning isn't about how many times a bell rings; it's about how much the bell tells you that you didn't already know.
Blocking
The first stimulus "uses up" the limited capacity of surprise available in the system.
Overshadowing
Two stimuli "fight" for surprise based on their inherent physical intensity (saliency).
Common Pitfalls
Students may confuse \(\Sigma V\) with \(\lambda\). Clarify that \(\lambda\) is the goal, \(\Sigma V\) is the current status.
Thinking learning is linear. Emphasize the "diminishing returns" seen in the curve.
Overlooking the \(\alpha\) parameter. Saliency is often fixed in simple models but critical for overshadowing.
Socratic Facilitation Prompts
"Imagine a trial where the US is removed (Extinction). How does the model mathematically handle the reduction in associative strength?"
Key: \(\lambda\) becomes 0, making the error term negative.
"If \(\alpha\) (CS Saliency) were 0, what would happen to the learning curve?"
Key: Zero learning occurs regardless of surprise (\(\Delta V = 0\)).
"How does this model explain 'Unlearning' versus 'Suppression'?"
Key: Note that R-W model assumes true unlearning (erasure of V), which is a limitation (doesn't explain renewal).
Workshop Key: Simulation (Phase 1)
Trial V (Start) Error (1 - V) \(\Delta V\) (0.5 * Err) V (End) 1 0.00 1.00 0.50 0.50 2 0.50 0.50 0.25 0.75 3 0.75 0.25 0.125 0.875 4 0.875 0.125 0.0625 0.9375 5 0.9375 0.0625 0.03125 0.96875
Instructor Note: Focus on how the change (\(\Delta V\)) gets smaller. This is the visual representation of "learning being completed."
Neural Circuitry Slides Neurobiology of Fear
Amygdala and Hippocampal Circuitry
Lesson 2: Mapping the Associative Brain
The Hub of Convergence
The Lateral Amygdala is the primary site of CS-US convergence.
Auditory/Visual stimuli (Thalamus/Cortex)
Nociceptive info (US - Pain)
Mechanism: LTP
Co-activation leads to strengthened synaptic efficacy via NMDA-dependent pathways.
Convergence Zone
"Neurons that fire together, wire together."
Contextual Gating
Hippocampus
Encodes the context (environment/background) in which learning occurs.
Projects to the Basal Amygdala to gate the expression of fear based on environmental cues.
Clinical Key
Hippocampal dysfunction is linked to over-generalization of fear in PTSD.
"Extinction is NOT Erasure"
Original Trace
CS \(\rightarrow\) US Memory
Stored in the Lateral/Basal Amygdala permanently.
Extinction Trace
CS \(\rightarrow\) No US Memory
Involves the Infralimbic Cortex (mPPC) inhibiting the amygdala.
Relapse occurs when the Inhibitory Trace fails to activate, leaving the original fear memory unopposed.
Integrated Threat Circuit
IL
Infralimbic
The "Stop" signal for fear expression.
LA
Amygdala
The engine of associative learning.
HC
Hippocampus
The "Where" signal gating the response.
SYNTHESIS: Context + Inhibition = Stability
Literature Analysis Form Primary Literature Critique
NEUROBIOLOGICAL SYSTEMS ANALYSIS | PSYC-802
"Inquire. Analyze. Synthesize."
Reviewer:
Date:
Target Publication
Select a paper focusing on NMDA receptors in the Lateral Amygdala or Hippocampal-Amygdala projections.
1 Neural Substrate Identification
Which specific neural populations or circuits were isolated? Detail the methods used for isolation (e.g., optogenetics, pharmacological blockade, DREADDs).
2 Mechanistic Evidence of Learning
How did the authors demonstrate that the observed neural change was specific to associative learning rather than sensitization or general motor changes?
3 Extinction Paradox
If the study involves extinction, what evidence is provided to support the "Inhibition Theory" versus the "Unlearning Theory"? Look specifically for spontaneous recovery or renewal data.
4 Theoretical Synthesis
Relate these biological findings back to the Rescorla-Wagner model. Where in the brain is the "error term" \((\lambda - \Sigma V)\) physically computed, according to this research?
Psychology Department | Associative Learning Lab | Document ID: CRIT-02
Amygdala Circuits Reference Sheet Circuit Reference Sheet
Anatomy of Threat and Safety
Reference V1.2
1. Information Streams
Thalamus / Cortex
Transmits CS (Tone/Light) properties. Thalamus = Fast/Crude; Cortex = Slow/Detailed.
Somatosensory
Transmits US (Pain/Shock) information to the Lateral Amygdala.
Hippocampus
Contextual encoding. Sends environmental snapshots to the Basal Amygdala.
2. Core Processing (The Amygdala)
Lateral Nucleus (LA)
The convergence zone . CS + US meet here. Site of NMDA-dependent LTP during acquisition.
Basal Nucleus (BLA)
Integrates HC contextual info. Passes signal to Central Nucleus.
Central Nucleus (CeA)
The output station . Projects to Brainstem/Hypothalamus to trigger CR (Freezing, Heart Rate).
3. Executive Control
Infralimbic Cortex (IL)
The Safety Center. Activates during extinction. Sends inhibitory signals to suppress CeA output.
Prelimbic Cortex (PL)
The Fear Center. Promotes fear expression. Enhances Amygdala activity in high-threat contexts.
Intercalated Cells (ITC)
GABAergic inhibitory neurons. The physical "gate" that the IL uses to shut down the CeA.
Neuro-Associative Principle
Associative learning is not stored in a single node, but as a distribution of synaptic weights across the LA, BLA, and CeA, gated by the IL (extinction) and HC (context). Extinction training creates a competing trace in the IL-ITC pathway that competes for control of behavior.
Relapse Mechanisms Slides The Context of Relapse
Renewal, Reinstatement, & Spontaneous Recovery
Lesson 3: Why Extinction Fails
Spontaneous Recovery
The passage of time alone is sufficient to weaken the extinction memory.
If you extinguish a fear on Monday, and test the response on Friday, the fear will partially return without any further pairings.
Learning Trace Competition
Excitation vs. Inhibition
Renewal Paradigms
ABA
Context A (Acquisition)
Strong Relapse
ABC
Context A (Acquisition)
Moderate Relapse
AAB
Context A (Acquisition)
Weak/Moderate Relapse
"Extinction is context-specific; Acquisition is context-independent."
Reinstatement
Exposure to the US alone (the shock or the drug) can trigger a relapse of the CR to the CS.
"Once a person experiences the trauma or the high again, the previously extinguished triggers become active once more."
Triggering the entire associative network via the primary outcome stimulus.
The Clinical Imperative
Therapy Problem
Therapy occurs in a safe office (Context B). Real life occurs at home (Context A). This is a perfect setup for ABA Renewal.
Strategy
Extinction training must be conducted in multiple contexts to increase the "generalizability" of the inhibitory trace.
NEXT: Translating these mechanisms into exposure protocols.
Clinical Relapse Case Studies Clinical Relapse Casebook
APPLIED ASSOCIATIVE LEARNING | SEMINAR SERIES
Instructions: Analyze the following clinical scenarios using the terminology of modern extinction theory (Renewal, Reinstatement, Spontaneous Recovery). For each case, identify the Acquisition Context, the Extinction Context, and the mechanism of relapse.
Case A: The Social Anhedonia Relapse
REF: SUD-042
A 28-year-old male receives treatment for Alcohol Use Disorder at an intensive residential facility (isolated from previous social circles). During the 30-day program, he successfully undergoes cue exposure therapy, showing significantly reduced cravings when shown bottles of beer. However, on the first night back at his favorite downtown bar with old friends, he experiences an intense, overwhelming urge to drink and relapses within hours.
Relapse Mechanism
Identify if this is ABA, ABC, or AAB renewal.
Theoretical Explanation
Why did cue exposure in the clinic fail in the bar?
Case B: The Re-traumatization Effect
REF: PTSD-109
A survivor of a severe motor vehicle accident (MVA) successfully completes Prolonged Exposure (PE) therapy. Her fear of driving (CS) is extinguished, and she drives daily for six months without incident. However, while walking in a park, she is bitten by an aggressive dog (a separate, unrelated traumatic US). The following day, she finds herself unable to get behind the wheel of her car, experiencing a full return of MVA-related panic.
Relapse Mechanism
Is this Renewal or Reinstatement?
Theoretical Explanation
How does an unrelated US trigger an extinguished CS-CR?
Synthesizing a Solution
Based on Bouton’s (2002) research, propose ONE clinical modification to traditional therapy that would decrease the likelihood of relapse in Case A.
Bouton, M. E. (2002). Context, ambiguity, and unlearning: Sources of relapse after behavioral extinction. Biological Psychiatry.
Relapse Seminar Cards Paradox 01
"If extinction were true 'unlearning,' why does the passage of time (Spontaneous Recovery) bring the behavior back? What does this imply about the physical architecture of memory?"
Discussion Focus: Inhibitory Learning Theory
Paradox 02
"Why is Acquisition usually 'context-free' while Extinction is 'context-specific'? Evolutionarily, what is the survival advantage of this asymmetry?"
Discussion Focus: Adaptive Evolutionary Psychology
Paradox 03
"Reinstatement shows that a US-only exposure can revive an extinguished CS-CR. Does this mean the CS becomes a 'pointer' to a general state of arousal, or is the specific association being re-indexed?"
Discussion Focus: Associative Indexing Models
Paradox 04
"Most exposure therapy aims for 'Habituation' (feeling less fear). If Inhibitory Learning theory is correct, should we actually want the patient to stay scared to maximize learning?"
Discussion Focus: Expectancy Violation vs. Habituation
Advanced Associative Chains Slides Beyond the Bell
Higher-Order Conditioning & Sensory Preconditioning
Lesson 4: Building Associative Chains
Second-Order Conditioning
A neutral stimulus (CS2) becomes a predictor of a conditioned stimulus (CS1), thereby acquiring the ability to elicit the CR.
Phase 1: CS1 \(\rightarrow\) US
Direct Link
The CS2 is never paired with the US, but it piggybacks on the existing associative strength of CS1.
Sensory Preconditioning
Learning an association between two neutral stimuli before any reinforcement occurs.
Phase 1: CS2 \(\leftrightarrow\) CS1 (Neutral)
Phase 2: CS1 \(\rightarrow\) US (Reinforced)
Test: CS2 \(\rightarrow\) CR!
Latent Learning
Evidence that the brain map is being built constantly, even in the absence of biological rewards or threats.
Conditioned Inhibition (\(V < 0\))
The Procedure
A CS+ is paired with US.
The "Safety" Signal
The CS- predicts the absence of the expected US. It acquires negative associative value.
Summation Test: (CS+) + (Inhibitor) = Reduced CR
Design Challenge
How can we distinguish between Sensory Preconditioning and Higher-Order Conditioning if the final test (CS2 \(\rightarrow\) CR) looks identical?
Variable Isolation / Control Groups / Temporal Order
Experimental Design Blueprint Experimental Design Blueprint
Isolation of Higher-Order Associative Mechanisms
The Challenge
Design a controlled experiment to isolate the phenomenon of Sensory Preconditioning from Second-Order Conditioning. Your design must account for temporal sequencing and demonstrate that associative strength is transferred without direct reinforcement.
1. Experimental Hypothesis
Define the predicted behavioral outcome for the Experimental group vs. the Control group.
2. Procedural Design
Phase Experimental Group (SP) Control Group I: Pre-exposure II: Conditioning III: Test
Dependent Variable (DV)
How will you quantify the transfer of associative strength?
Potential Confounds
What environmental factors must be held constant?
3. Theoretical Implications
If the test results show a strong CR in the experimental group, what does this tell us about the Rescorla-Wagner model's ability to explain learning without US-presence (where \(\lambda = 0\))?
Associative Research Protocol | Lab Material 4.2
Higher Order Conditioning Matrix Associative Structures Matrix
Ref: LVL-4 Cheat Sheet
Mechanism The Protocol Underlying Concept 2nd Order Conditioning 1. CS1-US 2. CS2-CS1 Associative transfer via CS predictive value. Sensory Preconditioning 1. CS2-CS1 2. CS1-US Integration of neutral associative links. Latent Inhibition 1. CS1 alone 2. CS1-US Familiarity slows future learning (low saliency). Conditioned Inhibition 1. CS1-US 2. (CS1+CS2)-No US The CS2 becomes a "safety" signal (\(V < 0\)). Blocking 1. CS1-US 2. (CS1+CS2)-US Lack of prediction error halts learning for CS2. Overshadowing 1. (CS1+CS2)-US Salience competition for associative strength.
Study Note: Note the difference between Inhibition (CS predicts NO outcome) and Extinction (repeated exposure until CR stops). They represent different mathematical states in the Rescorla-Wagner model.
Inhibitory Learning Slides Theory to Therapy
The Inhibitory Learning Model
Lesson 5: Rethinking Exposure
Emotional Processing Theory
Traditional goal: Fear Reduction within the session.
Assumes that if the patient feels "calm" during exposure, the fear is habituated and the session was successful.
"The Habituation Myth"
Research shows within-session fear reduction does NOT predict long-term outcomes.
Inhibitory Learning Model (ILM)
The goal is Expectancy Violation, not fear reduction.
Exposure should maximize the surprise (prediction error) regarding the feared outcome.
"The patient stays scared, but learns that the feared disaster does not happen."
This creates a powerful inhibitory trace (\(V_{neg}\)) that competes with the fear memory.
Optimizing Extinction
Variability
Varying time, place, and stimulus properties during exposure to prevent context-specificity.
Deepened Extinction
Extinguishing multiple cues together to use up all associative value (Rescorla-Wagner principle).
No Safety Signals
Removing medication or therapist presence to ensure the patient attribute safety to the CS-US link failure.
Synthesis
Old View
"Stay in the situation until you feel calm."
Modern View
"Stay in the situation until you learn what happened."
Craddock et al. (2012): Extinction is about information, not emotion.
Exposure Protocol Project Protocol Designer
INHIBITORY LEARNING INTERVENTION | PROJECT 05
Status: Research Draft
Clinical Objective
"Traditional exposure for a patient with Panic Disorder (Fear of heart attack) focuses on staying in the panic until the heart rate drops. Your task is to design a protocol that focuses on Expectancy Violation . How can we prove to the patient's amygdala that the heart rate increase does not lead to death, even if they remain highly anxious?"
1. Identification of Expectancy
Define the catastrophic belief (the US) and the specific conditions (\(\lambda\)) that must be violated.
Primary CS (Triggers)
Predicted US (Disaster)
2. Maximizing Prediction Error (\(\lambda - \Sigma V\))
Describe the exposure trials. How will you ensure the 'surprise' is high? (e.g., using "multiple cue extinction" or "shorter, higher-intensity trials").
3. Prevention of Renewal
How will you apply the 'multiple context' principle to ensure the inhibitory trace is not restricted to the therapy room?
4. Safety Gating
Identify 3 potential "Safety Signals" (CS inhibitors) that must be removed to ensure the patient attributes the lack of disaster to the CS itself, not a protective factor.
Signal 1
Signal 2
Signal 3
Protocol Framework v4.0 Department of Clinical Psychology
Theory to Practice Rubric Project Evaluation Criteria
Translational Research Rubric
Theoretical Grounding (30%)
[EXEMPLARY / COMPETENT / EMERGING]
The protocol clearly identifies the expectancy violation goal and correctly applies the Rescorla-Wagner error term to the proposed trials. Does the student explain WHY surprise is being prioritized over habituation?
Mechanistic Application (30%)
[EXEMPLARY / COMPETENT / EMERGING]
Identification and systematic removal of Safety Signals. The protocol demonstrates an understanding of how inhibitors (GABAergic/IL-ITC circuitry) gate the fear response and how to bypass them.
Generalization Strategy (25%)
[EXEMPLARY / COMPETENT / EMERGING]
The plan includes robust multi-context training to address Renewal and Spontaneous Recovery. Proactive measures against ABC and ABA renewal are explicitly detailed.
Clinical Feasibility (15%)
[EXEMPLARY / COMPETENT / EMERGING]
The protocol is ethically sound and realistically implementable within a clinical setting while maintaining the high intensity required for inhibitory learning.
Instructor Feedback Note: Focus feedback on the student's ability to pivot from "feeling better" as a metric to "learning better" (informational acquisition) as the primary success indicator.
