Bias Breakdown Worksheet Bias Breakdown Worksheet
Lesson 1: Psychology of Trust in Automation
STUDENT NAME: ____________________
DATE: ____________________
Case Study: The "Plausible Patch" Failure
In a 2024 experiment, 100 experienced developers were tasked with fixing a critical memory leak. They were provided with an AI-generated solution that included a detailed, authoritative-sounding comment block explaining why the fix worked.
"Implementing a thread-safe mutex lock with lazy initialization to prevent the race condition identified in the buffer allocation logic."
The code, however, actually introduced a Time-of-Check to Time-of-Use (TOCTOU) vulnerability. 72% of participants approved the code for production without running a stress test.
1. Cognitive Mechanism Identification
Identify which specific cognitive bias (e.g., automation bias, halo effect, fluency heuristic) is most prominent here. Why does the 'authoritative comment' bypass technical skepticism?
2. The "Plausibility Gap"
Explain how 'Vibe Coding' (relying on the aesthetic and narrative quality of code) creates a false sense of security. Contrast 'Correctness' vs. 'Coherence' in this context.
3. Resistance Strategy
Propose one procedural change to a code-review workflow that would force a developer to break the "Bias Loop" when reviewing AI-generated patches.
Synthesis & Critical Evaluation
High-Trust Environment
Rapid deployment cycles
High volume of boilerplate
AI provides helpful explanations
Social proof (others use it)
Skeptical Framework
"Trust but Verify" is insufficient
Adversarial code review
Testing-first generation
Zero-knowledge review
4. Mapping the Automation Bias Spectrum
Where do you personally fall on the spectrum of AI trust (1: Paranoia to 10: Pure Reliance)? Justify your position based on your professional experience or current academic work.
5. Future Implications: The "Skill Decay" Paradox
If engineers stop manually writing logic and instead focus on 'vibe-checking' AI output, what core technical competencies are at risk of atrophy? How does this impact the ability to perform an audit in 5 years?
Vibe Coding Guardrails: Graduate Seminar
CS-702: Engineering Ethics & AI
Trust Blindness Slides Trust Blindness
The Psychology of Automation Bias
Lesson 01 | CS-702
The 70% Paradox
In recent studies, over 70% of developers accepted insecure AI code without verification.
Why does expertise fail when AI sounds confident?
70%
Acceptance Rate of Flawed Code
What is "Vibe Coding"?
Technical Coding
Logic-first approach
Step-by-step verification
Documentation as evidence
Vibe Coding
Intuition-first approach
Aesthetic verification
Narrative-driven trust
Automation Bias
"The propensity for humans to favor suggestions from automated decision-making systems and ignore contradictory information made without automation."
Errors of Omission
Failing to notice an error because the AI didn't flag it.
Errors of Commission
Following an AI instruction even when it contradicts your logic.
The Halo Effect
Assuming AI is "smarter" because it is computationally superior.
Discussion: The "Plausible Patch"
If an AI gives you a perfect explanation of why a piece of code works, does that increase the likelihood you will skip the test?
"The explanation is logically sound, so the implementation must be too."
VS.
"The explanation is a hallucinated justification for a flawed logic path."
Hallucination Hunt Lab Hallucination Hunt Lab
LESSON 2: SECURITY AUDITING
AUDITOR: _________________________
DATE: _________________________
// SYSTEM ALERT: AI GENERATED PROJECT INITIALIZED
Target: secure-auth-gateway-v2
AI Assistant: VibeCoder-Alpha
Status: Build Successful (Warning: 0% dependency validation performed)
The Mission
You are conducting a security audit on a mission-critical authentication gateway. The previous lead developer used a generative AI to build the dependency tree. While the code compiles and passes local tests, three "hallucinated packages" are suspected to be hidden in the package.json. These non-existent libraries could be registered by an attacker at any moment to execute a Supply Chain Takeover .
Task 1: The Dependency Manifest Scan
package.json snippet
"dependencies": {
"express": "^4.18.2",
"jsonwebtoken": "^9.0.2",
"argon2-crypto-lite": "^2.1.0",
"zod": "^3.22.4",
"helmet-shield-pro": "^1.0.5",
"dotenv": "^16.4.5",
"react-auth-hooks-secure": "^0.8.2"
}
Audit Instructions:
Search public registries (simulated via your local 'registry truth' list).
Identify which three packages are hallucinations.
Predict the vulnerability type if an attacker registers the name.
Suspect #1
Suspect #2
Suspect #3
Task 2: Heuristic Analysis
Explain the "Linguistic Plausibility" of the hallucinated packages identified in Task 1. Why did the LLM likely suggest these specific names instead of random strings?
Task 3: The Counter-Prompt Strategy
Design a "Negative Constraint" prompt that forces the AI to cross-verify its library suggestions against a verified trust-list or explicitly mention when it is 'predicting' a package name.
Prompt Editor
Ethical Deep Dive: Liability
If a hallucinated package is exploited in a production environment, where does the legal and ethical liability lie? Rank the following (1-4) and provide a brief justification for your #1 choice.
[ ] The AI Model Provider
[ ] The Individual Developer
[ ] The Security Auditor
[ ] The Sponsoring Organization
Audit Facilitation Guide Hallucination Facilitation Guide
Teacher Resource | Lesson 2: Security Auditing
Confidential
Answer Key & Plan
Learning Goal
Students identify how LLMs predict probable but non-existent dependency names.
Pacing
60-minute workshop: 15m Intro, 30m Lab, 15m Debrief.
Risk Factor
Focus on 'Linguistic Plausibility' over technical syntax.
Task 1 Answer Key: The Registry Truth
Package Name Status Explanation for Hallucination express REAL Standard Node.js framework. argon2-crypto-lite Hallucination The LLM combined "Argon2" (real hashing algorithm) with common suffixes "crypto" and "lite". helmet-shield-pro Hallucination Synthesized from "helmet" (real security middleware) + "shield" + "pro". Very common AI naming pattern. react-auth-hooks-secure Hallucination A "semantic drift" error where the AI predicts a hook library for auth that sounds logical but doesn't exist.
Facilitation Notes
Key Concept: Supply Chain Takeovers
Remind students that attackers monitor LLM hallucination patterns. If a specific "hallucinated package" becomes common in AI outputs, an attacker will register that exact name on NPM/PyPI with malicious code, turning a "hallucination" into a "backdoor."
Observation Check
During the lab, look for students who only check if the package name "sounds right." Redirect them to perform actual manual searches (simulated) or use dependency checkers like npm view.
Task 3 Grading Rubric: Counter-Prompt Strategy
Exemplary Prompt Response
"Generate the dependency list, but for every package suggested, you MUST provide a direct link to the official repository. If you are unsure of the package name or it is a predictive suggestion, label it [UNCERTAIN] and suggest the closest verified alternative from the following list: [X, Y, Z]."
Poor Prompt Response
"Don't hallucinate packages. Only use real ones."
FAIL: LLMs lack self-awareness of hallucinations. Negative constraints without verification methods are ineffective.
Workshop Debrief Questions
How did the "build successful" warning in the lab intro contribute to your trust level?
If an AI hallucination is 90% correct syntax but 10% non-existent library, is it more dangerous than a completely broken script?
IP Ethics Case File IP & Ethics Case File
Ref: 2026-AI-IP-ETHICS | Confidential Analysis
CASE OFFICER: ____________________
RESTRICTED ACCESS
Case Study: The Ghost in the Machine
A multinational fintech firm utilizes an AI assistant to refactor a critical payment processing module. The AI generates a 500-line sorting algorithm that is 98% identical to a proprietary routine used by a competitor, which was originally published in a private repo but accidentally leaked in 2023.
The competitor sues for copyright infringement. The fintech firm argues they didn't 'copy' anything—the AI simply 'suggested' a logical solution.
Legal Precedents
Thaler v. Perlmutter: AI-generated works without human authorship cannot be copyrighted.
GPL Viral Clause: If AI outputs code from a GPL-licensed repo, does the whole project become GPL?
Analysis 1: Authorship vs. Attribution
If an engineer 'prompts' the AI and then 'edits' 5% of the code, who is the author of record? Define the "Human-in-the-Loop" threshold for legal ownership.
Analysis 2: Licensing Contamination
You are a Senior Architect. Your team has inadvertently committed AI-generated code that resembles AGPL-3.0 licensed code. Evaluate the risk of "License Poisoning" for your proprietary product.
Analysis 3: The Ethics of Training Data
Open-source developers contribute code with specific expectations of attribution or reciprocal sharing. Using that code to train a commercial AI that then sells the 'suggestions' back to developers is viewed by some as an Ethical Supply Chain Violation . Argue for OR against this perspective below.
ARGUMENT: "EXPLOITATION"
ARGUMENT: "EVOLUTION"
The Licensure Risk Matrix
Categorize the following license types by their 'Risk Profile' when used in AI training/output contexts.
Permissive
MIT, Apache 2.0
Assign Risk Level
Weak Copyleft
LGPL, MPL
Assign Risk Level
Strong Copyleft
GPL, AGPL
Assign Risk Level
REF ID: CS702-MODULE-03-IP
© 2026 GRADUATE ENGINEERING SEMINAR
License Logic Slides Legal Lines
IP, Licensing & AI Attribution
The Attribution Conflict
Traditional Software
"Author writes code → Author owns copyright → Author grants license."
AI-Assisted Software
"AI digests $N$ authors → AI 'predicts' snippet → Human 'accepts' code → Who owns it?"
The "Human Authorship" Bar
NOT Copyrightable
Code purely output by an LLM
"Copy-pasted" AI suggestions
Functionally generic logic (sorting, basic API routes)
Copyrightable
The selection and arrangement of AI snippets
Significant manual refactoring
Human-written architecture & logic
License "Contamination"
The Training Set
AI models are trained on both permissive (MIT) and copyleft (GPL) codebases.
The Leakage
The AI may output a "unique" 20-line block that actually belongs to a GPL repo.
The "Viral" Risk
If that 20-line block is deemed derivative, your entire proprietary app may legally require open-sourcing.
How to Defend Your IP
01
Implement AI-blocking 'scanners' for known licenses (e.g., GitHub Copilot filters).
02
Maintain a rigorous audit log of human vs. AI contributions.
03
Use 'Clean Room' refactoring for critical proprietary algorithms.
Audit Speed Heuristics Audit Speed Heuristics
TACTICAL REFERENCE: HIGH-VOLUME AI VALIDATION
The Triage Protocol
Level 1: Structural Scan
"Does the architecture make sense?"
Verify imports & dependencies.
Check function signatures.
Look for "comment-to-code" alignment.
Level 2: Logic Validation
"Can this code actually fail?"
Trace the "Happy Path."
Locate state changes (mutation).
Identify recursive base cases.
Level 3: Safety Audit
"Where are the backdoors?"
Input sanitization check.
Boundary condition stress.
Permission/Auth leakage.
Speed Heuristics
01
The Pattern Match
Scan for repetitive boilerplates. AI often repeats logic with slight variations that introduce subtle bugs.
02
The "Magic" Constant
Flag hardcoded IDs, salts, or buffer sizes. AI loves "perfect numbers" that break in production.
03
The 'Hallucinated' Property
Check object properties against their interfaces. LLMs often "wish" properties into existence.
Critical Focus Areas
Spend 80% of your time on these high-risk areas:
Regex Patterns
Database Queries
Auth Handlers
File System I/O
Cryptographic Ops
Loop Terminations
VIBE CODING GUARDRAILS | MODULE 4
VER: 1.0.4 AUTH: GRADUATE SEMINAR
The 10-Minute Audit Challenge The 10-Minute Audit Challenge
Lesson 4: Advanced Code Review Simulation
Limit: 600 Seconds
Simulation Brief
You are the lead architect for a high-frequency trading firm. An AI has generated a mission-critical Order Matching Engine refactor. You have exactly 10 minutes to identify the four critical vulnerabilities hidden in the code snippet below before the deployment window closes.
OrderMatcher.cpp READ-ONLY
1 void matchOrders(Order* newOrder, OrderBook* book) {
2 std::lock_guardstd::mutex lock(book->mtx);
3 for (auto& existingOrder : book->orders) {
4 if (newOrder->price >= existingOrder.price && newOrder->side != existingOrder.side) {
5 double totalValue = newOrder->quantity * newOrder->price;
6 if (existingOrder.quantity >= newOrder->quantity) {
7 processTrade(newOrder, &existingOrder);
8 existingOrder.quantity -= newOrder->quantity;
9 newOrder->quantity = 0; // Filled
10 } else {
11 processTrade(&existingOrder, newOrder);
12 newOrder->quantity -= existingOrder.quantity;
13 existingOrder.quantity = 0;
14 }
15 logTrade(totalValue, existingOrder.id);
16 }
17 }
18 if (newOrder->quantity > 0) book->orders.push_back(*newOrder);
19 }
Vulnerability Audit Log
Issue #1: Arithmetic/Logic
Issue #2: Race Condition / Mutex
.
Issue #3: Data Integrity / Precision
Issue #4: Performance / Complexity
Simulation Reflection
1. Heuristic Effectiveness
Which heuristic from your reference sheet helped you spot the most critical bug? If none did, what new heuristic would you add to your toolkit specifically for C++ order-matching logic?
2. The "Acceptance vs. Audit" Threshold
In a real production environment, would you have approved this code if the AI provided a 100-line unit test suite that all passed green? Why is a "passing test suite" a dangerous signal for AI code?
Performance Metric
BUGS FOUND: ____ / 4
TOTAL TIME: ____ MIN ____ SEC
STAMP: [ APPROVED / REJECTED ]
The Evolution Seminar Cards The Architect Evolution
Lesson 5: Capstone Seminar Provocation Cards
Seminar Instructions: In your groups, select one card to debate for 10 minutes. Each card presents a "future-state" scenario or ethical dilemma. You must defend your position using the technical and legal frameworks established in Lessons 1-4.
Economics
The "Junior Gap" Crisis
If senior engineers use AI to do the work of three juniors, how does the next generation of engineers learn "deep coding" skills? Is the entry-level role permanently replaced by AI oversight?
PROMPT #01 KEYWORD: ATROPHY
Liability
The Auditor's Oath
Should "AI Auditor" become a legally protected title like Professional Engineer (PE)? Should engineers be personally liable for AI logic they didn't write but "approved"?
PROMPT #02 KEYWORD: MALPRACTICE
Culture
Death of the 'Code Monkey'
Engineering is shifting from "writing syntax" to "defining intent." Does this make software engineering a branch of Philosophy or Linguistics rather than Mathematics?
PROMPT #03 KEYWORD: INTENTIONALITY
Security
The Zero-Knowledge Future
In 10 years, software will be too complex for any one human to understand. We will rely entirely on "AI Security Auditors" to check "AI Developers." Where is the human in this loop?
PROMPT #04 KEYWORD: OPAQUENESS
Print on heavy cardstock for best results
Professional Ethics Rubric Professional Ethics Rubric
Lesson 5: Architect Evolution Seminar Assessment
COURSE ID
CS-702-ETH
Assessment Context
This rubric is designed to evaluate graduate-level engagement in the capstone seminar. Students are assessed on their ability to synthesize technical security risks (hallucinations, bias) with legal and ethical frameworks (IP, professional liability) to project the future of the engineering profession.
Criterion Emerging (1) Proficient (2) Advanced (3) Synthesis of Risk Identifies AI risks as separate, isolated incidents. Connects technical hallucinations to broader security and legal liabilities. Develops a comprehensive framework where psychological, technical, and legal risks are interdependent. Role Definition Describes the future role as simply "using AI more." Defines specific new responsibilities (e.g., Lead Auditor, Prompt Architect). Articulates a paradigm shift from "Author" to "Architect," detailing new skill sets required for seniority. Ethical Reasoning Focuses solely on individual responsibility or "following orders." Analyzes liability from both an individual and organizational perspective. Critically evaluates systemic ethical failures (e.g., training data exploitation) and proposes industry-wide standards. Argumentative Rigor Opinions are shared without grounding in seminar materials. Uses case studies and precedents from previous modules to support claims. Anticipates counter-arguments and provides nuanced, evidence-based rebuttals.
Instructor Evaluation Notes
STRENGTHS
AREAS FOR GROWTH
Seminar Debrief & Meta-Analysis
Post-Seminar Checklist for Teacher
Did students acknowledge the "Skill Decay" paradox?
Was there a consensus on who holds liability for AI hallucinations?
Did students differentiate between "Permissive" and "Copyleft" risks in the final debate?
How many students suggested new professional certifications for AI auditing?
Recommended Extension Activity
"The Manifest of the Responsible Architect": Have students draft a 10-point code of ethics for engineers working in AI-dominant environments. This document should serve as their personal 'north star' as they enter the workforce.
VIBE CODING GUARDRAILS | FINAL ASSESSMENT RESOURCE
