Interesting Things with JC #1835: "The AI Safety Problem Nobody Can Agree On"
Facial recognition errors have contributed to wrongful arrests, while AI assisted military targeting raises questions about human oversight. Researchers have also demonstrated how AI systems can follow malicious instructions and misuse automated tools. Yet predictions about future AI escaping human control remain unproven. As AI advances medicine and scientific discovery, the debate continues over which dangers are already documented, which are technically credible, and which remain predictions.
Curriculum - Episode Anchor
Series: Interesting Things with JC
Episode Number: 1835
Episode Title: The AI Safety Problem Nobody Can Agree On
Host and Creator: JC
Curriculum System: Version 5.1.1
Audience: Grades 9–12, introductory college, homeschool, lifelong learners
Subject Areas: Artificial intelligence, computer science, cybersecurity, civics, ethics, media literacy, technology policy
Instructional Duration: 50–60 minutes, expandable to 90 minutes
Primary Resource: Interesting Things with JC #1835
Curriculum Access: Open educational resource
Central Inquiry: How should society distinguish documented artificial intelligence harms, demonstrated technical vulnerabilities, and predictions about future AI systems while preserving the technology's benefits?
Essential Understanding: Artificial intelligence safety involves different categories of evidence. Documented harms establish what has occurred, technical demonstrations establish what systems can do under specified conditions, and future scenarios require reasoned predictions about capabilities and consequences that have not yet been observed.
Instructional Position: Evidence-based, nonpartisan, technically grounded, and constructive. Students evaluate competing claims without being directed toward either unconditional acceptance or rejection of artificial intelligence.
Lesson Overview
Lesson Purpose
Students investigate why researchers, governments, and technology companies disagree about AI safety. They examine real-world failures, controlled security experiments, uncertain predictions, and efforts to improve reliability and oversight.
Learning Objectives
By the end of the lesson, students will be able to:
Distinguish documented harms from demonstrated technical vulnerabilities and speculative future scenarios.
Explain how facial recognition errors can contribute to wrongful identification.
Describe the distinction between AI decision support and autonomous decision-making.
Identify prompt injection, hallucination, and unauthorized tool use as separate technical concerns.
Evaluate claims using evidence quality, uncertainty, and potential consequences.
Explain how human oversight, independent testing, and restricted permissions can reduce certain risks.
Identify beneficial AI applications in science, medicine, and research.
Construct a defensible safety recommendation supported by credible sources.
Essential Questions
What evidence establishes that an AI system has caused harm?
Does demonstrating a vulnerability prove that it has been exploited in the real world?
How should researchers evaluate risks involving technology that does not yet exist?
When should humans retain authority over automated decisions?
Can stronger safety requirements support technological progress?
Success Criteria
Students demonstrate mastery when they can correctly classify at least 8 of 10 evidence scenarios, identify the limitations of their sources, and defend one practical safety recommendation using at least 2 credible references.
Key Vocabulary
Foundational Terms
Artificial Intelligence (AI): Computer systems capable of performing tasks associated with human cognitive abilities, including prediction, language processing, pattern recognition, and decision support.
Machine Learning: Computational methods that identify patterns in data and use those patterns to make predictions or generate outputs.
AI Safety: Research and engineering intended to reduce harmful behavior, failures, and unintended consequences from AI systems.
AI Risk: The possibility of harmful outcomes arising from an AI system's design, operation, deployment, or misuse.
Facial Recognition: Technology that analyzes facial features to compare or identify individuals.
Algorithmic Bias: Systematic differences in an algorithm's outcomes that may produce unfair or inaccurate results.
False Positive: A result that incorrectly indicates a condition, match, or event is present.
Automation Bias: The tendency to place excessive confidence in recommendations generated by automated systems.
Technical and Security Terms
Hallucination: An AI-generated output containing unsupported, fabricated, or incorrect information.
Prompt Injection: An attack in which malicious instructions embedded in lower-trust material attempt to redirect an AI system's behavior.
AI Agent: A system that uses an AI model to plan or execute actions, potentially involving external tools.
Tool Permissions: Restrictions defining which operations an automated system is authorized to perform.
Adversarial Testing: Evaluating a system using inputs or conditions designed to expose weaknesses.
Red Teaming: Structured testing that attempts to discover vulnerabilities, misuse opportunities, or failures.
Human Oversight: Processes through which people supervise, review, intervene in, or authorize system behavior.
Autonomous Weapon System: A weapon system that, once activated, can select and engage targets without further human intervention, according to the operational definition used in many international discussions.
Research and Policy Terms
Demonstrated Risk: A capability or failure established through observation or experimentation.
Speculative Scenario: A possible future outcome that has not been demonstrated and depends on assumptions about future conditions.
Loss of Control: A proposed situation in which humans cannot effectively constrain or redirect an AI system's behavior.
Risk Mitigation: Actions intended to reduce the probability or severity of harm.
Independent Evaluation: Assessment conducted by parties sufficiently separate from a system's developers or operators to provide meaningful scrutiny.
Narrative Core
The Documented Problem
AI safety debates begin with consequences already experienced by people.
Facial recognition systems have contributed to wrongful arrests when incorrect algorithmic matches were used in criminal investigations. These cases demonstrate the danger of treating automated outputs as conclusive evidence.
The technology does not independently arrest a person. Human investigators, procedures, and institutional decisions determine how a match is used.
The resulting harm can therefore involve both technological failure and inadequate human verification.
The Military Dimension
AI can assist military analysts by processing imagery, identifying patterns, and supporting intelligence assessments.
Decision-support systems and autonomous weapons are not interchangeable.
A decision-support system supplies information or recommendations. An autonomous weapon system may independently select and engage targets after activation.
Both raise safety concerns, but their technical behavior, legal implications, and human-control requirements differ.
International humanitarian law continues to apply to military operations involving AI.
Demonstrated Technical Vulnerabilities
Researchers have shown that AI systems can:
Generate inaccurate information with convincing language.
Produce insecure or defective computer code.
Follow malicious instructions embedded in documents or websites.
Make inappropriate tool calls when permissions and safeguards are insufficient.
These findings establish that certain failures are possible under tested conditions. They do not establish that every AI system will fail in the same way or that every demonstrated attack has occurred in a real deployment.
Future Risk and Scientific Uncertainty
Some researchers are concerned that future AI systems may acquire combinations of planning, autonomy, and strategic capabilities that make meaningful human control difficult.
Such scenarios remain uncertain.
The 2026 International AI Safety Report distinguishes existing capabilities from possible future developments and recognizes substantial uncertainty about the likelihood and timing of severe loss-of-control outcomes.
Scientific uncertainty does not justify declaring a prediction inevitable. It also does not automatically justify ignoring a potentially severe outcome.
The Constructive Response
AI safety research is producing practical approaches to evaluating systems, identifying vulnerabilities, limiting access, and improving oversight.
AI also contributes to medical research, scientific analysis, accessibility, and other useful applications.
A sound policy must consider both the risks of deploying insufficiently tested technology and the potential costs of unnecessarily restricting beneficial uses.
Transcript
Square podcast cover for Interesting Things with JC, episode #1835, “The AI Safety Problem Nobody Can Agree On.” A man studies surveillance monitors in a dimly lit operations center. One screen shows pedestrians identified by facial recognition boxes, another displays aerial imagery with targeting crosshairs, and an overhead monitor shows a military drone. The episode title appears across the upper portion in bold white and yellow lettering. The lower third contains no text.
Interesting Things with JC #1835
“The AI Safety Problem Nobody Can Agree On”
Police have arrested innocent people after facial recognition software identified the wrong suspect. In some cases, investigators treated a computer-generated match as stronger evidence than it actually was.
That is an artificial intelligence safety failure we can document.
AI is also helping military analysts process surveillance footage and identify possible targets. The International Committee of the Red Cross warns that unreliable information and excessive trust in automated recommendations can put civilians at risk. A system assisting a human decision is not necessarily a weapon making that decision itself.
Other dangers have been demonstrated in controlled tests. AI can invent convincing information, produce flawed computer code, and follow malicious instructions concealed inside material it reads. Give an AI agent permission to use outside tools, and an error can turn into an action.
But another debate concerns systems that don't yet exist at the necessary level of capability. Some researchers fear future AI could plan independently, evade oversight, and resist human efforts to regain control. The 2026 International AI Safety Report says current systems cannot reliably combine those abilities. It also warns that relevant capabilities are improving.
There is progress worth recognizing. Researchers test models against deliberate attacks, study their failures, and develop ways to restrict what automated agents can access. The National Institute of Standards and Technology provides guidance for evaluating and managing these risks. None of these measures guarantees safety, but each gives developers and users something concrete to test.
Meanwhile, AI is supporting medical research, scientific discovery, and faster analysis of complex information. Protecting those benefits requires attention to what has happened, what technology can already do, and what future systems might become.
These are interesting things, with JC.
Student Worksheet
Student Name: ____________________
Date: ____________________
Course: ____________________
Part A — Listening Comprehension
What example of a documented AI safety failure opens the episode?
Why can facial recognition results become dangerous when investigators rely on them too heavily?
How does military AI decision support differ from an autonomous weapon?
Identify 3 technical vulnerabilities discussed in the episode.
What does the 2026 International AI Safety Report say about the capabilities of current systems relative to extreme loss-of-control scenarios?
Part B — Evidence Classification
Classify each scenario using:
D: Documented real-world harm
T: Demonstrated technical vulnerability
S: Speculative future scenario
B: Documented beneficial application
A person is wrongfully arrested following an incorrect facial recognition match.
Researchers demonstrate that a malicious document can redirect an AI assistant in a controlled test.
A future AI system becomes capable of resisting every human attempt to deactivate it.
AI helps researchers analyze large collections of medical images.
An experimental AI agent attempts an unauthorized operation during a laboratory evaluation.
An AI system provides a false recommendation that contributes to a documented operational error.
Researchers predict that future AI may become capable of independently developing long-term strategies.
An AI model successfully assists scientists in identifying promising protein structures.
Part C — Analysis
Explain why a laboratory demonstration does not necessarily establish widespread real-world exploitation.
Identify one possible benefit and one possible risk of using AI in military intelligence analysis.
Why might a human supervisor accept an incorrect AI recommendation?
Explain how limiting tool permissions can reduce the consequences of an AI error.
Why is the potential severity of a future risk relevant even when its probability remains uncertain?
Part D — Evidence Evaluation
Read one primary research or institutional source from the References section.
Identify the source's author or responsible institution.
State the principal claim you are evaluating.
Describe the evidence presented.
Identify one limitation or uncertainty.
Explain whether the evidence supports a present-day harm, a demonstrated technical capability, or a future prediction.
Part E — Reflection
Should AI systems used in high-consequence decisions require independent evaluation before deployment? Defend your position with evidence.
Propose one safeguard that could improve AI safety without eliminating a useful application.
Difficulty Scaling
Level 1 — Foundational: Complete questions 1–13 with vocabulary support.
Level 2 — Intermediate: Complete questions 1–23 and cite one authoritative source.
Level 3 — Advanced: Complete all 25 questions and construct a counterargument addressing uncertainty, implementation costs, or technical limitations.
Student Output: A completed worksheet and a 150–250-word evidence-based response.
Academic Integrity Guidance: Students may use AI as a research assistant when permitted, but must independently check factual claims, cite original sources, and disclose substantial AI assistance.
Teacher Guide
Quick Start: Play episode #1835 without first presenting the three risk categories. Ask students to identify the different kinds of evidence introduced as the story progresses.
Instructional Goals
Develop scientific and technological literacy.
Strengthen source evaluation and evidence classification.
Encourage informed debate without ideological assumptions.
Demonstrate how safety engineering can coexist with technological innovation.
Materials
Episode #1835 audio or transcript.
Student Worksheet.
Internet access for primary-source research.
NIST AI Risk Management Framework.
International AI Safety Report 2026.
Paper or digital worksheet for evidence classification.
Pacing Guide — Audio First
0–5 minutes — Opening: Ask students to describe an AI failure they have encountered or read about. Distinguish personal experience from verified reporting.
5–10 minutes — Listening: Play the episode. Students identify examples of harm, technical failure, and future predictions.
10–20 minutes — Concept Instruction: Introduce the vocabulary and explain the differences among the three risk categories.
20–30 minutes — Evidence Classification: Students complete worksheet questions 6–13 individually, then compare answers.
30–40 minutes — Source Investigation: Students examine a primary source and identify its claims, supporting evidence, and limitations.
40–50 minutes — Discussion: Students consider human oversight, independent evaluation, and beneficial applications.
50–60 minutes — Assessment: Students write a short recommendation for one safety measure, supported by evidence.
Discussion Prompts
Can an AI system be technically accurate but still harmful when deployed?
How does human judgment affect the consequences of an automated error?
Should a technology's potential benefits influence acceptable risk?
What kinds of evidence would strengthen predictions about future loss of control?
How can researchers test dangerous capabilities without unnecessarily exposing the public to harm?
Formative Checkpoints
Students accurately distinguish an incorrect prediction from a documented harmful outcome.
Students recognize that AI-assisted targeting does not necessarily involve autonomous weapon engagement.
Students explain prompt injection as an attempt to redirect system behavior through untrusted input.
Students identify uncertainty in predictions about future capabilities.
Students propose safeguards appropriate to the risks identified.
Common Misconceptions
- Misconception: Every AI error represents intentional deception.
Correction: Many errors arise from statistical limitations, training data, system design, or unreliable outputs without evidence of intent.
- Misconception: Demonstrating a security vulnerability proves it has been widely exploited.
Correction: Controlled demonstrations establish feasibility under particular conditions, not prevalence.
- Misconception: All military AI systems independently select targets.
Correction: Some systems provide analysis or recommendations while humans retain operational authority.
- Misconception: Future catastrophic AI outcomes are established scientific facts.
Correction: They remain uncertain scenarios requiring further investigation.
- Misconception: Safety restrictions necessarily prevent innovation.
Correction: Appropriate testing and engineering controls can improve reliability and public confidence, although restrictions may also impose costs.
Differentiation
Additional Support: Provide a three-category evidence reference sheet and preselect shorter source excerpts.
Advanced Learners: Require comparison of technical research findings with institutional risk assessments.
English Learners: Provide vocabulary definitions, sentence frames, and opportunities for paired discussion.
Homeschool Adaptation: Replace group discussion with a written comparison of two competing research perspectives.
Introductory College: Require an additional discussion of risk probability, severity, uncertainty, and regulatory tradeoffs.
Answer Key — Student Worksheet
Wrongful arrests involving incorrect facial recognition matches.
An incorrect match may be treated as stronger evidence than warranted.
Decision support informs human choices; autonomous weapons may select and engage targets without further human intervention.
Hallucinations, insecure code, prompt injection, and inappropriate tool use are acceptable examples.
Current systems cannot reliably combine the capabilities associated with extreme loss-of-control scenarios.
D
T
S
B
T
D, provided the harm and causal contribution are documented.
S
B
Controlled experiments may involve artificial conditions, limited samples, or configurations unlike real deployments.
Benefits include faster imagery analysis; risks include false identification, automation bias, and inadequate verification.
Automation bias, time pressure, insufficient training, or mistaken confidence.
Permissions can prevent unauthorized actions or limit their consequences.
Even an uncertain event can warrant research when its potential consequences are severe.
19–25. Evaluate using the assessment criteria below. Accept different conclusions when supported by accurate evidence and sound reasoning.
Quiz
Instructions: Select the best answer. Each question is worth 1 point.
1. Which example represents a documented AI-related harm?
A. A prediction about future superintelligence
B. A wrongful arrest involving a false facial recognition match
C. A hypothetical system resisting shutdown
D. A proposed future AI architecture
2. What does prompt injection involve?
A. Physically damaging a computer
B. Improving a model's training accuracy
C. Embedding malicious instructions in material an AI system processes
D. Replacing a computer's processor
3. What is automation bias?
A. Refusing to use automated technology
B. Excessively trusting automated recommendations
C. Training a system on too little data
D. Measuring computing performance
4. Which statement best describes AI decision support?
A. It always makes final decisions independently
B. It cannot produce inaccurate information
C. It provides information or recommendations to support decisions
D. It eliminates human accountability
5. What does a controlled security demonstration establish?
A. Every deployed system is vulnerable
B. The demonstrated behavior can occur under tested conditions
C. The attack has occurred worldwide
D. Future systems will necessarily fail
6. Which organization developed the AI Risk Management Framework?
A. NASA
B. UNESCO
C. NIST
D. World Health Organization
7. Which statement about future loss-of-control scenarios is most accurate?
A. They have all been demonstrated in deployed systems
B. They are scientifically impossible
C. Their probability and timing remain uncertain
D. They require no additional research
8. Why are restricted tool permissions useful?
A. They eliminate every model error
B. They can limit the actions an AI agent is authorized to perform
C. They guarantee factual accuracy
D. They remove all cybersecurity threats
9. Which is a beneficial AI application discussed in the episode?
A. Medical research assistance
B. Unauthorized access to private accounts
C. Incorrect suspect identification
D. Malicious prompt injection
10. Which principle best reflects evidence-based AI safety?
A. Treat every future prediction as certain
B. Ignore potential risks until harm occurs
C. Evaluate documented harms, technical capabilities, and future scenarios using appropriate evidence
D. Assume human oversight guarantees safety
Quiz Answer Key
B
C
B
C
B
C
C
B
A
C
Scoring
9–10: Advanced understanding
7–8: Proficient understanding
5–6: Developing understanding
0–4: Additional instruction recommended
Assessment
Summative Assessment — AI Safety Evidence Brief
Task: Prepare a 500–750-word written brief addressing the question:
How should governments, researchers, and technology developers manage demonstrated AI harms while preparing for uncertain future risks?
Required Components
Introduction: Identify the central disagreement about AI safety.
Documented Harm: Explain one verified real-world example, including the role of human decisions.
Technical Vulnerability: Analyze one experimentally demonstrated AI failure or security weakness.
Future Scenario: Evaluate one predicted risk and explain the associated uncertainty.
Constructive Application: Identify one beneficial use of AI that responsible safety practices should preserve.
Recommendation: Propose 2 practical safety measures, explaining their benefits and limitations.
References: Cite at least 3 credible sources, including 2 primary or authoritative institutional sources.
Evidence Requirements
Students must distinguish between:
Observed events and documented outcomes.
Controlled experimental findings.
Expert forecasts or theoretical scenarios.
Policy recommendations and value judgments.
Assessment Rubric — 100 Points
Factual Accuracy — 25 points: Correct technical concepts and reliable descriptions of evidence.
Evidence Evaluation — 25 points: Appropriate sources, accurate classification, and acknowledgment of limitations.
Technical Reasoning — 20 points: Clear explanations of vulnerabilities, oversight, and safeguards.
Balanced Analysis — 15 points: Consideration of both potential harms and beneficial applications.
Organization and Communication — 10 points: Logical structure, clarity, and effective academic writing.
Citation Integrity — 5 points: Traceable references and appropriate attribution.
Performance Levels
90–100: Advanced; rigorous evidence evaluation and technically sound recommendations.
80–89: Proficient; accurate reasoning with minor gaps.
70–79: Developing; some unsupported claims or incomplete analysis.
Below 70: Requires additional instruction or revision.
Optional Advanced Extension
Design a safety evaluation protocol for a hypothetical AI research assistant that can search documents and create files.
Students must specify:
Authorized operations.
Restricted operations.
Possible prompt injection entry points.
Human approval requirements.
Logging and monitoring requirements.
At least 3 adversarial test scenarios.
Conditions requiring the system to stop operating.
Safety Boundary: This activity is conceptual. Students do not need to develop malicious prompts, compromise systems, or interact with real military or surveillance infrastructure.
Standards Alignment
Alignment Method: The following standards are instructional connections, not claims that a single lesson independently satisfies every element of each standard. Where a standard requires investigation, design, or extended research, the associated worksheet, assessment, or extension supplies that activity.
NGSS — Next Generation Science Standards
HS-ETS1-1 — Engineering Design: Students analyze a technological problem involving criteria, constraints, public safety, and competing needs. Evidence: Assessment recommendation.
HS-ETS1-2 — Engineering Design: Students break a complex safety problem into manageable components, including model reliability, tool permissions, and human oversight. Evidence: Advanced extension.
HS-ETS1-3 — Engineering Design: Students evaluate proposed safeguards against constraints and tradeoffs. Evidence: Assessment and extension.
Science and Engineering Practice — Engaging in Argument from Evidence: Students construct evidence-based positions concerning present and future AI risks. Evidence: Worksheet questions 19–25.
Science and Engineering Practice — Obtaining, Evaluating, and Communicating Information: Students investigate institutional reports and communicate conclusions. Evidence: Evidence brief.
Common Core State Standards — English Language Arts
CCSS Reading
CCSS.ELA-LITERACY.RI.9-10.1 — Cite strong and thorough textual evidence: Students support claims about AI risks using primary-source documents.
CCSS.ELA-LITERACY.RI.11-12.7 — Integrate and evaluate multiple sources of information: Students compare technical, institutional, and policy evidence.
CCSS Writing
CCSS.ELA-LITERACY.WHST.9-10.1 — Write arguments focused on discipline-specific content: Students defend an AI safety recommendation.
CCSS.ELA-LITERACY.WHST.11-12.7 — Conduct short as well as more sustained research projects: Students investigate documented harms and technical vulnerabilities.
CCSS.ELA-LITERACY.WHST.11-12.8 — Gather relevant information from multiple authoritative print and digital sources: Students evaluate reliability and cite evidence.
CCSS Speaking and Listening
CCSS.ELA-LITERACY.SL.9-10.1 — Initiate and participate effectively in collaborative discussions: Students debate oversight and risk management.
CCSS.ELA-LITERACY.SL.11-12.3 — Evaluate a speaker's point of view, reasoning, and use of evidence and rhetoric: Students assess claims made during AI safety discussions.
C3 Framework — College, Career, and Civic Life
D2.Civ.2.9-12 — Analyze the role of citizens in the U.S. political system: Students examine public participation in decisions about AI oversight.
D2.Civ.10.9-12 — Analyze the impact and appropriate roles of personal interests and perspectives on the application of civic virtues, democratic principles, constitutional rights, and human rights: Students evaluate civil liberties and surveillance concerns.
D3.1.9-12 — Gather relevant information from multiple sources: Students investigate technical research and institutional reports.
D3.2.9-12 — Evaluate the credibility of a source by examining how experts value the source: Students distinguish research evidence from unsupported predictions.
D4.1.9-12 — Construct arguments using precise and knowledgeable claims: Students develop a defensible safety recommendation.
ISTE Standards for Students
1.2 — Digital Citizen: Students evaluate responsible technology use, privacy, and the consequences of automated decisions.
1.3 — Knowledge Constructor: Students curate credible research and evaluate information quality.
1.4 — Innovative Designer: Students design a conceptual safety protocol under specified constraints.
1.5 — Computational Thinker: Students identify technical failure points and evaluate safeguards.
1.7 — Global Collaborator: Students examine how AI safety challenges involve international institutions and shared responsibilities.
Career and Technical Education — CTE
Career Readiness Competencies
Risk Assessment: Identify hazards, estimate consequences, and propose controls.
Cybersecurity: Explain prompt injection, permissions, and system monitoring.
Systems Engineering: Evaluate reliability, human oversight, and failure prevention.
Technical Communication: Translate research findings into recommendations for nontechnical audiences.
Professional Ethics: Consider public safety, accountability, and the consequences of technological deployment.
Career Connections
AI safety researcher
Cybersecurity analyst
Machine learning engineer
Systems safety engineer
Technology policy analyst
Digital forensics specialist
Research scientist
ACRL — Framework for Information Literacy for Higher Education
Authority Is Constructed and Contextual: Students examine why technical researchers, government agencies, and international organizations may evaluate AI risks differently.
Information Creation as a Process: Students distinguish peer-reviewed research, institutional reports, policy commentary, and speculation.
Information Has Value: Students consider the consequences of inaccurate or misleading technological claims.
Research as Inquiry: Students investigate unanswered questions about future AI capabilities.
Scholarship as Conversation: Students compare differing interpretations of emerging technical evidence.
Searching as Strategic Exploration: Students locate primary research and evaluate its relevance.
Bloom's Revised Taxonomy
Remember: Define AI safety, hallucination, and prompt injection.
Understand: Explain the distinction between observed harms and future predictions.
Apply: Classify evidence scenarios.
Analyze: Identify relationships among technological failures, human decisions, and consequences.
Evaluate: Judge the strength and limitations of competing claims.
Create: Develop an evidence-based safety recommendation or evaluation protocol.
UDL — Universal Design for Learning
Engagement: Offer meaningful choices in case studies and research questions.
Representation: Provide audio, transcript, vocabulary support, and accessible source excerpts.
Action and Expression: Permit written briefs, oral presentations, or structured evidence analyses where assessment objectives allow.
Accessibility: Provide clear heading hierarchy, readable text, and descriptive alternatives for any supplementary visuals.
United Kingdom — National Curriculum and Qualifications
Key Stage 3
Computing: Understand computational systems, digital safety, and the responsible use of technology.
Citizenship: Examine rights, responsibilities, and informed participation in public decisions.
English: Evaluate arguments, distinguish facts from opinions, and communicate evidence.
Key Stage 4
GCSE Computer Science: Connect system reliability, cybersecurity, and ethical considerations to real-world applications.
GCSE Citizenship Studies: Investigate privacy, civil liberties, public accountability, and technology governance.
GCSE English Language: Analyze informational texts and construct supported arguments.
Key Stage 5
A Level Computer Science: Examine artificial intelligence, system limitations, security, and ethical implications.
A Level Politics: Evaluate government responsibilities and regulatory approaches.
Extended Project Qualification: Conduct independent research into AI risk, technical evidence, and public policy.
International Baccalaureate — IB
Diploma Programme Computer Science: Analyze AI applications, technical limitations, and responsible system design.
Theory of Knowledge: Evaluate how evidence, uncertainty, and expert disagreement influence knowledge claims.
Digital Society: Investigate surveillance, automated decisions, rights, and governance.
Extended Essay: Develop a focused research question involving AI safety or technological ethics.
Cambridge International
Cambridge IGCSE Computer Science: Explore the effects of computing technologies, security, and system reliability.
Cambridge IGCSE ICT: Consider responsible technology use and information security.
Cambridge International AS & A Level Computer Science: Examine system design, AI concepts, security, and societal implications.
Cambridge Global Perspectives: Evaluate competing perspectives, research quality, and evidence-based conclusions.
Standards Implementation Note: Teachers should confirm the applicable edition, course specification, jurisdiction, and local assessment requirements before claiming formal compliance.
Show Notes
Episode: Interesting Things with JC #1835
Title: The AI Safety Problem Nobody Can Agree On
Host: JC
Website: https://JimConnors.net
Format: Short-form educational nonfiction podcast
Primary Themes: Artificial intelligence, technological risk, cybersecurity, civil liberties, military applications, scientific uncertainty, human oversight
Episode Summary
Artificial intelligence has already contributed to documented failures involving facial recognition, while researchers continue to identify vulnerabilities in automated systems. Other concerns involve possible future AI capabilities that have not yet been demonstrated.
Episode #1835 examines the difference between established harms, experimentally demonstrated risks, and predictions about future systems. It also explores how safety research and independent evaluation can help preserve AI's benefits in medicine, science, and other fields.
Key Takeaways
Documented AI-related harms require investigation and accountability.
Laboratory demonstrations establish capabilities under specified conditions, not necessarily widespread exploitation.
Military decision support and autonomous weapons have different operational characteristics.
Future loss-of-control scenarios remain uncertain and require continued research.
Independent evaluation, restricted permissions, and human oversight can reduce certain risks.
AI safety and beneficial innovation are not inherently opposing goals.
Suggested Classroom Use
Technology ethics
Computer science and cybersecurity
Civics and public policy
Media literacy and evidence evaluation
Scientific reasoning and uncertainty
Introductory engineering design
Suggested Discussion Question
How can society make responsible decisions about AI when some risks are already documented, others are experimentally demonstrated, and still others depend on uncertain future developments?
Content Advisory
This lesson discusses wrongful arrests, surveillance, military targeting, cybersecurity vulnerabilities, and potential technological harms. It does not provide instructions for conducting cyberattacks or operating weapons.
Open Curriculum: https://JimConnors.net
References
Primary Research and Institutional Resources
1. International AI Safety Report 2026
Publisher: International AI Safety Report
Resource Type: International scientific assessment
Relevance: General-purpose AI capabilities, documented harms, technical vulnerabilities, emerging risks, and uncertainty surrounding future loss-of-control scenarios.
https://internationalaisafetyreport.org/
2. National Institute of Standards and Technology — AI Risk Management Framework
Publisher: NIST, U.S. Department of Commerce
Resource Type: Federal technical guidance
Relevance: Framework for identifying, measuring, managing, and governing AI risks.
https://www.nist.gov/itl/ai-risk-management-framework
3. NIST — Artificial Intelligence Risk Management Framework: AI RMF 1.0
Publication: NIST AI 100-1, January 2023
Resource Type: Primary technical framework
Relevance: Trustworthy AI characteristics, governance, measurement, and risk management practices.
https://doi.org/10.6028/NIST.AI.100-1
4. NIST — AI RMF Generative AI Profile
Publication: NIST AI 600-1, July 2024
Resource Type: Technical risk management guidance
Relevance: Risks associated with generative AI, including information integrity, security, and evaluation.
https://doi.org/10.6028/NIST.AI.600-1
5. International Committee of the Red Cross — Artificial Intelligence and Machine Learning
Publisher: International Committee of the Red Cross
Resource Type: Humanitarian law and policy resources
Relevance: AI-assisted military decisions, autonomous weapons, civilian protection, and human responsibility.
6. National Academies of Sciences, Engineering, and Medicine
Publisher: National Academies
Resource Type: Scientific and engineering research
Relevance: Technology assessment, scientific uncertainty, responsible innovation, and evidence-based public policy.
https://www.nationalacademies.org/
Additional Educational and Technical Resources
7. UNESCO — Recommendation on the Ethics of Artificial Intelligence
Publication: 2021
Resource Type: International ethics framework
Relevance: Human rights, oversight, accountability, fairness, and governance.
https://www.unesco.org/en/artificial-intelligence/recommendation-ethics
8. OECD — AI Principles
Publisher: Organisation for Economic Co-operation and Development
Resource Type: International policy principles
Relevance: Trustworthy AI, human rights, transparency, robustness, and accountability.
https://oecd.ai/en/ai-principles
9. MITRE ATLAS — Adversarial Threat Landscape for Artificial-Intelligence Systems
Publisher: MITRE
Resource Type: AI security knowledge base
Relevance: Documented adversarial techniques, security weaknesses, and defensive research.
10. OWASP — Top 10 for Large Language Model Applications
Publisher: Open Worldwide Application Security Project
Resource Type: Application security guidance
Relevance: Prompt injection, insecure output handling, excessive agency, and related AI application vulnerabilities.
11. National Institute of Justice — Facial Recognition Technology
Publisher: U.S. Department of Justice, National Institute of Justice
Resource Type: Criminal justice research and technology information
Relevance: Facial recognition applications, identification accuracy, and law enforcement considerations.
12. Stanford University — AI Index Report
Publisher: Stanford Institute for Human-Centered Artificial Intelligence
Resource Type: Annual research synthesis
Relevance: AI capabilities, investment, adoption, research developments, and reported incidents.
https://hai.stanford.edu/ai-index
Reference Use Guidance
Prioritize original technical research and official institutional publications.
Distinguish direct evidence from secondary interpretation.
Check publication dates when discussing rapidly changing AI capabilities.
Avoid presenting hypothetical outcomes as observed events.
Treat institutional guidance as recommendations unless it establishes a binding legal requirement.
Recognize that scientific reports may describe uncertainty rather than definitive predictions.
