/CAPS-Spring-2019

The spring 2019 iteration of the CAPS seminar

Computational Applications to Policy and Strategy

The CAPS AI Policy and Strategy (CAPS) skills course is an immersive six-part seminar that trains participants in the role of a fictional Forward-Deployed Policy Engineer tasked with overseeing AI systems deployed in an ongoing conflict zone.

The course begins with an AI bootcamp and quickly moves participants “into the field” to confront case studies rooted in real needs in the test environment of Afghanistan's contested territories. The seminar was most recently taught at Johns Hopkins SAIS in Spring 2019.

Updates

We are excited to announce that CAPS will continue to be taught at Johns Hopkins SAIS in the Fall 2019 semester. The Spring 2019 course materials can be found in the primary "Docs" directory.

Contents

Logistics

CAPS takes place over three weeks, with two 90 minute sessions per week. The first session will be on Wednesday, April 3.

  • Wednesdays 6:00–7:30 pm in Nitze 507

  • Fridays 6:00–7:30 pm in BOB 736

📧 For enrollment or questions, contact capsseminar@gmail.com

Syllabus

A pdf version of the syllabus can be found here (last updated, April 01 2019).

Course Materials and Overview

All materials not linked below are forthcoming. All can also be found in and downloaded from the Docs folder.

📘 Lecture 1 - Foundations Bootcamp: Introduction to Human Factors and Reinforcement Learning

  • Guiding question: How do human teams make decisions and how does this decision-making compare to autonomous decision processes, such as those of reinforcement learning algorithms?

  • Topics covered: Learning from interaction; Decision-making in human teams; Reinforcement learning; Markov decision processes; Bellman equation

  • Case: Learning in a counterinsurgency team

  • Notes | Slides | Case

📘 Lecture 2 - Rule-Based Decision Making in a Fuzzy World

  • Guiding question: How can we transform fuzzy descriptions of specialized human performances into computable knowledge for an autonomous system to act on?

  • Topics covered: Rule-based systems; Finite-state machines; Basic search algorithms; State space complexity; System requirements in fuzzy environments

  • Case: Designing and evaluating a rule-based system to clear a conflict zone

  • Slides | Case

📘 Lecture 3 - Learning to Make Decisions with and without a Model of the World: From Learning Architectures to Decision Profiles

  • Guiding question: How can we conceptualize core differences in learning architectures and apply this knowledge to augment partial observations of a reinforcement learner’s performance?

  • Topics covered: Q-learning; Value iteration; Basic inverse reinforcement learning; Black box problems

  • Case: Determining and evaluating possible learning architectures of an enemy drone

  • Notes | Slides | Case: Base Code and Supplementary Code

📘 Lecture 4 - Goal Specification and Reward Design

  • Guiding question: How can we use human input throughout the reinforcement learning process and during deployment to ensure optimal system performance and what are the trade-offs of this approach?

  • Topics covered: Goal specification; Reward design; Human-in the-loop reinforcement learning; Shared autonomy

  • Case: Improving the operation of a semi-autonomous supply convoy in a contested environment

  • Slides

📘 Lecture 5 - Implementing Value Iteration in Python

  • Guiding question: How can we synthesize partial observations into stable predictions about the longterm development of AI and its impact on operational ecosystems?

  • Topics covered: Policy iteration vs. value iteration; Challenges in making inferences about live AI systems based on partial observations; Foundations of Python implementation (added per participant request)

  • Case: Developing recommendations on what form(s) of autonomous decision-making to implement in scenarios related to a counterinsurgency campaign

  • Slides

📘 Lecture 6 - Conclusion and Guest Presentation by Professor Sarah Sewall

Guiding question: How can we provide meaningful insights into a complex, technical domain for a senior policymaker to determine high-level strategy?

  • Topics covered: Holistic review (design challenges of autonomous systems, model-based vs. model-free learning, multi-agent learning, degrees of autonomy, agent-environment interaction); Guest presentation and discussion led by Professor Sarah Sewall

  • Slides

Skills Course Policy

CAPS is an official SAIS skills course. Participants who attend all sessions and pass the requirements obtain formal certification for the course on their Johns Hopkins academic transcripts.

For the course to appear on one’s trascript, they must attend all sessions. If a session is missed due to illness or an emergency, the CAPS team will arrange a make-up session.

Team and Contact

CAPS is created and taught by SAIS MAs Leo Klenner, Henry Fung, Cory Combs and JJ Lee.

The course is sponsored by Sarah Sewall, Former Undersecretary of State for Civilian Secuirty, Democracy and Human Rights (2014-2017) and Speyer Family Foundation Distinguished Scholar at the Henry A. Kissinger Center for Global Affairs.

CAPS is funded through a grant from Johns Hopkins Technology Ventures.

📧 To contact us, reach out to capsseminar@gmail.com.