2020 IEEE Conference on Cognitive and Computational Aspects of Situation Management (CogSIMA)

Tutorial 1: Human-AI Teaming

Abstract: Artificial Intelligence (AI) has produced an incredible portfolio of capabilities. Research has shown that some of the greatest impacts of AI come when it is successfully integrated with a human cognitive workflow in a teaming paradigm. In this way, Human-AI teaming may offer an optimization in the coupling of AI Situation Management software with human insight and strategy maintenance skills resulting in an amplification of existing capability. This tutorial will introduce concepts in Human-AI teaming including "teaming intelligence" and Human Aware AI (HAAI). A historic perspective of Human-AI teaming will be presented culminating in a description of the state of the art and current research challenges. The benefits of designing for Human-AI integration will be presented, as well as the risks of not considering human teaming for AI. Real world examples will be offered. An architecture for Human-AI teaming will be described, and domain specific frameworks based on the architecture will be presented for cyber security, predictive maintenance, and multi-domain operations.

Instructor's biography: Dr. Alicia Ruvinsky is a Research Computer Scientist in the Scientific Software branch of the Information Technology Laboratory (ITL) within the US Army Corp of Engineers' Engineer and Development Center (USACE ERDC). She has 17 years of experience in government funded research and development in computer science, software engineering, and data science. Dr. Ruvinsky has participated in various innovative research efforts funded by DARPA, IARPA, OSD, and Joint Artificial Intelligence Center. Dr. Ruvinsky received her PhD in Computer Science from the University of South Carolina in 2009 with dissertation work in multiagent systems and pro-social behavior. She received her MS in Mathematics and Computer Science, and her BS in Mathematics. Prior to joining ERDC in 2016, Dr. Ruvinsky held a senior software engineering position with Lockheed Martin Advanced Technology Labs since 2009.

Instructor's biography: Dr. Cody Salter is a Research Mechanical Engineer in the Institute for Systems Engineering Research (ISER) of the Information Technology Laboratory (ITL) within the U.S. Army Engineer Research and Development Center (ERDC). He holds a Ph.D. in Systems Engineering and an MS and BS in Mechanical Engineering, all from the University of South Alabama. Dr. Salter joined the ERDC team after completing his Ph.D. in Systems Engineering under the SMART Scholarship program. At ERDC, Dr. Salter has been involved with research projects ranging from the aerostructural defeat of unmanned aerial vehicles to applications of systems engineering in big data analytics. Currently, Dr. Salter leads a team of researchers investigating applications of data science and machine learning in mission engineering and cybersecurity.

Instructor's biography: Mr. Trent May is a Computer Scientist in the Scientific Software Branch of the Information Technology Laboratory (ITL) within the U.S. Army Corps of Engineers' Engineer Research and Development Center (USACE ERDC). He holds a BS in Computer Science and a MS in Cybersecurity and Information Assurance from Mississippi College. Mr. May's experience includes software application development and contributions in research initiatives for Cyber Data Analytics and Human-Artificial Intelligence Teaming. Mr. May is an active Capture-the-Flag (CTF) cybersecurity competition participant and enjoys researching offensive cyber tactics, secure computing practices, and low-resource computing constructs and strategies.

Tutorial 2: Design, build, assess, and refine a synthetic teammate: Lessons and guidance from over a decade of research

Abstract: Led by Dr. Nancy Cooke and Dr. Chris Myers, our team will present this tutorial with contributions from collaborators from government, industry, and academia. Together we will cover the effort necessary to create a synthetic teammate that can perform a complex task as part of a team with humans. Topics will include:

• Defining the goals of the research and selecting appropriate tasks
• Task analysis and scoping
• Building an ontology of the task and team interactions
• Designing the components of the teammate
• Complications to look out for: executive control, testing a unique system, natural language processing, etc.
• Assessing the teammate: individual performance, team performance, team coordination dynamics, and cognitive plausibility

Instructor's biography: Nancy J. Cooke is a professor of Human Systems Engineering at Arizona State University and directs ASU's Center for Human, AI, and Robot Teaming, and the Advanced Distributed Learning DOD Partnership Lab. She received her PhD in Cognitive Psychology from New Mexico State University in 1987. Dr. Cooke is a past President of the Human Factors and Ergonomics and recently chaired a study panel for the National Academies on the Enhancing the Effectiveness of Team Science. Dr. Cooke was a member of the US Air Force Scientific Advisory board from 2008-2012. Dr. Cooke's research interests include the study of individual and team cognition and its application to the development of cognitive and knowledge engineering methodologies, human-AI-robot teaming, cyber security, intelligence analysis, remotely-piloted aircraft systems, healthcare systems, and emergency response systems. Dr. Cooke specializes in the development, application, and evaluation of methodologies to elicit and assess individual and team cognition. Her work is funded primarily by DoD.

Instructor's biography: Dr. Christopher Myers is an experienced computationally oriented cognitive scientist who leverages modeling and simulation to improve the foundational understanding of human cognition and to develop intelligent systems capable of partnering with humans within teams. This research agenda is manifest across two broad areas of current research: synthetic teammates and physiocognitive models. Within each area, Dr. Myers has led multiple interdisciplinary research teams across multiple collaborations within AFRL as well as with academia and industry. Dr. Myers obtained his doctorate from the Cognitive Science department under the advisement of Dr. Wayne Gray at Rensselaer Polytechnic Institute. He was a postdoctoral scholar with Dr. Nancy Cooke where he collaborated with AFRL to develop a synthetic teammate capable of interacting with humans. Dr. Myers continues this research within the Cognitive Science, Models, & Agents branch and is the Lead for the Cognitive Models core research area within the Airman Systems Directorate, AFRL. Dr. Myers Co-Chaired the 2018 International Conference on Cognitive Modeling and edited the Best Papers from ICCM2018 for TopiCS in Cognitive Science.

The Team
Michelle Caisse, L3Harris
Mustafa Demir, Postdoctoral Research Scholar and Faculty Associate, Ira A. Fulton Schools of Engineering, Arizona State University
Mary Freiman, Senior Scientist at Aptima, Inc., Performance Assessment Technologies
Jamie Gorman, Associate Professor of Psychology, Engineering Psychology, Director of the Systems Psychology Lab at Georgia Tech
David Grimm, PhD student at Georgia Tech
Tim Halverson, Senior Research Engineer at Aptima, Inc., Performance Assessment Technologies
Craig Johnson, PhD student in Human Systems Engineering at Arizona State University
Nathan McNeese, Assistant Professor and Director of Team Research Analytics in Computational Environments Research Group, School of Computing, College of Engineering, Computing and Applied Sciences at Clemson University
Steve Shope, President, Sandia Research Corporation & President at Cognitive Engineering Research Institute

Tutorial 3: Evaluating Uncertainty Techniques for Situation Awareness Information Fusion Systems

Abstract: One of the main goals of information fusion is uncertainty reduction, which is highly dependent on the representation chosen. Uncertainty representation differs across the various levels of Information Fusion (as defined by the JDL/DFIG models). High-level information fusion of hard and soft information from diverse sensor types still depends heavily on human cognition. This results in a scalability conundrum that current technologies are incapable of solving. Although there is widespread acknowledgment that an HLIF framework must support automated knowledge representation and reasoning with uncertainty, there is no consensus on the requirements an HLIF framework must meet, on the most appropriate technologies to satisfy these requirements, and on how to evaluate how well they are being met. A clearly defined, scientifically rigorous evaluation framework and metrics are needed to help information fusion researchers assess the suitability of various approaches and tools to their applications. In this tutorial we will explore the Uncertainty Representation and Reasoning Evaluation Framework (URREF), which has been under active development by the Uncertainty Techniques for Uncertainty Representation working group of the International Society of Information Fusion (ISIF ETURWG). We will cover use cases involving Situation Awareness challenges for uncertainty representation and explore the use of URREF to address these challenges.

Instructor's biography: Paulo Cesar G. Costa is Associate Professor at the Department of Systems Engineering and Operations Research at George Mason University. His research focus is on integrating semantic technologies and uncertainty management with applications in multidisciplinary areas. He is a pioneer in the field of probabilistic ontologies and the creator of PR-OWL, a probabilistic ontology language for the Semantic Web, and a key contributor to UnBBayes-MEBN, a Java-based, open source implementation of the PROWL language. Dr. Costa's expertise derives from both his prior work as a practitioner and then researcher in the areas of transportation safety and security, operations research, information fusion, and cybersecurity; which he currently applies in research grants from DARPA, AFRL, U.S. DoT, and U.S. DoE. He is an IEEE Senior Member, member of the International Council of Systems Engineering (INCOSE), member of the Institute for Operations Research and the Management Sciences (INFORMS), and currently serves as the President of the International Society of Information Fusion (isif.org).

Tutorial 4

Dealing with Uncertainties in Data Processing: from Probabilistic and Interval Uncertainty to Combination of Different Approaches, with Application to Geoinformatics, Bioinformatics, and Engineering

Abstract: Most data processing techniques traditionally used in scientific and engineering practices are statistical. These techniques are based on the assumption that we know the probability distributions of measurement errors etc. In practice, often, we do not know the distributions, we only know the bound D on the measurement accuracy - hence, after the get the measurement result X, the only information that we have about the actual (unknown) value x of the measured quantity is that x belongs to the interval [X-D, X+D]. Techniques for data processing under such interval uncertainty are called interval computations; these techniques have been developed since the 1950s. In many practical problems, we have a combination of different types of uncertainty: interval, fuzzy, probabilistic. The purpose of this talk is to describe the theoretical background for interval and combined techniques, to describe the existing practical applications, and ideally, to come up with a roadmap for such techniques.

Deep Learning (Partly) Demystified

Abstract: Successes of deep learning are partly due to the appropriate selection of activation function, pooling functions, etc. Most of these choices have been made based on empirical comparison and heuristic ideas. In this paper, we show that many of these choices - and the surprising success of deep learning in the first place - can be explained by reasonably simple and natural mathematics.

Instructor's biography: Vladik Kreinovich is Professor of Computer Science at the University of Texas at El Paso. His main interests are the representation and processing of uncertainty, especially interval computations and intelligent control. He has published eight books, 24 edited books, and more than 1,500 papers. Vladik is Vice President of the International Fuzzy Systems Association (IFSA), Vice President of the European Society for Fuzzy Logic and Technology (EUSFLAT), Fellow of International Fuzzy Systems Association (IFSA), Fellow of Mexican Society for Artificial Intelligence (SMIA), Fellow of the Russian Association for Fuzzy Systems and Soft Computing. He is Treasurer of IEEE Systems, Man, and Cybernetics Society.