2020 IEEE Conference on Cognitive and Computational Aspects of Situation Management (CogSIMA)

Dawn of the Multi-Species Systems: Human Performance Engineering in the Age of AI

Abstract: We are entering an age whereby the combination of human and machine intelligences is driving us to think beyond the classical paradigm of human-machine interaction design and into one of engineering a teaming or fusion between these capabilities. Human performance engineering is the collection of scientific and design methods that apply deep expertise in how humans think, learn, and behave to the goal of optimizing system performance and mission readiness. The insertion of advanced AI capabilities in those systems is a game changer, as it introduces system elements that learn, adapt, and change continuously as a result of reasoning about the observations it collects about the work environment and human behavior. Insuring that these hybrid systems behave within the performance envelop they have been designed for is a key challenge for the system designer. Using examples from the real world and ongoing research projects, we will highlight some key trends in human performance engineering and illustrate how the future of work, in domains ranging from training and education to manufacturing, defense, aviation, and medicine, will be highly dependent on our ability to thoughtfully engineer these new and autonomous human-AI "multi-species" systems.

Bio: Daniel is Aptima's Principal Founder, establishing and implementing a vision for Aptima as the premier Human-Centered Engineering business in the world. Mr. Serfaty's current work involves the technical leadership and coordination of inter-disciplinary projects for government agencies and private industries, in conjunction with several academic, industrial, and government teams. These efforts investigate factors that drive organizational performance, expertise development, and human-systems integration in large-scale, technology-rich socio-technical systems.

Prior to founding Aptima in 1995, Daniel was engineering group leader and program manager at Alphatech (now BAE Systems), where he coordinated projects in the decision-making, training, and human engineering areas.

For the last 25 years, Daniel's research interests have included the application of rigorous modeling and experimental methods to improve decision-making performance, develop expertise in field settings, and apply systems engineering methods to the design of large-scale organizations. In addition to his technical pursuits, his many industry activities include participating in various technology leadership forums and serving on the board of directors for small technology businesses.

Daniel's academic background includes undergraduate degrees in Mathematics/Physics, Psychology, and Aeronautical Engineering from the Université de Paris and the Technion, Israel Institute of Technology, an MS in Aeronautical Engineering (Technion), and a M.B.A. in International Management from the University of Connecticut. His doctoral work at the University of Connecticut pioneered a systematic approach to the analysis of distributed decision-making in dynamic and uncertain environments.

Learning, Reasoning, and Fusion in Human-Machine Teams

Abstract: As we increasingly seek to automate cognitive processes to perform activities in medicine, industry, transportation, and defense, we are gaining insights into the challenges we face. This keynote provides a brief overview of three key areas and the research challenges that confront us in each. First, we continue to expand and refine our methods to represent the cognitive processes to learn, represent the world in computational models, and then reason about our observations. Meanwhile we are currently learning the vulnerabilities to machine learned knowledge, and the means to protect that knowledge. Second, we discuss the broad state of automating the cognition process to discriminate objects, their relations to each other, and their behaviors over time. Finally, it is important to recognize the challenge of achieving synergy between humans and their cognitive machines, using computational models to represent shared, transactive knowledge. We summarize the key research challenges in each of these three areas.

Bio: Edward Waltz is Professor of Practice (Intelligence) in the Center for MultiINT Studies at the Naval Postgraduate School (NPS) where he lectures and conducts research into intelligence processing from multiple intelligence sources. Prior to this position, he was Division Chief at the Advanced Concepts National Reconnaissance Office and Distinguished Member of the Technical Staff at Virginia Tech. He also held senior positions with BAE Systems Advanced Information Technologies (Chief Scientist), General Dynamics, and Veridian (Technical Director, Senior Scientist) where he was developing and deploying signal processing, data fusion and intelligence analysis capabilities. Mr. Waltz also led numerous hard target Multi-INT studies and tool developments for different agencies of the IC. He has given over 45 international lectures on intelligence. He was a regular invited lecturer at the National Intelligence University on Advanced Analytics. Mr. Waltz is a recipient of the National Intelligence Unit Meritorious Citation and the DoD Joe Mignona Data Fusion Award among others. He is also the author of numerous books: Quantitative Intelligence Analysis, Knowledge Management in the Intelligence Enterprise, Information Warfare Principles and Operations and the coauthor of Counterdeception Principles and Applications for National Security and Multisensor Data Fusion.

The Artful Mind, Machines, Codes and the Wonder of It All

Bio: Dr. Bob Deutsch is a cognitive neuroscientist (Ph.D., Albert Einstein College of Medicine) and was a professor of psychiatry and cultural anthropology (Rutgers University), who is founder of the consulting practice, Brain-Sells.(www.brain-sells.com). Bob has worked in primitive tribal societies, in the strategy rooms along Pennsylvania Avenue (both as a State Dept. Foreign Service Officer and a consultant) and in the business war rooms on Madison Ave. His focus has been on understanding how real people, living real lives, on-the-ground, real-time, experience their everyday life, and how they use that self-narrative to propel their attachments to and decisions about products, persons and ideas. Bob's work has been applied to business, politics, education, entertainment and technology.

Command and Control

Abstract: The DoD Dictionary of Military and Associated Terms defines Command and Control (C2) as: The exercise of authority and direction by a properly designated commander over assigned and attached forces in the accomplishment of the mission. C2 is exemplified in a relation connecting human beings who have different sorts of capabilities: to exercise authority, to execute plans, to direct actions, to engage in joint actions, and many more. I will present a definition of capability and of related terms such as authority, operation, direction, plan, and so forth, and use these definitions as the basis for a general ontology of command and control. I will then compare the sorts of capabilities realized by humans in C2 with the capabilities of the computer systems with which human commanders and assigned forces are required to interoperate in military action. This comparison, finally, will serve as the basis for a discussion of the hypothesis that intelligent machines might one day be responsible for commanding and controlling human beings.

Bio: Barry Smith is a prominent contributor to both theoretical and applied research in ontology. He is SUNY Distinguished Professor of Philosophy in the University at Buffalo, with joint appointments in the Departments of Biomedical Informatics, Neurology, and Computer Science and Engineering. Smith is the author of some 300 peer-reviewed publications, with over 35,000 citations. His research has been funded by the National Institutes of Health, the US, Swiss and Austrian National Science Foundations, the Volkswagen Foundation, the European Union, and the US Department of Defense. Smith is the creator of Basic Formal Ontology (BFO), the most commonly adopted upper-level ontology development framework and recently approved to become international standard ISO/IEC:21838-2. His work led also to the formation of the OBO (Open Biomedical Ontologies) Foundry, a suite of interoperable ontology modules designed to support information-driven research in biology and biomedicine. More recently, his work has led to the creation of analogous BFO-driven foundry efforts in a range of different domains, including military intelligence, space situational awareness, digital manufacturing, and model-based systems engineering.

Human-Centered Artificial Intelligence: A Second Copernican Revolution

Abstract: The Human-Centered Artificial Intelligence (HCAI) model clarifies how to (1) design for high levels of human control and high levels of computer automation so as to increase human performance, (2) understand the situations in which full human control or full computer control are necessary, and (3) avoid the dangers of excessive human control or excessive computer control.

HCAI represents a second Copernican revolution. In the past, researchers and developers focused on building AI algorithms and systems, stressing the autonomy of machines rather than human control through user interfaces. In contrast, HCAI puts the human users at the center of design thinking, emphasizing user experience design. Researchers and developers for HCAI systems stress measuring human performance and satisfaction, valuing customer and consumer needs, and ensuring meaningful human control. Achieving HCAI will increase human performance, while supporting human self-efficacy, creativity, and responsibility.

Bio: BEN SHNEIDERMAN (http://www.cs.umd.edu/~ben) is an Emeritus Distinguished University Professor in the Department of Computer Science, Founding Director (1983-2000) of the Human-Computer Interaction Laboratory (http://hcil.umd.edu), and a Member of the UM Institute for Advanced Computer Studies (UMIACS) at the University of Maryland. He is a Fellow of the AAAS, ACM, IEEE, and NAI, and a Member of the National Academy of Engineering, in recognition of his pioneering contributions to human-computer interaction and information visualization. His widely-used contributions include the clickable highlighted web-links, high-precision touchscreen keyboards for mobile devices, and tagging for photos. Shneiderman's information visualization innovations include dynamic query sliders for Spotfire, development of treemaps for viewing hierarchical data, novel network visualizations for NodeXL, and event sequence analysis for electronic health records.

Ben is the co-author with Catherine Plaisant of Designing the User Interface: Strategies for Effective Human-Computer Interaction (6th ed., 2016). He co-authored Readings in Information Visualization: Using Vision to Think (1999) and Analyzing Social Media Networks with NodeXL (2nd edition, 2019). His book Leonardo's Laptop (MIT Press) won the IEEE book award for Distinguished Literary Contribution. The New ABCs of Research: Achieving Breakthrough Collaborations (Oxford, 2016) describes how research can produce higher impacts