Cognitive Task Analysis of Teams (Team CTA)

Slideshow Transcript

1. Slide 1: Team CTA
2. Slide 2: Team CTA Cognitive Task Analysis of Teams -Yogesh Tadwalkar March 22, 2004
3. Slide 3: Team CTA Traditional Task Analysis Task Analysis aims to  optimize work performance by matching tasks with human capabilities and limitations  Human work is comprised of both physical and cognitive activities  Physical tasks are observable (Overt)  However, cognitive processes that lead to physical actions are not (Covert)
4. Slide 4: Team CTA Cognitive Task Analysis (CTA)  Extension of traditional task analysis techniques to uncover information about: – - Work domain, mental demands on operator – - Goal Structures – - Knowledge and Strategies – - Thought Processes and Decision Making  Thus, providing a holistic, complete view of human work
5. Slide 5: Team CTA Goals of CTA  Performance improvement through: – Better Task and Interface Design – Ideal Personnel Selection – Comprehensive Training and Instructions – Systematic Performance Assessment – Improved Planning
6. Slide 6: Team CTA Cognitive Task Analysis of Teams
7. Slide 7: Team CTA Why Conduct Team CTA?  Because 2 + 2 ≠ 4 – Individuals are often required to perform in teams – Optimum individual performances may not lead to final mission accomplishment if team co-ordination is unsatisfactory  Thus, individual CTA would not be sufficient for designing optimum task performance
8. Slide 8: Team CTA Team CTA  As defined by Klein: – A reflection and representation of the team mind / Knowledge – Description of the cognitive skills required for effective team performance – Elaboration of the team decision requirements – Determination of types of expertise found in effective teams
9. Slide 9: Team CTA Objectives of Team CTA  Team performance enhancement through: – Restructuring and Resizing the team – Designing better: • Information technology • Information management strategies • Human-computer interfaces • Decision support systems and Communication – Developing methods for team training and selection
10. Slide 10: Team CTA Team Environment
11. Slide 11: Team CTA Team Environment  Tasks are decomposed and assigned to different team members  Team members may or may not have direct contact with each other (co-located vs. dispersed teams)  Tasks can be performed serially or in parallel or overlapping fashion  Team members possess, acquire and process information, make decisions, solve problems and make plans
12. Slide 12: Team CTA Team Environment  Teams are intelligent entities (Thordsen and Klein, 1989) – Team members possess, acquire and process information – Communicate with each other – Make decisions – Solve problems and – Make plans
13. Slide 13: Team CTA Types of Teams  Planning Teams – Military Command – Emergency Response at nuclear plants  Action Teams – Air Traffic Control – SWAT
14. Slide 14: Team CTA Cognitive Processes of Teams What to measure through Team CTA methods
15. Slide 15: Team CTA Cognitive Processes of Teams Methods for team CTA should focus at  capturing these 5 processes (Klein): 1. Shared mental models - Team Goals, team member roles 2. Control of attention - Information seeking, filtering, communication 3. Shared situation awareness - Assessment of dynamism, prediction of future 4. Application of strategies - To make decisions, solve problems and plan 5. Metacognition - Self-monitoring, sensing trouble, making alternative plans
16. Slide 16: Team CTA A Model for Team Cognition Dynamic Knowledge Pre-task Knowledge • Control of Attention • Shared Mental Models • Shared Situation Awareness • Strategies • Metacognition Team Performance Mental Models Member A Expectations Knowledge Member C Member B Attitudes Skills (Stout, Cannon-Bowers, and Salas, 1996)
17. Slide 17: Team CTA 1. Shared Mental Models  The degree to which team members have the same understanding of key processes (Cannon-Bowers, Sales, & Converse, 1992) - Knowledge about overall team goals and mission objectives e.g. Football game - Roles and responsibility of each team member - Team-mate characteristics and preferences - Task Procedures, sequences and timing - Use and relationship of equipment
18. Slide 18: Team CTA Shared Mental Models and Team Performance  Shared mental models affect team performance significantly (Oransu, 1990): – Anticipating team member behaviours and information requirements – Performing tasks from a common frame of reference – Attending to, interpreting, communicating about, and responding to the world more similarly than individuals with discrepant knowledge (Rentsch & Hall, 1994)
19. Slide 19: Team CTA Methods to Elicit and Represent Shared Mental Models
20. Slide 20: Team CTA Goal-Directed Analysis Variation of (Woods & Hollnagel, 1987) To solicit knowledge of Team Goals and Task Objectives – Document Analysis and Observation – Separate interviews with each team member, followed by group interview – Representation: Agreement Metric For Whole Team Sub-Goal 1 For Each Member Team Task Task 1 Sub-Goal 2 Agreement Goals Goal Metric Task 2 Sub-Goal 3
21. Slide 21: Team CTA Task-Action Mapping Variation of (Coury et al., 1991) To solicit knowledge of Task Procedures, Sequence and Timing – Document Analysis and Observation – Ask each team member to describe key team concepts, associated tasks and specific actions – Representation: Concept Maps Action 1 Task 1 • What (Steps) • Who • When Concept 1 Description Task 2 e.g. Evacuation Action 1 • What (Steps) • Who Task 3 • When
22. Slide 22: Team CTA Bootstrapping (Hoffman, Shadbolt, Burton & Klein 1995) To solicit knowledge of Roles and Responsibilities Bootstrapping Documentation Group Observation Interviews Individual Interviews May not adequately capture dynamic, contextual data like misinterpretations, omissions due to environmental factors
23. Slide 23: Team CTA Adapted Critical Incident Method (Flanagan, 1954; Critical Decision Method by Klein, 1989) To solicit knowledge of Roles and Responsibilities, Strategies, Attitudes and Personal Characteristics – Document Analysis and Recall of a critical past event e.g. fire, accident, high-stake mission. – Ask each team member to describe various decisions taken, action triggers, who did what vs. who was supposed to do what – Describe how each teammate reacted to the situation (perception, prediction, response time, stress, flexibility in co-ordination, etc.)
24. Slide 24: Team CTA 2. Control of Attention  The way a team engages in information management - Information seeking - Filtering irrelevant information - Allocating attention to important functions - Properly distributing messages  Refers to the team’s working memory – The way it uses limited resources for processing simultaneous messages (Klein)
25. Slide 25: Team CTA Methods to Elicit and Represent Control of Attention
26. Slide 26: Team CTA Scenario based Questionnaires (Blickensderfer, 1997) To solicit knowledge of Information Management Strategies, and Communication Skills – Document Analysis, observations and interviews to create a questionnaire covering a team work scenario, given to each team member – Answers to outline relevant information protocol for the scenario, missing information, cues for identifying missing data, strategies to acquire and distribute information, and possible communication bottlenecks
27. Slide 27: Team CTA 3. Shared Situation Awareness  Extent to which team members have the same interpretation of ongoing events (Cannon-Bowers, Sales, & Converse, 1992) –How different team members perceive dynamic situations and events –How they form divergent impressions e.g. police in a riot situation –How they update each other  Is giving full data to each member a remedy for discrepant situation Awareness?
28. Slide 28: Team CTA Methods to Elicit and Represent Shared Situation Awareness
29. Slide 29: Team CTA Situation Awareness Global Adapted Assessment Technique (SAGAT) (Endsley, 1995) To solicit knowledge of compatible situation awareness – Bootstrapping to create a mission simulation – Asking team members to perform mission – Freezing mission after specific intervals – Asking each team member to analyze the situation and predict outcome and strategize – Compare results for all members and assess compatibility and reasons for discrepancy
30. Slide 30: Team CTA Situation Awareness Linked Indicators Adapted to Novel Tasks (SALIANT) (Muniz et al. , 1998) To solicit knowledge of compatible situation awareness 1. Delineate behaviours theoretically linked to team situations (24 behaviours, 5 clusters). • Being able to predict next event • Managing resources (technical, system , internal, team) • Managing time • Feeling of being in control; able to implement elegant solutions • Taking the right decision at the best moment • Detecting mismatches, etc. 2. Development of task scenarios 3. Identification of specific, observable responses for each scenario 4. Development of a script (to be administered to a team member) 5. Observation form: Presence or Absence of response behavior
31. Slide 31: Team CTA 4. Application of Strategies  Every team has a prepared list of steps, and routines for decision making, problem solving and planning  However, a team also develops shortcuts and workarounds not codified in procedures, as it gains experience  A skillful team knows its: – Key decision makers (to be supported) – Key sources of expertise (to be consulted)  Time is critical in strategy execution
32. Slide 32: Team CTA Methods to Elicit and Represent Team Strategies
33. Slide 33: Team CTA Retrospective Protocol Analysis (Means, 1993) To solicit knowledge of agreement on strategies – Based on non-interruption – Use video-tapes of similar mission or actual team performance recorded previously – Asking each team member to analyze the situation and justify strategies taken – Analyze team agreement
34. Slide 34: Team CTA 5. Metacognition  A team needs to monitor itself during ongoing task performance  To determine whether and when it is running into difficulty  Predict consequences of the difficulty  Determine when and how it needs to shift its strategies e.g. Police at riot situation
35. Slide 35: Team CTA Methods to Elicit and Represent Metacognition
36. Slide 36: Team CTA Mission Simulation (Klein) To solicit knowledge of self-monitoring or metacognition – Bootstrapping to create a mission simulation – Asking team members to perform mission – Observing how team members recognize and collect more information about critical incidents – Conducting individual interviews with each team member to analyze the video-recording – Analyzing gaps in team agreement
37. Slide 37: Team CTA Representing Team CTA Results
38. Slide 38: Team CTA Methods for Representing Team CTA Results  Dealing with the ‘Envisioned World Problem’ (Woods, in press) Prototypes Team CTA World of Changed World – Creating prototypes and simulations to gauge how team CTA would impact the future – Iterative testing
39. Slide 39: Team CTA Limitations / Challenges in Applying Team CTA Methods
40. Slide 40: Team CTA Challenges in Applying Team CTA Methods  Knowledge elicitation – Knowledge is internalized and even experts find it hard to verbalize it – Retrospective account of work may differ from reality  Capturing dynamism – In many cases, team strategies differ vastly from one situation to another – Generalizing based on a few situations may be inappropriate
41. Slide 41: Team CTA Summary Team CTA is critical to obtain a holistic (both  overt and covert tasks) view of human work Teams are intelligent entities with 5 key  cognitive processes (Klein) 1. Shared mental models 2. Control of attention 3. Shared situation awareness 4. Application of strategies to make decisions, solve problems and plan 5. Metacognition
42. Slide 42: Team CTA Summary The 5 key processes can be spread over  2 types of team knowledge – Pre-task knowledge – Dynamic Knowledge Team CTA Methods should aim at  eliciting team knowledge of these 5 processes
43. Slide 43: Team CTA Summary Team CTA can yield significant  performance improvement of teams by: – Restructuring and Resizing the team – Designing better: • Information technology • Information management strategies • Human-computer interfaces • Decision support systems and Communication – Developing methods for team training and selection
44. Slide 44: Team CTA References  Gary Klein – Cognitive Task Analysis of Teams (1995)  Cannon-Bowers, Sales, Baker – Analyzing knowledge requirements in team tasks – Shared Mental Models in Expert Team Decision Making (1993)  Hoffman, R.R. – Use of Critical Incidents to Elicit Knowledge (1995)  Oransu, J. – Shared Mental Models and Crew Performance (1990)  Coury, B.G., Motte,S., & Seiford, L.M. – Capturing and representing decision processes in the design of an information system  Muniz, E.J., Stout, R.J., Bowers & Salas – A methodology for measuring team situation awareness: Situation Awareness Linked Indicators Adapted to Novel Tasks (SALIANT) (1998)  Woods, D.D. & Hollnagel, E. – Mapping cognitive demands in complex problem solving worlds (1987)
45. Slide 45: Team CTA Questions ?
46. Slide 46: Team CTA Cognitive Task Analysis of Teams Thank You !