Read this document on Scribd: IDT873 Maddrell Cognitive TA Abstract 3
IDT 873 Abstract: Cognitive Task Analysis Jennifer Maddrell Schaafstal, A., Schraagen, J. M., & van Berlo, M. (2000). Cognitive task analysis and innovation of training: The case of the structured troubleshooting. Human Factors, 42(1), 75–86. Research Overview. Following an instructional design evaluation of an existing Royal Netherlands Navy maintenance training course, Schaafstal, Schraagen, and van Berlo (2000) observed a gap between the instruction and the practice of troubleshooting the subject system. They observed that the existing instruction was based largely on the technical equipment documentation from engineers which focused exclusively on the system’s components. Following a comprehensive cognitive task analysis (CTA), Schaafstal et al. revised the instruction under the assumption that maintenance system troubleshooting is a complex cognitive task requiring not only knowledge about the system’s components, but also knowledge about how the system functions and how to consider possible causes and solutions to maintenance problems. The CTA consisted of several observational studies of troubleshooting with technicians of varying expertise levels. Based on information from the CTA, a modified course was prepared which focused on a functional understanding of the system versus the component orientation of the prior course. In addition, general troubleshooting strategies were incorporated which gave learners instruction on how to a) describe the problem, b) generate causes, c) test causes, d) repair, and e) evaluate solutions. Purpose. The purpose of the presented research was to evaluate the modified structured troubleshooting training course and to compare it with the exiting maintenance training course. Schaafstal et al. predicted superior outcomes from the revised course. Methodology. A series of experimental studies compared the learning outcomes of maintenance trainees taking the new structured troubleshooting training course with groups of maintenance trainees taking the existing training course. Outcome measures included malfunction identification, reasoning, and functional understanding of the system. Conclusions. The modifications in the course reduced the course duration by 33% (from six to four weeks). Even at the shortened length, those participating in the new course achieved statistically superior results as compared to those in the original course. Based on the results of the study, Schaafstal et al. suggest that novice technicians lack both a systematic approach to troubleshooting, as well as a functional understanding of the equipment. As seen in prior research, they observed that novices face information overload (lose the forest for the trees), lack hierarchically organized cognitive frameworks, lack functional understanding, possess inadequate mental models of underlying system, and lack the ability to isolate causes of the problem. Therefore, based on the results of their evaluation, they suggest that training in troubleshooting should focus on three areas: 1) system independent troubleshooting strategies to be used across systems, 2) system specific functional models, and 3) system specific domain knowledge. Heuristics Results of this research suggest the importance of moving away from a purely component oriented analysis to what the researchers term a functional decomposition when designing troubleshooting skills instruction. While analysis and instruction on the components is necessary, it is not sufficient. Analysis and instruction should also focus on the functional processes, including likely causes of potential problems and paths to solutions, in order for learners to know what to do when troubleshooting. Further, the results indicate that training in system independent troubleshooting skills can further augment the troubleshooting skills instruction. IDT 873 Abstract: Cognitive Task Analysis Jennifer Maddrell Critique The presented research is important for two reasons. The research suggests a positive influence of CTA on outcomes in troubleshooting training. By revising the instruction to focus on a functional understanding of the system from information gleaned in the CTA, the instruction appears to have been significantly improved. In addition, the findings suggest a positive impact from teaching system independent troubleshooting skills. Also, the paper is valuable for the information provided about the evolution of the authors’ CTA process. This information will be helpful to future designers and researchers. Unfortunately, the written presentation of this paper is horribly disjointed. It is doubtful that most readers will devote the time necessary to weave a coherent narrative out of the broken threads of theory, prior research, CTA processes, instructional design considerations, research methodologies, and conclusions. There is a wealth of information included in the paper, but unfortunately the reader must devote an unnecessary amount of effort to piece it all together.