Representing human-automation challenges A model based approach for human factors

168 Pages·2015·14.32 MB·English

by  Jonas Andersson

THESIS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY

Representing human-automation challenges
A model based approach for human factors engineering
in industrial practice

JONAS ANDERSSON

Division Design & Human Factors
Department of Product and Production Development
Chalmers University of Technology

Gothenburg, Sweden, 2014
Representing human-automation challenges
A model based approach for human factors engineering in industrial practice

JONAS ANDERSSON
ISBN 978-91-7597-005-9

© JONAS ANDERSSON, 2014.

Doktorsavhandlingar vid Chalmers tekniska högskola
Ny serie Nr 3686
ISSN 0346-718X

Division Design & Human Factors
Department of Product and Production Development
Chalmers University of Technology
SE-412 96 Gothenburg
Sweden
Telephone + 46 (0)31-772 1000

Cover by Pia Koskela
www.piakoskela.com

Printed by
Reproservice at Chalmers University of Technology
Göteborg, Sweden 2014
Abstract
Automation technology is widely implemented in process control domains due to its benefits
of improving efficiency and enhancing control. However, use of automation also introduces
an often complex intermediary between the human and the controlled domain, which can
obscure from the operator how system functioning is achieved. The difficulty for operators to
perceive and understand what the automatic system is doing has a potentially negative impact
on overall system performance, since the human operator perform important functions in the
work system related to both safety and production.

In this thesis it is argued that there are few approaches that address the problem of
specifically, and each existing approach might individually not cover the entire problem scope
in full detail. Further, current methodologies seem to have difficulties in reaching applications
apart from narrow human factors engineering practices.

With this background in mind, the research work presented in this thesis has focused on how
human-automation related challenges can be addressed to improve preconditions for operators
in understanding automatic system functioning. Creating the appropriate preconditions in
control environments is a multidisciplinary design challenge striving for safe and efficient
work systems. The purpose of this thesis was to aid human factors engineering practitioners in
industry in dealing with this challenge.

To fulfil the purpose, an existing theoretical model was adapted and used to describe human-
automation challenges in general. This led to a theoretical unification of human-automation
related challenges and a way to describe challenges systematically. The unified format
enables description and analysis of automated human-machine systems in order to identify
representational gaps and matches in the work system. The theoretical model was then used as
a basis for developing a method named the “System Representation Matrix”. The System
Representation Matrix enables description and analysis of the dynamic domain, the control
system, the control system user interface and the necessary operator knowledge, in a unified
representation.

Conclusions from testing and evaluating the method are that the System Representation
Matrix can aid creating an overview of automated human-machine systems. The overview has
potential as an aid for reasoning about matters of system functioning and design. In practice,
the matrix could provide support for design decisions, help define necessary operator
knowledge and become a tool to aid human factors engineering in multidisciplinary teams.
This has the potential to lead to improved aid for human factors engineers when dealing with
human-automation challenges in industrial practice.

i

Acknowledgement
First of all I would like to thank my main supervisor Professor Anna-Lisa Osvalder for giving
me the opportunity to enter into the world of research. Your kind encouragement, support and
guidance have been very helpful. Maybe sometime in the future I will learn to take care of my
s’s…

Thanks to my co-supervisor Lars-Ola Bligård, whose support and contributions to my
research work has been invaluable. Cooperation with you during the development of the
models and method has been a driving force. I often had to struggle to keep up with your pace
in thought during our discussions and collaborating with you has been a pleasure that I hope
will continue. Without your encouragement I would not have been able to push myself all the
way to the finishing line.

Co-supervisors Adjunct professor Stig Franzén and Professor Håkan Alm; I am very grateful
for how you have contributed to my research work with your contrasting perspectives and
guiding comments. Time and again you have forced me to clarify my thoughts and sharpen
my arguments. Your support has been very important.

I would also like to thank Anna Thunberg for our cooperation during my early years as a
Ph.D. candidate. Your introduction to research gave me inspiration to pursue my studies.

I have to thank all operational personnel that have participated in my projects. You have
endured endless questions, interviews, questionnaires, being observed and long and
intentionally boring simulator sessions. Without you, there would have been no research
worth mentioning.

My thanks too, to all practitioners who agreed to work with me on the projects. By sharing
your experience and knowledge you made me a better researcher and a wiser man. And
special thanks to those who helped me evaluate the developed method on such a short notice.

Moreover, I have to thank the library at Chalmers University of Technology. Your service is
impeccable and an invaluable aid for every researcher at Chalmers.

Further, I would like to thank the financiers of the projects in which I have worked: the
Nordic Nuclear Safety Research (NKS), the Thermal Engineering Research Institute
(Värmeforsk), the Swedish Maritime Administration, CGM AB, and Vinnova.

I would like to thank all my colleagues at the division Design & Human Factors at Chalmers,
headed by Professor MariAnne Karlsson, for making all the years of work and studies a true
pleasure.

Also, thank you to all the lovely and talented people at the Human-Technology-Design
research school. I will surely miss the late night discussions at the spring meetings, not to
mention the candle light, frozen food dinners on the fifth floor when finishing the thesis.

Jon-Anders, Ann-Sofie, Sigrid and Vilhelm; thank you for having a man staying in the
basement from time to time. You might not attach much importance to it, but having a family
breakfast before going to work when you are away from your own family is an energy boost.
iii
The past few years prior to finishing my PhD have been the most challenging of my life in
many ways. Writing a PhD thesis and being away from your loved ones could have been
enough, one might think. At times, life is a maelstrom and we are but autumn leafs within it.
Anywho… Tor, you kept me floating.

Roland and Lena, thank you for taking care of us, always.

Ulla, thank you for your love, care and sincere interest in my research work. We all miss you
terribly.

Dad, you taught me to seek my own answers. It is the most precious gift I have ever received.
Mum and Erika, without your endless support I would not be where I am now. Thank you for
looking after me and my own little family.

Emilia, sing me one of your songs and I have peace of mind. It was the sweetest relaxation
when my head was full of levels, matrices, boxes and lines.

Alfred, finally, FINALLLY the darn thesis is ready. Let’s play with LEGO!

To those of you who have read this far, I’m glad you managed. To those of you to have the
intention to read further, I would like you to know that the effort put into this work would not
have been possible without one person. Her name is Julia.

iv
Table of contents

1 Introduction .................................................................................................................................... 1
1.1 Background ............................................................................................................................. 1
1.2 Problem description ................................................................................................................. 2
1.3 Purpose and aim ...................................................................................................................... 3
1.4 Research process – to acquire understanding by shifting roles ............................................... 3
1.5 Research approach ................................................................................................................. 12
1.6 Delimitations ......................................................................................................................... 15
1.7 Outline of thesis ..................................................................................................................... 15
2 Modelling automated human-machine systems ............................................................................ 17
2.1 Research context - framing the research................................................................................ 17
2.2 The benefits of automatic process control ............................................................................. 17
2.3 The triadic view of human-machine systems ........................................................................ 19
2.4 Adapting the triadic model to process control ....................................................................... 23
2.5 Model incongruence in operator-control system-dynamic domain interaction ..................... 25
2.6 Automation related problems in process industry control rooms .......................................... 29
2.7 Prescribed solutions to automation related challenges .......................................................... 42
2.8 Relating prescribed solutions to the dynamic domain-control system-operator triad ........... 50
2.9 Modelling automated human-machine systems - conclusions .............................................. 52
2.10 Summary of chapter 2 ........................................................................................................... 53
3 What system designers need to know in order to deal with automation related challenges ......... 55
3.1 Shifting perspective from gaps to matches ............................................................................ 57
3.2 Existing methodologies in Cognitive Systems Engineering .................................................. 58
4 Requirements elicitation for development of a viable method ..................................................... 65
4.1 Defining method requirements .............................................................................................. 66
4.2 Summary of established method requirements ...................................................................... 81
5 The System Representation Matrix – description and use ............................................................ 83
5.1 Integrating the means-ends hierarchy with the triadic model of human-machine systems ... 83
5.2 Description of the System Representation Matrix ................................................................. 85
5.3 How to use the matrix for analysis ...................................................................................... 100
5.4 How to create a System Representation Matrix .................................................................. 103
5.5 Summary of chapter 5 ......................................................................................................... 108
6 Evaluation of the System Representation Matrix ....................................................................... 109
6.1 Modelling example - the OPUS paint factory simulator ..................................................... 109
6.2 Test session with human factors engineering practitioners ................................................. 125
6.3 Application in industry ........................................................................................................ 128
6.4 Reflections from a fellow method developer ...................................................................... 130
v
6.5 Summary of chapter 6 ......................................................................................................... 131
7 Fulfilment of method requirements ............................................................................................ 133
8 Discussion ................................................................................................................................... 139
8.1 Contributions ....................................................................................................................... 139
8.2 Threats to validity ................................................................................................................ 140
9 Conclusions ................................................................................................................................ 145
References ........................................................................................................................................... 147

vi
List of figures
Figure 1. Research context ......................................................................................................... 2
Figure 2. Research process ......................................................................................................... 3
Figure 3. Projects and associated domains ................................................................................. 9
Figure 4. The relationship between research and practice is used to gain understanding and
create useful designs. ........................................................................................................ 14
Figure 5. A simplified description of process control ............................................................. 18
Figure 6. The Automation pyramid ......................................................................................... 18
Figure 7. The ISA-95 functional hierarchy model .................................................................. 19
Figure 8. The triadic-semiotic model as presented by Bennet and Flach ................................ 20
Figure 9. The triadic model overlaid on the traditional regulation model ............................... 22
Figure 10. A model describing process control ........................................................................ 23
Figure 11. Gaps in model congruence across the dynamic domain-control system-operator
triad (example) ................................................................................................................. 26
Figure 12. Clumsy automation ................................................................................................. 30
Figure 13. Out-of-the-loop (I) .................................................................................................. 32
Figure 14. Out-of-the-loop (II) ................................................................................................. 33
Figure 15. High functional specificity ...................................................................................... 35
Figure 16. Miscalibrated trust .................................................................................................. 36
Figure 17. Brittleness ............................................................................................................... 38
Figure 18. Configuration error ................................................................................................. 39
Figure 19. Mode confusion ...................................................................................................... 41
Figure 20. Factors affecting model incongruence .................................................................... 51
Figure 21. Example of designer roles associated with each system part ................................. 55
Figure 22. When moving from left to right, each part needs knowledge of the preceding parts
.......................................................................................................................................... 56
Figure 23. Moving from gaps to matches ................................................................................ 57
Figure 24. Multilevel Flow Modelling mapped onto the model of process control ................. 60
Figure 25. Cognitive Work Analysis mapped onto the model of process control ................... 61
Figure 26. Applied Cognitive Work Analysis mapped onto the model of process control ..... 63
Figure 27. Method requirements .............................................................................................. 69
Figure 28. The figure shows a generic product development process. Activities move back
and forth between design and requirements specification ............................................... 76
Figure 29. The collaborative process in requirements elicitation. ........................................... 77
Figure 30. Question matrix ....................................................................................................... 84
Figure 31. The toaster described in the System Representation Matrix .................................. 93
Figure 32. Upper left quadrant of the toaster system representation matrix ............................ 94
Figure 33. Lower left quadrant of the toaster system representation matrix ............................ 95
Figure 34. Control column of the toaster system representation matrix .................................. 96
Figure 35. Interface column of the system representation matrix ............................................ 97
Figure 36. Knowledge column of the toaster representation matrix ........................................ 98
Figure 37. Reading the system representation (part of Figure 31) ........................................... 99
Figure 38. Example of interpretation of the System Representation Matrix of a toaster ....... 101
Figure 39. Example of interpretation of the System Representation Matrix of a toaster ....... 102
Figure 40. System representation is built by iteratively moving between three phases:
Identifying entities, Describing relations, and Creating a system narrative. .................. 103
Figure 41. Matrix orientation ................................................................................................. 104
Figure 42. Paint factory simulator setup during experiments ................................................ 109
Figure 43. Overview display of tanks in the OPUS paint factory .......................................... 111
vii
Figure 44. Preparation tank and the associated faceplate ....................................................... 111
Figure 45. Completion tank and the associated faceplate ...................................................... 111
Figure 46. Storage tank .......................................................................................................... 112
Figure 47. The Paint factory system representation (boxes are intentionally blank) ............. 113
Figure 48. Upper left quadrant of the paint factory system representation ............................ 114
Figure 49. Lower left quadrant of the paint factory system representation ........................... 115
Figure 50. Control column of the paint factory system representation .................................. 116
Figure 51. Control system user interface column of the paint factory system representation 117
Figure 52. Knowledge column of the paint factory system representation ............................ 118
Figure 53. Dosing control loop .............................................................................................. 119
Figure 54. Brittle failure in the paint factory system representation ...................................... 120
Figure 55. Viscosity control loop ........................................................................................... 122
Figure 56. Out-of-the-loop (I) in the paint factory system representation ............................. 122
Figure 57. Out-of-the-loop discussion between the operator (green), the process engineer
(black), the automation engineer (red), and the interface designer (blue) ..................... 124

viii