STRUCTURING EXPLORATORY TESTING THROUGH TEST CHARTER DESIGN AND DECISION

216 Pages·2017·12.58 MB·English

S
T
R
U
C
T
U
ABSTRACT R STRUCTURING EXPLORATORY TESTING
IN
G
Context: Exploratory testing (ET) is an approach to most frequently used technique mentioned by the E THROUGH TEST CHARTER DESIGN AND
X
test software with a strong focus on personal skills practitioners is ET which is not a much researched P
L DECISION SUPPORT
and freedom of the tester. ET emphasises the simul- topic. We also found many interesting claims about O
R
taneous design and execution of tests with minimal ET in grey literature produced by practitioners in A
T
test documentation. Test practitioners often claim the form of informal presentations and blogs but O
R
that their choice to use ET as an important alterna- these claims lacked any empirical evidence. There- Y
T
tive to scripted testing is based on several benefits fore, a controlled experiment was conducted with E
S
ET exhibits over the scripted testing. However, these students and industry practitioners to compare ET T
IN
claims lack empirical evidence as there is little rese- with scripted testing. The experiment results show G
arch done in this area. Moreover, ET is usually consi- that ET detects significantly more critical defects T
H
dered an ad-hoc way of doing testing as everyone compared to scripted testing and is more time effi- R Ahmad Nauman Ghazi
O
does it differently. There have been some attempts cient. However, ET has its own limitations and there U
G
in past to provide structure to ET. Session based test is not a single way to use it for testing. In order to H
T
management (SBTM) is an approach that attempts provide structure to ET, we conducted a study whe-
E
S
to provide some structure to ET and gives some re we proposed checklists to support test charter T
basic guidelines to structuring the test sessions. design in ET. Furthermore, two more industrial fo- C
H
However, these guidelines are still very abstract cus group studies at four companies were conduc- A
R
and are very open to individuals’ interpretation. ted that resulted in a taxonomy of exploration levels T
E
in ET and a decision support method for selecting R
D
Objective: The main objective of this doctoral exploration levels in ET. Lastly, we investigated dif- E
S
thesis is to support practitioners in their deci- ferent problems that researchers face when con- IG
N
sions about choosing exploratory versus scripted ducting surveys in software engineering and have
A
testing. Furthermore, it is also aimed to investi- presented mitigation strategies for these problems. N
D
gate the empirical evidence in support of ET and D
find ways to structure ET and classify different Conclusion: The taxonomy for levels of explo- E
C
levels of exploration that drive the choices made ration in ET, proposed in this thesis, provided test IS
IO
by exploratory testers. Another objective of this practitioners at the companies a better understan-
N
thesis is to provide a decision support system to ding of the underlying concepts of ET and a way S
U
select levels of exploration in overall test process. to structure their test charters. A number of in- P
P
fluence factors elicited as part of this thesis also O
R
Method: The findings presented in this thesis are help them prioritise which level of exploration T
obtained through a controlled experiment with par- suits testing more in their product’s context. Fur-
A
ticipants from industry and academia, exploratory thermore, the decision support method provided
h
m
surveys, literature review, interviews and focus groups the practitioners to reconsider their current test
a
conducted at different companies including Erics- focus to test their products in a more effective way. d

N
son AB, Sony Mobile Communications, Axis Com-
a
u
munications AB and Softhouse Consulting Baltic AB. m
a
n

Results: Using the exploratory survey, we found G
three test techniques to be most relevant in the ha Blekinge Institute of Technology
z
context of testing large-scale software systems. The i Doctoral Dissertation Series No. 2017:06
2
0
ISSN:1653-2090 1
2017:06 ISBN: 978-91-7295-339-0 7:06 Department of Software Engineering
Structuring Exploratory Testing
Through Test Charter Design and
Decision Support
Ahmad Nauman Ghazi

Blekinge Institute of Technology Doctoral Dissertation Series
No 2015:09
Blekinge Institute of Technology Doctoral Dissertation Series
No 2017:06
Social Sustainability within
the Framework for Strategic
Sustainable Development
Structuring Exploratory Testing
Through Test Charter Design and
Decision Support
Merlina Missimer
Ahmad Nauman Ghazi
Doctoral Dissertation in
Strategic Sustainable Development
Doctoral Dissertation in
Software Engineering
Department of Software Engineering
Department of Strategic Sustainable Development
Blekinge Institute of Technology
Blekinge Institute of Technology
SWEDEN
SWEDEN
2017 Ahmad Nauman Ghazi
Department of Software Engineering
Publisher: Blekinge Institute of Technology
SE-371 79 Karlskrona, Sweden
Printed by Exakta Group, Sweden, 2017
ISBN: 978-91-7295-339-0
ISSN:1653-2090
urn:nbn:se:bth-14121
Structuring Exploratory Testing
Through Test Charter Design and
Decision Support
Ahmad Nauman Ghazi

Blekinge Institute of Technology Doctoral Dissertation Series
No 2015:09
Blekinge Institute of Technology Doctoral Dissertation Series
No 2017:06
Social Sustainability within
the Framework for Strategic
Sustainable Development
Structuring Exploratory Testing
Through Test Charter Design and
Decision Support
Merlina Missimer
Ahmad Nauman Ghazi
Doctoral Dissertation in
Strategic Sustainable Development
Doctoral Dissertation in
Software Engineering
Department of Software Engineering
Department of Strategic Sustainable Development
Blekinge Institute of Technology
Blekinge Institute of Technology
SWEDEN
SWEDEN
2017 Ahmad Nauman Ghazi
Department of Software Engineering
Publisher: Blekinge Institute of Technology
SE-371 79 Karlskrona, Sweden
Printed by Exakta Group, Sweden, 2017
ISBN: 978-91-7295-339-0
ISSN:1653-2090
urn:nbn:se:bth-14121
ToAllah(swt),forblessingmewiththeabilitiesandopportunities;
Tomylatefather;
Tomymotherandsisters,fortheirconstantsupport,loveandprayers;
TomywifeSarahandsonsAjlaanandGhazwan,forbeingacontinuoussourceof
peaceandjoy.
v