Usability and User Experience Evaluation of EUDAT Services Use Case

159 Pages·2016·5.88 MB·English

by  Taufik Akbar Sitompul

Aalto University
School of Science
Degree Programme in Computer Science and Engineering
Taufik Akbar Sitompul
Usability and User Experience Evalu-
ation of EUDAT Services
Use Case: Aalto Data Repository
Master’s Thesis
Espoo, September 30th, 2016
Supervisor: Assoc. Prof. Keijo Heljanko, Aalto University
Advisors: Assoc. Prof. Antonella De Angeli, University of Trento
Altti Ilari Maarala, M.Sc. (Tech.), Aalto University
Aalto University
School of Science ABSTRACT OF
Degree Programme in Computer Science and Engineering MASTER’S THESIS
Author: Taufik Akbar Sitompul
Title:
Usability and User Experience Evaluation of EUDAT Services
Use Case: Aalto Data Repository
Date: September 30th, 2016 Pages: 159
Major: Service Design and Engineering Code: T-110
Supervisor: Assoc. Prof. Keijo Heljanko
Advisors: Assoc. Prof. Antonella De Angeli
Altti Ilari Maarala, M.Sc. (Tech.)
The amount of generated research data is growing exponentially. Following this
trend, many universities and research institutes require their researchers to plan
research data management before commencing a new research project. Research
data is a valuable product of research process. Without proper management, the
value of research data cannot be utilized properly.
As the largest university in Finland in the field of engineering, Aalto University
has defined its research data management policy. The policy aims to make re-
search data management easier and it requires Aalto University to find the most
suitable platforms to be used for its research data management. Currently Aalto
University has four available options for its research data management platform
and EUDAT services are considered as one of the best candidate for Aalto Uni-
versity research data management platform. EUDAT is a pan European project
which offers common data services and it is funded by European Union. However,
it is still unclear how useful EUDAT services would be for researchers in Aalto
University.
This thesis evaluates the current state of usability and user experience of four
EUDAT services: B2DROP, B2SHARE, B2FIND, and B2ACCESS. In addition
to that, this thesis also evaluates the functionalities that are available on those
services. There are two usability evaluation methods that are used to evaluate
EUDAT services: feature inspection and usability testing. The results from those
twoevaluationsshowthatB2DROPandB2FINDhavefewusabilityproblemsand
the available functionalities are good enough for researchers in Aalto University.
On the other hand, the results show that B2SHARE and B2ACCESS need to
be improved significantly before it can be widely used by researchers in Aalto
University. This mainly caused by existing usability problems with significant
impact that need to be fixed first.
Keywords: usability, user experience, EUDAT, research data, common
data services, data management
Language: English
2
Aalto-yliopisto
Perustieteiden korkeakoulu DIPLOMITYO¨N
Tietotekniikan koulutusohjelma TIIVISTELMA¨
Tekij¨a: Taufik Akbar Sitompul
Ty¨on nimi:
EUDAT-palvelujen k¨aytett¨avyyden ja k¨aytt¨okokemusten arviointi
K¨aytt¨otapaus: Aalto-yliopiston datavarasto
P¨aiv¨ays: 30. syyskuuta 2016 Sivum¨a¨ar¨a: 159
P¨a¨aaine: Service Design and Engineering Koodi: T-110
Valvoja: Professori Keijo Heljanko
Ohjaajat: Professori Antonella De Angeli
Diplomi-insin¨o¨ori Altti Ilari Maarala
Luodun tutkimusdatan m¨a¨ar¨a kasvaa eksponentiaalisesti. T¨at¨a suuntausta seura-
ten monet yliopistot ja tutkimuslaitokset vaativat tutkijoitaan suunnittelemaan
tutkimusaineiston hallintaa ennen uuden tutkimuksen aloittamista. Tutkimusda-
ta on arvokas tutkimusprosessin tuote. Ilman asianmukaista datan hallintaa t¨at¨a
arvokasta dataa ei voida kunnolla hy¨odynt¨a¨a.
Aalto-yliopisto on Suomen suurimpana insin¨o¨oritieteiden yliopistona linjannut
tutkimusdatapolitiikkansa, jonka tarkoituksena on tehd¨a tutkimusdatan hallin-
nasta helpompaa. T¨am¨a my¨os edellytt¨a¨a sen, ett¨a Aalto-yliopiston on l¨oydett¨av¨a
sopivimmat alustat tutkimusdatan hallintaan. T¨all¨a hetkell¨a Aalto-yliopistolla
on nelj¨a vaihtoehtoa datan hallinnan alustaksi ja EUDAT-palveluja pidet¨a¨an yh-
ten¨a parhaimpana vaihtoehtoina Aalto-yliopiston tutkimusdatan hallintaan. EU-
DATonyleiseurooppalainenprojekti,jokatarjoaayhteisi¨adatapalveluitajaseon
Euroopan Unionin rahoittama. Viel¨a on kuitenkin ep¨aselv¨a¨a, kuinka hy¨odyllisi¨a
EUDAT-palvelut olisivat Aalto-yliopiston tutkijoille.
T¨am¨a tutkielma arvioi nelj¨an eri EUDAT-palvelun t¨am¨anhetkist¨a k¨aytett¨avyytt¨a
ja niiden k¨aytt¨okokemuksia: B2DROP, B2SHARE, B2FIND ja B2ACCESS.
Lis¨aksi tutkielma arvioi kyseisten palvelujen toiminnallisuutta. Kahta arvioin-
timenetelm¨a¨a k¨aytet¨a¨an arvioimaan EUDAT-palveluja: ominaisuuksien tarkas-
telu sek¨a k¨aytett¨avyyden testaus. N¨aiden kahden arvioinnin tulokset osoittavat,
ett¨a B2DROP ja B2FIND -palveluilla on vain v¨ah¨an k¨aytett¨avyysongelmia ja nii-
den toiminnallisuudet ovat tarpeeksi hyvi¨a Aalto-yliopiston tutkijoille. Tulokset
osoittavat kuitenkin B2SHARE:n ja B2ACCESS:n tarvitsevan viel¨a merkitt¨av¨a¨a
kehitt¨amist¨a ennen kuin Aalto-yliopiston tutkijat voivat k¨aytt¨a¨a niit¨a laajalti.
Syyn¨a t¨ah¨an on p¨a¨aasiassa nykyiset k¨aytt¨oongelmat, joilla on merkitt¨avi¨a vaiku-
tuksia ja jotka t¨aytyy ensin korjata.
Asiasanat: k¨aytett¨avyys, k¨aytt¨okokemukset, EUDAT, tutkimusdata, yh-
teiset datapalvelut, datan hallinta
Kieli: Englanti
3
Acknowledgements
This thesis would not have been completed without the contribution of many
people. I would like to use this opportunity to express my gratitude to those
who have directly or indirectly involved in completion of this master’s thesis.
First of all, I would like to thank my supervisor, Assoc. Prof. Keijo Hel-
janko for giving me an opportunity to work with a big project like EUDAT
and supervising this thesis. Working on a big project like EUDAT has been a
long and exhausting, but really rewarding experience for me. Special thanks
to my two advisors, Assoc. Prof. Antonella De Angeli for her guidance on
usability and user experience and Altti Ilari Maarala for proof-reading this
thesis for many times.
Thank you very much to thirteen researchers and six students of Aalto
University who have participated as test users and pilot users, respectively.
ManythankstodevelopersofEUDATfortheircooperationduringthesepast
seven months. I also would like to thank my two colleagues, Farouk Salem
and Asad Javed for teaching me to write this thesis in LaTeX.
Last but not least, I would like to thank my family for their unconditional
and infinite support during my master studies.
Espoo, September 30th, 2016
Taufik Akbar Sitompul
4
Abbreviations
EUDAT EuropeanAssociationforDatabasesinEducationand
Training
ACRIS Aalto Current Research Information System
CC Creative Commons
MIT Massachusetts Institute of Technology
CDI Collaborative Data Infrastructure
API Application Programming Interface
CKAN Comprehensive Kerbal Archive Network
CESSDA ConsortiumofEuropeanSocialScienceDataArchives
CLARIN Common Language Resources and Technology Infras-
tructure
ENES European Network for Earth System
GUI Graphical User Interface
HCI Human-Computer Interaction
ISO International Organization for Standardization
UEM Usability Evaluation Method
WebDAV Web Distributed Authoring and Versioning
SQL Structured Query Language
LAMP Linux, Apache, MySQL, and PHP
CSC Center for Science
OAI-PMH Open Archives Initiative Protocol for Metadata Har-
vesting
REST Representational State Transfer
PID Persistent Identifier
EPIC European Persistent Identifier Consortium
XML Extensible Markup Language
JSON JavaScript Object Notation
SAML Security Assertion Markup Language
LDAP Lightweight Directory Access Protocol
SUS System Usability Scale
5
CSUQ Computer System Usability Questionnaire
QUIS Questionnaire for User Interaction Satisfaction
SCI School of Science
ENG School of Engineering
ARTS School of Arts, Design, and Architecture
HIIT Helsinki Institute for Information Technology
6
Contents
Acknowledgements 4
Abbreviations and Acronyms 5
1 Introduction 10
1.1 Aalto University research data management policy . . . . . . . 11
1.2 Introduction to EUDAT services . . . . . . . . . . . . . . . . . 14
1.3 Problem statement . . . . . . . . . . . . . . . . . . . . . . . . 17
1.4 Thesis scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
1.5 Thesis structure . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2 Background 19
2.1 Introduction to usability and user experience . . . . . . . . . . 19
2.2 Usability evaluation methods . . . . . . . . . . . . . . . . . . 21
2.2.1 Usability testing . . . . . . . . . . . . . . . . . . . . . 21
2.2.2 Usability inspection . . . . . . . . . . . . . . . . . . . . 33
2.3 Combining usability evaluation methods . . . . . . . . . . . . 40
3 EUDAT services 42
3.1 B2FIND . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
3.2 B2DROP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3.3 B2SHARE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.4 B2ACCESS . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4 Conducting usability evaluation 53
4.1 Selected usability evaluation methods . . . . . . . . . . . . . . 53
4.2 Feature inspection . . . . . . . . . . . . . . . . . . . . . . . . 53
4.3 Usability testing . . . . . . . . . . . . . . . . . . . . . . . . . . 55
4.3.1 Test plan . . . . . . . . . . . . . . . . . . . . . . . . . 55
4.3.2 Test users . . . . . . . . . . . . . . . . . . . . . . . . . 57
4.3.3 Task scenario . . . . . . . . . . . . . . . . . . . . . . . 57
7
4.3.4 Test materials . . . . . . . . . . . . . . . . . . . . . . . 58
4.3.5 Pilot test . . . . . . . . . . . . . . . . . . . . . . . . . 59
4.3.6 Test site and equipment . . . . . . . . . . . . . . . . . 60
4.3.7 Real Usability Test . . . . . . . . . . . . . . . . . . . . 61
4.3.8 Data analysis . . . . . . . . . . . . . . . . . . . . . . . 62
5 Results 64
5.1 Overall results . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
5.1.1 Simple and natural dialogue . . . . . . . . . . . . . . . 67
5.1.2 Speak the user’s language . . . . . . . . . . . . . . . . 67
5.1.3 Minimize the user’s memory load . . . . . . . . . . . . 69
5.1.4 Be consistent . . . . . . . . . . . . . . . . . . . . . . . 69
5.1.5 Provide feedback . . . . . . . . . . . . . . . . . . . . . 70
5.1.6 Provide clearly marked exits . . . . . . . . . . . . . . . 70
5.1.7 Provide shortcuts . . . . . . . . . . . . . . . . . . . . . 71
5.1.8 Provide good error messages . . . . . . . . . . . . . . . 72
5.1.9 Error prevention . . . . . . . . . . . . . . . . . . . . . 72
5.2 Results for B2FIND . . . . . . . . . . . . . . . . . . . . . . . . 73
5.3 Results for B2DROP . . . . . . . . . . . . . . . . . . . . . . . 78
5.4 Results for B2SHARE . . . . . . . . . . . . . . . . . . . . . . 82
5.5 Results for B2ACCESS . . . . . . . . . . . . . . . . . . . . . . 85
5.6 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
6 Conclusions 90
Bibliography 99
A Test Script 100
A.1 General introduction . . . . . . . . . . . . . . . . . . . . . . . 100
A.2 Introduction and background questionnaire for each EUDAT
service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
B Test users’ profile 103
C List of test tasks for the test users 106
C.1 List of test tasks to be completed on B2FIND . . . . . . . . . 106
C.2 List of test tasks to be completed on B2DROP . . . . . . . . . 107
C.3 List of test tasks to be completed on B2SHARE . . . . . . . . 108
D Post-task questionnaires 109
D.1 Modified System Usability Scale (SUS) . . . . . . . . . . . . . 109
D.2 Modified Computer System Usability Questionnaire (CSUQ) . 110
8
D.3 Modified Questionnaire for User Interaction Satisfaction (QUIS)111
E List of problems and suggestions 113
E.1 B2FIND . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
E.1.1 List of problems found on B2FIND . . . . . . . . . . . 114
E.1.2 List of proposed suggestions for B2FIND . . . . . . . . 115
E.2 B2DROP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
E.2.1 List of problems found on B2DROP . . . . . . . . . . . 118
E.2.2 List of proposed suggestions for B2DROP . . . . . . . 119
E.3 BSHARE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
E.3.1 List of problems found on B2SHARE . . . . . . . . . . 125
E.3.2 List of proposed suggestions for B2SHARE . . . . . . . 128
E.4 B2ACCESS . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
E.4.1 List of problems found on B2ACCESS . . . . . . . . . 134
E.4.2 List of proposed suggestions for B2ACCESS . . . . . . 135
F Results from post-task questionnaires 136
F.1 Results from System Usability Scale (SUS) . . . . . . . . . . . 136
F.2 ResultsfromComputerSystemUsabilityQuestionnaire(CSUQ)142
F.3 Results from Questionnaire for User Interaction Satisfaction
(QUIS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
9
Chapter 1
Introduction
With millions of researchers and scientists pursuing their research across the
globe, the amount of generated research data is growing exponentially1. Fol-
lowing this trend, many universities and research institutes require their re-
searcherstohavearesearchdatamanagementplanbeforecommencinganew
research project2. Evidence of good practices of research data management
has become an expected requirement for granting research funding3. Re-
search data management is the procedure that covers research data lifecycle,
includingcreating, storing, delivering, maintaining, archiving, andpreserving
research data4.
To have a better understanding of research data management, researchers
also need to understand the nature of research data. Boston University
stated that ”research data is data that is collected, observed, created for
purposes of analysis to produce original research results”5. Research data
can be in physical or digital format and it also comes in various types, such
as text, image, audio, video, specimens, samples, and so on. Depending on
the research purposes, research data can be classified into four categories,
such as observational data, experimental data, simulation data, and derived
or compiled data6. Different category and type of research data may require
different research data management plan.
The advancement of technology revolutionizes the way researchers cap-
ture, store, manage, analyse, and visualize research data, thus making re-
1http://researchanalytics.thomsonreuters.com/m/pdfs/1003903-1.pdf
2https://blogs.ntu.edu.sg/lib-datamanagement/introduction
3http://www.dcc.ac.uk/events/workshops/introduction-research-data-management
4http://datalib.edina.ac.uk/mantra/datamanagementplans
5http://www.bu.edu/datamanagement/background/whatisdata
6https://www.nottingham.ac.uk/research/research-data-management/what-is-
research-data.aspx
10