Table Of ContentInteractingwithComputers19(2007)196–205
www.elsevier.com/locate/intcom
Design and evaluation of a tactile memory game for visually
impaired children
Roope Raisamo *, Saija Patoma¨ki, Matias Hasu, Virpi Pasto
TampereUnitforComputer-HumanInteraction(TAUCHI),DepartmentofComputerSciences,FIN-33014UniversityofTampere,Finland
Availableonline27September2006
Abstract
Visuallyimpairedpeoplehavealackofproperuserinterfacestoallowthemtoeasilymakeuseofmoderntechnology.Thisproblem
maybesolvedwithmultimodaluserinterfacesthatshouldbedesignedtakingintoaccountthetypeanddegreeofdisability.Thepurpose
ofthestudypresentedinthisarticlewastocreateusablegamesforvisuallyimpairedchildrenmakinguseoflow-costvibro-tactiledevices
inmultimodalapplications.Atactilememorygameusingmultimodalnavigationsupportwithhigh-contrastvisualfeedbackandaudio
cueswasimplemented.Thegamewasdesignedtobeplayedwithatactilegamepad.Differentvibrationsweretoberememberedinstead
of sounds or embossed pictures that are common in memory games for blind children. The usability and playability of the game was
tested with a group of seven 12–13-year-old visually impaired children. The results showed that the game design was successful and a
tactile gamepad wasusable.The gamegot a positive responsefrom the focusgroup.
(cid:2)2006Elsevier B.V.All rightsreserved.
Keywords: Visuallyimpairedchildren;Multimodaluserinterfaces;Low-costhapticdevices;Tactilegames;Tactilefeedback;Usabilitytesting
1. Introduction The International Game Developers Association has
addressed the question of game accessibility and has
When making information technology accessible formed a Game Accessibility Special Interest Group
for disabled people an answer is to use customized devices (GA-SIG). The ambitious goal of the GA-SIG is to make
and design the interfaces according to established guide- all game genres equally accessible despite the disability of
linesthataremaintainedbytheorganizationsrepresenting theuser(IGDAAccessibilityGroup,2004).Theystatethat
disabledpeople(Tiresias.org,2006).Anothersolutionisto the game industry should be pushed to design games that
implement multimodal applications making use of the lat- take into account persons with various disabilities.
est interface technology. When lacking of a certain ability There isspecialassistivetechnology(VisionConnection,
the rest of the abilities are used to substitute the impair- 2006) designed and developed particularly for the needs of
ment in the best possible ways (Goldstein, 1999). In multi- blindcomputerusers.Forexample,withtheaidofascreen
modal user interfaces the loss of sight, hearing or mobility reader it is possible for a blind person to discover the tex-
can be replaced with other modalities. Well designed mul- tual contents of a program. Information on the screen is
timodal user interfaces can be invaluable aids for people read aloud through speech synthesizer. Screen magnifiers
withdisabilitiesinmanydifferentareas,suchaseducation, are handy when a partially sighted user has to enlarge a
training, rehabilitation, and communication. They can portionofthedisplay.ABrailledisplaypassesinformation
offer access to information, entertainment and games. through the user’s fingertips as its mechanical parts physi-
callychangepositionandformBraillesentences.However,
these aids are expensive, and mainly designed for blind
* Correspondingauthor.Tel.:+358335517056;fax:+358335516070.
adultusers.Assuch,theyarenotenoughtomeettheneeds
E-mail addresses: [email protected].fi (R. Raisamo), [email protected].fi
(S.Patoma¨ki),[email protected].fi(M.Hasu),[email protected].fi(V.Pasto). of blind children.
0953-5438/$-seefrontmatter (cid:2)2006ElsevierB.V.Allrightsreserved.
doi:10.1016/j.intcom.2006.08.011
R.Raisamoetal./InteractingwithComputers19(2007)196–205 197
We chose visually impaired children as the main focus play for partially sighted and sighted users. Even in games
group. By visually impaired we are referring to both blind for blind children use of visual feedback can be justified
and partially sighted children. They are a very challenging because it also benefits blind children who can share the
groupofusers.Thesechildrenareevenmoredifferentfrom excitement of playing with their sighted friends.
each other than average children, since there are so many Sonification, visualization by sound, has been studied
differentkindsofvisualimpairment.Itisalsohardtodiag- widelyfordecadesandithasalsobeenappliedinsoftware
nose the impairment of children in a young age, as they forblindpeople.Theaimistodisplay,forexample,graph-
cannot explain accurately what they see, especially if they ics, line graphs or even pictures using non-speech sound
have always seen that way. (Meijer, 2006). By altering the various attributes of sound,
VisuallyimpairedchildrenstarttolearnBrailletypically for example pitch, volume and waveform, the sound is
in the elementary school. Before that they have to master changed according to its visual counterpart. In the Sound-
some fine-motor, directional and tactile skills that are View application (van den Doel, 2003) a colored surface
taughtatapre-schoolageoffivetosix.Wechosetocreate couldbeexploredwithapointingdevice.Theideawasthat
a tactile memory game, because it would teach these vital thecharacteristicsofcolors,hue,saturationandbrightness
skills to the children in a simple way and in the form of a were mapped into sounds. Roth et al. (2002) used audio
possibly already familiar classical memory game. and kinesthetic rendering jointly and separately. The
In this paper, we present a case study of designing and audio-kinesthetic encoding of the graphs was found to be
evaluating the tactile memory game to be played with the most usable approach in visualizing shapes to blind
low-cost tactile feedback devices and multimodal naviga- people.
tion support. There is a wide selection of haptic feedback The Swedish Library of Talking Books and Braille
devicesonthemarket.Thereisanearlierexperiment(Rai- (Barnens, 2006) has audio games suitable for young blind
samoetal.,2005)investigatingthelimitsoflow-costhaptic children.Thegames areplayedwithanordinarykeyboard
feedback devices, such as force feedback steering wheels, andtheyarebasedonsoundfeedback,butvisualfeedback
joysticks, and gamepads. In the present study these earlier is also present. A memory game is played by recalling
results have been used to design the tactile effects. When animal sounds. Based on the tests (Eriksson and
designing game interfaces for blind children the essential Ga¨rdenfors,2006)thatfourvisuallyimpairedchildrenpar-
sensory channels should be hearing and touch. Further- ticipated in, the auditory memory game was difficult for
more, based on earlier research partially sighted children them. Non-visual computer games can not provide such a
need to have clear, high-contrast visual stimuli (Patoma¨ki good overview of the play area as graspable cards can.
et al., 2004). Their advice was to provide proper instructions that help
Benefitsofmultimodalgamesforblindchildrenareevi- a blind child to understand the mental model of the game.
dent. Game play may motivate to use computers also for AudioBattleship(Sa´nchezetal.,2003)wasbasedonthe
otherpurposesandthussupportsthedevelopmentofcom- classical battleship game using only audio cues. User tests
puting skills. Therefore, the special interfaces presented in withfourvisuallyimpairedchildrenshowedthatcollabora-
this paper help visually impaired children to gain entrance tion between blind children was enhanced in the game.
in the modern information society dominated by the AudioDoom(LumbrerasandSa´nchez,1999)isaninterac-
sighted. tive game where spatial sound is used to present the envi-
ronment that the player experiences from a first-person
2. Previous work perspective. It was found that blind children preferred
playingthegamewithajoystickoverakeyboard;obvious-
Inthis section, we give an overview ofprevious work in lythecleardirectionsofthejoystickgaveadditionalspatial
interface technology and applications used by blind com- information. As the game was played for several times the
puter users. The emphasis is on interfaces especially children could construct the environment accurately with
designed for blind children. Applications for blind adult LEGO bricks. All of these earlier results helped us to
users are also presented when applicable. We have divided design the use of audio in our game.
the previous work in three categories: auditory interfaces,
tangible tablets and keyboards, and dynamic haptic 2.2. Tangible tablets and keyboards
interfaces.
Oneway ofcreatingtangiblehuman-computer interfac-
2.1. Auditory interfaces esforvisuallyimpairedyoungchildrenistousealternative
keyboardsortablets.Thegoalistocombinetheuseofreal
In auditory interfaces the audio can be the main or the tangiblematerialswithadvantagesofinformationtechnol-
only feedback channel and it can be used in various ways. ogy.Thebenefitthesedeviceshavecomparedtomosthap-
There may or may not be visual feedback that can be tic devices is that they are used in a simple and natural
designed in two different ways: it can just be present with- manner. The application can be manipulated directly with
out giving any benefit in gaming that is based on audio fingers by touching and pressing the surface of the device.
hints only, or visual feedback significantly eases up game Hand-made or industrially manufactured overlays can be
198 R.Raisamoetal./InteractingwithComputers19(2007)196–205
set on top of these devices, such as Flexiboard (Flexi SensAble PHANTOM devices (SensAble, 2006) have
Forum, 2006) and IntelliKeys (Inclusive Technology, been used in applications for blind computer users. Sjo¨s-
2006). Overlays are customizable and can be constructed tro¨m (2002) implemented several games for blind children.
individually for a particular child. For example, fabrics, Forexample,ahapticbattleshipgame,ahapticmathemat-
pearlsandwaxedstringscanbegluedonoverlays,orprint- ics program, and a painting application in which textures
ed embossed graphics and figures can be used. correspond to different colors have been developed. Pato-
Computergamesoriginallyintendedforsightedchildren ma¨ki et al. (2004) constructed several simple games for
can be played with minor adjustments with the aid of tan- visually impaired 4–6-year-old children making use of
gibletablets.IntheTIMproject(TIM,2001)severalgames dynamic haptic effects with the PHANTOM device. A
(X-tune, Reader Rabbit, Can You Hear It?) were made graph visualization tool for blind students using a PHAN-
usable forblind children as appropriate overlays were pre- TOM device has been studied by Ramloll et al. (2000).
pared.Hammarlund(1999)suggeststhatspecialgamesfor They found that haptic graphs should be visualized as
blindchildrenshouldbedesignedfromthescratchtomake groove lines rather than embossed lines.
full use of the combination of real-world materials, sound The previous research presented in this section gave us
illustrations and recorded speech. insight and some empirical results to design the auditory
Geometricshapesandmapscanbeshowntoblindpeo- andhapticfeedbackmethodsforthetactilememorygame.
pleusingtablets.InastudybyRothetal.(2000),geometric However, many of the design decisions needed to be made
shapesweretaughttovisuallyimpairedpupilsasraisedfig- during the iterative development based on our own pilot
ures using a printed swell paper overlay on a tablet. When tests and preliminary investigations, as directly applicable
shapes were explored audio feedback was simultaneously results did not exist. Next, we describe this process.
provided.AstudybyLandauetal.(2003)showedthatwith
the Talking Tactile Tablet (Touch Graphics, 2006) their 3. Design of a tactile memory game
studentslearnedgeometricalshapesbetterthanwhenusing
common mathematics materials intended for blind pupils. Visually impaired children are very dependent on their
Holmes and Jansson (1997) presented a street network in parentsandotherpeoplewhohelpthemintheirdailylives.
an audio-enhancedvirtual map. Itwas foundthat atactile Blindchildrenmaythereforehaveproblemswithgettingin
map overlay needed less exploration time than a matrix contactwithotherchildren.Ourgoalwastocreateagame
overlay. Based on these previous results we decided to thatwouldbeplayedbybothsightedandvisuallyimpaired
includeatangiblemodelinthetestingprocedurepresented children, and would encourage visually impaired children
in this article. to use computers independently. Our primary focus group
isvisually impaired young children who do not have other
2.3. Dynamic haptic interfaces disabilities.
We also wanted to know how low-cost tactile devices
The benefit of creating dynamic haptic applications suchasagamepad,a joystick orawheel cansupportboth
compared to using of physical materials is that the con- navigation and game experience for visually impaired
tent can be designed freely and changed dynamically child-users. Existing memory games for visually impaired
making it more interactive. Nemec et al. (2004) used a children are either tangible games, made of touchable or
force feedback joystick and a haptic mouse together with sound-making materials, or auditory computer applica-
spatial sound that was found to be applicable when cre- tions. Instead of graspable objects or sounds, we used tac-
ating a mental model of indoor environments. In their tileeffectsproducedbyatactilegamepadastheitemstobe
study a haptic mouse was proven to be usable in haptic memorized: images were replaced by tactile effects. The
sensing of the virtual model, but a force feedback joy- Swedish Library of Talking Books and Braille’s Djur
stick was confusing for all of the users. In a pilot study Memory sound memory game (Barnens, 2006) was an
by Feintuch et al. (2004) a blind girl, after training with inspiration for this design decision, as they used audio in
navigation using a force feedback joystick, could walk a similar way.
quickly and confidently in real surroundings, not known Toourknowledgeatactiledevicehasnotbeenusedthis
to her before. These examples emphasize a good design way before. Compared to tangible games computer-based
of the navigation methods. haptic interfaces give more freedom to easily adjust haptic
Baptiste-Jessel et al. (2004) used a Logitech Wingman and sound feedback adapting to each user. Furthermore,
force feedback mouse to present graphical data to blind thecontentofthevirtualgamecanbericherasitisnotlim-
users in web-based documents. For example, a geographi- ited to real artifacts. Dynamically changing contents and
calmapwaspossibletobeexploredwiththemousebygiv- advancing in the game makes the game more expressive
ing a magnetized effect, when the center of the region was and interactive.
approached, and a textured effect, when the mouse was in The aim of the classical memory game is to find image
the periphery of the area. The test subjects told them that pairsfromcardsthatarelaidoutonaplane.Atthebegin-
haptic feedback helped to get a mental model of the geo- ningallthecardsarefaceddown.Theuserstartsthegame
graphical area. by choosing one card and turning it over. Then the user
R.Raisamoetal./InteractingwithComputers19(2007)196–205 199
triestofindapairforitbyturninganothercardover.Ifthe effects. The larger motor is located within the left handle
imageofthesecondcarddoesnotmatchwiththefirstone, of the pad, and the smaller one within the right handle.
both cards are returned to the face down position. If they Although the game was designed for blind users, we
do match, the cards are removed. The game ends when offered simple, high contrast visual feedback as well. Each
all the pairs have been found. The less there are turns of cardwasrepresentedwithuniformvisualfeedback(afilled
pairs the better the score is. box).Thegameinterfaceconsistedof12darkbluecardsin
Inapreviousstudy(Raisamoetal.,2005)18adultusers a3·4gridonawhitebackground,asshowninFig.2.The
took part in a series of tests aimed to find out distinguish- width of a card was 270 pixels and the height was 150 pix-
ingthresholdsfortactileeffects withagamepad,ajoystick els.Thecursorwaspresentedasa385·215-pixelbounding
and a wheel. The study was aimed to roughly estimate the black box around the card. The game area was 1280·925
tactile parameters that would be usable for visually pixels on a 1700 LCD monitor. A turned card was marked
impaired children. The results supporting the selection of with the cyan color. As the pairs were found their visual
tactile gamepad were the following: if the difference in and haptic presentations were removed from the game.
the balance of the motors was high enough (more than Navigation information was presented with a digitized
50%), all the users could differentiate the effects without xylophonesoundthatwasconsideredpleasant,butneutral.
errors, which was clearly not the case in other conditions. The column of the cursor was indicated by panning the
In addition, the type of motor used had a clear effect on sound from the left to the right audio channel based on
the recognition of force magnitude. With the small motor thehorizontalcursor position. The middle column divided
the small magnitudes were the easiest, while with the large the sound equally in both channels. The pitch of the xylo-
motorthelargemagnitudesweretheeasiest.Thisisdueto phoneindicatedtherow;thelowestrowproducedthelow-
their mechanical construction. The gamepad also clearly estsoundandthehighestrowproducedthehighestsound.
outperformed the force feedback devices in differentiating A negative-sounding hoot sound indicated a wrong pair
frequency of the effects. and a melodious chimes sound was played when the user
Considering also the game type, easy availability, and found a right pair. The hoot sound was followed by a
inexpensive price, we chose to use a Logitech WingMan sound imitatinga cardturn toindicatethat thecards were
Rumblepad (Fig. 1) as the tactile controller for the new returned to the face-down position. As the cards disap-
game.Itssmallsizeandtwo-handed(robust)usewerealso peared when the pairs were found, an empty card slot
important criteria in selecting the device, as the game was was indicated by a sound with the corresponding pan
designedforvisuallyimpairedsmallchildren.Thegamepad and pitch, but with 50% of the volume of a full card slot
wasconnectedtothecomputerusingastandardUSBcon- sound. The edges of the game grid were indicated by a
nectoranditwasinterfacedinthesoftwareusingMicrosoft louder knocking sound.
DirectX 9.0. The tactile effects to be memorized were designed with
The game cursor was controlled with the four-way dig- Immersion Studio 4.1 (Immersion, 2006). Initially we
ital‘‘D’’pad(Fig.1)asitclearly representsphysicaldirec- assumed that plain tactile effects without sounds would
tions. The cards were turned with the ‘‘C’’ button (Fig. 1) be too hard for the children to recognize. So in the pilot
which is easily accessible with the player’s right thumb. tests we used effects imitating the feel of animals with
The gamepad has two vibrating motors to produce tactile sounds associated to them. These effects consisted of,
e.g., dog barking and chicken cackling. In the pilot tests
the animal sounds turned out to be too dominant, so we
Fig.1. LogitechWingManRumblepad. Fig.2. Thetactilememorygameinterface.
200 R.Raisamoetal./InteractingwithComputers19(2007)196–205
left them out from the actual tests to focus on tactile feed- possible score would have been six, if only it would have
back. Yet audio cues for navigation were found to be required a lot of luck. Scores of 12 pairs or less were con-
working.Therefore,theywerekeptunchangedinthegame. sidered as very good.
Fortheactualtestwedesignedanewtactileeffectsetwith-
out sounds, making use of earlier empirical research on 4. Usability and playability evaluation
similardevices (Raisamo etal., 2005), as theanimaleffects
worked well with the corresponding sounds, but were not Although our final users for the memory game were
the most easily recognizable effects if only tactile feedback visually impaired young children, in this study the target
were used. group consisted of visually impaired teenagers, since they
The exact parameter values that are reported in earlier were able to tell us more accurately what they feel with
studies of tacton research (e.g., Chan et al., 2005; van the device to help us in iterative development of the tactile
Erp and Spape´, 2003; Brown, 2005) were not usable for game, and we did not want tooverload our final test users
us, since the results are specific to the devices reported in with testing of versions of the application that might not
the papers. We tried to use the results based on high-qual- have been optimal. Testing of the final prototype with
itySensAblePHANTOMdevices(SensAble,2006),butthe theyoungchildrenisleftforfuturework.Aspecialusabil-
cheapdevicewasnotabletoproducethefrequenciesdeter- ity testing procedure developed in an earlier project (Pato-
mined to work best in these studies without causing dis- ma¨ki et al., 2004) was refined for these studies. It follows
turbing resonance or other technical problems. So the the principles of usability studies (Nielsen, 1993), but
effects needed to be specifically designed for the device includes specific support for visually impaired young chil-
used.Nevertheless,thesestudiesgaveususefulbackground dren, as they often cannot manage alone. It is highly
information and basic rules when designing the tactons important not to give too much support during the test,
used. but to enable the child to use the computer system.
Thecriteriaforselectingtheeffectsinthegamewasthat
they had to be very short and at the same time clearly dis- 4.1. Testing environment
tinguishablefromeachother.Inaddition,theeffectshadto
be designed so that they could have clear names that ease In Finland, as in several other countries, the aim isthat
up their memorability in the game, as Brown et al. (2005) visuallyimpairedchildrenwithoutadditionalhandicapsgo
suggested in their work. They also suggested that instead to a normal school with the sighted children. Twice in a
of vibro-tactile parameters (frequency, magnitude, wave- year these blind children belonging to the same age group
form or duration) more complex parameters should be are invited to take part in a special teaching period lasting
used. They found roughness and rhythm to be usable as for a week in Jyva¨skyla¨ School for the Visually Impaired
such parameters, level of roughness being produced with (Jyva¨skyla¨, 2006). These courses are also convenient for
magnitude-modulatedsignalsandrhythmswithpulsesthat testing purposes, since an entire target user group is gath-
have different durations. Accordingly, four of our effects ered at the same time in the same place from all over the
were based on altering magnitude and two were based on country.
rhythmic pulses. To make the effects clearly distinctive The testing was carried out in an empty classroom,
and therefore easy to label, we designed three effect pairs where the test setup was built (see Fig. 3). The equipment
and named them as follows: decreasing and increasing, consistedofacomputer,aflatdisplay,apairofloudspeak-
two-rhythm and four-rhythm, and gentle and intense. ers,avibro-tactilegamepad andtheequipmentforrecord-
Characteristics of the effects are listed in Table 1. All the ing and videotaping the test situation.
effects used were based on square waveform.
Whenthegame ended, ascore windowappearedonthe 4.2. Test users
screen. The number of pairs turned was written on the
screenandforeverypairturnaredcircleappearedsequen- Beforetestingthegamewiththetargetusersitwaspilot
tiallytothescreenwithapingsoundsothattheblindchild tested several times with sighted adults and children with
couldunderstandthescore.Asthereweresixpairs,thebest andwithouttheaidofgraphicalfeedback.Thetwosighted
Table1
Theeffectsusedinthetactilememorygame
Effectname Description Magnitude Motor Duration(ms)
Gentle Gentleconstantvibration 2000 Small 2000
Intense Strongconstantvibration 10000 Large 2000
Increasing Increasingvibration From2000to10000 Large 2000
Decreasing Decreasingvibration From10000to2000 Large 2000
2-Rhythm Two250msimpulsesseparatedwitha250msbreak 10000 Large 750
4-Rhythm Four250msimpulsesinarow 10000 Large 1000
R.Raisamoetal./InteractingwithComputers19(2007)196–205 201
reader and screen magnifier. Every child had tried out the
PHANTOM haptic feedback device (SensAble, 2006) in
the tests carried out earlier in the same school (Patoma¨ki
et al., 2004; Saarinen et al., 2006). Some of the children
were also familiar with haptic gaming devices such as a
forcefeedbackwheel,aforcefeedbackjoystick,oratactile
gamepad.
4.3. Testing procedure
The testing was carried out in Jyva¨skyla¨ School for the
Visually Impaired (Jyva¨skyla¨, 2006). Demographic infor-
mation on the children was gathered with a questionnaire
thattheirparentsfilledinbeforethetests.Inthebeginning
of each session the test assistant chatted casually with the
child to suppress the tension that situations like this often
Fig. 3. A blind girl is playing the tactile memory game with the vibro- cause. Our test assistant was a nursery school teacher
tactilegamepadinherhands.
whose task was to motivate and interact with the children
based on the needs of the situation.
pilot child-users were at ages of 9 and 12 years. A blind 9- Aninformal interviewconsisting ofquestions aboutthe
year-old girl was also invited to pilot test the game. child’sbackgroundfollowednextbasedontheinformation
A group of seven 12–13-year-old visually impaired chil- acquiredwiththequestionnaire.Itwasextremelybeneficial
drenparticipatedinthetests:wehadthewholeageclassof to go through the questions with the child considering his
Finnishvisuallyimpairedchildrenwithoutanyothermajor or her skills and experiences on applications and haptic
disabilities.Allthechildrenwerevisuallyimpairedbutthey devices, since clearly the parents had not been able to
had different levels of impairment. Three of the children describe them accurately.
were totally blind and four were partially sighted. We Based on earlier experiences in studies with visually
asked the parents to write down a description of the level impaired children (Patoma¨ki et al., 2004) we provided the
ofremainingeyesightforthosewhowerepartiallysighted. children with visual stimuli. Before testing, the seeing abil-
Two of the partially sighted children saw to a degree with ityofpartiallysightedchildrenwasevaluatedbyobserving
oneeyewhiletheirothereyewasblind.Theothertwopar- theiruseofgraphicalfeedback.Thebestdistanceandposi-
tiallysightedchildrenhadsomeaccuratesightabilitywhen tion of the display was determined together by the child
they looked at the target at a close distance. There were and the test assistant. The length of one test session was
three boys and four girls (see Table 2). normally less than 45min. If needed, a session could be
Fourofthechildrenwerecongenitallyblindorpartially extended up to 60min.
sighted. Three of the children had gone blind or partially First, the child got used to playing the memory game
sighted later in their life. It is common for a blind child with a tangible model. Following the advice by Eriksson
tohaveotherdisabilitiesaswell.Astheschoolhasspecial- andGa¨rdenfors(2006)weorganizedtrainingtoensurethat
izedineducationofblindpupils,exclusivelyfocusingatten- the children had a correct mental model of the game. We
tiontochildrennotbeingmultiplydisabled,therewasonly also used elements of tangible interfaces. The purpose
onechildinthegroupwhohadotherdisabilities.Thechild and rules of the memory game were rehearsed with a tan-
inquestionsufferedfromproblemsinconcentrationandin gible model having a similar form as we had implemented
spatial conceptualization. as a virtual multi-modal application. The virtual cards in
Allthechildrenhadfrequentlyusedordinarykeyboards thetactilememorygamewerepresentedintheexactlysame
and assistive technology, such as Braille keyboard, screen layout as in the tangible model. A tangible memory game
modelwasconstructedofsmallboxeswithcoversthatwere
gluedonthecardboardplane(seeFig.4).Fabricswithvar-
Table2 ioustextileswereattachedinsidethecoversoftheboxes.In
Basicinformationonusers
the tangible memory game an overturn of a cover and
Age Gender Visualdisability Inceptionage Otherhandicaps active exploration of the fabric texture in it corresponds
ofblindness
toa card turn and seeing of apicture in thememory game
13 Boy Blind Congenital Yes of the sighted.
12 Girl Partiallysighted Congenital No
Next, each child was introduced to the vibro-tactile
13 Boy Partiallysighted Congenital No
gamepad (Fig. 1). The proper operating position and
12 Girl Partiallysighted 2 No
12 Girl Blind 11 No the use of push buttons in the gamepad were shown to
12 Boy Partiallysighted Congenital No the child. In the training mode of the game the cards
13 Girl Blind 9 No were pre-arranged. In this way, the test assistant could
202 R.Raisamoetal./InteractingwithComputers19(2007)196–205
and turned their head to turn their better eye towards the
display.
After the test the children were asked about the various
soundsthegamecontained.Thechildrenrecalledtheaudio
cues fairly accurately and could explain the meaning of
sounds in the game to the test assistant. The game con-
tained sound illustrations and audio cues for navigation.
Theknockingsoundwasrememberedbyallsevenchildren.
Location sounds of the cards were remembered by four of
them. Five of the children remembered the melodious
chimes sound. Four of the children remembered the hoot
sound. The card turning sound was remembered by two
of the children. The empty card slot sound was remem-
bered by three. Two of the children mentioned the ending
sound of the game and three of the children noted about
the stereo sound used for localizing. The navigation
Fig.4. Mentalmodelofthegamefieldwasunderstoodwiththeaidofa sounds,knockingsoundandlocationsounds,wereremem-
tangiblemodel.
beredmoreoftenthanothersoundsinthegame.Thisindi-
cates that the navigation sounds were useful for the
explain and show all the various action sounds that children.
occur during the game play. The idea and rules of the
game were gone through once more in this virtual gam- 5.2. Localizing in the game
ing context.
After the rehearsal of playing the virtual memory game The tactile memory game sets different demands for a
the child continued with the actual testing. The goal was blind user in contrast to a partially sighted user. In the
that the child would playthe game independently,without gamethefirstchallengeforablindpersonistoformamen-
any help. The child was expected to complete the game tal model of a 3·4 matrix. This means that the spatial
with as few card turns as possible. information gained with the tangible model is represented
After completing thetesting phaseeachchild wasasked andsymbolizedinmemory.Whenmovinginthegamearea
asetofquestions,buttheinterviewthatwascarriedoutas theuserhastoconstantlykeephisorherpositioninmind.
a normal conversation was not limited to them. In short, Another possible way for a blind user to localize is to
theissuesdiscussedincludedthetactiledevice,theauditory concentrate in the aid of location sounds. To get benefits
andtactilefeedbackofthegame,andothergameideasthat oftheseaudiocuesablindpersonhastohaveafairlyaccu-
arose in the child’s mind. rate ability to discriminate between different pitches of
tone. With a good ear for tones it is possible to know the
5. Results and discussion exact position in the matrix. One adult pilot user said that
he could make localization based on the location sounds
In this section, we present the results of the usability rather than keeping constant track of movements.
study in the following main categories: use of visual and Thegameplaydoesnotputasgreatacognitiveloadfor
auditory feedback, localizing in the game, success in the partially sighted users as it does for blind users. Partially
game play, navigation strategies, and memorizing vibra- sightedusersactuallyseetheirownpositionandatthetime
tions. The results are based on video logs, observations of a card turn they see the cell being touched. During the
of the children, event logs, and the children’s answers in gameplaypartiallysighteduserscanconcentrateonrecall-
questionnaires. ing the vibration and use the visual memory to localize it,
whereas blind users do not have these external visual aids
5.1. Visual and auditory feedback for remembering. A blind user has to substitute visual
memoryaidswithamentalimageand/orwiththelocation
In the tests it was noticed that all four partially sighted sounds of the game.
childrencouldmakeuseofthehigh-contrastgraphicaluser Weconsideredusinga4·4gridinthegame,butdecid-
interface ofthe game. The flat display proved tobe practi- edagainstit.Basedonourpilotstudiesalargernumberof
cal for them, since it can be placed on the very edge of the tactile effects would have been recognizable. Navigation
desk.Itisimportantthatthedisplayiseasilyadjustablefor would have become more difficult. As our final target user
every child to have an optimal distance to the source of group was visually impaired small children, we aimed at
visual feedback.Two of the partially sighted children were finding a game configuration that would work well for
able to use visual feedback in a normal sitting position 12–13-year-old visually impaired teenagers. This way the
headturned directlytowardsthedisplay.Thetwochildren game would become challenging, but presumably not too
who could see partially only with one eye leaned forward challenging for the 5–6-year-old children.
R.Raisamoetal./InteractingwithComputers19(2007)196–205 203
5.3. Success in the game play that the difficulties in game play derived from the reason
thatchildrendidnotcomprehendtheplayarea.Onesignif-
Using the tangible model blind children understood the icantdifferencebetweenourandtheirtestprocedureisthat
notion oftheplayareamatrix:therowandcolumn layout we taught the children the game idea and layout with the
in which the touchable boxes had been arranged. The tangible model (Fig. 4). This may be the most important
choice of the interaction device was a success: there were reason for the better results. Of course, other reasons can
not any difficulties in operating the push buttons and han- betheageofthechildren,occurrenceofotherundiagnosed
dling the gamepad. We considered a tactile memory game disabilities, and the level of visual impairment in the test
with 12 cards, that is six different vibrations, as rather group. There may also have been differences in memory
demanding, especially for totally blind children. Our games and testing procedures, since they did not give
assumption was that memorizing various vibrations is details on these.
more difficult than memorizing plain sounds or real mate-
rialsthatthechildrenarealreadyfamiliarwithinordinary 5.4. Navigation strategies
memory games.
However, the result was that blind children managed to As the adult users played the game we noticed that the
finish the tactile memory game in fewer steps than expect- majorityofthemnavigatedinthematrixrowbyrow.Some
ed. With adult pilot users the average result was 16 pair preferredtonavigatefromlefttoright,whilesomenavigat-
turns,buttherewasagreatvariationintheresultsbetween ed every other row from left to right and every other row
theusers.Withthesightedpilotchild-userstheresultswere from right to left. At the same time when moving forward
24and38pairturns.The9-year-oldblindpilotuserplayed in the matrix they systematically turned cards over. They
thegamewith21pairturns,butfromtimetotimeshehad returned back only to pick up a card as they found a card
tobeassisted.Intheactualteststhechildrentypicallycom- that they recalled having a vibration felt earlier.
pletedthegame with15pair turns,thetwoextremesbeing It was somewhat surprising that the children did not
11 and 24 pair turns (see Table 3). have this kind of systematic navigation strategy. Only
It was evident that the children could play the game one child navigated in the game row by row from left to
without external help, since six of them played the game right. She was partially sighted and got the best result in
effortlesslybythemselves.Theblindchildwhohadaprob- thewholetestseries,11pairturns.Therestofthechildren
lem with conceptualizing the space had serious difficulties seemed to wander around in the matrix and turned the
withthegame.Inthetestsheconcentratedwellandplayed cards every now and then in a seemingly random order.
the game with great enthusiasm, but he clearly could not When investigating the game area, the user can choose
manageinvisualizingthegamementally.Thetestassistant to turn the cards so that they are located either closely or
helped him to find pairs card by card in the game that led distantly in the matrix in relation to each other. In our
toaresultof24pairturns.Withtheothersixchildrenthere analysis,thepairstheuserchoseduringthegameplaywere
was no great difference in results. Even total blindness grouped in three categories: distant, close and matching
seemednottobeacrucialfactorinsucceedinginthegame. pairs (see Table 3). The term ‘distant pair’ means that the
Resultsofthepartiallysightedchildrenwere11,12,15and cards were scattered in the matrix. The term ‘close pair’
15 pair turns, whereas blind children had 13 and 15 pair meansthatthechosencardsweresidebyside,nexttoeach
turns. other. The term ‘matching pair’ means that the cards had
Eriksson and Ga¨rdenfors (2006) had a similar memory the same effect in them, forming a right pair. Distant pairs
game to ours, but they used only sound feedback. Howev- and close pairs tell about the spatial separation of the
er, our experiences on the usability of the memory game cards,whereas‘matchingpair’hasbeenaddedtothechart
differed quite much from theirs. In their user tests blind to show the total number of pair turns.
children, aged from 9 to 12, did not manage well with the With five of the children the separation can be made
game, whereas in our tests children managed better than based on their preferences on choosing the cards when
expected. Eriksson and Ga¨rdenfors made an assumption investigating the game area. The users 3, 4 and 6 tended
tochoosethecardsthatweredistantinthematrixwhereas
the users 5 and 7 chose to more often turn cards that were
Table3
closetoeachother.Gameplayoftheusers1and2differed
Thepairturnstheusersmadeduringthegame
fromtheotherfivedistinctively.Theuser1obviouslycould
Subject 1 2 3 4 5 6 7
not locate the vibrations and for the most part the game
Gender M F M F F M F
was played together withthe assistant. The user2 navigat-
Vision B* P P P B P B
edthecardssequentiallyfromlefttorightrowbyrow.Part
Distantpairs 11 3 7 8 3 4 2
Closepairs 7 2 2 1 4 2 7 of the cards in pairs were situated distantly because the
Matchingpairs 6 6 6 6 6 6 6 other card was the last in a row and the other one the first
Pairturnsintotal 24 11 15 15 13 12 15 in a row.
Abbreviations in the table: M, male; F, female; B, blind, P, partially Itcanbenotedthatthetotallyblindusers5and7tend-
sightedand*,otherhandicaps. ed to slightly more often choose to turn cards that were
204 R.Raisamoetal./InteractingwithComputers19(2007)196–205
close toeachotherinthematrix. The children withpartial was found last by three of seven children. Two-rhythm
vision were more able to choose cards that were distant in and gentle vibrations were the easiest to remember,
the matrix. This can be explained with visual memory that whereasthesimilarcharacteristicsofdecreasing,increasing
isastrongeraidinpositioningthanaudiocues(Goldstein, and intense vibrations made them feel alike.
1999).
6. Conclusions
5.5. Memorizing vibrations
In this paper, we have presented design and evaluation
Beforethetestsweassumedthattheremightbeadiffer- ofatactilememorygamethatisplayedwithatactilegame-
ence in memorability of various vibration effects. In the padbyvisuallyimpaired children.Thegamepad provedto
game there were six different vibration effects: decreasing, be a usable device for 12–13-year-old children. The device
increasing, two-rhythm, four-rhythm, gentle and intense had limited possibilities to produce the effects, but it is an
(seeTable1).Thememorabilitywasmeasuredintwoways: example of devices that are widely available and can be
theorderthevibrationeffectswerecorrectlymatchedinthe boughtbyaveragecomputerusers.Theeffectsforthegame
game, and the number of times a vibration effect was tried were created by using more complex parameters like
out (Fig. 5). rhythm. The duration of the effects had to be short, yet
On average, gentle vibration was tried out the smallest the effects had to be distinguishable. To make the effects
number of times and intense vibration the largest number easily memorized they were given names that described
of times. During the game each child tried out a gentle the feel of them.
vibration card from two to six times, while an intense Although one might consider the quality of tactile feed-
vibration was tried out from 3 to 10 times. The reason back of the gamepad to be inadequate it was found to be
for the gentle vibration demanding less tries than the sufficient for the games like the one studied. Thus, tactile
intensevibrationmaybethatthedecreasingandincreasing low-cost devices have potential in game design for visually
vibrations had common qualities with the intense vibra- impaired children. Since these children differ from each
tion. All these three effects had the large gamepad motor other in many aspects, the equipment setup, particularly
in use and they had the same magnitude value (10,000). the position of the display, should be easily adjustable.
In this sense the gentle vibration making use of the small The tangible model of the game was found to be highly
motorwasadistinctiveeffectinthisgroup.Gentle,intense, importantintraining,sinceitclearlyaidedintheconceptu-
decreasingandincreasingvibrationshadanequalduration alization of the game space. The positioning in the game
distinguishing them from two rhythmic effects. Both was enhanced with navigational audio cues that were
decreasingandintensevibrationsweretriedoutfromthree remembered well by the children. In general, the children
to ten times. gotexcellentscoresinthegameandsucceededingameplay
Typicallythefirstpairfoundwasthetwo-rhythmvibra- better than expected.
tion.Asmanyasfourchildrenoutofsevenfoundthetwo- Between blind and partially sighted children there did
rhythm vibration pair first. The decreasing vibration pair not seem to be any notable difference in the results. Still,
there was a difference in investigation strategies of the
play area between blind and partially sighted children.
The totally blind children tended to turn cards that were
close to each other more often than the partially sighted
children, who chose the cards that were scattered in the
area. Based on the results gained we have adjusted the
game to be suitable for younger, under 7-year-old visual-
ly impaired children. Our further research will also
continue with designing, implementing and testing of
various other multimodal games, including educational
games.
Acknowledgements
This work was funded by the National Technology
Agency of Finland (Tekes), Grant 40263/04, by the
Academy of Finland (Grant 105555), and by Nordic
Development Centre for Rehabilitation Technology
(NUH). We thank the children and their parents for par-
ticipating in the study and the personnel of the Jyva¨skyla¨
School for the Visually Impaired for helping to organize
Fig.5. Averagememorabilityofdifferentvibrationsinthegame. testing.
R.Raisamoetal./InteractingwithComputers19(2007)196–205 205
References for Universal Access in the Information Society: Eighth ERCIM
Workshop on User Interfaces for All. Lecture Notes in Computer
Science,vol.3196,pp.452–459.
Baptiste-Jessel,N.,Tornil,B.,Encelle,B.,2004.UsingSVGandaforce
Nielsen,J.,1993.UsabilityEngineering.AcademicPress.
feedback mouse to enable blind people to access ‘‘Graphical’’ Web
Patoma¨ki, S., Raisamo, R., Salo, J., Pasto, V., Hippula, A. 2004.
baseddocuments.In:ProceedingsofNinthInternationalConference
Experiencesonhapticinterfacesforvisuallyimpairedyoungchildren.
on Computers Helping People with Special Needs (ICCHP’04).
In: Proceedings of Sixth International Conference on Multimodal
LectureNotesinComputerScience,vol.3118,pp.228–235.
Interfaces(ICMI’04).ACMPress,pp.281–288.
Barnens, 2006. The Swedish Library of Talking Books and Braille.
Raisamo,R.,Patoma¨ki,S.,Raisamo,J.,Hasu,M.,Pasto,V.,Kosonen,
<http://www.tpb.se/barnens_tpb/spel/ljud/>.
K., Pakkanen, T., 2005. Determining the usefulness of affordable
Brown,L.M.,Brewster,S.A.,Purchase,H.C.,2005.Afirstinvestigation
haptic feedback devices: from games to applications helping the
into the effectiveness of tactons. In: WorldHAPTICS 2005, IEEE
visually impaired. In: Proceedings of HCI International 2005. Law-
Press,pp.167–176.
renceErlbaumAssociates.
Chan,A.,MacLean,K.E.,McGrenere,J.,2005.Learningandidentifying
Ramloll, R., Yu, W., Brewster, S.A., Riedel, B., Burton, A.M.,
hapticiconsunderworkload.In:WorldHAPTICS2005,IEEEPress,
Dimigen, G. 2000. Constructing sonified haptic line graphs for the
pp.432–439.
blind student: first steps. In: Proceedings of ASSETS 2000. ACM
Eriksson,Y.,Ga¨rdenfors,D.,2006.Computergamesforpartiallysighted
Press, pp. 17–25.
and blind children. Project report. <http://www.tpb.se/barnens_tpb/
Roth, P., Petrucci, L., Pun, T., 2000. From dots to shapes:an auditory
spel/projekt/report.html>.
haptic game platform for teaching geometry to blind pupils. In:
Feintuch, U., Rand, D., Kizony, R., Weiss, P.L., 2004. Promoting
ProceedingsofInternationalconferenceoncomputershelpingpeople
Research and Clinical Use of Haptic Feedback in Virtual Environ-
withspecialneeds(ICCHP2000),pp.603–610.
ments.In:ProceedingsofFifthInternationalConferenceonDisability,
Roth,P.,Kamel,H.,Petrucci,L.,Pun,T.,2002.Acomparisonofthree
VirtualRealityandAssociatedTechnologies(ICDVRAT’04).Univer-
nonvisualmethodsforpresentingscientificgraphs.JournalofVisual
sityofReading,UK,pp.141–147.
ImpairmentandBlindness96(6),420–428.
Flexi Forum 2006. Meeting place for Flexiboard users. <http://
Saarinen, R., Ja¨rvi, J., Raisamo, R., Tuominen, E., Kangassalo, M.,
www.handitek.se/flexi/english.htm>.
Peltola,K.,Salo,J.,2006.Supportingvisuallyimpairedchildrenwith
Goldstein, E.B., 1999. Sensation and Perception, fifth ed. Brooks/Cole
softwareagentsinamultimodallearningenvironment.VirtualReality
PublishingCompany,PacificGrove,CA.
9(2–3),108–117,Springer,2006.
Hammarlund, J., 1999. Computer Play for Children Who Are Severely
Sa´nchez,J.,Baloian,N.,Hassler,T.,Hoppe,U.,2003.AudioBattleship:
VisuallyImpaired.TRCreportnr.20,1999.<http://www.handitek.se/
blindlearnerscollaborationthroughsound.In:ConferenceonHuman
flexi/text/eng/trcnr20.htm>.
Factors in Computing Systems (CHI’03), Extended abstracts. ACM
Holmes, E., Jansson, G., 1997. A touch tablet enhanced with synthetic
Press,pp.798–799.
speech as a display for visually impaired people’s reading of virtual
SensAble, 2006. SensAble Technologies, Inc. PHANTOM Devices.
maps. In: Proceedings of CSUN 12th Annual Conference on
<http://www.sensable.com/products/phantom_ghost/phantom.asp>.
Technology for People with Disabilities. California State University,
Sjo¨stro¨m, C., 2002. Non-visual haptic interaction design: guidelines and
Northridge.
applications. Doctoral dissertation, Certec, Lund Institute of
IGDAAccessibilityGroup,2004.Accessibilityingames:motivationsand
Technology.
approaches.WhitePaper,InternationalGameDevelopersAssociation.
TIM(2001),TactileInteractiveMultimediaComputerGamesforVisually
<http://www.igda.org/accessibility/IGDA_Accessibility_WhitePaper.
Impaired Children, project IST-2000-25298. Deliverable 12: Game
pdf>.
functionreport.<http://filarkiv.sitrec.kth.se/timd12.pdf>.
Immersion,2006.ImmersionStudioforGaming:SoftwareDevelopment
Tiresias.org(2006).Guidelines fortheDesignofAccessible Information
Kit.<http://www.immersion.com/developer/>.
and Communication Technology Systems. <http://www.tiresias.org/
Inclusive Technology, 2006. IntelliKeys Transparent Overlays. <http://
guidelines/index.htm>.
www.inclusive.co.uk/catalogue/acatalog/transparent_overlays.html>.
TouchGraphics,2006.Touchgraphics,inc.Talkingtactiletablet.<http://
Jyva¨skyla¨, 2006. Jyva¨skyla¨ School for the visually impaired. <http://
www.touchgraphics.com/ttt.htm>.
www.jynok.fi/mainpage.htm>.
vandenDoel,K.(2003)SoundView:sensingcolorimagesbykinesthetic
Landau,S.,Russell,M.,Gourgey,K.,Erin,J.,Cowan,J.,2003.Useofthe
audio. In: Proceedings of the International Conference on Auditory
talking tactile tablet in mathematics testing. Journal of Visual
Display (ICAD 2003), Boston University, USA. <http://www.icad.
ImpairmentandBlindness97(2),85–96.
org/websiteV2.0/Conferences/ICAD2003/paper/74%20Doel.pdf>.
Lumbreras,M.,Sa´nchez,J.,1999.Interactive3Dsoundhyperstoriesfor
van Erp, J.B.F., Spape´, M.M.A., 2003. Distilling the underlying dimen-
blind children. In: Proceedings of Conference on Human Factors in
sionsoftactilemelodies.In:ProceedingsofEurohaptics2003,Dublin,
ComputingSystems(CHI’99).ACMPress,pp.318–325.
Ireland,pp.111–120.
Meijer, P.B.L., 2006. The vOICe Java applet. <http://www.seeingwith-
VisionConnection (2006). Accessibility and assistive technology, a com-
sound.com/javoice.htm>.
pendiumofavailableassistivetechnologyproducts.VisionConnection,
Nemec,V.,Sporka,A.J.,Slavik,P.,2004.Hapticandspatialaudiobased
Lighthouse International, Inc. <http://www.visionconnection.org/
navigationofvisuallyimpairedusersinvirtualenvironmentusinglow
Content/Technology/AssistiveTechnologies/default.htm>.
costdevices.In:ProceedingsofUser-CenteredInteractionParadigms
