Structural Motion Grammar for Universal Use of Leap Motion: Amusement and Functional Contents ...

16 Pages·2017·8.52 MB·English

Hindawi
Journal of Sensors
Volume 2018, Article ID 6073786, 15 pages
https://doi.org/10.1155/2018/6073786
Research Article
Structural Motion Grammar for Universal Use of Leap Motion:
Amusement and Functional Contents Focused
Byungseok Lee, Donghwe Lee, and Seongah Chin
DivisionofMediaSoftware,SungkyulUniversity,AnyangCity,RepublicofKorea
CorrespondenceshouldbeaddressedtoSeongahChin;[email protected]
Received 19 May 2017; Revised 17 November 2017; Accepted 27 November 2017; Published 9 January 2018
AcademicEditor:StefanoStassi
Copyright©2018ByungseokLeeetal.ThisisanopenaccessarticledistributedundertheCreativeCommonsAttributionLicense,
whichpermitsunrestricteduse,distribution,andreproductioninanymedium,providedtheoriginalworkisproperlycited.
MotionsusingLeapMotioncontrollerarenotstandardizedwhiletheuseofitisspreadinginmediacontents.Eachcontentdefines
its own motions, thereby creating confusion for users. Therefore, to alleviate user inconvenience, this study categorized the
commonly used motion by Amusement and Functional Contents and defined the Structural Motion Grammar that can be
universally used based on the classification. To this end, the Motion Lexicon was defined, which is a fundamental motion
vocabulary, and an algorithm that enables real-time recognition of Structural Motion Grammar was developed. Moreover, the
proposedmethodwasverifiedbyuserevaluationandquantitativecomparisontests.
1. Introduction [4],musicalinstruments[5],mixedreality[6],andrehabili-
tationandmedicalapplications[7].
Interfacetechnology,whichsupportstheinteractionbetween Inparticular,LeapMotiongesturerecognitioninAmuse-
contentandusers,iscontinuouslybeingdeveloped.Recently, ment (game) Contents plays a crucial role in keeping the
the technology is transforming into a natural user interface playerengrossedinthegame.Italsoincreasestheimmersive
(NUI) method that provides users with a bigger sense of senseoftheAmusementContentbecauseLeapMotionuses
reality compared with the conventional method, which the player’s gestures without any controllers in real time as
focusses on the use of mouse and keyboard. NUI is an theplayerinteractswiththecontent.Gamesthatusegesture
up-to-date means of interacting with computers that has recognitioncancapturetheplayer’sattentioneasilythrough
graduallydrawnmoreinterestsinhuman-computerinterac- theprogressofthegame[8].
tion(HCI).NUIcomprisesvoiceinterface,sensoryinterface, Research on the recognition of Leap Motion has been
touchinterface,andgestureinterface.LeapMotionisafinger carried out in technical studies. Some studies on the use of
gesture interface-supported device [1, 2]. The infrared ray SVM were reported in [9], and studies using HMM were
cameras attached to the Leap Motion controller capture investigatedin[10–12].However,thesestudiesusemachine
and analyse the hand gesture, and the content recognizes learning, which requires feature extraction, normalization,
the motion. The Leap Motion controller introduces a new andtime-consumingtrainingprocedures.
novel gesture and position tracking system with submilli- Asdescribedabove,werealisetheuseofLeapMotionin
meter accuracy. Its motion sensing precision is unmatched contentsisexpandingandthetechnologyofrecognitionhas
by any depth camera currently available. It can track all 10 a cumbersome preprocessing task. Although many studies
of the human fingers simultaneously. As stated by the investigated movement recognition through Leap Motion
manufacturer, the accuracy in the detection of each finger- and content application, authors have not found any litera-
tip position is approximately 0.01mm, with a frame rate ture reported on standardized motion grammar. This study
of up to 300fps. actuallyisdesignedtotargetleapmotiongesturesthathave
For these benefits, the Leap Motion controller is widely beenusedingamessincegameusersareinconveniencedby
usedinvariousapplicationssuchasgames[3],signlanguages having to learn different motions for content because they
2 JournalofSensors
allhavetheirownmotions.Apreliminaryconferencepaper in interactions for architecture students in mixed reality
isshownin[13]. environments. The menu interface design supported the
Tothisend,thisstudydefinedtheMotionLexicon(ML) real-timedesignoflargeinteriorarchitecturalspacesexperi-
that can be universally used in Amusement and Functional enced in mixed reality environments. Iosa et al. [7] con-
Contents and designed the Structural Motion Grammar ducted a study to test the feasibility, the compliance, and
(SMG) composed ofthecombinationofML.Then,thetree thepotentialefficacyofusingLeapMotioncontroller-based
of SMG was recognized in real time thorough coupling a system for progressing the recovery of elderly patients with
motion API without using complex procedures such as ischemicstroke.
feature extraction and training process like a machine
learning algorithm. Then, the defined motions were then
3. Methods
tested for verification.
To accomplish theproposedmethod,thecommon motions
2. Related Work used in Amusement and Functional Contents were first
classified. Based on the classifications, the ML that can be
Researchers have studied the accuracy and robustness of used universally was defined. Then, SMG was defined
Leap Motion [14, 15]. Weichert et al. [14] analysed the
through the combination of ML. Also, the recognition step
accuracy and robustness of Leap Motion and applied the
isprovided.
research on industrial robots. Guna et al. [15] conducted Figure 1shows theoverall flowoftheproposed method
research on the accuracy and reliability of the static and
as an example. Leap Motion, which is a form of NUI,
dynamic movements of Leap Motion.
enables the free movement of hands and its recognition.
TheLeapMotion’smovementrecognitionhasalsobeen To define ML, the contents were divided into Amusement
investigated[16–21].Marinetal.[16,17]conductedresearch
and Functional Contents. The representative motions were
on the multiclass classifier by coupling Leap Motion with a thenselected.Then,wedefinedtheStructuralMotionGram-
Kinect and depth camera, while Vikram et al. [18] studied
mar (SMG) composed of the combination of ML. Every
the recognition of handwritten characters. Lu et al. [19]
motion can be represented in the SMG that is visualized in
proposed the Hidden Conditional Neural Field (HCNF) a tree structure. Prior to defining the selected motions, the
classifiertorecognizethemovinggestures.Boyalietal.[20] features of Leap Motion API were analysed to define the
researched the robotic wheelchair control, which applied
ML. For instance, Leap Motion API translated the orders
blocksparse,sparserepresentative,andclassification.Seixas
byspecifyingthemfromthetoptothebottomclassesbased
et al. [21] compared the screen tab and selected gesture of on the top-down method. When the first condition of
bothhands.
identifying static or dynamic movement was applied, the
The use of Leap Motion on sign language is also being identification of the motion whether it is static or dynamic
investigated [22–24]. Chuan et al. [22] investigated the
will be possible. When the second condition of hand API
recognition of English sign language using the 3D motion was applied, the information on hands can be classified.
sensor, while Khan et al. [23] researched the prototype that When the last condition of finger API was applied, the
can convert sign language to text. Mohandes et al. [24] informationonfingerscanbeclassified.Basedonthisinfor-
investigatedtheArabicsignlanguagerecognition. mation,thedifferentiatedmotionscanbedefinedandcanbe
ResearchersalsoinvestigatedcontentusingLeapMotion
laid out in diverse forms of SMG. More comprehensive
[25–27]. The research evaluated 3D pointing tasks using gestureswillbedefinedinthefollowingsections.
Leap Motion sensor to support 3D object manipulation
[25]through some controlledexperiments including expos-
3.1. Content Classification. To define the universal motions
ing test subjects to pointing task and object deformation,
thatuseLeapMotion,therepresentativemotionsareneeded
measuring the time taken to perform mesh extrusion and
to be extracted by each content classification. The digital
object translation. Sutton [26] presented the air painting
contents where leap motion is applicable can be classified
method using Leap Motion that could be an input data to
intoAmusementContentandFunctionalContentbasedon
the Corel Painter Freestyle application. The painting pro-
their purposes. Both types of contents have subgenres, and
cess through gestures was implemented. A study about
commonlyusedmotionswereextractedthroughtheclassifi-
sound synthesis and interactive live performance using
cationandanalysisofthegenres.
Leap Motion was also reported [27]. The study imple-
mented a 5-grain granular synthesizer making users trigger
individual grains. 3.1.1. Amusement Content. Amusement Content is also
And there were also studies about various contents and known as the game content. This content can be classified
techniquesusingLeapMotion,aforementionedinIntroduc- into the following subgenres based on the motions:
tion[3,6,7].Leeetal.[3]studiedagamemodelusingLeap Action, FPS (First Person Shooing), Simulation-Racing/
Motionthatcombinedgesture-dependentimpactlevelswith Flight, Arcade, Sports, and Role-playing. Of the six genres,
the physical characteristics of players. A game was realised Sports and Role-playing were excluded because they did
in which the gesture was recognized to be associated with not fit in the current study. Sports games were not fit for
a player’s gesture with a gesture-specific weight. Davis Leap Motion usage because multiple players need to be
et al. [6] proposed the work to establish new modalities controlled simultaneously.
JournalofSensors 3
Content classification Motion recognition
Amusement content
Static/dynamic
Dynamic, static distinction
Functional content
Hand recognition Hand
Finger
Structural motion
grammar
Lefthand, righthand
ML: Motion lexicon distinction
Action performed
AML: Adverb ML
CML: Compound ML
ACML: Adverb CML
ML AML CML ACML
Finger distinction
Go Stop Go Left Go Shoot Go Left Shoot
direction direction
Figure1:Schematicoftheproposedmethod.
For Role-playing games, which have a high level of Content and Functional Content using the hand and finger
freedom, defining the motion has limitations because its API. To define ML, the hand and finger API reflecting the
interface and the number of possibilities are very complex features of the genres have been analysed. Tables 3 and 4
and diverse. show the defined ML by contents. More specifically,
To this end, the four genres, namely, FPS, Action, Table3definesthemotionsforbothleftandrighthandsto
Simulation-Racing/Flight and Arcade, were analysed and be used for FPS, Action, Simulation-Racing/Flight, Arcade,
common motions were extracted. Table 1 shows the repre- Sports, and Role-playing games. For Action, Simulation-
sentative motions by game genres. Within the FPS genre of Racing/Flight and Arcade games, the left hand was defined
AmusementContent,the“SuddenAttack(NexonCo.)”isa for movement and the right hand was defined for action
representative game, and its motions are: “Move,” “Jump,” because both motions occur simultaneously. Table 3 shows
“Run,” “Sit,” “Shot,” “Reload,” and “Weapon Change.” In the details of ML, image, and motion principle. “Go” for
Table 1, three games (Sudden Attack, King of Fighters, the left hand was denoted by having all fingers straight,
and Cookie Run) are just representative examples of games whereas “Stop” was denoted by having all fingers folded.
from lots of games that we looked into to define the When defining ML, “Jump” or “Sit” motions were linked
common motions. to the up and down directions. “Shot,” “Reload,” and
The motions can be comprehensively categorized into “Weapon Change” motions for the right hand were also
movement and action. In this study, ML is defined based defined by linking with the actual motion.
on the framework that the left hand is the movement while For Functional Content, “Zoom In,” “Rotation,” “Play,”
therighthandistheaction. “Pause,” and “Rewind” were representative motions. Given
a very wide range of motions, not all of them can be
3.1.2.FunctionalContent.FunctionalContentwasclassified defined. Therefore,motions thatwere commonly used have
into Experience and Creation Content and Teaching and been defined.
Learning Content. With the recent expansion of the virtual Table 4 shows the Experience and Creation Contents
reality market, numerous Experience contents or disaster of ML and explains their ML, image and motion princi-
reactiontrainingcontentsuseNUI.Arepresentativeexample ple. Here, ML comprises “Zoom In,” “Zoom Out,” and
oflecturecontentise-Learning,whichisaformofTeaching “Rotation.” For “Zoom In” and “Zoom Out,” the Vector3
andLearningContentthatprovideslecturevideosonlineto coordinate was applied to both hands, and the movement
overcome the drawbacks of offline education, such as being of the x-axis was recognized as the distance. “Zoom Out”
closed and collective. Table 2 shows the representative was defined as having both hands together, and “Zoom
motionsusedbyeachFunctionalContent.TheVRMuseum In” as having both hands apart. For “Rotation,” the hor-
of Fine Art (Steam VR Co.) is a representative example of izontal and vertical condition was applied to the hand to
Experience and Creation Content, and its motions are identify whether the hand was horizontally positioned.
“ZoomIn,”“ZoomOut,”“UsingTool,”and“Rotation.” The rotating counterclockwise motion was defined when
the left hand moved towards the right side of the x-axis,
3.2. Motion Lexicon. Motion Lexicon (ML) consists of the and the clockwise rotation motion was defined when the
motions that have been analysed within the Amusement right hand moved to the left side of the x-axis.
4 JournalofSensors
Table1:MotioncategorybyAmusementContentgenres.
Gamegenre Motion Gameexample
FPS Move,Jump,Run,Sit,Shot,Reload,WeaponChange
SuddenAttack(NexonCo.)
Action Move,Jump,Run,Attack(Skill)
KingofFighters(SNK)
Arcade Move,Jump,Function
CookieRun(Devsisters)
Table2:MotioncategorybyFunctionalContentgenres.
FunctionalContent Motion Example
ZoomIn,ZoomOut,UsingTool(Drawing,Attaching,
ExperienceandCreation
Cutting,andsoforth),Rotation,andsoforth
VRMuseumofFineArt(SteamVRCo.)
TeachingandLearning Play,FastPlay,Stop,Rewind
ICTinEducation(KRCSGroup)
JournalofSensors 5
Table3:AmusementContentMotionLexicon.
ML Image Motionprinciple
(i)Dynamicandstaticclassificationcondition
Go(G) (ii)Leftandrighthandclassificationcondition
(iii)Numberoffingercondition
(i)Dynamicandstaticclassificationcondition
Stop(ST) (ii)Leftandrighthandclassificationcondition
(iii)Numberoffingercondition
(i)Dynamicandstaticclassificationcondition
LeftDirection(LD) (ii)Leftandrighthandclassificationcondition
(iii)Degreecondition
LeftHand
(i)Dynamicandstaticclassificationcondition
RightDirection(RD) (ii)Leftandrighthandclassificationcondition
(iii)Degreecondition
(i)Dynamicandstaticclassificationcondition
Jump(J) (ii)Leftandrighthandclassificationcondition
(iii)Vector3coordinatecondition
(i)Dynamicandstaticclassificationcondition
Sit(S) (ii)Leftandrighthandclassificationcondition
(iii)Vector3coordinatecondition
(i)Dynamicandstaticclassificationcondition
Roll(R) (ii)Leftandrighthandclassificationcondition
(iii)Vector3coordinatecondition
6 JournalofSensors
Table3:Continued.
ML Image Motionprinciple
(i)Dynamicandstaticclassificationcondition
(ii)Leftandrighthandclassificationcondition
Shot(sh) (iii)Numberoffingercondition
(iv)Degreecondition
(i)Dynamicandstaticclassificationcondition
Reload(r) (ii)Leftandrighthandclassificationcondition
(iii)Numberoffingercondition
(i)Dynamicandstaticclassificationcondition
WeaponChange(ch) (ii)Leftandrighthandclassificationcondition
(iii)Horizontalandverticalcondition
RightHand
(i)Dynamicandstaticclassificationcondition
Kick(k) (ii)Leftandrighthandclassificationcondition
(iii)Vector3coordinatecondition
(i)Dynamicandstaticclassificationcondition
Punch(p) (ii)Leftandrighthandclassificationcondition
(iii)Vector3coordinatecondition
(i)Dynamicandstaticclassificationcondition
Function1(F1) (ii)Leftandrighthandclassificationcondition
(iii)Numberoffingercondition
(i)Dynamicandstaticclassificationcondition
Function2(F2) (ii)Leftandrighthandclassificationcondition
(iii)Numberoffingercondition
JournalofSensors 7
Table3:Continued.
ML Image Motionprinciple
(i)Dynamicandstaticclassificationcondition
(ii)Leftandrighthandclassificationcondition
Drift(D)
(iii)Horizontalandverticalcondition
(iv)Numberoffingercondition
(i)Dynamicandstaticclassificationcondition
Booster(B) (ii)Leftandrighthandclassificationcondition
(iii)Numberoffingercondition
Table4showstheMLdefinedinrelationtotheTeaching ACML is a combination of ML and ML and Adverb
andLearningContentmotions.Here,MLcomprises“Play,” vocabularies and was used when three motions were exe-
“Fast Play,” “Rewind,” and “Pause.” The horizontal and cuted. Forinstance, thelefthandresponsibleformovement
vertical conditions and Vector3 coordinate condition were recognizes the ML of “Go” and also recognizes the Adverb
applied to both hands, and “Play” was defined when the of “Left Direction” simultaneously. The right hand can
distance of the two hands on the x-axis was 0. Specifically, express“Shot”withtheintegrationofML.Ontheschematic
thisisthesamemotionasclapping. tree, theSMG leads toACML,which then leads to theML/
For “Fast Play” and “Rewind,” the movement was the ML/Adverb. The process of “Left Direction+Go+Shot”
same, but with different hands and directions. For “Fast was identified with arrows on the schematic tree. In this
Play,”thelefthandwasmovedtotherightsideofthex-axis; study,thevocabularycombinationsbasedontheaforemen-
whilefor“Rewind,”therighthandwasmovedtotheleftside tioned schematic tree have been used to define the SMG.
ofthex-axis.For“Pause,”thenumberoffingerconditionwas The red dotted arrows indicate the recognition procedures
appliedtoidentifywhethertherearetwoleftfingers. that satisfy SMG. For example in Figure 2, Go and Shot
literally means that a game player wants to make a tank
3.3. Structural Motion Grammar. Structural Motion Gram- go forward and shot at the same time. Thus, SMG can
mar is a combination and grammaticalization of the be classified into CML broken down into ML (Go) and
aforementionedMLthathasbeendefined.ItconsistsofML ML (Shot).
(Motion Lexicon), AML (Adverb and ML), CML (Com- Aformal representationof SMG is theform ofcontext-
pound ML), and ACML (Adverb and Compound ML). freegrammar(CFG)sinceSMGcanbebrokendownintoa
Figure2isaschematictreeoftheclassificationandcoupling setofproductionrules.SMGillustratesallpossiblemotions
oftheMLthathasbeendefined. ingivenformalmotions.WealsodefineSMGasatheoretical
MLcanbeSMGbyitself,suchasthe“Rotation”motion formasbelow.
of the Experience and Creation Content. SMG is connected SMG:=AML∥CML∥ACML∥ML,
toML.Theprocessof“Rotation”motionhasbeenidentified AML:=ML+Adverb,
witharrowswithintheschematictree. CML:=ML+ML,
AML is a combination of ML and Adverb and Adverb ACML:=ML+ML+Adverb,
wasusedasapartofspeechthatsupportsML.Forinstance, ML:=G∥ST∥S∥LD∥RD∥J∥S∥R∥sh∥r∥ch∥k∥p∥
forthelefthandmotionthatwasresponsibleformovement, F1∥F2∥D∥B∥ZI∥ZO∥RO∥p∥fp∥rw∥PA,
the ML of “Go” was recognized and, at the same time, the Adverb:=LD∥RD.
SMG of the “Right Direction+Go” was expressed with the
coupling of the Adverb of “Right Direction.” Within the 3.4.MotionRecognition.GiventhatSMGhasacombination
schematic tree, theSMG leads toAML,which thenleads to ofMLthatrepresentsamotioneitherusingonehandortwo
the ML/Adverb. The process of “Right Direction+Go” hands,theSMGisdecomposedintofourchildrenML,AML,
motionhasbeenidentifiedinarrowsontheschematictree. CML,orACML;then,therecognitionstepsofMLarecarried
CMLwasusedwhentwotypesofmotionswereexecuted out.Recognitionreferstotheconditionsthatcanexplainthe
using ML and ML. For example, the left hand that was recognizable APIon theLeap Motion device anddefine the
responsible for movement recognizes the ML of “Go,” and motions. Leap Motion, which is a form of NUI, provides
at the same time, the right hand can express the “Shot” various APIs [2]. Among the numerous APIs, most of the
motion with the integration of ML. On the schematic tree, contents in the market use the hand and finger API. These
SMGleadstoCML,whichthenleadstoML/ML.Theprocess contentsreceivetheirdatafromtheupper-mostframe,where
of “Go+Shot” motion has been identified with arrows on thehandisrecognizedtotrackandcollectinformation.The
the schematic tree. hand API that has received the data can recognize the
8 JournalofSensors
Table4:FunctionalContentMotionLexicon.
FunctionalContent ML Image Motionprinciple
ZoomIn(ZI)
(i)Dynamicandstaticclassificationcondition
(ii)Leftandrighthandclassificationcondition
(iii)Vector3coordinatecondition
ZoomOut(ZO)
ExperienceandCreation
(i)Dynamicandstaticclassificationcondition
(ii)Leftandrighthandclassificationcondition
Rotation(RO)
(iii)Horizontalandverticalcondition
(iv)Vector3coordinatecondition
(i)Dynamicandstaticclassificationcondition
(ii)Leftandrighthandclassificationcondition
Play(p)
(iii)Horizontalandverticalcondition
(iv)Vector3coordinatecondition
(i)Dynamicandstaticclassificationcondition
TeachingandLearning FastPlay(fp) (ii)Leftandrighthandclassificationcondition
(iii)Vector3coordinatecondition
(i)Dynamicandstaticclassificationcondition
Rewind(rw) (ii)Leftandrighthandclassificationcondition
(iii)Vector3coordinatecondition
JournalofSensors 9
Table4:Continued.
FunctionalContent ML Image Motionprinciple
(i)Dynamicandstaticclassificationcondition
Pause(PA) (ii)Leftandrighthandclassificationcondition
(iii)Numberoffingercondition
SMG
(Structural motion grammar)
AML CML
(Adverb & ML) (Compound ML)
Right direction+Go Go+Shot
(RD+G) (G+sh)
ML ML ML
(Motion lexicon) Adverb (Motion lexicon) (Motion lexicon)
Go (G) Stop (ST) Sit (S) Left Right Go (G) Stop (ST) Sit (S) Shot (sh) Weapon
direction (LD) direction (LD) change (ch)
ML ACML
(Motion lexicon) (Adverb compound ML)
Left direction+Go+Shot
(LD+G+sh)
Rotation
(RO) (MotioMn lLexicon) (MotioMn lLexicon) Adverb
Rotation Zoom
(RO) out (ZO)
Fast play (fp) Go (G) Stop (ST) Sit (S) Shot (sh) Weapon Left Right
change (ch) direction (LD) direction (LD)
Figure2:Structureofthemotiongrammartree.
existence of the left or right hand and distinguish the left wastoapplythedynamicandstaticclassificationconditions
from the right hand. In addition, the API can identify the to the shooting motion. Then, using the data on hand API,
speed, location, and degree of the hand. Finger API can theclassificationconditionsfortheleftandrighthandswere
distinguish each finger andidentify thespeed, location,and applied.Finally,usingthedataonfingerAPI,theconditions
degree of the fingers. While the data on speed and location on the number of fingers on the right hand, as well as the
were continuously updated, the former data was compared degrees of the fingers, were applied. When two fingers of
with the current data by tracking the hands and fingers. the right hand were used, the API identifies whether the
These comparison results can help distinguish whether ML fingers are thumb and forefinger and distinguishes the x-z
isdynamicorstatic. axis degree of the thumb. The shooting motion was recog-
The algorithms SMG (mr_SMG), ML or ML_Adverb nized only when all of the aforementioned conditions were
(mr_ML and mr_ML_Adverb), Hand Count (HC), Hand set. Given that the shooting motion has been defined only
Feature (HF), Finger Count (FC), and Finger Feature (FF) for the right hand, the direction and movement motions
aredefinedasshowninFigure3.Supposingthattheshooting will be defined to the left hand, enabling the use of both
motionhasbeendefinedwithintheFPScontent,thefirststep hands for manipulation.
10 JournalofSensors
Algorithm : motion recognition of
Algorithm : motion recognition of SMG: mr_SMG ML or ML_Adverb
: mr_ML or mr_ML_Adverb
(i) Input : Motion Lexicon (ML), Adverb ML (AML), (i) Input : Motion Lexicon (ML)
(ii) Compound ML (CML), Adverb Compound ML (ii) Output : ML Recognition
(iii) (ACML), Structural Motion Grammar (SMG) (iii)
(iv) Output : SMG Recognition
(iv) for ML do
(v)
(v) Call HC() ;
(vi) switch (SMG)
(vii) case CAML: Call mr_ML() ; (vi) Call HF() ;
(viii) case AAML: Call mr_ML() + mr_ML_Adverb() ; (vii) Call FC() ;
(ix) case ACML: Call mr_ML() + mr_ML() ; (viii) Call FF() ;
(x) case ACML: Call mr_ML_Adverb() + mr_ML() (ix) end for
(xi) + mr_ML() ;
(xii) end switch
Algorithm : Hand_Count: HC Algorithm : Hand_Feature: HF
(i) Input : Left Hand (LH), Right Hand (RH) (i) Input : NH, LH, RH, H
(ii) Output : the number of Hand (NH) (ii) Output : Hand Feature Vector
(iii) (iii)
(iv) if (1) then (iv) for NH do
(v) Using either LH or RH ; (v) Using the Direction of H ;
(vi) NH = 1 ; (vi) Using the Speed of H ;
(vii) else (vii) Using the Location of H ;
(viii) Using both LH and RH ; (viii) Using the Normal Vector of H ;
(ix) NH = 2 ; (ix) end for
(x) end if
Algorithm : Finger_Count: FC Algorithm : Finger_Feature: FF
(i) Input : Finger (F) (i) Input : NF, F
(ii) Output : the number of Finger (NF) (ii) Output : Finger Feature Vector
(iii) (iii)
(iv) for F do (iv) for NF do
(v) Increasing the number of F (NF) ; (v) Using the Direction of F ;
(vi) end for (vi) Using the Touch of F ;
(vii) Using the Speed of F ;
(viii) Using the Location of F ;
(ix) end for
Figure3:Motionrecognitionalgorithm.
4. Experiments for the hardware. The motion recognition module was
developed using C#.
4.1. SMG Recognition Rate Comparison Test. The following For the test method, the Amusement and Functional
experimental environment was set up to evaluate the Contentsmotionsdefinedinthisstudyandestablishedinto
SMG suggested in this study. The desktop used for simula- grammar (ours) were compared with the Leap Motion
tion was installed with Window 7 64bit OS, with Geforce SVM [28] method through a quantitative evaluation. Each
GTX 770 as the graphics card. For software, Unity 5.3.1f1 motion was tested 20 times as the number of input. The
version was installed, and Leap Motion was established inputs are composed of the features of each gesture that