Intelligent Robot: Implementation and Applications

293 Pages·2023·5.752 MB·English

by  Feng Duan, Wenyu Li, Ying Tan

Feng Duan
Wenyu Li
Ying Tan
Intelligent
Robot
Implementation and Applications
Intelligent Robot
(cid:129) (cid:129)
Feng Duan Wenyu Li Ying Tan
Intelligent Robot
Implementation and Applications
FengDuan WenyuLi
CollegeofArtificialIntelligence CollegeofArtificialIntelligence
NankaiUniversity NankaiUniversity
Tianjin,China Tianjin,China
YingTan
CollegeofArtificialIntelligence
NankaiUniversity
Tianjin,China
ISBN978-981-19-8252-1 ISBN978-981-19-8253-8 (eBook)
https://doi.org/10.1007/978-981-19-8253-8
JointlypublishedwithBeijingHuazhangGraphics&InformationCo.,Ltd,ChinaMachinePress
TheprinteditionisnotforsaleinChina(Mainland).CustomersfromChina(Mainland)pleaseorderthe
printbookfrom:BeijingHuazhangGraphics&InformationCo.,Ltd,ChinaMachinePress.
©TheEditor(s)(ifapplicable)andTheAuthor(s),underexclusivelicensetoSpringerNatureSingapore
PteLtd.2023
Thisworkissubjecttocopyright.AllrightsaresolelyandexclusivelylicensedbythePublisher,whether
thewholeorpartofthematerialisconcerned,specificallytherightsofreprinting,reuseofillustrations,
recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission
or information storage and retrieval, electronic adaptation, computer software, or by similar or
dissimilarmethodologynowknownorhereafterdeveloped.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Thepublishers,theauthors,andtheeditorsaresafetoassumethattheadviceandinformationinthis
book are believed to be true and accurate at the date of publication. Neither the publishers nor the
authors or the editors give a warranty, expressed or implied, with respect to the material contained
herein or for any errors or omissions that may have been made. The publishers remain neutral with
regardtojurisdictionalclaimsinpublishedmapsandinstitutionalaffiliations.
ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSingaporePteLtd.
The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721,
Singapore
Preface
Robotics has developed rapidly since the mid-twentieth century when research on
modern robots was conducted. Nowadays, robots are integrated into our work and
daily life. With the development of computer, Internet, and artificial intelligence
technologies,therearemoreandmoretypesofrobotswithenhancedfunctionsand
animprovedexperienceofusingthem.
Atthesametime,newadvancesinroboticsresearchanddevelopmentcontinueto
be made. In particular, the Robot Operating System (ROS) is of great strategic
importancetothedevelopmentoftheintelligentroboticsindustry.Robotoperating
systemcanprovideaunifiedplatformforrobotdevelopment,sothatmoreuserscan
conveniently study and verify robot algorithms and develop robot applications on
thisplatform,whichgreatlypromotesthedevelopmentofrobottechnology.In2010,
WillowGaragereleasedtheopen-sourcerobotoperatingsystemROS(RobotOper-
atingSystem).Ratherthanbeinganoperatingsystem,ROSisadistributed,modular
open-source software framework.Withtheadvantages ofpeer-to-peer design,pro-
gramming language independence, and open source, ROS has become a new hot
spotforlearninganduseinroboticsresearch.
TheLaboratoryofIntelligentPerceptionandHuman–RobotInteraction,College
ofArtificialIntelligence,NankaiUniversity,wheretheauthors’workisdedicatedto
theresearchofintelligentrobots.Thisbookisbasedonourlong-termexperiencein
developing robots using ROS, and we hope to provide a useful reference book for
university students who are interested in learning intelligent robotics and technical
staffwhoareengagedinintelligentrobotdevelopmentwork.
Thisbookisdividedintothreeparts:thefirstpartmainlyintroducesthebasicsof
robotics,includingthedefinitionofrobotics,developmenthistory,keytechnologies,
frameworkanduseofROS,etc.;thesecondpartintroduceshowtodeveloparobot
withrelativelycompletefunctionsintermsofrobothardwareandsoftwarecompo-
sition,visionfunctionimplementation,autonomousnavigationfunctionimplemen-
tation,voiceinteractionfunctionimplementation,graspingfunctionimplementation,
etc.;thethirdpartgivescomprehensivecaseswithdifferentapplicationscenariosto
showhowtodeveloprobotswithdifferentfunctions.
v
vi Preface
The cases in this book are based on the robot program that the author's team
participated in and won the RoboCup Robotics World Open. Beginners following
the explanations in this book and practicing with the hands-on resources that
accompany this book can not only master the advanced concepts of the software
framework involved in robot development but also develop a fully functional
intelligentrobotstepbystep.
Tianjin,China FengDuan
WenyuLi
YingTan
Contents
PartI BasicKnowledge
1 OverviewofRobots. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1 DefinitionandClassificationofRobots. . . . . . . . . . . . . . . . . . . 3
1.1.1 DefinitionofRobots. . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1.2 ClassificationofServiceRobots. . . . . . . . . . . . . . . . . 4
1.2 TheEvolutionofModernRobots. . . . . . . . . . . . . . . . . . . . . . . 5
1.2.1 EarlyStagesofModernRobotsResearch. . . . . . . . . . . 5
1.2.2 The1970s. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2.3 The1980s. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.2.4 The1990s. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.2.5 Twenty-FirstCentury. . . . . . . . . . . . . . . . . . . . . . . . . 15
1.3 CompositionoftheRobots. . . . . . . . . . . . . . . . . . . . . . . . . . . 23
1.3.1 ActuatorsfortheRobot. . . . . . . . . . . . . . . . . . . . . . . 24
1.3.2 DriveUnits. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
1.3.3 SensingDevices. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
1.3.4 ControlSystems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
1.3.5 IntelligentSystems. . . . . . . . . . . . . . . . . . . . . . . . . . . 25
1.3.6 IntelligentHuman-MachineInterfaceSystems. . . . . . . 26
1.4 KeyTechnologiesforRobots. . . . . . . . . . . . . . . . .. . . . . . . . . 26
1.4.1 AutonomousMobileTechnologies. . . . . . . . . . . . . . . 26
1.4.2 PerceptualTechnologies. . . . . . . . . . . . . . . . . . . . . . . 27
1.4.3 IntelligentDecision-MakingandControl
Technologies. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
1.4.4 CommunicationTechnologies. . . . . . . . . . . . . . . . . . . 27
1.5 TrendsinRobots. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
1.5.1 HierarchicalandHumanizedHuman-Computer
Interaction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
vii
viii Contents
1.5.2 InteractionIntelligencewiththeEnvironment. . . . . . . . 28
1.5.3 NetworkingofResourceUtilization. . . . . . . . . . . . . . . 28
1.5.4 Standardization,ModularizationandSystematization
ofDesignandProduction. . . . . . . . . . . . . . . . . . . . . . 29
FurtherReading. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2 GettingStartedwithROS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
2.1 IntroductiontoROS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.1.1 WhyUseROS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.1.2 WhatIsROS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.1.3 DifferencesBetweenROSandComputerOperating
Systems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.1.4 KeyFeaturesofROS. . . . . . . . . . . . . . . . . . . . . . . . . 33
2.2 InstallationandUninstallationofROS. . . . . . . . . . . . . . . . . . . 35
2.2.1 VersionsofROS. . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.2.2 InstallingandConfiguringROSIndigo. . . . . . . . . . . . 35
2.2.3 InstallingandConfiguringROSMelodic. . . . . . . . . . . 38
2.2.4 UninstallationofROS. . . . . . . . . . . . . . . . . . . . . . . . 40
2.3 ResourcesforFurtherLearning. . . . . . . . . . . . . . . . . . . . . . . . 40
FurtherReading. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3 TheFrameworkandFundamentalUseofROS. . . . . . . . . . . . . . . . 43
3.1 ROSFramework. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.1.1 FileSystemLevel. . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.1.2 CalculationofGraphLevels. . . . . . . . . . . . . . . . . . . . 44
3.1.3 CommunityLevel. . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.2 BasicsofROSUse. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
3.2.1 OverviewofCatkin. . . . . . . . . . . . . . . . . . . . . . . . . . 47
3.2.2 WorkspacesandTheirCreationMethods. . . . . . . . . . . 47
3.2.3 CreatingROSProjectPackages. . . . . . . . . . . . . . . . . . 50
3.2.4 CompilingtheROSProjectPackage. . . . . . . . . . . . . . 51
3.2.5 CreatingROSNodes. . . . . . . . . . . . . . . . . . . . . . . . . 51
3.2.6 CompilingandRunningROSNodes. . . . . . . . . . . . . . 52
3.2.7 UseofRoslaunch. . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
3.2.8 CreatingROSMessagesandServices. . . . . . . . . . . . . 55
3.2.9 WritingaSimpleMessagePublisherandSubscriber
(C++Implementation). . . . . . . . . . . . . . . . . . . . . . . . 59
3.2.10 WritingaSimpleMessagePublisherandSubscriber
(PythonImplementation). . . . . . . . . . . . . . . . . . . . . . 61
3.2.11 TestingMessagePublisherandSubscriber. . . . . . . . . . 63
3.2.12 WritingaSimpleServerandClient(C++
implementation). . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
3.2.13 WritingaSimpleServerandClient(Python
Implementation). . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
3.2.14 TestingaSimpleServerandClient. . . . . . . . . . . . . . . 68
FurtherReading. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
Contents ix
4 ROSDebugging. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.1 CommonCommandsforROSDebugging. . . . . . . . . . . . . . . . 71
4.2 CommonToolsforROSDebugging. . . . . . . . . . . . . . . . . . . . 73
4.2.1 Usingrqt_consoletoModifytheDebugLevel
atRuntime. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
4.2.2 UsingroswtftoDetectPotentialProblemsinthe
Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
4.2.3 DisplayingNodeStateGraphsUsingrqt_graph. . . . . . 76
4.2.4 PlottingScalarDataUsingrqt_plot. . . . . . . . . . . . . . . 78
4.2.5 Displaying2DImagesUsingimage_view. . . . . . . . . . 78
4.2.6 3DDataVisualizationUsingrqt_rviz(rviz). . . . . . . . . 80
4.2.7 RecordingandPlayingBackDataUsingrosbagand
rqt_bag. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
4.2.8 rqtPluginsandrxApplications. . . . . . . . . . . . . . . . . . 84
4.3 SummaryofBasicROSCommands. . . . . . . . . . . . . . . . . . . . . 84
4.3.1 CreatingaROSWorkspace. . . .. . . . .. . . . . .. . . . .. 85
4.3.2 PackageRelatedOperations. . . . . . . . . . . . . . . . . . . . 85
4.3.3 RelatedOperationsofNodes. . . . . . . . . . . . . . . . . . . . 86
4.3.4 RelatedOperationsoftheTopic. . . . . . . . . . . . . . . . . 87
4.3.5 RelatedOperationsoftheService. . . . . . . . . . . . . . . . 88
4.3.6 RelatedOperationsofrosparam. . . . . . . . . . . . . . . . . . 89
4.3.7 Bag-RelatedOperations. . . . . . . . . . . . . . . . . . . . . . . 89
4.3.8 RelatedOperationsofrosmsg. . . . . . . . . . . . . . . . . . . 90
4.3.9 RelatedOperationsofrossrv. . . . . . . . . . . . . . . . . . . . 91
4.3.10 OtherCommandsofROS. . . . . . . . . . . . . . . . . . . . . . 91
PartII ImplementationofCoreRobotFunctions
5 InstallationandInitialUseoftheRobots. . . . . . . . . . . . . . . . . . . . 95
5.1 IntroductiontotheTurtlebotRobot. . . . . . . . . . . . . . . . . . . . . 95
5.2 CompositionandConfigurationoftheTurtlebotRobot
Hardware. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
5.3 InstallationandTestingoftheTurtlebotRobotSoftware. . . . .. 98
5.3.1 InstallingfromSource. . . . . . . . . . . . . . . . . . . . . . . . 98
5.3.2 DebInstallationMethod. . . . . . . . . . . . . . . . . . . . . . . 99
5.3.3 ConfigurationAccordingtoKobukiBase. . . . . . . . . . . 100
5.4 LaunchingTurtlebot. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
5.5 ManualControlofTurtlebotviaKeyboard. . . . . . . . . . . . . . . . 102
5.6 ControllingTurtlebotThroughScripting. . . . . . . . . . . . . . . . . . 103
5.7 MonitoringtheBatteryStatusoftheKobuki. . . . . . . . . . . . . . . 104
5.8 ExtensionstotheTurtlebotRobot. . . . . . . . . . . . . . . . . . . . . . 106
FurtherReading. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
x Contents
6 ImplementationofRobotVisionFunctions. . . . . . . . . . . . . . . . . . . 109
6.1 VisionSensors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
6.1.1 KinectVisionSensor. . . . . . . . . . . . . . . . . . . . . . . . . 110
6.1.2 PrimesenseVisionSensors. . . . . . . . . . . . . . . . . . . . . 111
6.2 DriverInstallationandTesting. . . . . . . . . . . . . . . . . . . . . . . . . 112
6.3 RunningTwoKinectsattheSameTime. . . . . . . . . . . . . . . . . . 113
6.4 RunningKinectandPrimesenseattheSameTime. . . . . . . . . . 116
6.5 RGBImageProcessingWithOpenCVinROS. . . . . . . . . . . . . 117
6.5.1 InstallingOpenCVinROS. . . . . . . . . . . . . . . . . . . . . 117
6.5.2 UsingOpenCVinROSCode. . . . . . . . . . . . . . . . . . . 118
6.5.3 UnderstandingtheROS-OpenCVConversion
Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
6.5.4 ROSNodeExample. . . . . . . . . . . . . . . . . . . . . . . . . . 121
6.6 PointCloudLibraryandItsUse. . . . . . . . . . . . . . . . . . . . . . . . 125
6.6.1 IntroductiontoPointCloudsandPointCloud
Libraries. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
6.6.2 DataTypesofPCL. .. . . . . .. . . . . .. . . . . .. . . . . .. 125
6.6.3 PublishandSubscribtoPointCloudMessages. . . . . . . 127
6.6.4 TutorialonHowtoUsePCLinROS. . . . . . . . . .. . . . 129
6.6.5 ASimpleApplicationofPCL-DetectingtheOpening
andClosingStateofaDoor. . . . . . . . . . . . . . . . . . . . 136
FurtherReading. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
7 AdvancedImplementationofRobotVisionFunctions. . . . . . . . . . . 139
7.1 ImplementationoftheRobotFollowFunction. . . . . . . . . . . . . 139
7.1.1 TheoreticalFoundations. . . . . . . . . . . . . . . . . . . . . . . 139
7.1.2 OperationalTestingofFollowFunctions. . . . . . . . . . . 140
7.2 ImplementationoftheRobotWavingRecognitionFunction. . . 144
7.2.1 ImplementationFrameworkandDifficultiesAnalysis
ofRobotHandWavingRecognition. . . . . . . . . . . . . . 145
7.2.2 FaceDetectionBasedonAdaBoostandCascade
Algorithms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
7.2.3 IdentifyingHumanHandsWiththeTemplate
MatchingAlgorithm. . . . . . . . . . . . . . . . . . . . . . . . . . 149
7.2.4 SkinToneSegmentationBasedonYCrCbColor
Space. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
7.2.5 OperationalTestingoftheWaveRecognition
Function. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
7.3 ImplementationofObjectRecognitionandLocalization
FunctionsoftheRobot. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
7.3.1 SlidingWindowTemplateMatchingMethod
BasedonHueHistogram. . . . . . . . . . . . . . . . . . . . . . 152
7.3.2 ObjectLocalizationMethodsBasedonSpatial
PointCloudData. . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
7.3.3 ImplementationandTestingofObjectRecognition
andLocalization......................... .... 156