Table Of ContentSPRINGER BRIEFS IN COMPUTER SCIENCE
Oge Marques
Image Processing
and Computer
Vision in iOS
1 23
SpringerBriefs in Computer Science
SeriesEditors
StanZdonik,BrownUniversity,Providence,RI,USA
ShashiShekhar,UniversityofMinnesota,Minneapolis,MN,USA
XindongWu,UniversityofVermont,Burlington,VT,USA
LakhmiC.Jain,UniversityofSouthAustralia,Adelaide,SA,Australia
DavidPadua,UniversityofIllinoisUrbana-Champaign,Urbana,IL,USA
XueminShermanShen,UniversityofWaterloo,Waterloo,ON,Canada
BorkoFurht,FloridaAtlanticUniversity,BocaRaton,FL,USA
V.S.Subrahmanian,DepartmentofComputerScience,UniversityofMaryland,
CollegePark,MD,USA
MartialHebert,CarnegieMellonUniversity,Pittsburgh,PA,USA
KatsushiIkeuchi,Meguro-ku,UniversityofTokyo,Tokyo,Japan
BrunoSiciliano,DipartimentodiIngegneriaElettricaedelleTecnologie
dell’Informazione,UniversitàdiNapoliFedericoII,Napoli,Italy
SushilJajodia,GeorgeMasonUniversity,Fairfax,VA,USA
NewtonLee,InstituteforEducation,Research,andScholarships,LosAngeles,
CA,USA
SpringerBriefs present concise summaries of cutting-edge research and practical
applicationsacrossawidespectrumoffields.Featuringcompactvolumesof50to
125pages,theseriescoversarangeofcontentfromprofessionaltoacademic.
Typicaltopicsmightinclude:
(cid:129) Atimelyreportofstate-of-theartanalyticaltechniques
(cid:129) A bridge between new research results, as published in journal articles, and a
contextualliteraturereview
(cid:129) Asnapshotofahotoremergingtopic
(cid:129) Anin-depthcasestudyorclinicalexample
(cid:129) Apresentationofcoreconceptsthatstudentsmustunderstandinordertomake
independentcontributions
Briefsallowauthorstopresenttheirideasandreaderstoabsorbthemwithminimal
time investment. Briefs will be published as part of Springer’s eBook collection,
withmillionsofusersworldwide.Inaddition,Briefswillbeavailableforindividual
print and electronic purchase. Briefs are characterized by fast, global electronic
dissemination, standard publishing contracts, easy-to-use manuscript preparation
and formatting guidelines, and expedited production schedules. We aim for pub-
lication 8–12 weeks after acceptance. Both solicited and unsolicited manuscripts
areconsideredforpublicationinthisseries.
**Indexing:ThisseriesisindexedinScopus,Ei-Compendex,andzbMATH**
Moreinformationaboutthisseriesathttp://www.springer.com/series/10028
Oge Marques
Image Processing and
Computer Vision in iOS
OgeMarques
DepartmentofComputer&Electrical
EngineeringandComputerScience
FloridaAtlanticUniversity
BocaRaton,FL,USA
ISSN2191-5768 ISSN2191-5776 (electronic)
SpringerBriefsinComputerScience
ISBN978-3-030-54030-2 ISBN978-3-030-54032-6 (eBook)
https://doi.org/10.1007/978-3-030-54032-6
©TheAuthor(s)2020
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology
nowknownorhereafterdeveloped.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Thepublisher,theauthors,andtheeditorsaresafetoassumethattheadviceandinformationinthisbook
arebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsor
theeditorsgiveawarranty,expressedorimplied,withrespecttothematerialcontainedhereinorforany
errorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregardtojurisdictional
claimsinpublishedmapsandinstitutionalaffiliations.
ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSwitzerlandAG.
Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland
ToIngrid,witheternalgratitude forher
unfailing support.
Preface
Thisbookwasmotivatedbythesynergybetweentwomaindrivingforcesintoday’s
technological landscape: (1) the maturity and popularity of image processing and
computer vision (IPCV) techniques and algorithms; and (2) the unprecedented
successofiOS-poweredmobiledevices,suchasiPhoneandiPad.Thecombination
of these techniques and skills has been driving the emerging era of mobile visual
computing.
This book presents a technical overview of some of the tools and technologies
currentlyavailablefordevelopingiOSapplicationswithIPCVcapabilities,includ-
ing the latest advances on the use of machine learning and deep learning to build
intelligentIPCVapps.Itsmaingoalistoprovidethereaderaguidedtourofwhatis
currentlyavailableandapathtosuccessfullytacklethisrathercomplex,buthighly
rewarding,task.
This book is targeted at developers, researchers, engineers, and students who
might need a roadmap to navigate the ever-changing maze of languages, libraries,
frameworks,andAPIsinvolvedindevelopingIPCVapplicationsforiOS.
Thefollowingaresomeofthehighlightsofeachchapterinthisbook:
(cid:129) Chapter1introducesthefieldofmobilevisualcomputingandprovidessupport-
ingfactsandstatisticsthatmotivatethedevelopmentofiOSappsinthisspace.
(cid:129) Chapter 2 provides an overview of the basic steps in image processing and
computer vision as well as the main aspects of the process of developing iOS
applications for IPCV, with a particular emphasis on libraries and frameworks
relatedtoacquiring,storing,processing,anddisplayingimages.
(cid:129) Chapter 3 introduces the Core Image library and shows the examples of using
CoreImageforimagefilteringandfacedetection.
(cid:129) Chapter 4 provides a quick conceptual review of Machine Learning and Deep
Learning as well as an up-to-date coverage of how to develop intelligent IPCV
appsusingCoreMLandCreateML.
(cid:129) Chapter5presentsApple’sVisionFrameworkandshowshowitcanbeintegrated
withCoreMLtodevelopintelligentIPCVapps.
vii
viii Preface
(cid:129) Finally, Chap.6 provides an overview of the widely popular OpenCV library,
withaspecialemphasisontheintegrationofOpenCVintoiOSapps.
Attheendofeachchapter,Ihaveincludedusefulresourcesforthereaderwho
wantstogofurtherandstartdevelopingIPCVappsforiOS.Theseincludebooks,
tutorials,videos,GitHubrepos,andselectedexamples.
ImageProcessingandComputerVisionremainextremelyrelevantfields,whose
techniquescanhelpsolvemanyproblemsandwhosegrowthhasbeenpropelledby
many recent developments, particularly the popularization of deep-learning-based
approaches for most IPCV tasks, which—thanks to Core Image, Core ML, Create
ML,andVisionFramework—have become relativelyeasytoincorporate intoiOS
apps.Iexpectthatthisbookwillfulfillitsgoalofservingasapreliminaryreference
onthesetopics.Readerswhowanttodeepentheirunderstandingofspecifictopics
willfindmorethanahundredreferencestoadditionalsourcesofrelatedinformation
throughoutthebook.
I would like to express my gratitude to Christian Garbin for his comments and
suggestions on an earlier version of the manuscript; Nataliia Neshenko, Humyra
Chowdhury, and Steven Alexander for their assistance with diagrams and figures;
andRoshniMeruguforherdiligentfact-checkingandbibliographyresearch.
I would also like to thank Paul Drougas and his team at Springer for their
patience,kindness,andsupportthroughoutthisproject.
BocaRaton,FL,USA OgeMarques
July2020
Contents
1 Introduction ................................................................... 1
1.1 Motivation................................................................ 1
1.2 TheAgeofMobileVisualComputing.................................. 2
1.3 MobileVisualComputingandiOS...................................... 4
References...................................................................... 5
2 ImageProcessingandComputerVisioniOSAppDevelopment:
TheBasics...................................................................... 7
2.1 ImageProcessing,ImageAnalysis,andComputerVision............. 7
2.2 TheiOSDevelopmentEnvironment .................................... 10
2.3 UsefulFrameworks,APIs,andLibraries ............................... 11
2.4 LearnMoreAboutIt..................................................... 16
References...................................................................... 17
3 CoreImage .................................................................... 19
3.1 Introduction .............................................................. 19
3.2 FundamentalClasses..................................................... 19
3.3 Filters..................................................................... 20
3.4 FaceDetectionUsingCoreImage ...................................... 24
3.5 AutoEnhancementFilters............................................... 25
3.6 LearnMoreAboutIt..................................................... 26
References...................................................................... 26
4 MachineLearningwithCoreML........................................... 29
4.1 MachineLearning:TheBasics.......................................... 29
4.2 TheEraofDeepLearning............................................... 31
4.3 CoreMLBasics.......................................................... 32
4.4 UsingPre-trainedModelswithCoreML............................... 34
4.5 TrainingYourOwnModelswithCreateML........................... 34
4.6 ComputerVisionandCoreML.......................................... 36
4.7 LearnMoreAboutIt..................................................... 38
References...................................................................... 39
ix
x Contents
5 ComputerVisionandImageAnalysiswiththeVisionFramework..... 41
5.1 Introduction .............................................................. 41
5.2 TheImageAnalysisPipeline............................................ 41
5.3 PracticalRecommendations............................................. 43
5.4 IntegrationwithCoreML ............................................... 44
5.5 ExamplesofIPCVSolutionsUsingVisionFramework ............... 46
5.6 LearnMoreAboutIt..................................................... 49
References...................................................................... 49
6 OpenCVandiOS.............................................................. 51
6.1 OpenCV:Overview ...................................................... 51
6.2 OpenCV:FundamentalClassesandOperations........................ 53
6.3 OpenCVandiOS......................................................... 54
6.4 LearnMoreAboutIt..................................................... 58
References...................................................................... 58
