Quantitative Remote Sensing at Ultra-High Resolution with UAV Spectroscopy

42 Pages·2017·5.48 MB·English

remote sensing
Review
Quantitative Remote Sensing at Ultra-High
Resolution with UAV Spectroscopy: A Review
of Sensor Technology, Measurement Procedures,
and Data Correction Workflows
HelgeAasen1,*,EijaHonkavaara2 ID,ArkoLucieer3 ID andPabloJ.Zarco-Tejada4
1 CropScienceGroup,InstituteofAgriculturalSciences,ETHZurich,8092Zurich,Switzerland
2 DepartmentofRemoteSensingandPhotogrammetry,FinnishGeospatialResearchInstitute,
NationalLandSurveyofFinland,Geodeetinrinne2,02431Masala,Finland;eija.honkavaara@nls.fi
3 DisciplineofGeographyandSpatialSciences,SchoolofTechnology,EnvironmentsandDesign,
CollegeofSciencesandEngineering,UniversityofTasmania,PrivateBag76,Hobart7005,Australia;
[email protected]
4 EuropeanCommission(EC),JointResearchCentre(JRC),DirectorateD—SustainableResources,
ViaE.Fermi2749—TP261,26a/043,I-21027Ispra,Italy;[email protected]
* Correspondence:[email protected]
(cid:1)(cid:2)(cid:3)(cid:1)(cid:4)(cid:5)(cid:6)(cid:7)(cid:8)(cid:1)
(cid:1)(cid:2)(cid:3)(cid:4)(cid:5)(cid:6)(cid:7)
Received:25May2018;Accepted:30June2018;Published:9July2018
Abstract: Inthelast10years,developmentinrobotics,computervision,andsensortechnologyhas
providednewspectralremotesensingtoolstocaptureunprecedentedultra-highspatialandhigh
spectralresolutionwithunmannedaerialvehicles(UAVs).Thisdevelopmenthasledtoarevolutionin
geospatialdatacollectioninwhichnotonlyfewspecialistdataproviderscollectanddeliverremotely
sensed data, but a whole diverse community is potentially able to gather geospatial data that fit
theirneeds. However,thediversificationofsensingsystemsanduserapplicationschallengesthe
commonapplicationofgoodpracticeproceduresthatensurethequalityofthedata. Thischallenge
canonlybemetbyestablishingandcommunicatingcommonproceduresthathavehaddemonstrated
successinscientificexperimentsandoperationaldemonstrations. Inthisreview,weevaluatethe
state-of-the-artmethodsinUAVspectralremotesensinganddiscusssensortechnology,measurement
procedures,geometricprocessing,andradiometriccalibrationbasedontheliteratureandmorethan
adecadeofexperimentation. Wefollowthe‘journey’ofthereflectedenergyfromtheparticlein
the environment to its representation as a pixel in a 2D or 2.5D map, or 3D spectral point cloud.
Additionally,wereflectonthecurrentrevolutioninremotesensing,andidentifytrends,potential
opportunities,andlimitations.
Keywords: imaging spectroscopy; spectral; unmanned aerial vehicles; unmanned aerial systems
(UAS);RemotelyPilotedAircraftSystems(RPAS);drone;calibration;hyperspectral;multispectral;
low-altitude;remotesensing;sensors;2Dimager;pushbroom;snapshot;spectroradiometers
1. Introduction
Overthepastdecade,thenumberofapplicationsofunmannedaerialvehicles(UAVs,alsoreferred
toasdrones,unmannedaerial/aircraftsystems(UAS),orremotelypilotedaircraftsystems(RPAS)
has exploded. Already in 2008, unmanned robots were envisioned to bring about a new era in
agriculture[1]. RecentstudieshaveshownthatUAVremotesensingtechniquesarerevolutionizing
forest studies [2], spatial ecology [3], ecohydrology [4,5] and other environmental monitoring
applications[6]. Themaindriverforthisrevolutionisthefastpaceoftechnologicaladvancesandthe
RemoteSens.2018,10,1091;doi:10.3390/rs10071091 www.mdpi.com/journal/remotesensing
RemoteSens.2018,10,1091 2of42
miniaturizationofsensors,airframes,andsoftware[7]. AwiderangeofUAVplatformsandsensors
have been developed in the last decade. They have given individual scientists, small teams, and
Remote Sens. 2018, 10, x FOR PEER REVIEW 2 of 42
the commercial sector the opportunity to repeatedly obtain low-cost imagery at ultra-high spatial
resoluadtivoannsc(e1s camndt oth1e mm)inthiaattuhriazvaetiobne eonf tsaeinlosorersd, taoirsfrpaemciefisc, aanreda sso,fptwroadrue c[t7s],. aAn dwdideeli vraenryget iomf eUsA[3V,8 –11].
Morepolvatefro,rmcosm anpdu steinnsgorpso hwaveer baenedn deeavseylotpoedu isne thcoe nlassut mdeecra-dger.a Tdheeys ohfatvwe agriveepn aincdkiavgidesu,alw schiiecnhtisitns,c lude
modesrmnacllo tmeapmust,e arnvdi stihoen caonmdmpehrcoiatol gsercatmorm theetr oypaplogroturinthitym tsos ruecpheaatsedsltyr uocbttuarine florowm-comsto itmioange(SryfM at) [12],
arebeucltorma-hinigghc shpeaatipael rreasnodluativoanisl a(1b lcemt otom 1a mny) tuhsaet rhsa.ve been tailored to specific areas, products, and
Bdeelfiovreeryth teimereas o[3f,8U–A11V].s ,Mthoeremovaejor,r ictoymopfustpinegct rpaolwdeart aasnedts ewasays ptor oudsue cceodnbsuymeexrt-egrrnaadled saotfatwsuarpep liers
packages, which include modern computer vision and photogrammetry algorithms such as structure
(companiesorinstitutions)inastandardizedwayusingafewtypesofsensorson-boardsatellitesand
from motion (SfM) [12], are becoming cheaper and available to many users.
mannedaircraft. Today,researchteamsownorevenbuildtheirownsensingsystemsandprocess
Before the era of UAVs, the majority of spectral datasets was produced by external data suppliers
theirdatathemselveswithouttheneedforexternaldatasuppliers. Technologyisdevelopingrapidly
(companies or institutions) in a standardized way using a few types of sensors on-board satellites
andofferingnewtypesofsensors. Thisdiversificationmakesdataqualityassuranceconsiderations
and manned aircraft. Today, research teams own or even build their own sensing systems and process
eventmheoirr edactrai ttihceaml—seilnveps awrtitihcuoulat rthfeo rneqeuda fnotri teaxttievrneaal nddatsap seucptpralilerrse.m Teocthensoelnogsyin igs daepvperloopaicnhge rsa,pgiidvlye nthe
compalnedx iotyffeorfintgh enegweo tmypeetsr iocf asnendsorrasd. iTohmise dtrivicercsoifrirceactitoionn msarkeeqsu diraetad qfuoarlaitcyc ausrsautreanspcee ccotrnossidcoerpayti-ofnosc used
envireovnemn emnotrael crreimticoatl—esienn psairntgic.ular for quantitative and spectral remote sensing approaches, given the
Scopmecptlreaxlitrye mof othtee sgeenosminegtrigc aatnhde rrsadiniofmoremtriact icoonrrbecytimonesa rseuqruiinregdt hfoer raacdcuiarnatcee sepmecittrtoesdco(pey.g-f.,oicnustehde case
ofchleonrvoirpohnymllenfltuaol rreemscoetne csee)n,srienflg.e cted,andtransmittedfromparticles,objects,orsurfaces. However,
Spectral remote sensing gathers information by measuring the radiance emitted (e.g., in the case
thisinformationisinfluencedbyenvironmentalconditions(mainlytheilluminationconditions)and
of chlorophyll fluorescence), reflected, and transmitted from particles, objects, or surfaces. However,
modifiedbythesensor,measurementprotocol,andthedata-processingprocedure. Thus,itiscritical
this information is influenced by environmental conditions (mainly the illumination conditions) and
tounderstandthefullsensingprocess,sinceundesiredeffectsduringdataacquisitionandprocessing
modified by the sensor, measurement protocol, and the data-processing procedure. Thus, it is critical
mayhaveasignificantimpactontheconfidenceofdecisionsmadeusingthedata[13]. Moreover,
to understand the full sensing process, since undesired effects during data acquisition and processing
itisamlsaoya hpavreer ae qsuiginsiiftiecatnot liamtepraucts oenp tihxee lcsotnofiudnendceer sotfa dnedcitshioenbs imolaodgei cuaslinpgr othcee sdsaetsa o[1f3t]h. eMEoarertohvesry, s tem
(c.f.[1it4 i]s) .also a prerequisite to later use pixels to understand the biological processes of the Earth system
R(ce.fc. e[1n4t]l)y. , several papers have reviewed the literature for UAV technology and its application
in Earth oRbesceenrvtlayt, isoenve[r7a–l 1p0a,p1e5r–s1 h7a]v.eW reivthiewtheed itshseu leitserpatoutreen ftoiar lUlyAaVr tiescinhgnoflroogmy aannd iitnsc arpepalsiicnagtiodni vine rsity
Earth observation [7–10,15–17]. With the issues potentially arising from an increasing diversity of
ofsmallspectralsensorsforUAVremotesensing,thereisalsoagrowingneedtospreadknowledgeon
small spectral sensors for UAV remote sensing, there is also a growing need to spread knowledge on
sensortechnology,dataacquisition,protocols,anddataprocessing. Thus,theobjectiveofthisreview
sensor technology, data acquisition, protocols, and data processing. Thus, the objective of this review
istodescribeanddiscussrecentspectralUAVsensingtechnology,itsintegrationonUAVplatforms,
is to describe and discuss recent spectral UAV sensing technology, its integration on UAV platforms,
andgeometricandradiometricdata-processingproceduresforspectraldatacapturedbyUAVsensing
and geometric and radiometric data-processing procedures for spectral data captured by UAV
systemsbasedontheliterature,butalsoonmorethanadecadeofourownexperiences. Ouraimisto
sensing systems based on the literature, but also on more than a decade of our own experiences. Our
followaimth eiss tiog nfoalllothwr otuheg hsitghneals ethnrsoinugghp trhoec essesn(sFinigg uprreo1ce)sasn (dFidguisrceu 1s)s aimndp odristacunstss itmeppsotrotaanct qsuteipres rteol iable
datawaciqtuhirUe AreVliasbpleec dtraatal sweinthsi UngA.VA sdpdecittiroanl saelnlys,inwge. Aredfldeictitoonnalltyh,e wceu rrerfelnecttr oenv othlue tciuornreinntr reemvooltuetisoenn isni ngto
identriefymtorteen sdenssainngd tpo oidteennttiifayl st.rends and potentials.

FigurFeig1u.rTe h1e. Tphaet hpaotfh ionff ionrfmoramtiaotinonfr formoma ap paarrttiiccllee ((ee..gg..,, ppigigmmenetnst wswithitinh itnhet hleeafl)e,a ofb),jeocbt,j eocr ts,uorrfascue rtfoa ceto
the data product. The spectral signal is influenced by the environment, the sensor, the measurement
thedataproduct.Thespectralsignalisinfluencedbytheenvironment,thesensor,themeasurement
protocol, and data processing on the path to its representation as a pixel in a data product.
protocol, and data processing on the path to its representation as a pixel in a data product.
In combination with metadata, this representation becomes information.
Incombinationwithmetadata,thisrepresentationbecomesinformation.
This review is structured as follows. Different technical solutions for spectral UAV sensor
Ttehchisnorleovgiye warei dsesstcrruibcetdu rine dSeactsiofno 2ll.o Twhes .geDomifefetrriec natndte rcahdnioimcaeltrsico lpurtoicoensssinfgo rstesppse acrtrea ellaUboArVatesde nsor
technionl oSgeyctiaornesd 3e sacnridb e4d, rinesSpeeccttiivoenly2.. ITnh eSegcetioomn e5t,r iwcea nbduirladd aio mmoerteri ccopmropcleestes inpgictsuterep soaf rseigenlaifbicoarnatt edin
RemoteSens.2018,10,1091 3of42
Sections3and4,respectively.InSection5,webuildamorecompletepictureofsignificantdevelopment
stepsandpresenttherecommendedbestpractices. TheconclusionsinSection6completethisreview.
2. SpectralUAVSensors
Spectral sensing can be performed using different approaches. Since in most countries the
thresholdtoflyUAVswithoutpermissionisbelowatake-offweightof30kg,thispaperwillfocus
on sensors that can be carried by these UAVs. Generally, spectral sensors capture information
in spectrally and radiometrically characterized bands. Commonly, they are distinguished by the
arrangementand/oranumberofbands[18,19]. Furthermore,sensorscanbeclassifiedonthebasis
ofthemethodbywhichtheyachievespatialdiscriminationandthemethodbywhichtheyachieve
spectraldiscrimination[20]. Inthefollowingsubsections,weintroduceseveraltypesofsensorsthat
areavailableforUAVswithsomeexampleapplications. Table1attheendofSection2summarizesthe
differentsensortypes.
2.1. PointSpectrometers
Pointspectrometers(referredtoasspectroradiometersiftheyarespectrallyandradiometrically
calibrated)captureindividualspectralsignaturesofobjects. Thespectrometer’sfieldofview(FOV)
andthedistancetotheobjectdefinethefootprintofthemeasurement.
Already in 2008, point spectrometers were mounted on flying platforms (e.g., [21]). Recently,
spectrometershavebeenminiaturizedsuchthattheirsizeandthepayloadcapacityofUAVsconverged.
In 2014, Burkart et al. [22] developed an ultra-lightweight UAV spectrometer system based on
the compact Ocean Optics STS [23] for field spectroscopy. It recorded spectral information in the
wavelengthrangeof338nmto824nm,withafullwidthathalfmaximum(FWHM)of3nmandaFOV
of12◦,andlateritwasusedasaflyinggoniometertomeasurethebidirectionalreflectancedistribution
functionofvegetation[24].Garzonioetal.[25]presentedasystemtomeasuresun-inducedfluorescence
intheoxygenAabsorptionbandwiththehigh-spectralresolutionspectrometerOceanopticsUSB4000
withanFWHMof1.5nmandaFOVof6◦. Furthermore,UAVpointspectrometershavebeenusedto
investigatetheimpactofenvironmentalvariablesonwaterreflectance[26]andforintercomparison
withdatafromamoderateresolutionimagingspectroradiometer(MODIS)inGreenland[27]. Besides,
UAVpointmeasurementshavebeenfusedwithmultispectral2Dimagingsensors[28]andapplied
fromfixed-wingUAVs[29],e.g.,tomonitorphytoplanktonbloom[30]. Additionally,firstattempts
have been made to build low-cost whiskbroom systems for UAVs, which are able to quickly scan
pointsonthesurface[31].
The benefits of point spectrometers are their high spectral resolution, high dynamic range,
highsignal-to-noiseratio(SNR),andlowweight. TheUSB4000weighsapproximately190g,andthe
STSspectrometeronlyweighsabout60g. Incombinationwithamicrocontroller,thetotalready-to-fly
payloadwiththeSTSisonly216g[22], allowingtheirinstallationonverysmallUAVs. However,
sincetheirdatacontainsnospatialreference,auxiliarydataisrequiredforgeoreferencing;thismakes
additionaldevicesnecessary,increasingtheoverallweightandcostofthesystem(c.f. Section3.1).
Inaddition,thedatawithintheFOVofthespectrometercannotbespatiallyresolved;i.e.,eachspectral
measurementincludesspectralinformationfromobjectswithintheFOVofthesensor.
2.2. PushbroomSpectrometers
Thepushbroomsensorrecordsalineofspectralinformationateachexposure. Byrepeatingthe
recordingofindividuallinesduringflight,acontinuousspectralimageovertheobjectisobtained.
EachpixelrepresentsthespectralsignatureoftheobjectswithinitsinstantaneousFOV(IFOV)[32].
Thepushbroomdesignhasbeenthe‘standard’designforlargeairborneimagingspectrometersfor
manyyears;however,miniaturizationforsmallUAVshasonlyrecentlybeenachieved.
Zarco-Tejada et al. [33] demonstrated the use of narrow-band indices acquired from a UAV
platformforwaterstressdetectioninanorchardusingaHeadwallmicro-HyperspecVNIRpushbroom
RemoteSens.2018,10,1091 4of42
scanner[34]withaFWHMof3.2nmor6.4nm,dependingontheslitused. Inaddition,pushbroom
scannershavebeenusedtodetectplantdiseases[35],estimategrossprimaryproduction(GPP)by
means of physiological vegetation indices [36], and retrieve chlorophyll fluorescence by means of
the Fraunhofer line depth method with three narrow spectral bands around the O2-A absorption
featureat760nm[35,36]. Pushbroomsensorshavealsobeenflowntogetherwithlightdetectionand
ranging(LiDAR)systemstofusespectraland3Ddata[37]. Mostofthesestudieswerecarriedoutwith
fixed-wingUAVsatflyingaltitudesof330–575mabovegroundlevel(AGL),resultinginaground
samplingdistance(GSD)of0.3–0.4m. Pushbroomsystemshavealsobeenmountedonmulti-rotor
UAVs,flyinglowerandslower,resultinginultra-highgroundsamplingdistances(<10cmpixelsize).
Lucieeretal.[38]andMalenovskyetal.[39]achievedresolutionsofupto4cmtomapthehealthof
Antarcticmossbeds.
State-of-the-artpushbroomsensorsforUAVsweighbetween0.5–4kg(typically~1kg)including
thelens. However,duetothelargeamountofdatacapturedbysuchasensor,amini-computerwith
considerablestoragecapacityhastobeflowntogetherwiththesensor,whichincreasesthepayload
(Section 3.2). Suomalainen et al. [40] built a hyperspectral mapping system based on off-the-shelf
components. Theirtotalsystemincludedapushbroomspectrometer,globalnavigationsatellitesystem
(GNSS)receiverandinertialmeasurementunit(IMU),ared–green–blue(RGB)camera,andaRaspberry
PItocontrolthesystemandactingasadatasink.Thetotalweightofthesystemwas2kg[40].Recently,
companiesthatformerlyfocusedonfull-sizeaircraftsystemshavealsoattemptedtominiaturizetheir
sensorsforUAVsandprovideself-containedsystemsincludingthesensor,GNSS/IMU,andcontrol
andstoragedevices(e.g.,[41]).
Thebenefitsofpushbroomdevicesincludetheirhighspatialandspectralresolution, andthe
abilitytoprovidespatiallyresolvedimages.Comparedtothepointspectrometers,theyareheavierand
requiremorepowerfulon-boardcomputers. Similartopointspectrometers,pushbroomsystemsneed
additionalequipmenton-boardtheUAVtoenableaccurategeoreferencingofthedata(c.f. Section3.2).
2.3. Spectral2DImagers
2Dspectralimagersrecordspectraldataintwospatialdimensionswithineveryexposure.Thishas
openedupnewwaysofimagingspectroscopy[42],sincecomputervisionalgorithmscanbeusedto
composeascenefromindividualimages,andspectraland3Dinformationcanberetrievedfromthe
samedataandcomposedto(hyper)spectraldigitalsurfacemodels[43,44]. Since[43]firstattempted
tocategorize2Dimagers(thencommonlyreferredtoasimage-framecamerasorcentralperspective
images [32]), new technologies have appeared. Today, 2D imagers exist that record the spectral
bandssequentiallyoraltogetherwithinasnapshot. Inaddition,multi-camerasystemsrecordspectral
bandssynchronouslywithseveralcameras. Inthefollowingsections, thesedifferenttechnologies
arereviewed.
2.3.1. Multi-Camera2DImagers
A multi-camera 2D imager uses several integrated cameras to record a multispectral or
hyperspectralimage. Often,thisisdonebyplacingfilterswithaspecificwavelengthconfiguration
in front of the detector. The first popular camera for this type for UAV application was the MCA,
whichhadfourorsixcameras. Someexamplesofapplicationsthathavebeendevelopedwiththis
firstbulkymodel(2.7kg)carriedouton-boardahelicopterUAVincludewaterstressdetectionand
precisionagriculturestudies[45–47]. FurtherminiaturizationoftheMCAmodelintothemini-MCA
cameraenableditsusefromlightweightplatformsforvegetationdetectioninherbaceouscrops[48]
andweedmapping[49]. However,duetoitstechnicalconfiguration,calibrationandpost-processing
ofdatawascomplex[50]. Additionally,thecamerahadarollingshutter,wherenotallpartsofthe
imagearerecordedatthesametime. Formovingscenes(e.g.,duetothemovementofthesensor),
thisresultsin“rolling-shutter”effectsthatdistorttheimages. Thus,rollingshuttercamerasarenot
RemoteSens.2018,10,1091 5of42
suitedfortakingimagesduringUAVmovement. NewerTetracamcamerasnowalsouseglobalshutter
intheMacawmodel.
Recently,similarbutmorecompactsystemshaveappearedonthemarket. Amongthemarethe
MicaSenseParrotSequoiaandRedEdge(-m)[51,52]withfourandfivespectralbands(blue,green,red,
rededge,near-infrared),andtheMAIAcamera[53],withninebandscapturedbyseparateimaging
sensorsthatoperatesimultaneously.Suchcameraswereusedtoassessforesthealth[54],leafareaindex
inmangroveforests[55]andgrapevinediseaseinfestation[56]. Inaddition,self-builtmulti-camera
spectral2Dsystemshavebeenusedtoidentifywaterstress[57]aswellascropbiomassandnitrogen
content[58].
2.3.2. Sequential2DImagers
Sequential band systems record bands or sets of bands sequentially in time, with a time
lag between two consecutive spectral bands. These systems have often been called image-frame
sensors[43,44,59,60].AnexampleofsuchasystemistheRikolahyperspectralimagerbySenopOy[61]
thatisbasedonthetunableFabry–PérotInterferometer(FPI).Thecameraweighs720g. Thedesired
spectralbandsareobtainedbyscanningthespectralrangewithdifferentairgapvalueswithinthe
FPI[44,62]. Thecurrentcommercialcamerahasapproximately1010×1010pixels,providingavertical
andhorizontalFOVof36.5◦ [61,63]. Intotal,380spectralbandscanbeselectedwitha1-nmspectral
stepinthespectralrangeofapproximately500–900nm,butintypicalUAVoperation,50–100bands
arecollected. TheFWHMincreaseswiththewavelengthiftheorderofinterferenceandthereflectance
of the mirrors remain the same [64]. However, in practical implementation of the Rikola camera,
theresultingFWHMsaresimilarinthevisibleandnear-infrared(NIR)ranges: approximately5–12nm.
TheRikolaHSIrecordsupto32individualbandswithinasecond;so,forexample,ahypercubewith
60 freely selectable individual bands can be captured within a 2-s interval. Recently, a sort-wave
infrared(SWIR)rangeprototypecamerawasdeveloped,withaspectralrangeof0.9–1.7µmandan
imagesizeof320×256pixels[65,66].
Benefitsofthesequential2Dimagersarethecomparablyhighspatialresolutionandtheflexibility
tochoosespectralbands. Atthesametime, themorebandsthatarechosen, thelongerittakesto
recordallofthem. Inmobileapplications,thebandsinindividualcubeshavespatialoffsetsthatneed
tobecorrectedinpost-processing(c.f. Section3.3.2)[44,67,68]. Theframerate,exposuretime,number
ofbands,andflyingheightlimittheflightspeedintunablefilter-basedsystems. TheFPIcamerashave
beenusedinvariousenvironmentalremotesensingstudies,includingprecisionagriculture[44,69–71],
peat production area moisture monitoring [65], tree species classification, forest stand parameter
estimation,biodiversityassessment[72–74],mineralexploration[68],anddetectionofinsectdamage
inforests[60,75].
2.3.3. Snapshot2DImagers
Snapshotsystemsrecordallofthebandsatthesametime[43,76,77],whichhastheadvantage
thatnospatialco-registrationneedstobecarriedout[43]. Currently,multi-pointandfilter-on-chip
snapshotsystemsexistforUAVs.
Multi-pointspectrometer
Multi-pointspectrometersuseabeamsplittertodividethe2Dimageintosections,ofwhichthe
signalisspreadinthespectraldomain[78]. AnexampleistheCubertFirefleye[79]. Thecameraand
acontrollingcomputerweighsabout1kgandrecordsanimagecubewith50×50pixelofspectral
datafrom450–900nmwithanFWHMof5nm(460nm)to25nm(860nm). Simultaneously,aone
megapixelgreyimagewiththesameextentistakenandcanbeusedtocollatetheimagesintoafull
scene[43]. Multi-pointsnapshotcamerashavebeenusedtoderivechlorophyll[42],plantheight[43]
andleafareaindex[80]incrops.
RemoteSens.2018,10,1091 6of42
Theadvantageofmulti-point2Dimagersisthattheyrecordallofthespectralinformationfor
each point in the image at the same time, and typical integration times are very short due to the
Rheimgohte lSiegnhs.t 2t0h18r,o 1u0,g xh FpOuRt .PEHERo RwEeVvIEeWr, t he disadvantage is the relatively low spatial resolution 6o foft h4e2
spectralinformation.
Mosaic filter-on-chip cameras
Mosaicfilter-on-chipcameras
In the mosaic filter-on-chip technology, each pixel carries a spectral filter that has a certain
transImnisthsieonm, osusachic afisl tthere- opnri-nchciipplet eocfh an oBlaoygeyr, peaatcthernp iixne lanc aRrGrieBs caamspeerac.t rTahlefi cltoemrbthinaetdh iansfoarmcearttiaoinn
otrfa tnhsem piisxseiolsn w,siuthcihna as mthoespaircin ocri ptilleeso fthaeBna ryeeprrpesaetntetrsn thine aspneRcGtraBl cinamfoermraa.tTiohne ocof mthbei nareeda isnefeonrm byat tiohne
toiflet.h Tehpei xteeclhsnwoiltohginy ais mbaosseadic oonr at itlhesint hweanferre porne tsoepn tosft ah emsopneocctrharloimnfaotricm caotmiopnloemftehnetaarryea mseeteanl–boyxitdhee
stielme.iTcohnedteucchtonro l(oCgMyOisSb)a sseednsoonr aint hwinhwichaf ethreo nwtaofpero fcoanmtaoinnos cbharnomd aptaicssc ofmiltperlesm theantt airsyolmateet aslp–eocxtirdael
wseamviecloenngdtuhcst oarcc(oCrMdiOngS )tos ethnes oFrabinryw–Phéicrhot tihnetewrfearfeerncceo pntrainincispblea n[8d1,p82a]s.s Tfihlete wrsaftehrast airseo lpartoedsupceecdtr ianl
aw raavnegleen ogft hspsaatciaclo crdoninfiggutoratthioenFsa, binrycl–uPdéirnogt ilninteearfre, rmenocseaipc,r ianncdip tliele[-8b1a,s8e2d]. fiTltheersw. Tahfeirss teacrhenpirqoudeu wceads
dinevaerlaonpgede obfys Ipmaetica lucsoinngfi gthuer aFtPioI nfisl,teinrsc ltuod pinrogvliidnee adri,ffmeroesnati cs,paenctdratli lbe-abnadsse d[8fi1]l.t eCrus.rrTehnitslyt,e tchhen ciqhuipe
iws aasvdaielavbelleo pfeodr tbhye Irmanegceu osifn 4g70th–e63F0P nImfil tienr s16to (4p r×o v4i dpeatdteifrfne)r ebnatnsdpse wctritahl ba asnpdasti[a8l 1r]e.soCluutriroenn tolyf,
5th1e2 c×h 2i5p6i spaixvealisl,a abnled fionr tthhee rraannggee ooff 640700––1603000n nmmi nin1 265( b4a×nd4sp (a5t ×te 5rn p)abttaenrdns) wwiitthh aa ssppaattiiaall rreessoolluuttiioonn
ooff 541029 ×× 221566 ppiixxeellss., Tanhde iFnWthHeMra nisg eboelfo6w00 1–51 0n0m0n fmori nbo2t5hb saynsdtesm(5s×. T5hpea ttwteorn c)hwipitsh aarsep ianttieaglrreasteodlu itnioton
coafm40e9ra×s b2y1 6sepvixerealsl .cTohmepFaWniHesM, anisdb weleoiwgh1 b5enlomwf o40r0b ogt h(e.sgy.s, t[e8m3,s8.4T])h. Seot wfaor,c ohniplys fairresti nattetegmraptetsd winittho
tchaims enraeswb ytescehvneoralolcgoym hpaavneie sb,eaennd pwuebiglihshbeedlo w[8450,806g], (ein.gc.l,u[8d3in,8g4 ])a. Ssotufdary, oonnly hfiorswt atttoe moppttsimwiizteh tthhies
dneemwotesachicninoglo ogfy thhea vimebaegeens [p8u7b].l iRsheceedn[t8ly5,, 8I6m],eicn hclausd ainnngoausntcueddy ao SnWhIoRw vteorsoipotnim ofi zteheth ceamdeemrao [s8a8ic].i ng
oftheThime aagdevsa[n8t7a]g.eR eocfe fnilttleyr,-Iomne-cchhipas caanmneoruansc iesd thaaStW thIReyv reercsoiornd oafllt hoef cthame beraand[8s8 ]a.t the same time.
AddiTtihoenaaldlyv,a tnhteayg earoef vfielrtye rl-iognh-tc, hainpdc caamne braes caisrrtiheadt bthye symraelclo UrdAValsl. oTfhteh edibsaanddvsanattatghee iss atmhaet teiamche.
bAadnddi tiiso njuasltl ym,tehaesyuraerde voenrcye lwigihtht,ina nedaccha ntilbee acnadr rtiheudsb, yacscmuraaltleU sApVecst.raTlh ienfdoirsmadavtiaonnt afogre oisneth baatneda cihs
obnanlyd aivsajiulasbtlme eoanscuer eevdeoryn cfeeww pitihxienlse. aCcuhrtrielentalyn,d ththisu iss, taaccckuleradt ebys pselicgthratllyi ndfeofromcuastiinogn tfhoer coanmeebraan adnids
ionntleyrpaovlaaitliaobnl eteocnhcneiqeuveesr. yTfheewy hpaixveel sa. hCiguhrreern stplya,titahli sreissotlauctkiolend thbayn smliguhlttil-ypodienfto scpuescintrgomtheetecrasm, beurat
tahned riandteiorpmoeltartiico npetercfohrnmiqaunecse. Tohf etyheh afivlteear-hoing-hcherips pteactihanlorelosgoylu thioasn tnhoatn ymetu rlteia-pchoeindt sthpee cqtruoamlietyte rosf,
ebsuttabthlieshreadd isoemnseitnrgic ppreinrfcoiprmleas ntcheato ifs tuhseefidl tienr -pooni-ncth aipndte lcihnne oslcoagnynihnags dneovticyeest. rTehaicsh medaitnhley qreusaulilttys
forfoemst tahbel itsehcehdnisceanl scihnagllpernignecsip dleusritnhga tthise umsaenduinfapctouirnitnagn pdrolicneesss c(ia.en.n, sintrgondgev viacreisa.tTiohni somf tahien tlhyicreksnuelstss
ofrfo tmhet hfeiltteerc henleicmalecnhtsa lbleentgweesednu ardinjagctehnet mpaixneulsfa) catundri ntghep rnoocveseslt(yi .eo.f, stthreo ntgecvhanriiqautieo.n Foifgtuhree t2h icshknowesss
iomftahgeesfi lctaeprteuleremde bnyts ab seetqwueeenntiaadl j2aDce nimtpaigxeerl,s a) amnudlttih-eponionvte slptyecotfrothmeetteecrh, nainqdu ea. fFilitgeur-roen2-cshhiopw ssniampsahgoest
ccaampteurrae.d byasequential2Dimager,amulti-pointspectrometer,andafilter-on-chipsnapshotcamera.

FFiigguurree 22.. EExxaammppllee iimmaaggeess ccaappttuurreedd bbyy tthhee sseeqquueennttiiaall RRiikkoollaa FFaabbrryy––PPéérroott IInntteerrffeerroommeetteerr ((FFPPII)) ((lleefftt)),,
mmuullttii--ppooiinntt ssppeeccttrroommeetteerr CCUUBBEERRTT FFiirreefflleeyyee ((cceenntteerr)) aanndd ffiilltteerr--oonn--cchhiipp IImmeecc NNIIRR ((rriigghhtt)) 22DD iimmaaggeerrss..
TThhee eexxcceerrpptt sshhoowwss oonnee 55 ×× 55 ttiliele uusseedd ttoo ccaappttuurree tthhee ssppeeccttrraall iinnffoorrmmaattiioonn..
SSppaattiioossppeeccttrraall ffiilltteerr--oonn--cchhiipp ccaammeerraass
TToo aaddddrreessss tthhee llaatttteerr,, aa mmooddiififieedd vveerrssiioonn ooff tthhee fifilltteerr--oonn--cchhiipp ccaammeerraa ccaalllleedd CCOOSSII CCaamm hhaass bbeeeenn
ddeevveellooppeedd [[8899]].. TThhiiss sseennssoorr nnoo lloonnggeerr uusseess aa ssmmaallll nnuummbbeerr ooff ssppeeccttrraall ffiilltteerrss iinn aa ttiilleedd oorr ppiixxeell--wwiissee
mmoossaaiicc aarrrraannggeemmeenntt.. IInnsstteeaadd, ,aa llaarrggeerr nnuummbbeerr ooff nnaarrrrooww bbaanndd ffiilltteerrss aarree uusseedd,, wwhhiicchh aarree ssaammpplleedd
ddeennsseellyy eennoouugghh ttoo hhaavvee ccoonnttiinnuuoouuss ssppeeccttrraall ssaammpplliinngg.. TThhee ffiilltteerrss aarree aarrrraannggeedd iinn aa lliinnee--wwiissee ffaasshhiioonn,,
wwiitthh aa nn aammoouunntt ooff lliinneess ((aa ssmmaallll nnuummbbeerr,, ffiivvee oorr eeiigghhtt)) ooff tthhee ssaammee ffiilltteerr nneexxtt ttoo eeaacchh ootthheerr,, ffoolllloowweedd
bbyy nn lliinneess ooff ssppeeccttrraallllyy aaddjjaacceenntt ffiilltteerr bbaannddss.. IInn tthhiiss aarrrraannggeemmeenntt,, ffiilltteerrss oonn aaddjjaacceenntt ppiixxeellss oonnllyy vvaarryy
slightly in thickness, leading to much cleaner spectral responses than the 4 × 4 and 5 × 5 pattern. The
COSI Cam prototype [89] was the first camera using such a chip, capturing more than 100 spectral
bands in the range of 600 nm–900 nm. In a further development, by using two types of filter material
on the chip, a larger spectral range of 475 nm–925 nm was achieved (ButterflEYE LS; [90]) with a
spectral sampling of less than 2.5 nm.
RemoteSens.2018,10,1091 7of42
slightlyinthickness, leadingtomuchcleanerspectralresponsesthanthe4 × 4and5 × 5pattern.
TheCOSICamprototype[89]wasthefirstcamerausingsuchachip,capturingmorethan100spectral
bandsintherangeof600nm–900nm.Inafurtherdevelopment,byusingtwotypesoffiltermaterialon
thechip,alargerspectralrangeof475nm–925nmwasachieved(ButterflEYELS;[90])withaspectral
samplingoflessthan2.5nm.
Physically, these cameras are filter-on-chip cameras, but their filter arrangement requires
adifferent mode of operation. Their operation includes scanning over an area, and is similar to
theoperationofapushbroomcamera. Therefore,thesesensorsarealsoreferredtoasspatiospectral
scanners. The2Dsensorcanbeseenasalargearrayof1Dsensors,eachcapturingadifferentspectral
band(infact,nduplicatelinesperband). Tocaptureallofthespectralbandsateverylocation,anew
imagehastobecapturedeverytimetheplatformhasmovedtheequivalentofnlines. Thisisachieved
by limiting the flyingspeed andoperating thecamera witha highframe rate(typically 30frames
persecond(fps)),whichmeansalargerportionoftheflyingtimeisusedforcollectinginformation.
Aspecializedprocessingworkflowthengeneratesthefullimagecubeforthescene[90].
Due to their improved design, the radiometric quality of the spectrospatial cameras is better
comparedtotheclassicalfilter-on-chipdesign. Atthesametime,theirdataenablesthereconstruction
ofthe3Dgeometrysimilartoother2Dimagersduetothe2Dspatialinformationwithintheimages.
Simaetal.[91]showedthatagoodspatialco-registrationcanbeachievedthatalsoallowsextraction
ofdigitalsurfacemodels(DSMs). Thedrawbackofthesesystemsisthattheyrequirelargestorage
capacityandalowerflyingspeedtoobtainfullcoverageoverthetargetofinterest. Afurtherchallenge
inthiskindofsensoristhateachbandhasdifferentanisotropyeffectsasaresultofhavingdifferent
viewanglestotheobject.
Characterized(modified)RGBcameras
RGB and modified RGB cameras (e.g., where the infrared filter is removed, and so called
color-infraredcameras(CIR)withgreen, redandnear-infraredbands)canalsobeusedtocapture
spectral data if they are spectrally and radiometrically characterized, and the automatic image
adjustmentisturnedoff.Thesecamerasonlyhaveaverylimitednumberofratherwidespectralbands,
buthaveaveryhighspatialresolutionatcomparativelylowcost. AnexampleistheCanons110NIR,
whichrecordsgreen(560nm,FWHM:50nm),red(625nm,FWHM:90nm),andnear-infrared(850nm,
FWHM:100nm)bandsat3000by4000pixels. Forsomecameras,firmwareenhancementssuchas
theCanonHackDevelopmentKit(CHDKCommunity,2017)areavailableandprovideadditional
functionality beyond the native camera firmware, which is potentially useful for remote sensing
activities. Berraetal.[92]demonstratedhowtocharacterizesuchcamerasandcomparedtheresultsof
multi-temporalflightstomapphenologytoLandsat8data.
While the main advantage of RGB and CIR cameras is their high spatial resolution and
comparativelylowcost,amainlimitationistheoverlapbetweenspectralbands. Additionally,these
bandsalsodonotcomplywiththebandsoriginallyusedinstandardvegetationindices(VIs). Thus,
thepseudo-normalizeddifferencevegetationindex(pseudo-NDVI)iscalculated,wheretheavailable
bandsareusedinsomecombination. However,thesecameraswerenotintentionallybuiltforprecise
radiometric measurements, and thus might lack stability. Thus, one needs to be careful with the
settings,sinceimage-processingprocedureswithinthecameramightaltertheinformationcapturedby
thecamera. Thebestwaytouseconsumer-gradecamerasistostoreimagesinarawdataformat(e.g.,
openDNG;[93]). Inaddition,someconsumer-gradesystemsonlyofferalowradiometricresolution
ofeightbitsperchannel,whichmightbeunabletoresolvesubtleradiometricdifferences.
RemoteSens.2018,10,1091 8of42
Table1.Differentspectralsensortypesforunmannedaerialvehicle(UAV)sensingsystemswithpropertiesofexamplesensors.Theexactnumbersmayvarybetween
Remote Sens. 2018, 10, x FOR PEER REVIEW 8 of 49
differenRtemmotoe Sdenesl.s 2,01a8n, 1d0, sx hFOoRu PldEEjRu RsEtVbIEeWt a kenasindication.Thevisualizationsindicatetheimagecubeslicesrecordedduringeachmeasure8m oef 4n9t ofasensortype(adapted
Remote Sens. 2018, 10, x FOR PEER REVIEW 8 of 49
fromdrawiTTnaagbbsllee p11.r. oDDviifffifeedrreeenndtt ssbppyeeccSttrrtaaell fssaeennnssooLrr ittvyyeppenesss ff)oo.rr Tuuhnnmmeaainnnnnfeeoddr maaeerraiiaatlli ovvneehhiiiccnllee t((hUUeAAtVVa)) bsseelennssiiisnnggc ossyymsstteepmmosss wwediitthho ppfrrooinppeefrrottiireemss ooaff teeixxoaanmmfppolleeu ssneenndssooirrnss.. tTThhheee leeixxtaaeccrtt anntuuummrbbeee,rross nmmaawyy evvbaarrsyyi tesofthemanufacturers,and
bTeatbwlee e1n. dDiiffffeerreenntt mspoedcetlrsa,l asnedn ssohro tuyldp ejus sfto br eu tnamkeann anse idn daiecraitaiol nv.e Thhicel ev i(sUuAalViz)a tsioennss iinngd iscyastete tmhes iwmiatghe p cruobpee srltiiceess orfe ceoxradmedp ldeu sreinngs oerasc.h T mhee aesxuarcetm neunmt obfe ar ss emnsaoyr vtyaprye
personalcob(braeerdttewwaspeepteeennod nddfiirdffoffeeemrrnee nncdtter ammwwooiddniteeghllsss ,, S paartnnoeddvf iassdhhneooduuL lldbdiy v jju ueSssnttt ebbsfeea fnttraa okkLeeminvn e aanVsss ii)Inn.T ddTiOicchaaett (iioofinnonf..r o TTrthhmheea evvtiiiossCuunaOa lliiinzzS aaIttthiiocoenna sstm aiinbn)dld,eii ccTiaasrtt eeoc ttonhhmede piimmLosaaøeggdkee eccouuffb brieeno ssfmlloiiccremeHss arrteyeiccoSoonrrp ddfeeeoddxu ndd(duuf orriiinnnrgg t t ehehaaeecc hhlMi tmmerjeeoaaatssluuunrrreieer,mm )o,eennna ttn w oodffe baa sR ssieetoennsbss ooeorrf rt ttyythppPeeea rkerfromMicasense(forthe
(adapted from drawings provided by Stefan Livens). The information in the table is composed of information found in the literature, on websites of the
RedEdge-mm(a)da.naFupWtfeadcHt ufrrMeorms:, fadunrldalw pwienrigsdos ntpharl ocavotirdrheeasdpl fobnymd Seatnexcfeiam nw iuLthimv Set.nesf)a.n T Lhive einnsf ofrromma tVioInT Oin ( fothr et htea bClOe SisI ccaomm),p Torsoendd o Lf øiknef ofrrommat HioynS pfoeuxn (dfo ri nth teh Me jloitlnerira)t,u arned, oRno bweretb Psaitrekse ro ff rothme
manufacturers, and personal correspondence with Stefan Livens from VITO (for the COSI cam), Trond Løke from HySpex (for the Mjolnir), and Robert Parker from
Mmaicnausfeancsteu r(feorrs ,t ahne dR pederEsdognea-lm co).r rFeWspHoMnd: efnuclle wwiidthth S atet fhaanl fL mivaexnism fruomm. VITO (for the COSI cam), Trond Løke from HySpex (for the Mjolnir), and Robert Parker from
Micasense (for the RedEdge-m). FWHM: full width at half maximum.
Micasense (for the RedEdge-m). FWHM: full width at half maximum.
DDDDaaaattttaaaa CCCCuuuubbbbeeee SSSSlllliiiicccceeee DDSSSiiDcccmmSaaainnnecemnannnnnssiieinnniinnoogggsinn ni ogn SSSpppaaatttiiiaaalll RRRSeeepsssaoootllliuuuatttliiioooRnnne soluSSStipppoeeenccctttrrraaalll BBBaaannndddsssS ***p***e ctrSSSapppleeeBcccattt((rrrnFFaaadWWlll RsRRHHeee*MMsss*ooo))lll u uutttiiiooonnnS pectBBB(rFiiiatttWl DDDRHeeeepppMstttohhh)l ution EEExxxBaaaimmmtDpppllleeee p SSSteeehnnnsssooorrrsss ExampleSensors
Dimension (FWHM)
Point Point Point Point ssspppasaaptttiiiaaaalllt ial nnnooonnneee none +++++++++++++++++++++++++++ ( ( ( (((111333000666222444444888)))))) ++++++++++++++++++(++(++++13++++++06++++++2 4 (((4(((8000111)...)–––111111–––222111 000nnn mmmnnnmmm))) ))) +++++++++111111(262626(01 bbbbbb.–1iiiiiitt1tttt– 21 0nnmm)) OOOcccOOOeeeaaacccnnneee aaa11OOOnnn62ppp OOObbtttiiipppiicccttssstttiii UcUcUcsssSSS SSSBBBTTT444SSS000 000 000 OcOeacneaOnpOtipcstiUcsSSBT4S000

HySpex Mjolnir V
Pushbroom Pushbroom Pushbroom Pushbroom ssspppasaaptttiiiaaaalllt ial VNIR SWIR VNIR SWIR VNIR SWIR VNIRSWIR++++++++++++++++++ ( ( ( (((111666222222444000000)))))) ++++++((16224++++++00++++++))++++++++++++ ((((((232323000000000000)))))) ++++++++++++++((++++++23+ + + 00(((+++33300 ...))(((222~~~–––666666 n.n.n.444mmm nnn)))mmm ))) +++++([phone].(2 ~bbbbbb–6iiiiii6tttttt .n4mn)mSSS)mmmpppHHHeeeiiicycyccyccBBBrrriiiSSSmmmooSSoaSaappppy-py-py- e ehehhAAsAsseeexxxyyyppHpHccci i i pppiiMiMeMsesesm(my(m(yaaacecece666SSj j KKj rKrr000 ooOOoOpsApsA11sA000EEElllpppe62CeCnCnn–––iiiSSSeexessxsi1i1i1IIIbbTTTaracrcarc ,,,666MM ///KKKiiRRRRSRSRS444nnntt 0EE0EE0jEEjeeeaaa(((oo 9 99ssSsLSLnSLnnnnnlloo7o77nTnTTo o o mmm1110n0n0niiR-R-R-0r00r–––hhohoo))) EEE,,,2 2 2 VVnynyn y HHHL5L5L5pp p PP0P00 ee eeeSe111000iiimaararr6k6k6kp ssHsdndndn aaaepppi mm wmww ccyeeeBricaca)c)a)Som S a,,,lllp l l l -py heAsexypHcipiesM(myeac6SrKj0OospA0EpleCn–iSesx1iIcTar,6M/KRRS4n0EEje(oa9sLSnnlo7nTmo1n0iR-0r–o)hE,2VnyHL5pP0e1e0ia6krdnsapwmea)cl,l

2DimagerR emo2D imager 2D imager 2D imager te SequentialSeMulti-cameranmulti-)bands.Sequential Sequential Sequential 2Multi-camera Multi-camera Multi-camera 01(multi-) band (multi-) band (multi-) band 8, 10, x FOR PEER REVIEW ssssssppppppseeeasaapcccpttttttiiierrraaaaaacallltllt l i raall SWIR VNIR VNIR VNIR ++++++++++++ +++((+(VNIR1+1+1++2 2 2 ((((888111+30000002+ 000××0×0+00 99×9 ×××66 6(2 1+00011152)))000+6 8000)+000 0)))( ×1 090600×)+++1++++0+++++++ 0 (((((((3015151500))0)0) )000 ))) ++++((++++51+++++++) 0 (+(+(+(1011 2 (((000)0555–––––––44431110000222 n n nnnnnmmmmmmm))))))) +++([phone](2222225– – bbbbbb41iiiiii0tttttt2 nnmm))PPPaaarrrrrrooottt SSSMMMeeePqqqiiircccRRRuuuoaaaoioioitssskkkoiiieeeaoaoaot11nnny,,,lllmmm22 sasasaTpTTeee aaabbFeFeeFe RRccRc tttPPiiPFrraraatteeeIIIaaaPwww dddccVcVVIEaEa Ea NSNNmmmdddWIIIggg RRRMMMeeeI R ---i mi mi mP nnnii a i MMMrr9oCCC tAoAASfM,,, e49qicR uaioskeioanl,msaTeaFeRctPraeIawdcVEaNmdIgRMe-imniMCA,
( R
WI ++(320×256) ++(30) ++(20–30nm) PrototypeFPISWIR
S
nt
oi
ulti-p none + (50 × 50) +++ (125) +++ (5–25 nm) 12 bit Cubert FireFleye
M

snapshot Filter-on-chip none VIS NIR ++++ ((541029 ×× 227126)) ++++ ((1265)) ++++ ((55––1100 nnmm)) 1100 bbiitt iimmeecc SSNNmm45xx45 **

d)
difie
o
m
zed (RGB none ++++ 3000 × 4000 + (3) + (50–100 nm) 12 bit canon s110 (NIR)
acteri
Char
Remote Sens. 2018, 10, x FOR PEER REVIEW 9 of 49
RReemmoottee SSeennss.. 22001188,, 1100,, xx FFOORR PPEEEERR RREEVVIIEEWW 99 ooff 4499
RemoteSens.2018,10,1091 9of42
R
WIR SWIWIR ++++++ (((333222000 ××× 222555666))) +++T+++ (a((333b000))) l e1.Con++t+++.+ (((222000–––333000 nnnmmm))) PPPrrroootttoootttyyypppeee FFFPPPIII SSSWWWIIIRRR
SS
Scanning SpectralResolution
DataCubeSlice SpatialResolution SpectralBands** BitDepth ExampleSensors
Dimension (FWHM)
Multi-pointMulti-point Multi-point Multi-point nnnooonnnneeeo ne +++ (((555000 ××× 5550+00))) (50×50) +++++++++ (((111222555))) +++(+++1+++2+++5 ((()555–––222555 nnnmmm))) +++111(2225 –bbb2iiittt5 nm) CCCuuubbb1eee2rrrttt b FFFiiiitrrreeeFFFllleeeyyyeee CubertFireFleye

er snapshot snapshot snapshot snapshot Filter-on-chipFilter-on-chip Filter-on-chip Filter-on-chip nnnooonnnneeeo ne VIS NIR VIS NIR VIS NIR ++++++NIRVIS++++++ ((((((545454101010292929 ×××××× 222222717171262626++))))))++ ((541029××22++++++71++++++26 ))((((((121212656565)))))) ++++((12++++++65++++++ )) ((((((555555––––––111111000000 nnnnnnmmmmmm)))))) ++++((11111155000000–– bbbbbb11iiiiii00tttttt nnmm)) iiiiiimmmmmmeeeeee11cccccc00 SSSSSSbbNNNNNNiittmmmmmm454545xxxxxx454545 ****** iimmeeccSSNNmm45xx45**
g
ma
2DiRemote SensCharacterized. 2(modified)RGB0Characterized (modified) 1Characterized (modified) Characterized (modified) 8, RGB RGB RGB 10, x FOR PEER REVIEW nnnooonnnneeeo ne ++++++++++++ 3330000+00000+ ×××+ 44+400003000000 00×4000 +++ (((333))) +(+++3 )(((555000–––111000000 nnnmmm))) +(50111–222 1 bbb0iiittt0 nm) cccaaannnooo1nnn2 sssb111i111t000 (((NNNIIIRRR))) 10 of 49c anons110(NIR)
alal
atiospectrpatiospectr spastpioastpioescptreacl tral ++++ 200+0 +++2000 ++ (160) ++(1+6+0 ()5–10 nm) ++(85 –b1it0 nm) CubertC 8BOubStiItt ecrafmlE Y E LS CubertCBOuStItecraflmEYELS
pS
S

* Sold by different companies, e.g., Cubert Butterfly VIS/NIR, ximea MQ022HG-IM-SM4X4-VIS/MQ022HG-IM-SM5X5-NIR, photonFocus MV1-D2048x1088-HS03-96-G2/
MV1-D2048x*1 S0o8l8d- HbyS d0i2ff-e9r6e-nGt c2o.m**pnanuimes,b ee.gr.o, Cfuspbeerctt Braulttbearfnlyd VsIdSe/NpIeRn, dxsimoena tMheQ0c2o2nHfiGg-uIMra-tSiMon4Xa4n-dVIbSi/nMnQin02g2oHfGt-hIMed-SeMv5icXe5s-NanIRd, pshhootuolndFojucusst bMeVu1s-Ded20a4s8xa10r8e8fe-HreSn0c3e-.
96-G2/MV1-D2048x1088-HS02-96-G2. ** number of spectral bands depends on the configuration and binning of the devices and should just be used as a reference.
RemoteSens.2018,10,1091 10of42
3. IntegrationofSensorsandGeometricProcessing
AccurategeometricprocessingisacrucialtaskinthedataprocessingworkflowforUAVdatasets.
Fundamentalstepsincludethedeterminationofthesensorinteriorcharacteristicsofthesensorsystem
(interiororientation),theexteriororientationofthedatasequence(positionandrotationofthesensor
duringthedatacapture),andtheobjectgeometricmodeltofindthegeometricrelationshipbetween
theobjectandtherecordedradiancevalue.
Accuratepositionandorientationinformationisrequiredtocomputethelocationofeachpixel
on the ground. Full-size airborne hyperspectral sensors follow the pushbroom design, and some
canuseasurvey-gradedifferentialGNSSreceiver(typicallymulti-constellationanddualfrequency
capabilities)andIMUtodeterminethepositionandorientation(pitch,roll,andheading)oftheimage
lines. Whenpost-processedagainstaGNSSbasestationestablishedoverasurveymarkatashort
baseline(within5–10km),apositioningaccuracyof1–4cmcanbeachievedfortheon-boardGNSS
antenna,afterwhichpropagationofallofthepose-relatederrorstypicallyresultsin5–10cmdirect
georeferencingaccuracyofUASimagedata. Thiscanbefurtherimprovedbyusinggroundcontrol
points(GCPs)measuredwithadifferentialGNSSroveronthegroundoratotalstationsurvey[94–97].
OneofthechallengesinhyperspectraldatacollectionfromUAVsisthelimitationinweightandsizeof
thetotalsensorpayload. Survey-gradeGNSSandIMUsensorstendtoberelativelyheavy,bulky,and
expensive,e.g.,fiberopticgyro(FOG)IMUsprovidingabsoluteaccuracyinorientationof<0.05◦ [98].
Developmentinmicroelectromechanicalsystems(MEMS)hasresultedinsmallandlightweightIMUs
suitableforUAVapplications;however,traditionally,theabsoluteaccuracyoftheseMEMSIMUshas
beenrelativelypoor(e.g.,typically~1◦absoluteaccuracyinpitch,roll,andyaw)[98].Theimpactofa1◦
errorataflyingheightof50mabovegroundlevel(AGL)isa0.87-mgeometricoffsetforapixelonthe
ground. IfweconsiderthekeybenefitofUAVremotesensingtobetheabilitytocollectsub-decimeter
resolutionimagery,thensuchalargeerrorispotentiallyunacceptable.Thecombinederrorinpitch,roll,
heading,andpositioncanmakethisevenworse. Thereisanimportantrequirementfortheoptimal
combinationofsensorstodetermineaccuratepositionandorientation(pose)ofthespectralsensor
duringacquisition(whichalsorequiresaccuratetimesynchronization),oranappropriategeometric
processingstrategybasedonimagematchingandgroundcontrolpoints(GCPs).
3.1. GeoreferencingofPointSpectrometerData
Whilepointspectrometersofferhighspectralresolution,theirdatacontainnospatialreference.
Thus,precisepositioningandorientationinformationaswellasanaccuratedigitalsurfacemodelare
necessarytoprojectthemeasurementpointsonthesurface.OneapproachistousepreciseGNSS/IMU
equipment,whichisstillexpensive. Analternativeistoalignandcapturedatasimultaneouslywith
a2Dimager,suchasforexamplewithaneithermonochromeorRGBmachinevisioncamera(e.g.,[25]).
Inanextstep,computervisionalgorithmssuchasSfMcanthenbeusedtoderivetheorientationand
positionoftheimagesandtheassociatedpointspectrometermeasurements,assumingthattheimages
containsufficientfeaturesandarecapturedwithenoughoverlap[99,100]. Whilethesecondapproach
ischeaperthanthefirst,bothaddadditionalpayloadtobecarriedbythesensingsystem. Forboth
approaches, accuratetimesynchronizationbetweenthespectroradiometerandtheGNSS/IMUor
cameraisrequired. Furthermore,eachspectroradiometerhasacertainFOVdeterminedbytheslitor
foreoptic,whichcanbeconstrainedwithadditionalaccessories,suchasaGershuntubeorcollimating
lens. Finally,theintegrationtimeofthespectroradiometerwillalsohaveanimpactonthesizeofthe
footprint. Foraspectroradiometerwitharelativelylongintegrationtime(e.g.,1s)onamovingUAV
platform,thecircularfootprintwillbe‘draggedout’intoanelongatedshape. Additionally,inoff-nadir
measurements,thecircularfootprintalsoelongatestoanellipticalshape[24]. Thecombinedeffects
oftheposition,orientation,FOV,integrationtimeofthespectroradiometer,flyingheightandspeed
oftheUAV,andthesurfacetopographywilldeterminethelocationandsize/shapeofthespectral
footprint. Finally, one should also consider that measurements of a field spectrometer are center