Spatial Features for Tree-crown Classification from 3-cm Resolution CIR Imagery

95 Pages·2015·5.28 MB·English

Spatial Features for Tree-crown Classification from
3-cm Resolution CIR Imagery
Linus Närvä
August 26, 2015
Key words: tree-crown classification, single tree remote sensing (STRS), feature se-
lection, support vector machine (SVM), Gray-level co-occurrence matrix (GLCM), Haar
wavelets.
Abstract
Speciescompositionisoneofthemostimportantfactorsinforestinven-
tory. Speciesisanextrinsicpropertyofanindividualtree. Therefore,afeasi-
ble remote-sensing approach to estimate the species composition of a forest
populationistoclassifyindividualtree-crowns. Onthesingle-treelevel,the
problem of tree-crown classification from aerial imagery has been studied
sincethebeginningofthenineties. However,speciesclassificationremainsa
bottleneckinremote-sensing-basedforestinventory.
Many of the studies in tree-crown classification deals with multispectral
features, i.e. measures of intensity in different spectral bands (e.g. colours).
Such features suffers drawbacks of being sensitive to factors such as view-
ing geometry, sun angle, time of day, and time of year. Another possible
approach is to base tree-crown classification on spatial features (Brandtberg,
2002;Erikson,2004),thatmeasureslocalchangesinpixelintensity.
Thisprimaryobjectiveofthisstudyistoevaluatealargesetofspatialfea-
turesandtocomparethesewithsomemultispectralfeatures,fortheproblem
ofdiscriminatingthetreespeciesspruce,pine,andbirch. Intotal366features
wereevaluatedonadatasetof875treesfromaborealforestinsouthernSwe-
den.
Thespatialfeaturesarederivedbytechniquesofimagemorphology,im-
age gradients, and from textural measures based on second-order statistics.
Specifically,thetexturalmeasuresarebasedontheGray-levelco-occurrence
matrix (GLCM) and on Haar wavelets. For the Support Vector Machine
(SVM) classifier the relative importance of each feature was established by
an SVM-based feature ranking algorithm proposed in Guyon et al. (2002).
In order to investigate what spectral information is necessary, the classifica-
tionforthebestfeatureswasmeasuredintherespectivesubsets;allfeatures,
spectralfeatures,spatialfeatures,andspatialfeaturesinasinglecolourchan-
nel.
1
The results demonstrates that the most powerful features were either
based on the GLCM, or on Haar wavelets, possibly because those features
analyses the tree-crown imagery at multiple scales. Another result was that
excluding spectral features did not cause a substantial drop in classifica-
tion accuracy if sufficiently many best-features were used. For smaller best-
feature sets, the spectral features can be replaced by spatial features if the
total number of best-features is increased. However, the spectral informa-
tion from utilizing multiple spectral bands was found to be necessary even
for spatial features. That is, the classification accuracy from limiting spatial
features to a single colour channel was impaired substantially, compared to
when all three CIR channels was utilized. The compound classification ac-
curacyismeasuredto76%,afigurethatiscomparabletootherstudiesusing
spatialfeaturesfortree-crownclassification.
The study shows that spatial features are powerful for imagery-based
tree-crownclassificationat3cmresolution.
2
Contents
1 Introduction 5
1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.2 Relatedwork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.3 Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2 Theory 11
2.1 Symbolsandnotation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.2 Digitalimagefundamentals . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.2.1 Imagehistograms . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.3 Imagefiltering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.3.1 Imagederivatives . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.4 CIRimagery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.5 Introductiontotexture . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.6 TheGray-levelco-occurrencematrix(GLCM) . . . . . . . . . . . . . . . 23
2.6.1 GLCMfeatures . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.6.2 GLCMatmultiplescales . . . . . . . . . . . . . . . . . . . . . . 28
2.7 Waveletsandmultiresolutiontheory . . . . . . . . . . . . . . . . . . . . 31
2.7.1 TheHaarwavelet . . . . . . . . . . . . . . . . . . . . . . . . . . 31
2.7.2 ExtensionoftheHaar-transformto2D . . . . . . . . . . . . . . . 34
2.8 Supervisedlearningandclassification . . . . . . . . . . . . . . . . . . . 36
2.8.1 Generalizationandoverfitting . . . . . . . . . . . . . . . . . . . 37
2.8.2 Classificationperformance . . . . . . . . . . . . . . . . . . . . . 40
2.8.3 Cross-validation . . . . . . . . . . . . . . . . . . . . . . . . . . 41
2.9 Thesupportvectormachine(SVM) . . . . . . . . . . . . . . . . . . . . 42
2.9.1 Extensiontomorethantwoclasses . . . . . . . . . . . . . . . . 42
2.10 Featureselectionandfeaturerankingforclassificationproblems . . . . . 44
2.10.1 FeaturerankingwiththeSVM . . . . . . . . . . . . . . . . . . . 45
2.11 TheFörstnerinterestpointoperator . . . . . . . . . . . . . . . . . . . . 48
3 Materials 51
3.1 Extractionofsingle-treeleveldata . . . . . . . . . . . . . . . . . . . . . 54
3.2 Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
4 Methodsoffeatureextraction 59
4.1 Spectralfeatures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
4.2 Morphologyfeatures . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
4.3 Gradientfeatures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
4.4 Second-ordertexture . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
5 Experiments 67
5.1 Classificationwithmultipleclasses . . . . . . . . . . . . . . . . . . . . . 67
5.2 Normalizedcross-validation . . . . . . . . . . . . . . . . . . . . . . . . 68
5.3 Experiment1: Identifyingthebestfeatures . . . . . . . . . . . . . . . . . 68
3
5.3.1 Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
5.4 Experiment2: Nestedfeatureselection . . . . . . . . . . . . . . . . . . . 69
5.5 Experiment3: Feasibilityofthevotingprocedure . . . . . . . . . . . . . 69
6 Results 71
6.1 Experiment1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
6.2 Experiment2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
6.3 Experiment3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
7 Discussion 85
7.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
7.2 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
7.3 Futurework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
8 Acknowledgements 89
4
1 Introduction
1.1 Background
Forest inventory is the process of estimating properties of total forest populations (West
and West, 2009). In particular, timber volume and species composition are two of the
mostimportantforestparameters(Walter,1998). Whatmethodofforestinventorythatis
suitable is determined by factors such as: the delineation of the population, the definition
of individuals in the population, what properties are to be measured, and what method of
analysisthatisused1 (WestandWest,2009). Manualmeasurementsarehowevercumber-
some. Even for small forests it becomes intractable to extract whatever information the
owner wants to know about the population, by manually measuring every tree (West and
West, 2009, p. 91). Manual small scale forest inventory is therefore primarily concerned
withmeasuringtreepropertiesofsamplesofthepopulation. Thesamplesareoftenasetof
trees within predetermined area plots called stands. The measurements of the stands can
then be generalized to the entire population, using statistics and sampling theory (West
andWest,2009,p. 91).
Large-scale forest inventory requires remote sensing techniques, in order to cover
largeareasatarelativelylowcost(comparedtomanualgroundmeasurements)(Shaoand
Reynolds, 2006, p. 4). Forest remote sensing is an umbrella term for analysis- and data
acquisition- techniques from aerial surveys (Walter, 1998). Two major data acquisition
methods of today are aerial imagery and airborne laser scanning (ALS) (Nordkvist and
Olsson, 2013). The Area based method (introduced in Scandinavia 2002) is a major
methodology for conducting forest inventory based on remote sensing. In the area based
method, the whole population is surveyed (usually by ALS). The measurements of the
survey is calibrated to manual ground measurements. Parameters of interest are then
estimated using different methods of analysis. The Area based method mainly estimates
parameters related to tree size. However, a particularly important property that is not
estimatedbytheAreabasedmethodisspeciescomposition(NordkvistandOlsson,2013).
In fact, to this day, no published results has produced a classification accuracy that is
suitable for operational use (Nordkvist and Olsson, 2013). Species classification remains
abottleneckinforestinventorybasedonremotesensing(Korpelaetal.,2010).
1Inthisarticleitisnaturaltothinkofthetreesasindividuals, andtheevaluatedstandsasconstituting
thepopulation.
5
1.2 Related work
Operational methods for forest inventory, based on remote sensing (e.g. the area based
method),mainlyestimatesforestparametersonthestandlevel(forgroupsoftrees). How-
ever, certain important forest parameters are extrinsic to individual trees and therefore
cannot be directly measured on the stand level. One particular such parameter is species.
Given remote sensing data of sufficient detail, the analysis can be conducted on a single-
tree level (Nordkvist and Olsson, 2013). This sub-field of forest remote sensing that
modelsindividualtreesiscalledSingletreeremotesensing(STRS).
An important sub-problem that arises in STRS is individual tree crown delineation2.
ALS data has proven valuable for automatically delineating individual tree crowns (see
e.g. Perssonetal.2002). ForacomparisonofdifferentALSbasedtreecrowndelineation
techniques,seeKaartinenetal.(2012);Vauhkonenetal.(2011). Itisalsopossibletouse
aerial 2D imagery for tree crown delineation. However, image based delineation algo-
rithmsingeneralsuffersfromcommissionerrors. Thereasonofthecommissionerrorsis
that in open areas, structures in the lower forest floor, such as shrubs or clumped vegeta-
tion,aremistakenfortrees. Ifhoweverheightinformationisavailable,thesecommission
errors can be removed (Leckie et al., 2003a). A review of 2D imagery based automatic
tree crown delineation is found in Brandtberg and Warner (2006). Aerial imagery is not
necessarily limited to 2D information though. It is possible to construct a digital canopy
model (DCM) of canopy elevation, using photogrammetry on overlapping aerial images
(Nordkvist and Olsson, 2013). To obtain a DCM from ALS data is straightforward, be-
cause laser scans produces 3D information directly. DCMs has shown especially useful
forautomatictreecrowndelineation(NordkvistandOlsson,2013).
The STRS community today regards aerial imagery mainly as a complement to ALS.
Inparticular,thisisthecaseforspeciesclassification(Leckieetal.,2003a;Packalénetal.,
2009). TheadvantageofALSstemsfromitsabilitytocapturestructuralinformation,be-
causeitproduceshighlydetailed3Dpointcloudsofthescannedenvironment. Aspecially
important property of ALS is that individual rays passes through the forest canopy, cap-
turing the ground and the internal structure of the trees (in contrast to photogrammetry
that only captures the uppermost surface of the forest canopy) (Nordkvist and Olsson,
2013). Thebenefitof2Dimagery,ontheotherhand,isitsabilitytocapturemultispectral
information and spatial details useful for species and health assessments (Leckie et al.,
2003a;BrandtbergandWarner,2006).
Even though research efforts in species classification based on ALS alone, demon-
strates the usefulness of ALS data in species classification3 (e.g. Brandtberg et al., 2003;
Brandtberg, 2007; Ørka et al., 2007, 2009; Li et al., 2010; Korpela et al., 2010), the ben-
efit of including aerial imagery must not be disregarded. That is, because aerial imagery
2Anoteonterminology.Tree-crowndelineationistheacceptedterminSTRSliterature,fordetermining
whichpixels/pointsbelongtorespectivetree-crowns. However, inimageprocessing, imageanalysis, and
computer vision in general, the corresponding term is tree-crown segmentation (Gonzalez, 2008, chapter
10).
3It is important to note that the features used in ALS based species classification are note exclusively
structural(i.e. derivedfromthegeometryofthe3Dpointcloud),butalsobasedonpropertiesofthelaser
echoes,e.g. themeanandstandarddeviationofechointensities(Ørkaetal.,2007).
6
in addition to ALS contributes to classification accuracy by a significant amount (Pack-
alén et al., 2009). Examples of laser-imagery hybrid systems for individual tree species
classification are found in Holmgren et al. (2008) and Ke et al. (2010), both which con-
cludesthatthesynthesizeddatayieldsbetterclassificationaccuracythantouseonlylaser
features, or only image features. In their analysis, Holmgren et al. (2008) includes the
classification accuracies from the respective feature sets used separately. Interestingly,
the resulting classification accuracies are comparable. However, the performance of the
imagefeature,specificallythemeanpixelintensitypercolourbandofthetreecrown,was
foundtovarybyasignificantamountbetweendifferentseasons.
On the single-tree level, species classification based on aerial imagery have a longer
history than methods based on ALS. Early work in species classification from aerial im-
agery is found in Gougeon et al. (1989); Pinz and Bischof (1990); Pinz et al. (1993). In
general, aerial imagery data can be separated into three categories of data; multispectral,
multitemporal, and spatial (Key et al., 2001). Multispectral information correspond to
the number of spectral bands used (e.g. colours). Multitemporal information correspond
to the number of dates (time points) the the scene has been photographed. Spatial infor-
mation correspond to local changes in pixel intensity. Generally, the spatial information
increaseswithincreasingresolution.
Important work in individual tree species classification from multispectral features is
foundinGougeon(1995). Gougeon(1995)evaluatedsevenspectralfeaturesforclassify-
ing five species of conifers4 in Canada. The study found two interesting results. Firstly,
the simplest features; the mean intensity values of the respective spectral bands con-
tributed the most to the classification accuracy. Secondly, repeating the experiment (ten
times),usingdifferenttrainingdata,resultedinalargevariationofclassificationaccuracy
ofupto22pp5.
Theideaofutilizingmultitemporaldatainindividualtreespeciesclassificationisthat
species-relevant phenological changes can be measured by collecting the imagery at op-
timal dates. Moreover, the phenological changes are often changes in spectral properties
(e.g. leaveschangingcolourinfall). Thisobservationmotivatestoutilizethecombination
ofmultitemporalandmultispectraldata(Keyetal.,2001). Keyetal.(2001)thereforein-
vestigated the relative contribution to species classification of adding spectral bands ver-
sus adding dates. The experiment uses data from a deciduous forest6 in West Virginia.
The study discovered that the contribution to classification accuracy per spectral band is
slightlygreaterthanthecontributionperdate.
Thelastcategoryofimagedataisspatialdata. Examplesofspatialdataincludestruc-
ture, texture7, and morphology. Human visual interpretation is to a large extent based on
the interpretation of spatial information (Brandtberg and Warner, 2006). It is therefore
sensible to assume that there should exist spatial descriptors that are able to discriminate
between tree species. Indeed, comprehensive catalogues of spatial descriptors such as
4BarrträdinSwedish.
5Percentage points (pp) is the unit of arithmetic difference between percentages. For example, the
differencebetween1%and5%is4pp.
6LövskoginSwedish.
7Strictlyspeaking,onlysecond-ordertextureisconsideredtobespatialinformation. Seesection2.5.
7
Fournier et al. (1995) have been produced for the purpose of species classification, from
manual interpretation of aerial images. However, to utilize spatial data in automatic clas-
sificationhasprovendifficult. Specifically,itishardtospecifyexactlywhatlocalchanges
tolookfor,andatwhatscaletolookforthem(BrandtbergandWarner,2006). Asaconse-
quence,muchoftheworkinspeciesclassificationfromspatialfeaturesalsousesfeatures
derived from multi-spectral information (e.g. Meyera et al., 1996; Brandtberg, 2002), or
indirectly uses spectral information by deriving spatial features from multiple spectral
bands (e.g. Erikson, 2004). In some cases, the simple multispectral features contribute
more to classification than the spatial features (e.g. Brandtberg, 2002). Still, there are
good reasons to omit spectral features in species classification. The problem of basing
features on absolute intensity properties (e.g. mean pixel intensity) is that such features
are sensitive to changes in illumination. For species classification from aerial images,
error sources include: different atmospheric conditions, or weather; different times of
day/year; and differences in viewing geometry for trees at different positions within the
image (Puttonen et al., 2009; Korpela et al., 2011; Lillesand et al., 2004). Furthermore,
the spectral signatures of trees varies within species (Leckie et al., 2005). Also, the spec-
tral signature of trees (even coniferous species) varies with time of year (Guyot et al.,
1989). Guyot et al. (1989) identifies a comprehensive list of factors affecting the spectral
signaturesofforestcanopies.
Tothebestoftheauthorsknowledge,texturalmeasuresbasedonsecond-orderstatis-
tics has never been tried in species classification on the single-tree level. However, on a
stand level, Franklin et al. (2000, 2001) investigated if features derived from the Gray-
level co-occurrence matrix (GLCM) in addition to multispectral features can be used to
improve classification accuracy, over multispectral features alone. The study found that
the incorporation of texture yielded a significantly greater improvement at a finer scale
(sub-meter resolution) compared with the improvement at a coarser scale. Textural mea-
sures based on first-order statistics are utilized at the single-tree level in (Meyera et al.,
1996), and in (Leckie et al., 2003b). Specifically, in the form of intensity standard devia-
tion of all pixels within the tree crown. The results of the two studies where ambiguous.
Leckieetal. foundthattexturedidnotcontributetoclassificationbeyondusingjustspec-
tralinformation. Incontrast,Meyeraetal. foundanimprovementwhenincludingtexture
in principal-component-analysis-derived features. Another first order textural measure;
theentropyofpixelintensitieswithinthetreecrown,wasutilizedatthesingle-treelevelin
(Brandtberg,2002). Thecontributiontoclassificationaccuracywasfoundtobemodest.
Morphological features in individual tree species classification was introduced by
Brandtberg (1997, 2002). The introduced feature aims to capture the radial branch pat-
tern of a Norwegian spruce crown. In later work by Erikson (2004), four morphological
features were suggested. Each of these features is supposed to single out a specific tree
class(Scotspine,Norwegianspruce,birch,andaspen). Themeasurescapturesproperties
apparentfrominspectionoftreeimagesofrespectivespecies.
A final class of spatial features for individual tree species classification describes the
geometry of the tree crowns. For example, in (Brandtberg, 2002), two boundary descrip-
tors and one regional descriptor was defined. The former two features were derived from
the curvature of the boundary pixels between the sunlit and shadowed regions of the tree
8
crown. Thelastfeatureistherelativecrownarea(seealsoBrandtberg,1998).
The studies mentioned this far evaluates classification from a relatively small subset
of well-chosen features, where each feature is expected to contribute information. An
alternative approach is to use a larger set of features, where only a subset of features is
expected to be useful, and to search for the useful features by means of feature selection,
ordatamining. Boman(2013)utilizesanSVM-basedmethodoffeatureselection,devel-
oped in Guyon et al. (2002), to evaluate (mainly) textural features for tree classification.
However, the classification problem treated in Boman (2013) is fundamentally different
fromtheoneinthisstudy. Boman(2013)usesterrestrialimageryandclassifies128×128
sub-imagesintotheclassesspruce,pine,andground.
1.3 Objective
Brandtberg (2002) concludes that it is a difficult, unsolved problem in tree-crown classi-
fication to find appropriate descriptors that contribute significant information to the clas-
sifier. Thirteen years has passed since then, but to the best of the author’s knowledge,
Erikson (2004) is the only study, after Brandtberg (2002), that uses spatial features in
theirownright.
Therefore, the primary objective is to systemically evaluate a large set of features
for discriminating three classes of trees in a Swedish, boreal forest. The tree classes are
specifically; spruce, pine, and deciduous trees8. Primarily, spatial features that repre-
sent wide categories of spatial information are considered, but a smaller set of spectral
features are also included for comparison. Because spatial features in general may have
advantages over spectral features, the relative contribution to classification performance
of respective feature subset (spatial and spectral) is of particular interest. What kind of
spectral information is required for classification and what kind of spectral information
theclassifiercandowithout,issubjecttoanalysis.
Spatialfeaturesisdefinedaslocalchangesinpixelintensity. Observethatthisdefini-
tion excludes textural measures based on first-order statistics, such as the standard devi-
ation in pixel intensity within the tree crown. Such measures are global in the sense that
they do not take into account where particular intensities occur. The distinction between
the two texture classes is further clarified in section 2.5. To avoid ambiguity, spectral
features are here defined as measures of intensity that are global to the tree-crown. This
definitionincludestexturalmeasuresbasedonfirst-orderstatistics.
8InSwedish: gran,tall,andlövträd,respectively
9
10