Table Of ContentStudies in Fuzziness and Soft Computing
Mahdi Eftekhari
Adel Mehrpooya
Farid Saberi-Movahed
Vicenç Torra
How Fuzzy
Concepts
Contribute
to Machine
Learning
Studies in Fuzziness and Soft Computing
Volume 416
SeriesEditor
JanuszKacprzyk,SystemsResearchInstitute,PolishAcademyofSciences,
Warsaw,Poland
The series “Studies in Fuzziness and Soft Computing” contains publications on
various topics in the area of soft computing, which include fuzzy sets, rough sets,
neural networks, evolutionary computation, probabilistic and evidential reasoning,
multi-valuedlogic,andrelatedfields.Thepublicationswithin“StudiesinFuzziness
and Soft Computing” are primarily monographs and edited volumes. They cover
significant recent developments in the field, both of a foundational and applicable
character. An important feature of the series is its short publication time and
world-wide distribution. This permits a rapid and broad dissemination of research
results.
IndexedbySCOPUS,DBLP,WTIFrankfurteG,zbMATH,SCImago.
AllbookspublishedintheseriesaresubmittedforconsiderationinWebofScience.
Moreinformationaboutthisseriesathttps://link.springer.com/bookseries/2941
· ·
Mahdi Eftekhari Adel Mehrpooya
·
Farid Saberi-Movahed Vicenç Torra
How Fuzzy Concepts
Contribute to Machine
Learning
MahdiEftekhari AdelMehrpooya
DepartmentofComputerEngineering DepartmentofComputerEngineering
ShahidBahonarUniversityofKerman ShahidBahonarUniversityofKerman
Kerman,Iran Kerman,Iran
FaridSaberi-Movahed VicençTorra
DepartmentofAppliedMathematics DepartmentofComputingSciences
GraduateUniversityofAdvanced UmeåUniversity
Technology Umeå,Sweden
Kerman,Iran
ISSN1434-9922 ISSN1860-0808 (electronic)
StudiesinFuzzinessandSoftComputing
ISBN978-3-030-94065-2 ISBN978-3-030-94066-9 (eBook)
https://doi.org/10.1007/978-3-030-94066-9
©TheEditor(s)(ifapplicable)andTheAuthor(s),underexclusivelicensetoSpringerNature
SwitzerlandAG2022
Thisworkissubjecttocopyright.AllrightsaresolelyandexclusivelylicensedbythePublisher,whether
thewholeorpartofthematerialisconcerned,specificallytherightsoftranslation,reprinting,reuse
ofillustrations,recitation,broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,and
transmissionorinformationstorageandretrieval,electronicadaptation,computersoftware,orbysimilar
ordissimilarmethodologynowknownorhereafterdeveloped.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Thepublisher,theauthorsandtheeditorsaresafetoassumethattheadviceandinformationinthisbook
arebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsor
theeditorsgiveawarranty,expressedorimplied,withrespecttothematerialcontainedhereinorforany
errorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregardtojurisdictional
claimsinpublishedmapsandinstitutionalaffiliations.
ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSwitzerlandAG
Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland
Tothesoulofmyfatherwhotaughtlikea
lovingteacheruntilthelastmomentsofhis
lifeanddedicatedtomymotheraliving
statueoflove.
MahdiEftekhari
Tomyparentswithlove,whogaveme“95%
helpandsupport”and“95%mentalhealth
issues”toenablemetohandlethis
formidabletask.No,itdidnotmakeup
190%,theymultitasked.
AdelMehrpooya
ToParvinRezaeiwhoopenedupnew
horizonsinmylife.
FaridSaberi-Movahed
Totheresearchersthathavebeenstudying
HFS.
VicençTorra
Preface
The aim of this book is to introduce some new trends of research regarding the
use of traditional fuzzy sets, particularly hesitant fuzzy sets, in machine learning.
Indata-drivensystems,weoftenneedtodealwithuncertainty.Thisuncertaintyis
causedbytheavailabledataandalsoinstochasticelementsinherenttotheproblems
beingmodeled.Machinelearningalgorithmsfacetheseproblemswhenbuildingthe
models. Fuzzy set theory provides some tools for this purpose. In this book, we
review some of the tools based on hesitant fuzzy sets that can be used in machine
learningproblems.Thisbookisaddressedtoaudienceswhoareinterestedinboth
machinelearningandfuzzysetextensions.
Thecontentsareorganizedintothreepartsincludingtenchaptersinadditionto
achapteronpreliminarieswhichisanintroductiontovariousconceptsusedinthe
nextthreeparts.Thatis,Chap.1presentsthepreliminaries.
PartIfocusesontheapplicationofbothfuzzysetandhesitantfuzzysetconcepts
in clustering algorithms and on other unsupervised learning approaches. This part
begins with Chap. 2 that describes the application of hesitant fuzzy concepts to
fuse the results of different clustering algorithms. This chapter also presents the
conceptofhesitantfuzzypartitions.Then,Chap.3introducesanunsupervisedfeature
selectionmethodbasedontheconceptsofsensitivityandcorrelation.Thedefinition
of sensitivity given in this chapter is based on the gradient of density function in
subtractiveclusteringwithrespecttoagivenfeature.
PartIIdiscussessupervisedlearningproblemsandexplainscasesforwhichfuzzy
andhesitantfuzzyconceptscanbeusedtoboosttheperformanceofsupervisedtasks.
It is composed of Chaps. 4–7 whose description is briefly provided here. Chap-
ters4and5studytwoextensionsoffuzzydecisiontreesdevelopedinrecentyears.
Chapter6focusesonusinghesitantfuzzysetsindecisiontreeswhenthedatatolearn
thetreesareimbalanced.Thatis,theuseoffuzzysetsforimbalancedclassification
problems,aparticulartypeofsupervisedlearningtasks.Moreprecisely,thechapter
describestheuseofvariousinformationgainmeasurestocombineinformation,as
well as the use of concepts related to hesitant fuzzy sets to combine the results
of fuzzy decision trees. Then, Chap. 7 discusses the application of hesitant fuzzy
sets to ensemble learning algorithms. In ensemble learning, and more particularly
indynamicensembleselectionproblems,situationsariseinwhichmultiple-criteria
vii
viii Preface
decision-makingnotionscanbeused.Chapter7concludesthispart.Itconsidersthe
problemofconsideringdifferentmachinelearningalgorithmsasasetofexpertsand
howtoprocesstheassociatedinformationusinghesitantfuzzyelements.
PartIIIprovidesabriefsurveyofrecentusesofhesitantfuzzysetandroughset
conceptsinsuperviseddimensionreductionproblems.InChap.8,somesimilarity
measuresaswellasfeatureevaluationmetricsaredefinedinformofhesitantfuzzysets.
Thenthesesetsareappliedtocombinedifferentmeasuresandcriteriaoffeatureselec-
tion.Themainideaofthischapteristointroducesomemethodsforcombiningdifferent
featurerankingalgorithmsandmetricsviahesitantfuzzysets.Adistributedversionof
hesitantfuzzy-basedalgorithmsisthenprovidedinChap.9.Thisdistributedversion
isappropriateforhandlingbigdataproblems.Then,Chap.10reviewsanapproach
tocombinedifferentroughset-basedfeatureselectionmetricsthroughhesitantfuzzy
sets.ThispartisconcludedwithChap.11.Inthislastchapter,weexplainhowhesitant
fuzzycorrelationcanbeusedtotacklesupervisedfeatureselectionissues.
Alltheapproachespresentedinthisbookhaveacommonmotif.Theyusehesi-
tantfuzzysetconceptsinmachinelearningproblems.Theyconsidertheseproblems
fromamulti-criteriadecision-makingperspective.Moreparticularly,theseproblems
are framed considering different machine learning algorithms as experts, and then
takingtheresultsofthesealgorithmsasexpert’sopinions.Then,hesitantfuzzysets
areusedtocombinetheseopinions.Thealgorithmspresentedinthisbookhavebeen
describedandtestedinasetofpapers.Thesepapersreportthegoodperformanceof
theapproaches.
Indata-drivenmachinelearning,modelbuildingandselectionisbasedonaset
of criteria. Naturally, efficiency and accuracy is one of them. This usually implies
thatmorecomplexmodelswithmoreparametersperformbetter.Thiscontrastswith
modelsimplicity(andOccam’srazor,aswementioninChap.5.1).ExplainableAI,
theneedtobuildtransparentmodelsandmaketransparentdecisionsaddadditional
constraints to how machine learning models are identified and how these models
areused.Privacyregulationsalsoaddadditionalconstraintstomodelbuildingand
selection.Theneedtoconsideralltheseaspectsintoaccountmakemachinelearning
problemschallenging.Inthisbook,wehavedescribedsomeofthefirstcontributions
ofhesitantfuzzysetsforthiscomplexproblem.Furthersolutionscanbedeveloped
totacklewiththeseothercompetingrequirements.
Theauthorsappreciateandvaluetheworkofstudentswhohavecontributedtothe
papersusedasthemajorreferencesofthisbookoverthecourseof6yearssince2014.
TheauthorsgratefullyacknowledgeMs.S.Barchinejad,Ms.L.Aliahmadipour,Ms.
M. Mokhtia, Ms. S. Sardari, Mr. M. Zeinalkhani, Mr. M. Mohtashami, Mr. M. K.
Ebrahimpour,andMr.J.Elmi.
Kerman,Iran MahdiEftekhari
Kerman,Iran AdelMehrpooya
Kerman,Iran FaridSaberi-Movahed
Umeå,Sweden VicençTorra
December2021
Contents
1 Preliminaries ................................................. 1
1.1 FuzzyNotions ........................................... 1
1.1.1 FuzzySets ....................................... 1
1.1.2 MembershipFunctions ............................ 2
1.1.3 HesitantFuzzySets ............................... 5
1.2 DecisionTree ............................................ 12
1.2.1 FuzzyDecisionTree .............................. 13
1.3 DimensionalityReductionandRelatedSubjects .............. 20
1.3.1 Pearson’sCorrelationCoefficientMeasure ........... 21
1.3.2 CorrelationCoefficientofHesitantFuzzySets ........ 22
1.3.3 Correlation-BasedMerit ........................... 23
1.3.4 SimilarityMeasures ............................... 24
1.3.5 RoughSetandFuzzy-RoughSetBasicConcepts ...... 25
1.3.6 WeightedRoughSetBasicConcepts ................ 27
1.3.7 FuzzyRoughSetBasicConcepts ................... 27
1.4 FuzzyClustering ......................................... 29
1.4.1 Fuzzyc-MeansClustering ......................... 29
1.4.2 SubtractiveClustering ............................. 30
1.4.3 FuzzyPartitions .................................. 31
1.4.4 I-FuzzyPartitions ................................. 32
1.5 LinearRegression ........................................ 33
1.5.1 RidgeRegression ................................. 34
1.5.2 LASSORegression ............................... 34
1.5.3 ElasticNetRegression ............................. 34
References .................................................... 35
PartI UnsupervisedLearning
2 ADefinitionforHesitantFuzzyPartitions ....................... 41
2.1 Introduction ............................................. 41
2.2 H-FuzzyPartition ........................................ 42
ix
x Contents
2.2.1 ConstructionofH-FuzzyPartitions .................. 43
2.3 Discussion .............................................. 49
2.4 ComputerProgrammingExercisesforFutureWorks ........... 49
References .................................................... 49
3 UnsupervisedFeatureSelectionMethodBasedonSensitivity
andCorrelationConceptsforMulticlassProblems ............... 51
3.1 Introduction ............................................. 51
3.1.1 GAforFeatureSelection .......................... 52
3.2 ProposedUnsupervisedFeatureSelectionMethod ............ 53
3.2.1 Feature Relevance Evaluation via Sensitivity
AnalysisBasedonSubtractiveClustering ............ 53
3.2.2 AGeneralSchemeforSensitivityandCorrelation
BasedFeatureSelection(SCFS) .................... 54
3.3 Discussion .............................................. 55
3.4 ComputerProgrammingExerciseforFutureWorks ........... 57
References .................................................... 57
PartII SupervisedLearningClassificationandRegression
4 FuzzyPartitioningofContinuousAttributesThroughCrisp
Discretization ................................................. 61
4.1 Introduction ............................................. 61
4.2 DiscretizationMethods .................................... 62
4.3 EmployingDiscretizationMethodsforFuzzyPartitioning ...... 64
4.3.1 DefiningMFsOverCrispPartitions ................. 66
4.3.2 FuzzyEntropyBasedFuzzyPartitioning ............. 75
4.4 Discussion .............................................. 77
4.5 ComputerProgrammingExercisesforFutureWorks ........... 78
References .................................................... 79
5 ComparingDifferentStoppingCriteriaforFuzzyDecision
TreeInductionThroughIDFID3 ................................ 81
5.1 Introduction ............................................. 81
5.2 StoppingCriteria ......................................... 82
5.3 IterativeDeepeningFID3 .................................. 84
5.3.1 TheStoppingCriterionofIDFID3 .................. 87
5.3.2 ComparisonMethodforVariousStoppingCriteria ..... 89
5.4 Discussion .............................................. 91
5.5 ComputerProgrammingExerciseforFutureWorks ........... 92
References .................................................... 92
6 Hesitant Fuzzy Decision Tree Approach for Highly
ImbalancedDataClassification ................................. 93
6.1 Introduction ............................................. 93
6.2 HesitantFuzzyDecisionTreeApproach ..................... 96
6.2.1 DataBalancing ................................... 96
