Table Of ContentOptimizingthe SOM NeuralNetwork Implementation forDataMiningwithDistributed
par
MemorySystemsandClusterComputing
PhilippTomsichandAndreasRauberandDieterMerkl
InstituteofSoftwareTechnology,ViennaUniversityofTechnology
{phil,andi,dieter}@ifs.tuwien.ac.at
Abstract performancecomputingprovidesthetechnologynecessary
The self-organizing map is a prominent unsupervised forthedesiredspeedup.
neural network model which lends itself to the analysis of In this paper we present optimizations to SOM, a
par
high-dimensionalinputdataanddataminingapplications. software-basedparallelimplementationoftheSOM.Itwas
However,thehighexecutiontimesrequiredtotrainthemap first introduced in [10] and provides a far better speedup
putalimittoitsapplicationinmanyhigh-performancedata compared to other implementation attempts such as [2].
analysisapplicationdomains. We have implemented and evaluated the SOM on a num-
par
In this paper we discuss the SOM implementation, berofplatforms,rangingfromoff-the-shelfdual-processor
par
a software-based parallel implementation of the self- Celeron systems, the 16-processor SGI Power Challenge,
organizing map, and its optimization for the analysis of to the 64-processor SGI Cray Origin 2000 using an appli-
high-dimensionalinputdatausingdistributedmemorysys- cationscenariofromthedigitallibrarydomainasatest-bed
temsandclusters.Theoriginal SOMalgorithmscalesvery for our performance analysis. However, the initial version
par
wellinaparallelexecutionenvironmentwithlowcommuni- didnotscaleverywellforNUMAarchitectures,otherdis-
cationlatenciesandexploitsparallelismtocopewithmem- tributedmemorysystemsandclusters.
ory latencies. However it suffers from poor scalability on Theremainderofthepaperisorganizedasfollows.Sec-
distributedmemorycomputers.Wepresentoptimizationsto tion 2 provides an introduction to the architecture and the
furtherdecouplethesubprocesses,simplifythecommunica- training algorithm of a self-organizing map, a discussion
tionmodelandimprovetheportabilityofthesystem. of the importance of SOMs for digital library applications
andsoftwareimplementationsofSOMs.The SOMsystem
par
is presented in Section 2.3, followed by an analysis of ex-
1 Introduction
perimentalresultsinSection3.Theoptimizationsto SOM
par
The self-organizing map (SOM) [5] is a prominent un- arepresentedinSection4.FutureworkisdiscussedinSec-
supervised neural network model providing a topology- tion5.FinalconclusionsaregiveninSection6.
preserving mapping from a high-dimensional input space
2 Self-organizingmaps
onto a two-dimensional map space. This capability al-
lowsanintuitiveanalysisandexplorationofunknowndata The self-organizing map [5] basically provides a form
spaces, with its applications ranging from the analysis of of cluster analysis by producing a mapping of high-
financial data, medical data, to time series data [3, 5, 9]. dimensional input data (cid:0)(cid:2)(cid:1)(cid:3)(cid:0)(cid:5)(cid:4)(cid:7)(cid:6)(cid:9)(cid:8) onto a usually 2-
Another ratherprominent application arena is textclassifi- dimensional output space while preserving the topological
cation, where documents in a collection are clustered ac- relationships between the input data items as faithfully as
cordingtotheircontent.However,inmanyapplicationsthe possible.Itconsistsofasetofunits,whicharearrangedin
dimensionalitiesofthefeaturespacestobeanalyzedaswell some topology where the most common choice is a two-
as the amount of data encountered are too large to allow a dimensional grid. Each of the units is assigned a weight
(cid:10)
fast,interactivetrainingoftheneuralnetwork.Still,interac- vector of the same dimension as the input data, (cid:4)
(cid:11)(cid:13)(cid:12) (cid:11)(cid:13)(cid:12)
tivetrainingandmappingarehighlydesirable,astheyopen (cid:6) (cid:8) . In the initial setup of the model prior to training, the
a huge field of applications in the context of data analysis weightvectorsarefilledwithrandomvalues.
in Data Warehouses or in the field of Digital Libraries, to Duringeachlearningstep,theunit withthehighestac-
(cid:14)
namejusttwoexamples.Inbothcases,interactiveresponse tivitylevel,i.e.thewinnercwithrespecttoarandomlyse-
timesareamust. lectedinputpattern(cid:0) ,isadaptedinawaythatitwillexhibit
Existing hardware implementations [4, 5, 6] have not an evenhigher activity level at future presentations of that
gained wide-spread acceptance because of the limitations specificinputpattern.Commonly,theactivitylevelofaunit
they put both on the dimensionality of the data to be an- isbasedontheEuclideandistancebetweentheinputpattern
alyzed or the size of the map. On the other hand, high- andthatunit’sweightvector,i.e.theunitshowingthelow-
1
theinputpattern (cid:0) interms ofthe inputspace.Apartfrom
thewinner,adaptationisperformedwithneighboringunits,
x(t)
too. Units that are subject to adaptation are depicted as
mc(t+1) shadedunitsinthefigure.Theshadingofthevariousunits
mc(t)
corresponds to the amount of adaptation and thus, to the
spatial width of the neighborhood-kernel. Generally, units
Output space Input space
Figure1.Architectureandtrainingprocessofaself-organizingmap in close vicinity of the winner are adapted more strongly
andaredepictedwithadarkershadeinthefigure.
est Euclidean distance between it’s weight vector and the 2.1 ImplementingSOMsinsoftware
distance between the input pattern and that unit’s weight
A serial implementation of the SOM iterates through
vector, i.e. the unit showing the lowest Euclidean distance
the training algorithm. This involves scanning all units to
betweenit’sweightvectorandthepresentedinputvectoris
find the winner and then modifying all units to reflect the
selectedasthewinner.Hence,theselectionofthewinner
(cid:14) change. As the map will usually be implemented as an ar-
maybewrittenasgiveninExpression(1).
ray of weight-vectors, this array is traversed twice. From
this,itshouldbeclearthatamajorbottleneckinasoftware-
(cid:0)(cid:21)(cid:20) (cid:0)#(cid:20) (1)
(cid:14)(cid:16)(cid:15)(cid:18)(cid:17)(cid:19)(cid:17) (cid:11)(cid:23)(cid:22)(cid:24)(cid:17)(cid:25)(cid:17)(cid:27)(cid:26)(cid:29)(cid:28)(cid:31)(cid:30)(cid:25) (cid:17)(cid:19)(cid:17) (cid:11)(cid:13)(cid:12)$(cid:17)(cid:19)(cid:17)% based implementation of a self-organizing map consists in
(cid:12)"!
thememorybandwidth.Duringeveryiterationofthetrain-
Adaptation takes place at each learning iteration and
ing algorithm every unit within the map is touched twice.
is performed as a gradual reduction of the difference be-
First,duringthesearchforthebestmatchingunitwithinthe
tween the respective components of the input vector and
mapandthenagain,whentheweightsareadjusted.Dueto
the weight vector. The amount of adaptation is guided by
thefactthatweareusinghigh-dimensionalinputswithmul-
a learning-rate thatis graduallydecreasinginthecourse
&
tiple thousands of inputs, it is not uncommon that a rather
of time. As an extension to standard competitive learning,
small map requires in excess of 5 MBs. A medium-sized
units in a time-varying and gradually decreasing neigh-
map may easily occupy tensof megabytes. This is beyond
borhood around the winner are adapted, too. This strategy
the capacity of most cache memories and therefore incurs
enables the formation of large clusters in the beginning
large overheadsinits memoryaccesses. Thecache thrash-
and fine-grained input discrimination towards the end of
ing can be lessened a little by using a technique known
thelearningprocess.Incombiningtheseprinciplesofself-
asloopinversion.Insteadoftraversingtheunit-arrayfrom
organizingmaptraining,wemaywritethelearningruleas
its beginning to its end when traversing it for the second
given in Expression (2). Please note that we make use of
time(i.e.,whenadjustingtheweights),thearrayistraversed
a discrete time notation with denoting the current learn-
’
backwardstoexploitthecache.
ingiteration.Theotherpartsofthisexpressionare repre-
&
Anothermajorbottleneckresultsfromthehugenumber
sentingthetime-varyinglearning-rate, representingthe
()(cid:22)*(cid:12)
offloatingpointoperationsrequired.Traininga10x15map
time-varying neighborhood-kernel, (cid:0) representing the cur-
of4000featuresin20000iterationsrequiresinexcessof72
rentlypresentedinputpattern,and denotingtheweight
(cid:11)+(cid:12)
billion arithmetic floating point instructions and 6 million
vectorassignedtounit .
(cid:10)
floatingpointcomparisons.Inordertoprovidehighperfor-
manceinsuchapplications,multipleprocessorscanbeused
(cid:0) (cid:20) (2) inparallel.Thisbecomesaviablealternative,asthedistance
(cid:11)(cid:13)(cid:12)(cid:3),-’/.(cid:29)0213(cid:26)"(cid:11)(cid:13)(cid:12)(cid:3),-’41/.5&6,-’4187(cid:24)()(cid:22)*(cid:12)(cid:3),-’41(cid:2)7:9 ,-’41 (cid:11)+(cid:12);,-’41=<
calculationandtheadaptionoftheweightscanbeexecuted
A simple graphical representation of a self-organizing independentlyfor eachunit. Theseindependentoperations
map’s architecture and its learning process is provided in constitutethemajorpartoftheruntime,callingforaparallel
Figure1.Inthisfiguretheoutputspaceconsistsofasquare implementation.
of36units,depictedascircles.Oneinputvector(cid:0) isran-
,-’41
2.2 SOMsandDigitalLibraries
domlychosenandmappedontothegridofoutputunits.In
thesecondstepofthelearningprocess,thewinner show- A prominent application area for SOMs can be found
(cid:14)
ing the highest activation is selected. Consider the winner with digital libraries, as information retrieval in digital li-
being the unit depicted as the black unit in the figure. The braries is often based on similarity search [1]. SOMs pro-
weightvectorofthewinner, ,isnowmovedtowards vide the capability to the high-dimensional feature vectors
(cid:11)(cid:23)(cid:22)>,-’41
the current input vector. This movement is symbolized in associated with documents into a low-dimensional Carte-
theinputspaceinFigure1.Asaconsequenceoftheadap- sianspace.Oncethismappingisconcluded,itcanbeused
tation,unit willproduceanevenhigheractivationwithre- todetermineaclusteringofdocuments.Similardocuments
(cid:14)
specttoinputpattern(cid:0) atthenextlearningiteration, , shouldbelongtothesameorneighboringclusters.Thispro-
’8.(cid:29)0
becausetheunit’sweightvector, ,isnownearerto videsauniqueopportunitytovisualizetheclustereddocu-
(cid:11)(cid:23)(cid:22)?,-’2.@021
2
L-CAFEIFHJIECK AAABBBCCCDDDEGEGEGFFFHJHJHJIIIEEECCCKKK
ments.However,themostimportantupcomingapplications
ofdigitallibrariesrequirejustthisinteractiveresponse:
select input-vector
Web-based search-engines. A visualization of search re-
sultsispossiblebymappingthedocumentsfromtheresult
broadcast input-vector wwwaaaiiittt uuunnntttiiilll ttthhheee iiinnnpppuuuttt---vvveeeccctttooorrr iiisss aaavvvaaaiiilllaaabbbllleee
setintoa2-dimensionalspace.
Fine-grained clustering of large document collections. find local-minimum
Various types of documents require a fine-grained clus-
tering of large document collections with a high num- wait until all local-minima are available bbbrrroooaaadddcccaaasssttt lllooocccaaalll mmmiiinnniiimmmuuummm
ber of relevant features (e.g., laws). A map trained us-
find the global minimum
ing SOM is available at http://www.ifs.tuwien.
par
ac.at/ifs/research/projects/parsom/forin-
broadcast the global minimum wwwaaaiiittt uuunnntttiiilll aaa ggglllooobbbaaalll mmmiiinnniiimmmuuummm iiisss aaavvvaaaiiilllaaabbbllleee
teractiveexploration.Foracloseranalysisoftheapplication
ofSOMsinthefieldofinformationretrieval,referto[7,8]. adjust weights of all local nodes
2.3 TheparSOMimplementation
Figure2.Aflowchartofthecomputationusinganasymmetricmodelofpar-
Two differentstrategiesforthe parallelizationofSOMs allelization:Twomajorsynchronizationpointsoccurinthisimplementation.
areavailable:
Vectorization. It is possible to use multiple processors or
A master thread selects an input vector and broadcasts
SIMD techniques to operate on the vector-components in
it to all its slave threads. All threads search for their local
parallel.Modernmicroprocessorsaresometimesfittedwith
minima, i.e. best-matching units concurrently. These best-
SIMDnodesforthebenefitofmultimediaapplications.Al-
matches are sent back to the master, which determines a
thoughSIMDunitscutdownthepureprocessingtime,the
globalmatch.Allslavesarenotifiedoftheglobalmatchand
loadonthememorybusisincreased.Specialcompilersup-
they modify the nodes relative to each node’s Eudclidean
portisnecessaryforthisoptimization.
distancetothisglobalmatchindependently.Thedatapack-
Partitioning.Themapispartitionedintomultiplesections, etsexchangedatthesynchronizationpointsareverysmall,
which are maintained by different processors. One of the consistingofaspatialcoordinateandanerrorvalueatmost.
majorbenefitofthisstrategyisthedistributionofrequired Thisalgorithmshouldscaleverywelltoanynumberofpro-
memoryacrossmultipleunits.Thisallowsbothforthehan- cessors in theory, as the search for the local minima and
dlingoffarlargermapsthanserialimplementationsandfor thetrainingprocessaccountforalmosttheentireexecution
afasterexecution,asmultipleprocessorsareused. time of the program. Remember, that both finding the lo-
For our current implementation, we use partitioning, cal minima and training involves operations on arrays of
withonemastercontrollingmultipleslaves.Asdepictedin largevectorsoffloatingpointnumbers.Forthisreason,the
Figure 2 three major synchronization points occur in this additionalworkofthemasterthreadisnegligibleanddoes
implementation. The data transfers are very small, as only notaffecttheoverallperformance.However,forsmallmaps
the offset of the input vector (within a replicated file) is andveryfastprocessors,scalabilityproblemsmayarisedue
broadcast;thedatapacketsforcomputingtheminimacon- tothesynchronizationoverheads.
sist of a Cartesian coordinate and a floating point number
3 AnalysisofExperimentalResults
signifyingtheerroratthatcoordinate.Theapproachchosen
inourworkisbasedonacommunication-orientedcompu- Table 1 summarizes the execution time for training the
tation and synchronization model. It uses a number of in- map using SOM. We used POSIX threads, a shared ad-
par
formation and condition broadcasts to synchronize its op- dressspaceanduser-levelspinlocksinourimplementation.
erations.Weviewamulti-dimensionalself-organizingmap The test were conducted on a number of multi-processor
asacollectionofunitswithassociatedcoordinatesallowing machineswithdissimilarmemoryarchitectures:
ustodistributetheunitsfreelybetweenthethreads. SGI PowerChallenge XL. A 16-processor system with a
Any subsets of the units can be easily represented in a shared-bus.Eachprocessorhasa2MBlevel2cachewitha
lineararray.BysegmentingtheSOMintomultipleregions, totalthroughputtothemainmemoryof1.2GB/s.
wecaneffectivelydistributethememoryusageacrossmul- SGI Cray Origin 2000. A 64-processor system which
tipleprocessingnodes.SincetrainingtheSOMrequiresthe adheres to a ccNUMA architecture using 32 SMP node
entire map segment to be operated on, the distribution of boards,whereeachprocessorhas4MBofcachememory;
the map across multiple processing nodes with indepen- Anentirelydifferentperformancecharacteristicwasob-
dentmemories(e.g.,onccNUMAarchitectures)reducesthe tained on our principal development platform. On the SGI
pressureonthememoryhierarchyandmakesitpossibleto PowerChallenge, which uses a shared memory architec-
fitthemapsegmentsentirelyintoL2cachesonthenodes. ture, a 22-fold increase in performance could be achieved
3
SGIPowerChallenge SGIOrigin2000 andhowtheseareimplementedinIRIX6.5.Insteadofal-
seconds seconds locating a separate heap for each thread, a shared heap is
threads elapsed speedup elapsed speedup used, which has its affinity hard-wired to the processor of
1 1555.591 1.0000 461.940 1.0000 the main thread. Understandably, performance degrades in
2 533.265 2.9171 209.790 2.2021 suchanenvironment,ifmanythreadsareused,asthetraf-
3 231.964 6.7069 150.223 3.0750 fic on the memory backbone increases. Nonetheless, the
4 180.035 8.6430 117.100 3.9448 speedup between the single-processor and dual-processor
5 128.025 11.3122 114.930 4.0178 implementation shows again how the availability of suffi-
6 117.228 13.2802 102.134 4.5216 cient cache memory to hold the entire SOM has a signif-
7 96.632 16.1030 90.817 5.0854 icant performance impact. We are currently working on a
8 91.481 17.0068 83.240 5.5493 MPI-basedimplementationtocreateanoptimalversionfor
9 83.333 18.6915 this architecture. We expect MPI to solve the performance
10 80.993 19.2307 problemonthisarchitecture,asitprovidesdistinct,cache-
11 76.701 20.2839 affineheapstothedifferentworkerprocesses.
12 70.390 22.1238
4 Optimizations
Table1.Executiontimesandspeedupsonmultiprocessorsystems Fromtheanalysisoftheresultspresentedintheprevious
100
actual speedup section it becomes apparent that the implementation does
90 linear speedup
not scale as expected on distributed memory systems and
80
for clusters, while providing considerable speed-ups and
me 70 exploiting cache effects due to unecessary communication
Relative execution tiM 34560000 tIvohnevecpestrueohtrei-nasfpedolureitncevtmaieoccuhntlo.tiirtpAselrulelasteipinxorgeoncc.aueTtrsiashoneendspoonhmreiagesnidenusamtolobfaaetlgrhgreogereoeintuheotmrenartalosonerolieipnnc:pttehudet
20 master thread. However, broadcasting this information to
10 all workers involved non-uniform latencies among differ-
0 ent nodes on distributed memory systems. Using multiple,
0 2 4 6 8 10 12
Number of processors used synchronized random number generators can alleviate this
Figure3.PerformanceonaSGIPowerChallenge:Cacheeffectsonashared-
memorysystemwithahighmemorybandwidthcanleadtoabetterthanlinear problem.
speedup. Integrationoflocalminima.Theoriginalcommunication
model was based on shared memory computing and inte-
with only 12 processors. Although this may seem para- grating the various local minima as soon as all workers
dox,itshouldbeexpected.Theperformanceincreasesvery had found their local minimum. However, with different
quicklyforthe firstfewthreads,untilthe amountofcache communicationlatencies and varying loads onthe compu-
memory exceeds the size of the map (compare Figure 3). tational nodes, unnecessary idle time was incurred on the
After that, the performance increase continues in a linear masterthread.
fashion. We provided time measurements up to 12 proces- As the local minimum of the the master thread is always
sors only, as the machine was not available for dedicated available and the search for the global minimum involves
testing.Theprecisionofthesemeasurementsformorethan multiplecomparisionsbetweentwolocalminima,theinte-
10 processors was also affected by the background load. grationofthelocalminimacanoccurincrementallywhen-
The scalability on this system remains unaffected by the ever another local minimum becomes available. However,
availablememorybandwidth:theentireSOMisdistributed thisassumesamessage-orientedcommunicationmodel.
throughthecachememoriesofthevariousprocessor,effec- Iterationsynchronization.Theworkersneedsynchroniza-
tivelyavoidingbuscontention.Interestingly,loopinversion tionregardingthecurrentiteration,asthelearningrate,the
leads to a large increase in performancefor the uniproces- neighbourhood radius and the input vector depend on the
sorversiononthismachine.Thisoptimizationresultedina currentiterationincurringcommunicationlatencies.
22%increaseinperformance. Inamessage-orientedcommunicationmodelwithreliable,
TheperformancegainsontheSGIOriginwerelessim- exact(i.e.non-overtakingandnon-duplicate)messaging,no
pressive than those on the PowerChallenge. Although the additionalsynchronizationisnecessaryforstartingorend-
currentimplementationscalesto8processorsandachieves inganiteration.
a5.5-foldperformanceincrease,itstilloffersroomforim- Byremedyingthesethreebottlenecks,astreamlinedversion
provement.ThisisduetoourdependenceonPOSIXthreads of SOMwasproduced.Onlyasinglesynchronizationpoint
par
4
L-CAFEJIFHJIECK AJBCDEGFHJIECK
5 LoadbalancingandSIMDprocessing
Although the current implementation performs remark-
broadcast random seed synchronize random seed
ablywell,anumberofpossibleimprovementsremaintobe
randomly select input-vector considered. Prominently, dynamic load balancing between
worker threads and the use of SIMD extensions of micro-
find local-minimum processorsandvectorcomputersremainasfuturework.
Dynamic load-balancing becomes a necessity, if non-
integrate local-minima incrementally broadcast local minimum
dedicatedcomputingresourcesareused,asthebackground
load may severely affect the performance of the system.
find the global minimum
Insuchenvironments,adynamicredistributionofthemap
broadcast the global minimum wait until a global minimum is available between nodes becomes necessary to provide peak perfor-
mance. We are currently investigating the use of a simple
adjust weights of all local nodes load balancing scheme, which measures the delay of the
various threads to complete their local computations and
Figure4.Theoptimizedalgorithmwithonlyonemajorsynchronizationpoint reassigns single nodes of the neural map from the slowest
remaining.
threadstothefastesthreads.However,asthisleadstoaddi-
tionaldatatransfersamongtheparticipatingcomputational
remains per iteration (see Figure 4), reducing the overall nodes,thresholdsfortheseoperationsneedtobedetermined
sensitivitytocommunicationlatencies. asnottoslowtheentiresystemdown.
Synchronizedrandomnumbergenerators. A second promising improvement planned for the near
The use of multiple synchronized random number genera- future is support for SIMD extensions to common micro-
torsposesaseriousproblem,ifmultipleheterogenoussys- processorsandvectorcomputers.Assumingahigh-enough
tems aretobeintegrated intoasingle resource,asitis the memorybandwidth,thespeedupfortheseshouldbelinear.
caseforclusters.Everyoperatingsystemmayprovideadif- It is of particular interest to us to test the combination of
ferentpseudo-randomnumbergenerator.Asaresult,weim- partitioningandvectorizationinasingletestrun.
plementedourownrandomnumbergeneratortoensuresyn-
6 Conclusions
chronizedoperation.ItisavariantoftheRC4streamcipher
featuringveryhighthroughput,anextraordinarilylongpe- Our experiments with parallel, software-based imple-
riodandasmallinternalstateofonly258bytes.Thissmall mentations of self-organizing maps for Digital Libraries
internal state has the benefit of not imposing on the scare haveshown that interactive response times—even for high
level2cachememory. dimensional feature spaces—can be achieved using super-
Adoptionofthemessagepassingparadigm. computing resources. The used algorithms scale well and
The use of a programming model oriented on shared- gain considerable performancedue tocache effects.Paral-
memory systems in the original version of SOM posed a lelismeffectivelybecomesamethodtoovercomethemem-
par
serious problem for porting the system, as condition vari- orylatenciesinself-organizingneuralnetworks.
ableswereused.Byadoptingthemessagepassingmetaphor
References
for the entire system, these condition variables were re-
[1] P.Adam,H.Essafi,M.-P.Gayrad,andM.Pic. High-performanceprogram-
duced to a much simpler model of synchronization. Addi- mingsupportformultimediadocumentdatabasemanagement. InHighPer-
tionally,thecommunicationdirectivesweremodelledafter formanceComputingandNetworking,1999.
[2] Y.Boniface,F.Alexandre,andS.Vialle. Abridgebetweentwoparadigms
MPI-primitives, supportingeasyportabilitytoawidevari- forparallelism:Neuralnetworksandgeneralpurposemimdcomputers. In
etyofplatforms.Basically,onlytwodirectivesarerequired: InternationalJointConferenceonNeuralNetworks,1999.
[3] G.DeBoeckandT.Kohonen.VisualExplorationsinFinance.SpringerVer-
broadcastandsend.Thesemanticdifferencebetweentheir lag,Berlin,Germany,1998.
implementation using MPI and their original implementa- [4] D.Hammerstrom. AVLSIarchitectureforhigh-performance,low-cost,on-
chiplearning.InInternationalJointConferenceonNeuralNetworks,1990.
tionisthatMPIcreateslocalcopiesofabroadcastmessage [5] T.Kohonen.Self-OrganizingMaps.SpringerVerlag,Berlin,Germany,1995.
[6] J.LiuandM.Brooke.Afullyparallellearningneuralnetworkchipforreal-
foreachreceiver.Thereforeadestructivereadoperationcan
timecontrol.InInternationalJointConferenceonNeuralNetworks,1999.
remove the message from the queue, simplifying the syn- [7] A.RauberandD.Merkl.Findingstructureintextarchives.InProc.European
Symp.onArtificialNeuralNetworks(ESANN98),Bruges,Belgium,1998.
chronizationlogic. [8] A. Rauber and D. Merkl. Usingself-organizingmaps toorganize docu-
Iterativeintegrationoflocalminima. mentcollectionsandtocharacterizesubjectmatters. InProc.10thConf.on
DatabaseandExpertSystemsApplications,Florence,Italy,1999.
Theorderthatlocalminimabecomeavailableinisindeter- [9] O.Simula,P.Vasara,J.Vesanto,andR.Helminen. Theself-organizingmap
ministic. For this reason, a queue of local minima is used inindustryanalysis.InL.JainandV.Vemuri,editors,IndustrialApplications
ofNeuralNetworks,Washington,DC.,1999.CRCPress.
inthemasterthread.Usingablockingreadonthismessage [10] P.Tomsich,A.Rauber,andD.Merkl. parSOM:Usingparallelismtoover-
comememorylatencyinself-organizingneuralnetworks. InHighPerfor-
queue the next local minimum is retrieved and integrated
manceComputingandNetworking,2000.
intotheglobalminimumcalculation.
5
