Review Article Military Simulation Big Data: Background, State of the Art, and Challenges - Hindawi

21 Pages·2015·3.86 MB·English

Hindawi Publishing Corporation
Mathematical Problems in Engineering
Volume 2015, Article ID 298356, 20 pages
http://dx.doi.org/10.1155/2015/298356
Review Article
Military Simulation Big Data: Background,
State of the Art, and Challenges
XiaoSong,YulinWu,YaofeiMa,YongCui,andGuanghongGong
SchoolofAutomationScienceandElectricalEngineering,BeihangUniversity,No.37XueYuanRoad,
HaiDianDistrict,Beijing100191,China
CorrespondenceshouldbeaddressedtoXiaoSong;[email protected]
Received5July2015;Revised5November2015;Accepted11November2015
AcademicEditor:AlessandroGasparetto
Copyright©2015XiaoSongetal. This is an open access article distributed under the Creative Commons Attribution License,
whichpermitsunrestricteduse,distribution,andreproductioninanymedium,providedtheoriginalworkisproperlycited.
Bigdatatechnologyhasundergonerapiddevelopmentandattainedgreatsuccessinthebusinessfield.Militarysimulation(MS)
is another application domain producing massive datasets created by high-resolution models and large-scale simulations. It is
usedtostudycomplicatedproblemssuchasweaponsystemsacquisition,combatanalysis,andmilitarytraining.Thispaperfirstly
reviewedseverallarge-scalemilitarysimulationsproducingbigdata(MSbigdata)foravarietyofusagesandsummarizedthe
maincharacteristicsofresultdata.Thenwelookedatthetechnicaldetailsinvolvingthegeneration,collection,processing,and
analysisofMSbigdata.Twoframeworkswerealsosurveyedtotracethedevelopmentoftheunderlyingsoftwareplatform.Finally,
weidentifiedsomekeychallenges andproposedaframeworkasabasisforfuturework.Thisframeworkconsideredboththe
simulationandbigdatamanagementatthesametimebasedonlayeredandserviceorientedarchitectures.Theobjectiveofthis
reviewistohelpinterestedresearcherslearnthekeypointsofMSbigdataandprovidereferencesfortacklingthebigdataproblem
andperformingfurtherresearch.
1.Introduction and Constructive simulations [8]. Besides, all kinds of data
about combat entitiesand events in battlefield are collected
Bigdatatechnologyisanemerginginformationtechnology together.
which discovers knowledge from large amounts of data to Modelsandsimulations(M&S)areoftenclassifiedbyUS
support decision-making. “Big” implies that the resulting DoD into four levels: campaign (theater), mission, engage-
datasetistoolargetobehandledwithtraditionalmethods. ment,andengineering,usuallydepictedasa“M&Spyramid”
Moreover,thedataproductionvelocityistypicallyfastwith [9, 10]. M&S applications span all levels of this pyramid,
various sources [1]. Big data methodology is even regarded with campaign models and simulations being applied in
as the fourth paradigm of exploring the world, since it is warfare analysis, mission-level simulations in such areas as
differentfromexperimentation,theoreticalapproaches,and JointOperationsforanalysisorexercise[11–15],engagement
computational science in terms of knowledge acquisition simulationsinconfrontationofsystemofsystemsforwarrior
methods[2]. training, and engineering-level models and simulations in
Recently, big data and simulation have been linked by equipmentacquirementanddevelopmentfortest andeval-
researchers to perform scientific discovery [3, 4]. Military uation(T&E).
domainhasalsodrawnagreatdealofattentiontothistrend. When these manifold models and simulations are exe-
The United States (US) Department of Defense (DoD) is cuted with high performance computing (HPC) to gain
carryingoutaseriesofbigdataprograms(e.g.,XDATA)to high efficiency, simulation data has the great potential to
enhancedefensecapabilities[5].Generally,militaryapplica- be generated with high volume and rapid speed. Recently,
tionsareproducingmassiveamountsofdatawithplentyof theterm“bigsimulation”wascoinedbyTayloretal.[16]to
Intelligence,Surveillance,andReconnaissance(ISR)sensors describethechallengethatmodels,data,andsystemsareso
[6, 7], and the data can also be generated by Live, Virtual, excessiveinscalethatitisdifficulttogovernthesimulation.
2 MathematicalProblemsinEngineering
Accordingly, we regard military simulation data as “MS big enable knowledge discovery. The term implies the trend of
data”todescribethehighvolumeofdatageneratedbymil- data explosion and its potential value for present society.
itarysimulations. DatascientistsoftenuseN-V(volume,velocity,variety,value,
MS big data can be produced by numerous simulation veracity,etc.)dimensionstoaccountforbigdata.
replications containing high-resolution models or large-
scale battle spaces or both. The data size increases rapidly Volume.Thesizeofbigdataisnotexactlydefined.Itvaries
with larger simulation and higher performance computer from different fields andexpands over time. Many contem-
resources. This poses significant challenges in management poraryInternetcompaniescangenerateterabytesofnewdata
andprocessingofMSbigdata.First,greateffortshavebeen everydayandtheactivedatabaseinusemayexceedpetabyte.
made to address the requirements from high performance Forexample,Facebookstored250billionphotosandaccessed
simulation,whileonlyafewistowarddataprocessing.How- morethan30 petabytesofuserdataeachday [21];Alibaba,
ever,theprocessingofMSbigdatamakessomedifferences thebiggestelectroniccommercecompanyintheworld,had
betweenbusinessdata;forexample,thecomputingresources storedover100petabytesofbusinessdata[22].Largevolume
areoftenusedfordataanalysistogetherwithsimulationexe- is the basic feature of big data. According to International
cutionandthismayleadtomorecomplexresourceschedul- Data Corporation (IDC), the volume of digital data in the
ing.Second,newrequirementswillemergewithavailability world reaches zetabytes (1 zetabyte = 230 terabytes) in 2013;
of high fidelity models and timeliness of large amounts of furthermore, it almost doubles every two years before 2020
data. Traditional data analytic methods are limited by tra- [23].
ditionaldatabasetechnologieswithregardtoefficiencyand
scalability. Now big data based new applications allow mil- Velocity.Dataexplosionalsomeansthatthedatageneration
itary analysts and decision-makers to develop insights or speedisveryquickandtheymustbeprocessedtimely.Take
understand the landscape for complex military scenarios Facebook, for example, again, millions of content items are
instead of being limited to only examining experimental shared and hundreds of thousands of photos are uploaded
results. An example is supporting the real-time planning every minute of the day. To serve one billion global users,
of real combat by simulating all kinds of possibilities. This over100terabytesofdataarescannedevery30minutes[24].
requires iterative simulation and analysis of result in very Velocity is also a relative concept and depends on practical
shorttime. application. For many Internet commerce applications, the
This paper serves as an introduction to the leading processed information must be available in a few seconds;
edge of MS big data. There are already many review papers otherwisethevalueofdatawilldiminishorbelost.
discussingtheconceptofbigdataanditsunderlyingtechnical
frameworks[17–19].However,fewofthemfocusonmilitary Variety. Big data has diverse types and formats. Tradition
simulation, yet related researchers should be interested in structured data saved in database possess regular format,
thegeneration,management,andapplicationofMSbigdata. that is, date, time, amount, and string, while unstructured
We will study some practices and development progress data such as text, audio, video, and image are main styles
by literature so that a general picture of MS big data can of big data. The unstructured data can be web page, blog,
be drawn. Meanwhile, for unique characteristics of MS big photo, comment on commodity, questionnaire, application
data, we will identify some technique challenges posed by log, sensor data, and so forth. These data express human-
the processing of MS big data. We will also demonstrate a readable contents but are not understandable for machine.
preliminaryframeworkaspossiblesolution. Usually, they are saved in file system or NoSQL (Not Only
Theremainderofthispaperisorganizedasfollows.The SQL) database which has simple data model such as Key-
next section introduces the background of MS big data. Value Pair. Variety also means that big data have various
Severalpracticalcasesarereviewed,alongwithasummaryof sources.Forexample,thedatafortrafficanalysismaycome
thecharacteristicsofMSbigdata.InSection3,thedetailed fromfixednetworkcamera,bus,taxi,orsubwaysensors.
advancementsintechnologyarediscussed.InSection4,we
investigatetwoplatformscloselyrelatedtoMSbigdata.Cur- Value. Big data can produce valuable insight for owner.
rentchallengesareidentifiedinSection5,alongwithapro- The insight help to predict the future, create new chance,
posed framework to integrate simulation and big data pro- and reduce risk or cost. As a result, big data can change
cessing.Finally,weconcludethepaperinSection6. or improve people’s life. A famous example of mining big
data is that Google successfully forecasted flu according to
2.Background the 50 million search records. Another example is that the
commodityrecommendationcanbefoundeverywherewhen
This section surveys the background that MS big data wesurfinInternetnowadays.Theserecommendationitems
problememerged,butfirstlywemustaddresswhatbigdata are generated according to large amounts of records about
is in general. Then we review several military simulations user access. Note that the value of big data is sort of low
from viewpoint of data. Finally the features of MS big data densityandmustbeextractedfromitshugevolume.
arediscussedandcomparedwiththoseinbusiness.
Veracity. Veracity means only that the trustworthy data
2.1.ConceptofBigData. Bigdatareferstothedatasetwhich shouldbeused;otherwisethedecisionmakermaygetfalse
is so huge that new processing methods are required to knowledgeandmakewrongdecision.Forexample,theonline
MathematicalProblemsinEngineering 3
review from customer is important to ranking system of experiments.Inthiscase,newdatamanagementtoolbased
commodity,andifsomepeoplemakefakecommentsorscore ongridcomputingtechnologywasproposedtoaddressthe
deceptivelyforprofit,theresultwillinfluencenegativelythe problemfromdataincrease[14].
ranking and customer’s choice. Veracity requires those fake
datatobedetectedandeliminatedpriortoanalysis. 2.2.2.DataFarmingProjects. Datafarmingisaprocessusing
BigdataemergedfromnotonlyInternetsocialmediaor numeroussimulationswithhighperformancecomputingto
commercebutalsogovernment,retailing,scientificresearch, generate landscapes of potential outcomes and gain insight
nationaldefense,andotherdomains.Nowadaystheyareall fromtrendsoranomalies[15].Thebasicideaistoplantdata
involvedinprocessingofmassivedata.Thebloomofbigdata in the simulation through various inputs and then harvest
application is driving the development of new information significantamountsofdataassimulationoutputs[27].Data
technology,suchasdataprocessinginfrastructure,dataman- farmingwasfirstappliedintheAlbertproject(1998–2006)of
agementtool,anddataanalysismethod.Recenttrendinthose theUSMarineCorps[28].Theprojectsupporteddecision-
enterprises and organizations owning large dataset in data makingandfocusedonquestionssuchas“whatif”and“what
centerisbecomingacorepartofinformationarchitecture,on factors are most important.” These kinds of questions need
whichscalableandhighperformancedataprocessingframe- holisticanalysiscoveringallpossiblesituations;howeverthe
work is running. Many of these frameworks are based on traditionalmethodsareunabletoaddressthembecauseone
ApacheHadoopecosystem,whichusesMapReduceparallel simulationprovidesonlyasingularresult[20].Bycontrast,
programming paradigm. At the same time, many new data data farming allows for understanding the entire landscape
miningandmachinelearningalgorithmsandapplicationsare bysimulatingnumerouspossibilities.
proposedtomakebetterknowledgediscovery.
The number of simulation instances running concur-
rentlythroughHPCusuallyislarge.Forexample,aForcePro-
2.2.MSBigDataCases. Thephenomenonproducingmassive tection simulation of German Bundeswehr created 241,920
data in military simulation can be traced back to the 1990s replicates[29].Furthermore,millionsofsimulationrunscan
when STOW97 was a distributed interactive simulation for be supported by the latest data farming platform built on
military exercises. It incorporated hundreds of computers heterogeneousresourcesincludingclusterandcloud[30].As
andproduced1.5TBofdataafterrunning144hours[25].As aresult,massivedatacouldbeproducedforanalysis[20].For
simulationtechnologyadvances,manymilitaryapplications example,theAlbertprojectgeneratedhundredsofmillionsof
areproducingmultipleterabytesofdataormore. datapointsinitsmiddlestage[31].
During the Albert project, many countries in the world
have leveraged the idea to study all kinds of military
2.2.1. Joint Forces Experiments. When DoD realized the
problems.Severalsimulationsystemsfordatafarmingwere
challenges of contemporary urban operation, the US Joint
established.SomeexamplesareMANAofNewZealand[32],
ForcesCommandcommissionedaseriesoflarge-scalecom-
PAXSEM of Germany [33], and ABSNEC of Canada [34].
bat simulations to develop tactics for the future battlefield
The research fields involved command and control, human
andevaluatenewsystems(e.g.,ISRsensors)deployedinan
factors, combat and peace support operations net-centric
urbanenvironment.AtypicalexperimentwasUrbanResolve
combat,andsoforth.
with three phases across several years [11]. It used the Joint
SemiautomatedForces(JSAF)systemtostudythejointurban
operation,setinthe2015–2020timeframe.JSAFisatypeof 2.2.3.CourseofActionAnalysis. Inordertoeffectivelycom-
ComputerGeneratedForces(CGF)softwarewhichgenerates pletemissionplanning,itiscrucialtorecognizecertainkey
andcontrolsvirtualinteractiveentitiesbycomputers. factors of the battle space via simulations. It is especially
Inthefirstphase,morethan100,000entities(mostwere important that the military commander evaluates possible
civilian) were simulated. Hundreds of nodes running JSAF plansandmultipledecisionpoints.Thiskindofexperiment
wereconnectedacrossgeographicalsites,and3.7TBofdata iscalledsimulation-basedCourseofAction(COA)analysis
was collected from models [12]. These data are relevant to and needs to test many situations with a large parameter
the dynamic environment, operational entities (including value space [35]. The simulation always runs faster than
liveandconstructive),andsensoroutputs[26].Theanalysis real-time, and it can be injected with real-time data from
involvedqueryandvisualizationforalargedataset,suchas actual command and control systems and ISR sensors. For
theKiller/VictimScoreboard,EntityLifecycleSummary,and high-levelCOAanalysis,alow-resolution,largeentity-count
TrackPerceptionMatrix. simulationisused.Furthermore,adeeperandmoredetailed
However, more data about civilian status which were COA analysis needs models with higher resolution.During
saved dozens of times were simply discarded because of peacetime, the COA can be planned carefully with more
unaffordableresources.Therefore,civilianactivitycouldnot details.Butduringacrisis,theCOAplanmustbemodified
beduplicatedforanalysis.Meanwhile,thedemandsforlarger asnecessaryinaveryshortamountoftime[36].
and more sophisticated simulations were increasing. With TheUSarmyusesCOAanalysisinoperationsaimingat
morepowerclusters,graphicsprocessingunit(GPU)accel- urbanenvironments.OneSAF(OneSemiautomatedForces)
eration, and high-speed wide area networks (WANs) using isasimulationsystemwhichfulfillsthistypeofrequirement.
interest-managedrouters[13],tensofmillionsofentitiesand As the latest CGF system of the US Army, it represents the
higher-resolution military models were supported in later stateoftheartinforcemodelingandsimulation(FMS)[37],
4 MathematicalProblemsinEngineering
includingcapabilitiessuchasbehaviormodeling,composite hoursintermsofwall-clocktime.Thesimulationwassuper-
modeling,multiresolutionmodeling,andagent-basedmod- real-time,andthesimulatedtimeitselfcouldbe90days.The
eling.Itisabletosimulate33,000entitiesatthebrigadelevel original simulation results data reached the terabyte level.
[38] and has been ported to HPC systems to scale up with Its analysis was measure of performance (MOP) based on
higherresolutionmodels[39].Realsystemcanbeintegrated recognizedparametersfromdailyoperations.Thesystemwas
with OneSAF through adapters so that the simulation is a networked program based on a load-sharing architecture
enhanced. Massive data can be generated to analyze and which was scalable and could include heterogeneous com-
comparedifferentCOAplans.Inthiscase,datacollectionand putationalresources.Toreducetheexecutiontime,SSNAM
analysisareidentifiedasitscorecapabilities[40]. has been ported to an HPC system and gained three times
The Synthetic Theater Operations Research Model increaseinperformanceasaresult.
(STORM) is also an analysis tool applicable to COA. It
simulates campaign level and is used by US naval force 2.2.6.TestandEvaluationoftheTerminalHighAltitudeArea
and air force. It can create gigabytes of output from single DefenseSystem. ThemissionoftheTerminalHighAltitude
replication, and one simulation experiment may contain a AreaDefense(THAAD)systemwastoprotecttheUShome-
set of replications running from several minutes to hours, landandmilitaryforcesfromshort-rangeandmedium-range
dependingonthecomplexityofscenarioandmodels[41,42]. ballistic missiles. The test and evaluation of the THAAD
system were challenged with analyzing the exponentially
2.2.4.AcquisitionofNewMilitarySystems. Australia’sacqui- expanding data collected from missile defense flight tests
sition of naval systems [43, 44] employed modeling and [46]. The system contained a number of components (e.g.,
simulation (M&S) to forecast the warfare capabilities of radars,launchers,andinterceptors)andwashighlycomplex
antiair, antisurface, and antisubmarine systems since the and software-intensive. Two phases have been evaluated
real experiment costs high or was unavailable. The M&S by experiment: deployment and engagement. The THAAD
played a significant role in the acquisition lifecycle of new program incorporated the simulation approach to support
platforms, such as missiles, radar, and illuminators. All systemlevelintegratedtestingandevaluationinreal-time.In
phases,includingrequirementsdefinition,systemdesignand this case, simulation was used to generate threat scenarios,
development, and test and evaluation, were supported by including targets, missiles, and environmental effects. In
M&S using computer technology. The simulation software addition,M&Swasalsousedfornormalexercises.
containedhighlydetailedmodels,whichcanbeusedforboth The THAAD staff developed a Data Handling Plan to
constructive and virtual simulation at a tactical level. The reduce,process,andanalyzelargeamountsofdata.ATHENA
simulation scenarios were defined by threat characteristics, software was designed to manage the terabytes of data
threatlevels,andenvironmentalconditions. generated by flight tests and concomitant simulations. The
ATHENA engine imported various files such as binary,
Theprojectspannedacrossseveralyears,involvingmulti-
comma-separated values, XML, images, and video format.
plephasesandstages.Italsoinvolvedmultipleorganizations
ThedatasourcescouldcomefromLAN,WAN,oracrossthe
acrossmultiplesites.Alarge-scalesimulationwithhundreds
Internet.
ofscenarios(eachscenarioranhundredsoftimeswithMonte
Carlo simulation) was executed on IBM blade servers and
2.3. Characteristics and Research Issues of MS Big Data.
terabytesofdataweregenerated.Complexrequirementsfor
Table1summarizesthemainfeaturesofMSbigdatawithin
analyses of the large dataset, such as verifying assump-
theabovecases.
tions, discovering patterns, identifying key parameters, and
We can draw the main characteristics of MS big data
explaining anomalies, were put forward by subject matter
fromtheabovecasesintermsofvolume,velocity,variety,and
experts. These works generated Measures of Effectiveness
veracity.
(MOE)todefinethecapabilitiesofthenewsystems.
Volume and Velocity. Table1 shows the data sizes. Almost
2.2.5. Space Surveillance Network Analysis. The SSNAM all cases are at the level of GB to TB per experiment (see
(Space Surveillance Network and Analysis Model) project Table1(b)). These simulation cases’ data volume is smaller
was sponsored by the US Air Force Space Command, and comparedwithcommerceandsocialmediaoninternetand
it used simulations to study the performance and charac- web, because simulation experiment can be well designed
teristics of the Space Surveillance Network (SSN) [45]. The and the needed data for analysis can be chosen carefully to
purposewasanalyzingandarchitectingthestructureofthe save. It suggests that the value density of simulation data
SSN.Anumberofconfigurationoptions,suchasoperation is higher. Another reason is that simulation data are not
time,trackcapacity,andweathercondition,wereavailablefor accumulatedacrossexperiments;differentexperimentshave
allmodeledsensorsintheSSN. different objectivities and are seldom connected to do data
OneresearchproblemwasCatalogMaintenance.SSNAM analysis.
provided capabilities to assess the impacts from catalog However, the simulation data size continues to increase
growthandSSNchangesrelatedtoconfigurationandsensors sharply because military simulation is advancing rapidly
(e.g.,addition,deletion,andupgrade).Thiskindofsimula- with bigger scale and higher resolution under the impetus
tionwasbothcomputationallyanddataintensive.Atypical from modern HPC system. Moreover, most business data
run simulated several thousands of satellites within a few are produced in real-time [47], but MS big data needs to
MathematicalProblemsinEngineering 5
Table1:Overviewofmilitarybigsimulations.
(a) Simulation-relatedproperties
Simulationlevel Thenumberof Thenumberof Model Simulation
Cases Timeadvancing
C M E G entities simulationreplications resolution executiontime
Jointforces ✓ ✓ ✓ Millions Several High Real-time Weeks
experiments
Datafarming ✓ ✓ ✓ Dozens∼hundreds Thousands∼millions Low Faster-than-real-time Hours
projects
COAanalysis ✓ ✓ ✓ Thousands Hundreds∼thousands Low Faster-than-real-time Minutes
medium hours
Acquisitionof
newmilitary ✓ ✓ Afew Tensofthousands High Faster-than-real-time Daysorweeks
orreal-time
system
SSNanalysis ✓ Tensofthousands Several High Faster-than-real-time Days
Testand
Evaluationofthe ✓ ✓ Afew Several High Real-time Months
THAADsystem
Simulationlevel:C,campaign;M,mission;E,engagement;G,engineering.
(b) Data-relatedproperties
Datageneration Datascaleper
Cases Datacontents Typicalanalyticalapplication
period experiment
Jointforces Sensordata,entitystatus(takeUrban EffectivenessofISRsensor.Statistical
1ms TB
experiments Resolve,e.g.) resultofmissionexecution
Dependonspecificproblemandcan
Correlation,trend,andoutlieranalysis
involveallkindsofphysicaland
Datafarming behavioraldataofsimulatedentities <1ms GB∼TB relatedtocommandandcontrol,
projects peacekeepingoperation,combat,and
andbattlefieldevents,suchasdamage
soforth
andsurvival
COAanalysis Civilbehavior,culture,weapon, <1sec GB∼TB Taskexecutionanalysis,operational
terrain,killing,victims,andsoforth effectivenessanalysis
Identifythekeyperformance
parameter,systemleveltrade-off
Acquisitionofnew Communication,sensor,command
N/A TB analysis,andMOEaboutraid,protect,
militarysystem andcontrol,weapon,andsoforth
detect,reaction,tracking,engagement,
andsoforth
SSNanalysis Sensordataandspecialeventssuchas <1sec TB MOPofsensordevice
launch,breakup,andin-orbit
Testandevaluation Maneuver,sustainment,commandand
oftheTHAAD control,communication,radartrack N/A TB MOE,MOPofdefensesystem
system data,andsoforth
begeneratedandanalyzedmanytimesfasterthanreal-time production.Fortime-sensitiveapplications,suchassituation
whentheobjectiveistorapidlyassessasituationandenhance awarenessorcommandandcontrol,bigdataisinjectedinto
decision-making. This requires simulation time to advance thesimulationanalysissystemintheformofastream,which
faster than real-time (see Table1(a)), and sometimes the requiresthesystemtoprocessthedatastreamasquicklyas
simulation generates data in a period of less than 1ms (see possibletomaximizeitsvalue.
Table1(b)).
Highvolumeandvelocityposetwoaspectsofchallenges Variety. Large-scale simulations can be built based on the
to MS big data applications. First, collecting massive data theory of system of systems (SoS) [48], which consists of
fromdistributedlarge-scalesimulationsmayconsumeextra manifoldsystemmodelssuchasplanes,tanks,ships,missiles,
resources in terms of processor or network, which is often andradars.Figure1presentsvariouskindsofdatainvolvedin
critical for simulation performance. Thus it is essential to militaryexperiments.
designahigh-speeddatacollectionframeworkthathaslittle FromFigure1,wecanobservethediversityofMSdata.
impactonthesimulationperformance.Second,thedatasets Forexample,MScanbelinkedwithlivepeople(e.g.,human-
must be analyzed at a rate that matches the speed of data in-the-loop) and live military systems (e.g., command and
6 MathematicalProblemsinEngineering
Military
simulation
Entity runtime data
Environment
(soldier, plane, vehicle, Scenario data Input data
data
ship, etc.)
Equipment Live data
Human Event data Mission data performance data (C4I, human, etc.)
Physical data
behavioral data
Forces ··· Natural data Social data
deployment data
Maneuver Sensor Intelligence Weapon control Terrain Weather Atmosphere Culture Political
Fire data
data data data data data data data data data
Communication ··· Decision-making data ··· Marinedata ··· Economics ···
data data
Figure1:Variouskindsofdatainsimulation-basedmilitaryexperiment.
controldevices).Assuch,valuableanalyticMSdataincludes
outputsofcomputationalmodels,aswellashumanactivity (1) Data generation (2) Data management (3) Data analysis
anddevicedata.Thedataformatsincludeunstructured(e.g.,
simulation log file), semistructured (e.g., scenario configu- (1.1) Modeling (2.1) Collection (3.1) Statistics
ration and simulation input), and structured (e.g., database (1.2) High performance
table)types.Allofthesefeaturesrequireversatileandflexible simulation (2.2) Storage (3.2) Data mining
toolstobedevelopedtominevaluefromthedataeffectively. (1.3) Distributed simulation (2.3) Processing (3.3) Visualisation
Thisimposesdifficultyonthedataprocessingtechnology.
Figure2:TechnologymapofMSbigdata.
Veracity. Veracity means that trustworthy data should be
created during the simulation. Because simulation data is
generatedbycomputerbutnothuman,thedatacouldnotbe 3.1.DataGeneration
fakebutcanbeincorrectbecauseofflawedmodel.Veracity
3.1.1. Combat Modeling. Modeling is a key factor to the
requiresthatthemodelandinputdatashouldbeverifiedand
veracityofMSbigdata[28].Incombatmodeling,themain
validated.However,thedifficultyisthatMSbigdataisoften
aspectsarephysical,behavioral,environmental,andsoforth,
involvedinhumanbehavior,whichisintelligent,yetintangi-
andthebehaviormodelisthemostsophisticatedpart.
bleanddiversebynature.Comparedwithanalyticphysicalor
Many research efforts focus on developing a cognitive
chemicalmodels,thereisnoprovenformulathatcanbeused
modelofhumans,aHumanBehaviorRepresentation(HBR),
forbehaviormodeling.Currentlyabehaviormodelcancreate
whichgreatlyaffectsthefidelityandcredibilityofamilitary
dataonlyaccordingtolimitedrulesrecognizedbyhumans.
simulation. HBR covers situation awareness, reasoning,
Therefore,thefidelityofsimulationmodelsisakeychallenge
learning,andsoforth.Figure3showsthegeneralprocessof
fortheveracityofMSbigdata.
humanbehavior.Recentresearchprojectsincludethefollow-
ThenextsectionwillgiveanoverviewoftheMSbigdata
ing: situation awareness of the battlefield [49]; decision-
technology, where we can see how the above problems are
making based on fuzzy rules, which captured the approxi-
resolvedtosomeextent.
mateandqualitativeaspectsofthehumanreasoningprocess
[50];commoninferenceengineforbehaviormodeling[51];
3.StateoftheArtofMSBigDataTechnology intelligentbehaviorbasedoncognitionandmachinelearn-
ing[52];modelingculturalaspectsinurbanoperations[53];
Fromtheviewpointofdatalifecycle,wecandividethesim- modeling surprise, which affects decision-making capabil-
ulation process into three consecutive phases: data gener- ities [54, 55]. Although it is still difficult to exhibit realistic
ation, data management, and data analysis. Figure2 shows human behaviors, the fidelity of these models can be en-
thedetailedtechnologymap.Datagenerationconcernswhat hancedbyadoptionofbigdataanalytictechnologies.
kindsofdatashouldbecreatedandhowtocreatevaliddata Meantime, some researchers focus on the emergent
inareasonableamountoftime.Datamanagementinvolves behaviorsofforcesasawhole.Abattlefieldiscoveredbyfog
howtocollectlargeamountsofdatawithoutdisturbingthe due to its nature of nonlinearity, adaptation, and coevolu-
normalsimulationandprovideavailablestorageandefficient tion [56]. Agent-based modeling (ABM) is regarded as a
processing capability. Data analysis utilizes various analytic promised technique to study such complexity [57] by sim-
methodstoextractvaluefromthesimulationresult. ulatingautonomousindividualsandtheirinteractionssothat
MathematicalProblemsinEngineering 7
andthesoftwareprovidesadvancedcapabilities,suchastime
Human behavior
synchronization based on an extra collaborative network
Memory Decision and efficient scheduling based on discrete events. Another
approach is to port an existing simulation system onto a
Perceived
truth common HPC platform. OneSAF had a good experience
Human withthisapproach[62–64].Inaddition,enhancingamodel’s
Situation factor
evaluation Plan performance (e.g., at performing line-of-sight calculations)
usingaGPUacceleratorisalsoaresearchhotspot[65].
Reactive
Sense Action
behavior 3.1.3. Distributed Simulation. Distributed simulation often
usesmiddlewaretointerconnectvarioussimulationresour-
Stimulate (ground truth) Response
ces,includingconstructive,virtual,andlivesystems.Middle-
waremeansthatthetierbetweenhardwareandsoftwareusu-
Environment allyisanimplementationofasimulationstandardforinter-
operation.Forhistoricalreasons,several standards are now
Figure3:Theprocedureofhumanbehavior. being used: distributed interactive simulation (DIS), high-
level architecture (HLA), and Test and Training Enabling
Architecture (TENA). HLA is an upgraded standard for
the whole system can be evaluated. The typical interactive distributedsimulationcomparabletoDIS.TENAisusedto
behaviors are command and control (C2), cooperation and testmilitarysystemsandpersonneltraining,butHLAisnot
coordination, and offensive and resistance. Recent research limitedtothemilitarydomain.
effortsincludelogicalagentarchitecture[58],artificialintel- Middleware enables large-scale simulations to execute
ligence based on agent [59], and multiagent framework for with a large number of entities from different nodes, such
war game [57, 60]. An interesting point is that agent-based as HPC resources. Middleware is the key to scalability of
systemsareconsideredtohavestrongconnectionswithbig the simulation, and many researchers have focused on the
data [16], because ABM provides a bottom-up approach performanceimprovementsitmakespossible.Forexample,
in which modeling large amounts of individuals and their enhancementworkontheRuntimeInfrastructure(RTI,the
emergent behavior must be identified from the big data softwareimplementationofHLA)withregardtoQualityof
generated. Serviceanddatadistributionmanagementisveryinteresting
becauseitmakesproperuseofHPCresources[68].
However,theinterconnectedapplicationsmayintroduce
3.1.2. High Performance Simulation. To address the enor-
geographically distributed data sources. This also poses the
mous computational requirements of large-scale military
complexity of managing and exploiting MS data for large
simulations, a high-performance computing (HPC) tech-
regions. To tackle this problem, [14] proposed a two-level
nique is employed as a fundamental infrastructure [61–65].
data model: the original collected data were organized by
Forexample,DoDhasdevelopedtheMauiHighPerformance
theloggingdatamodel(LDM)andthentransferredintothe
ComputingCenter(MHPCC)system,whichincludeslarge-
analysisdatamodel(ADM).ADMrepresentedthenotionof
scale computing and storage resources, to support military
ananalystandwasdefinedasMeasuresofPerformance(e.g.,
simulations.Inanothertypicalexample,thethree-levelparal-
sensoreffectiveness),whichinvolvedmultipledimensionsof
lelexecutionmodel(Figure4)wasproposed[61].InFigure4,
interests(e.g.,sensortype,targettype,anddetectionstatus).
thetoplevelisparallelexecutionofmultiplesimulationappli-
Another approach is to establish a standard format which
cations,themiddlelevelisparallelexecutionofdifferententi-
coversallkindsofdatainspecificapplications[69,70].
tieswithinthesamesimulationapplication,andthebottom
levelisparallelexecutionofdifferentmodelsinsideoneentity.
In spite of its huge processing power, the cluster-based 3.2.DataManagement
HPC systems still remain considerably challenged to han-
dle the data-intensive issues [66] presented by simulation 3.2.1. Data Collection. When a large-scale simulation is
applications. To address this issue, Hu et al. [67] proposed executed by an HPC system, the compute-intensive models
developingadvancedsoftwarespecifictosimulationrequire- cangenerateadelugeofdatatoupdateentitystateandenvi-
ments because there is a lack of software and algorithms ronmentconditions[71].Thetransferandcollectionofdata
to handle large-scale simulations. Also, many in the field generatedata-intensiveissues.Asthedataarecreatedbysim-
thinkthatsimulationmanagementsoftwareshouldprovide ulation programs, the collection is executing as simulation
functionssuchasjobsubmission,taskdeployment,run-time logging.Thisloggingisundergoinganevolutionarychange
monitoring,jobscheduling,andloadbalancing. fromastandaloneprocesstoafullydistributedarchitecture
Some approaches have begun to address this problem. (Figure5).
Forinstance,anewHPCsystemnamedYH-SUPEhasbeen Inthestandaloneloggermethod,oneorseveralnodesare
implementedtosupportsimulation[61]byoptimizingboth establishedtoreceiveandrecordthepublisheddataviathe
hardwareandsoftware.Thehardwarecontainsspecialcom- network[72].However,thisapproachaggravatestheshortage
ponents used to speed up specific simulation algorithms, of network resources because it requires large amounts of
8 MathematicalProblemsinEngineering
Parallel execution of Experiment
simulation sample
Parallel
execution of
entities Simulation Simulation
Parallel
execution of
inner entity Entity Entity Entity Entity Entity Entity
CPU CPU ··· CPU ··· CPU CPU ··· CPU
Figure4:Three-levelparallelexecutionofsimulationexperiment.
Stand-alone Distributed
Simulation Simulation
Simulation Simulation
Logger Logger
Data Data
Network Network
Simulation Logger Simulation
Logger
Logger
Logger
Data
Data
Figure5:Changefromstand-alonetodistributedloggingarchitecture.
data to be transferred. When the logger is moved into the calls, and this way ensured the independence of data col-
HPC platform, the network communication bottleneck is lection.WuandGong[74]proposedarecordingtechnique
alleviatedbyusinginter-memorycommunicationorahigh- with double buffering and scheduled disk operations based
speed network [39]. Furthermore, many useful data for onfuzzyinference,sothattheoverheadofdatacollectioncan
analysis (e.g., inner status of entities) are not transmitted besignificantlyreduced.
by the network so that they cannot be collected using the
standaloneloggermethod,buttheycanbecollectedwhenthe 3.2.2. Data Storage and Processing. Traditional databases
loggerisintegraltotheHPC. havelimitationsinperformanceandscalabilitywhenmanag-
By contrast, distributed loggers residing in most simu- ingmassivedata.Nowthatsimulationscalekeepsincreasing,
lation nodes are preferred since data are recorded in local the data management must be able to easily address the
node and thus network resource is saved [12]. Davis and requirements for future datasets. Distributed computing
Lucas [73] pointed out the principle for collecting massive technology could utilize a lot of dispersed resources to
data: minimize the network overhead by transferring only provide tremendous storage capacity and extremely rapid
required information (e.g., results of processing) and keep processingatarelativelylowcostpoint.Thismethodisvery
original data in the local node. To effectively organize the suitable for the case where the simulation data have been
disperseddata,adistributeddatamanagercanbeemployed, recordeddispersedly,andtheterm“in-situanalysis”isused
andwewillintroducethisconceptandrelatedworksinthe todescribethiskindofdataprocessing[75].Distributedtech-
nextsubsection. nologyalsoprovidesbetterscalability,whichisanimportant
performanceattributeforlarge-scalesystems.
Whenstandalonearchitectureturnsintodistributeddata
Currentlyseveraltypicalstorageandprocessingmethods
collection, the technical focus also shifts from the network
canbeusedforMSbigdata.
to the local node. The simulation execution should not be
disturbedintermsoffunctionandperformancebythisarchi- (i)DistributedFiles.Onesimplemethodistoutilizethe
tectural change. Wagenbreth et al. [14] used a transparent original file system by saving log files in each node
component to intercept simulation data from standard RTI and then creating a distributed application which
MathematicalProblemsinEngineering 9
Row oriented storage Column oriented storage
Column 1 Column 2 Column 3 Column 1 Column 2 Column 3
Row 1 Row 1
Row 2 Row 2
Row 3 Row 3
Row 4 Row 4
Row 5 Row 5
··· ···
Suitable for query of specific record Suitable for batch processing
Figure6:Roworienteddatabaseandcolumnorienteddatabase.
Table2:ComparisonofstoragetechnologiesforMSbigdata.
Storage Example Typicalprocessingmethod Advantages Disadvantages
Systemisnotgenerallyused;
Distributedfile Specificapplication Originalnetworkprogram Simple
Performanceisnotoptimized
High-performance;
Deploymentiscomplex;
Distributedfilesystem HDFS MapReduce scalable;
processingalgorithmiscomplex
reliable
Scalable;
Datagrid SDG Gridcomputing Complexprocessingislimited
easeofuse
Goodperformance;
scalable; Queryfunctionislimited;
NoSQLdatabase HBase MapReduce-wrappedinterface
dataprocessingis systemisnotmature
transparent
manipulates them. In this case, both the file format to manage tasks such as decomposing queries and
andtheapplicationarespecificallydesigned,andthe combiningtheresults.Datagridscanbeorganizedby
scalabilityofdataaccessingisrelativelylow.Inorder hierarchicaltopologyandexpandedondemand.The
to improve its universality and efficiency, a general- researchersinJointForcesExperimentsconstructed
ized list was designed to accommodate all kinds of a data grid platform (called SDG, and the details
datastructuresinlarge-scalesimulations[76]. willbediscussedinSection4)tomanagedistributed
databasesrunningMySQL[13,14].
(ii)Distributed File System. Another method is to use
a dedicated distributed file system. For example,
(iv)NoSQL Database. The NoSQL approach optimizes
theHadoopdistributedfilesystem(HDFS)provides
data accessing for data-intensive applications. There
commoninterfacestomanagefilesstoredindifferent
areseveralinstancesofNoSQLdatabasesystems,and
nodes,buttheuserdoesnotneedtoknowthespecific
we take HBase as a typical example. HBase is built
location.HDFSalsoprovidesadvancedfeatures,such
basedonHDFSandhascolumnorientedstyle,which
asreliabilityandscalability.MapReduceisadatapro-
meansthatthedataarenotorganizedbyrow,butby
cessingcomponentcompatiblewithHDFS.Itrealizes
column instead: values of records within the same
aflexible programmingframeworkandisprovedto
columnarestoredconsecutively(seeFigure6).That
be of high performance. The feasibility of handling
isbecausebigdataanalysisoftenconcernsthewhole
massivesimulationdatafromUSJointForcesExperi-
(in certain dimensions) but not a detailed record.
mentswasinvestigatedwithHadoop[77,78],andthe
NoSQL also optimizes data writing: simulation data
result was positive, though it is pointed out that the
donotinvolvecomplextransactions,andtherecords
performanceoftheWANmustbeimprovedbecause
are relatively independent of each other. However,
itisakeyfactoroflarge-scalemilitarysimulations.
relationaldatabasesystemstakeextratimeincheck-
(iii)DataGrid.Databasesystemsbringmanyadvantages, ingthedataconsistency.Theseunnecessaryfeatures
suchassimpledesign,powerfullanguage,structural arediscardedbyHBase.Inaddition,HBaseemploys
dataformat,maturetheory,andawiderangeofusers. a distributed architecture with load balancing capa-
Althoughasinglecentralizeddatabaseisnotsoprac- bilitysothatdataarestoredandprocessedatflexible
tical for distributed simulation, multiple databases scale.WuandGong[74]presentedanexamplewhich
canbeconnectedtogethertoaccommodatemassive transformed the data format of a simulated entity
data and process them in parallel to improve the statusfromdatabasetoHBase.
response speed. To build a cooperative mechanism
among databases, data grid technology can be used Table2comparestheprimaryfeaturesofabovemethods.
10 MathematicalProblemsinEngineering
3.3. Data Analysis. Data analysis includes algorithms and 3.3.2. Methods and Applications. The big data concept
toolswhichextractinformationfrombigdataandpresentthe emphasizesthevaluehiddeninmassivedata,sowefocushere
resultstoanalysts.WefirstdiscussthepurposeofMSbigdata ontheemergingtechnologiesfordataminingandadvanced
analysis and then review the emerging analytical methods visualization together with their applications. Nevertheless,
andapplications. traditionalstatisticaltechniqueshavebeenwidelyappliedin
militarysimulation,andtheyarestillusefulinbigdataera.
3.3.1.Purpose. Becauseofthediversityofmilitarydecision-
(1) Data Mining. Data mining refers to the discovery of
makingproblems,thepurposeofdataanalysisvariessignif-
previouslyunknownknowledgefromlargeamountsofdata.
icantly. It is difficult to get a complete view of all possible
Asawelldevelopedtechnology,itcouldbeappliedinmilitary
subjects and related methods. Generally, the purposes can
simulation to meet various kinds of analysis requirements.
beclassifiedintothreelevelsaccordingtothedegreeofdata
Thereareseveralcases:
usage:descriptive,predictive,andprescriptive[17].
(1)Association rules analysis shows the correlation
(1) Descriptive Analytics. Descriptive analytics is a primary
instead of causality among events. For example, in
use of a dataset and describes what has occurred. A typical
the context of a tank combat simulation, detecting
caseisthemeasureofweaponperformanceorsystemeffec-
therelationshipbetweentankperformanceandoper-
tiveness.Theanalyzeddatainvolveinteractiveeventsrelated
ationalresultsmaybeuseful.
tothesimulatedentityanditscurrentstatus.Descriptiveana-
lyticscanalsobeassistedbyvisualization,whichpresentsthe (2)Classificationanalysisgeneratesclassifierswithprela-
simulationresult(e.g.,scoreboardandallkindsofstatistical beleddataandthenclassifiesthenewdatabyprop-
charts)orthesimulationprocedure(e.g.,playbackwith2Dor erty.Forexample,itcanbeusedtopredicttheammu-
3Dsituationaldisplay).Descriptiveanalyticsusuallyemploy nitionconsumptionofatankplatoonduringcombat
traditionalstatistical methods when processing the original accordingtolargeamountsofsimulationresults.
results, but the analysis process can also involve complex
machinelearninganddataminingalgorithms.Forexample, (3)Clusteranalysisformsgroupsofdatawithoutprevi-
invirtualtrainingthehumanactiondataneedberecognized ouslabelingsothatthegroupfeaturescanbestudied.
forautomaticscoring. Forexample,itcanbeusedtoidentifythedestroyed
enemy groups by location and type. A normal data
(2)PredictiveAnalytics.Predictiveanalyticsisusedtoproject mining algorithm may be modified to comply with
thefuturetrendoroutcomebyextrapolatingfromhistorical the practical problem. The Albert project adopted a
data. For example, the casualty rate or level of ammunition characteristic rule discovery algorithm to study the
consumptioninacombatscenariocanbepredictedbymul- relationship between simulation inputs and outputs
tiplesimulations.Twotypicalmethodsarelinearregression [80].Acharacteristicruleissimilartoanassociation
and logistic regression. The basic idea is to set up a model rule, but its antecedent is predefined. Furthermore,
based on an acquired dataset and then calculate the result therelevanceiscalculatedbymeasuresof“precision”
forthesamescenariousingnewdatainputvalues.Predictive and“recall”insteadof“confidence”and“support”.
analytics can also utilize data mining tools to discover the
patterns hidden in massive data and then make automated In practice, the US Army Research Laboratory (ARL)
classifications for new ones [79]. However, it is recognized usedclassificationandaregressiontreetopredictthebattle
thatmilitaryproblemshavepervasiveuncertainty,andsoit result based on an urban combat scenario of OneSAF. The
isalmostimpossibletoproduceaccuratepredictions[28]. experiment executed the simulation 228 times and defined
435 analytic parameters. Finally, the accuracy of prediction
(3)PrescriptiveAnalytics.Asmentionedabove,accuratepre- reachedabout80%[81].TheIsraeliArmy’sBattle-Laboratory
dictions are difficult to obtain, but military users still need studied the correlations between events generated on a
to get valuable information or insights from simulations simulatedbattlefieldbyanalyzingthetime-series,sequence,
to improve their decision-making. This kind of analysis and spatial data. Various data mining techniques were
concerns all aspects of simulation scenarios. Prescriptive explored: frequent patterns, association, classification and
analytics focuses on “what if” analysis, which means the labelprediction,cluster,andoutlieranalysis[82].Inaddition,
processofassessinghowachangeinaparametervaluewill they proposed a process-oriented development method to
affectothervariables.Herearesomeexamples:Whatfactors effectively analyze military simulation data [83]. Yin et al.
aremostimportant?Arethereanyoutliersorcounterintuitive [84]minedtheassociatedactionsofpilotsfromaircombat
results? Which configuration is most robust? What is the simulations. The key involved a truncation method based
correlationbetweenresponses?Thesequestionsrequiredeep on a large dataset. As a result, interesting actions about
analysisofthedataandoftenemploydataminingmethods tactical maneuvers were found. The method was also used
andadvancedvisualizationtools.Usuallyusersareinspired tocontrolflyingformationsofaircraft,andthenaformation
by the system and become involved in the exploratory consistentwithhighqualitywaschosen[85].Acayexplored
process.Ontheotherhand,asmalldatasetcannotreflecta theHiddenMarkovModels(HMM)andDynamicBayesian
hiddenpattern,andonlyalargeamountofdatafrommultiple Network(DBN)technologiesinthesemiautomatedanalysis
samplescansupportthiskindofknowledgediscovery. ofmilitarytrainingdata[86].