Elastic Language in Persuasion and Comforting: A Cross-Cultural Perspective

12 Pages·2019·2.084 MB·

by  Grace Zhang, Vahid Parvaresh

Nuclear Engineering and Design 355 (2019) 110312
ContentslistsavailableatScienceDirect
Nuclear Engineering and Design
journal homepage: www.elsevier.com/locate/nucengdes
fi
The role of veri cation & validation process in best estimate plus
T
uncertainty methodology development
J. Zhang
Tractebel(ENGIE),BoulevardSimonBolivar34-36,1000Brussels,Belgium
ARTICLE INFO ABSTRACT
Keywords: Intheprocessofabestestimateplusuncertainty(BEPU)methodologyimplementationfornuclearpowerplant
Bestestimateplusuncertainty(BEPU) safetyanalyses,theverificationandvalidation(V&V)processincludingtheuncertaintyquantification(UQ)of
Industrialpractices theusedcomputercodesandplantmodelsplaysanessentialrole.Fromthetechnicalpointofview,aBEPU
Regulatoryrequirements methodologymustbebasedonfullyverifiedandvalidatedcodesandmodels,withquantifiedkeymodelun-
Technicalstandards
certainties.Fromtheregulatorypointofview,theadequacyofthesimulationcodesandplantmodelsforthe
Uncertaintyquantification(UQ)
Verificationandvalidation(V&V) intendedBEPUapplicationmustbeassessedthroughtheVVUQprocess.ThesehighlevelrequirementsofVVUQ
resultinhigh-costforBEPUmethodologydevelopment,preventingwiderapplications.Pragmatic,gradedap-
plicationsandpracticesofVVUQintheBEPUmethodologydevelopmentareneededtoallowtotakefullbenefit
ofBEPUapplications.
1. Introduction Finally,IAEAprovidesaguidanceonuncertaintyevaluationforbest
estimatesafetyanalysisfornuclearpowerplants(IAEA,2008).
Since 1990’s, best estimate plus uncertainty (BEPU) methodology TheadvantagesoftheBEPUmethodologyare:
for nuclear power plant design basis accident analysis (in particular,
(cid:129)
loss-of-coolantaccidentsorLOCAs)hasbeingdevelopedandappliedto It allows to realistically and accurately simulate phenomena that
meettheincreasingtechnicalandregulatoryrequirementsforlicensing governtheaccidentsandtransientsofinterest;
(cid:129)
newplantdesignormajorplantmodifications,poweruprate,newcore It is efficient to determine the plant limiting conditions in the in-
and fuel design, as well as for support to plant operation (Wilson, tegratedmulti-physicsanalysisenvironment;
(cid:129)
2013). Itallowstobetterassesstheactualmarginsandtoassurethatno-
According to International Atomic Energy Agency (IAEA) general cliff-edgeeffectsmayexist;
(cid:129)
safety requirement (GSR) Part 4 Requirement 18 (IAEA, 2016); “Any Itistransparentandaptforqualityassurance.
calculational method andcomputercodes used inthe safetyanalysisshall
undergoverificationandvalidation”. The BEPU might be thus the preferred methodology for licensing
IAEA specific safety guide SSG-2 (IAEA, 2019) recommends that safetyanalysisandsimulatingplantoperationandaccidentconditions.
“the methods used in the computer codes for the calculation should be However, the development and application of BEPU methodology re-
adequate for the purpose. The requirements for the validation and ver- quires a higher-level requirement on the verification and validation,
ificationdependonthetypeofapplicationandpurposeoftheanalysis.” and uncertainty quantification (VVUQ) of the used calculational
TheBEPUapproach“allowstheuseofabestestimatecomputercode method and simulation model. This may result in high-cost for BEPU
together with more realistic, that means best estimate and partially most implementation, and hence prevent the industry to take full benefit
unfavourable,initialandboundaryconditions.However,inordertoensure from the BEPU applications. Pragmatic and graded applications and
the conservatism required in analysis of design basis accidents the un- practicesofVVUQareneeded.
certaintiesneedtobeidentified,quantifiedandstatisticallycombined. Thispaperwillfocus ontheoverviewoftherelationship between
Availabilityofsystemsisusuallyassumedinaconservativeway.”TheBEPU theverification,validationanduncertaintyquantification(VVUQ)and
approach “contains a certain level of conservatism and is at present ac- theBEPUmethodologydevelopmentprocesses.Afterashortreviewof
ceptedforsomedesignbasisaccidentandforconservativeanalysesofan- theapplicabletechnicalstandardsandprocedures(§2),theregulatory
ticipatedoperationaloccurrences.” requirements and recommendations are reviewed (§3), followed by
E-mailaddress:[email protected].
https://doi.org/10.1016/j.nucengdes.2019.110312
Received17December2018;Receivedinrevisedform28June2019;Accepted21August2019
0029-5493/ © 2019 Elsevier B.V. All rights reserved.
J.Zhang Nuclear Engineering and Design 355 (2019) 110312
Nomenclature LWR Light-WaterReactor
MCDM Multi-CriteriaDecisionMaking
AHP AnalyticalHierarchicalProcess NEA NuclearEnergyAgency
AIAA AmericanInstituteofAeronauticsandAstronautics NPP NuclearPowerPlant
ANS AmericanNuclearSocietyorAmericanNationalStandards OECD Organization for Economic Co-operation and
ASME AmericanSocietyofMechanicalEngineering Development
BIC BoundaryandInitialConditions PCT PeakCladdingTemperature
BE Best-Estimate PDF ProbabilityDensityFunction
BEPU Best-EstimatePlusUncertainty PIRT PhenomenaIdentificationandRankingTable
CCVM CSNICodeValidationMatrix PMI PredictiveMaturityIndexes
CET CombinedEffectTests PREMIUM Post-BEMUSERefloodModelsInputUncertainty
CFD ComputationalFluidDynamics Methods
CSNI CommitteeontheSafetyofNuclearInstallations PWR PressurizedWaterReactor
ECCS EmergencyCoreCoolingSystem Q-PIRT QuantitativePhenomenaIdentificationandRankingTable
EM EvaluationModel RCS ReactorCoolantSystem
EMDAP EvaluationModelDevelopmentandAssessmentProcess SA SensitivityAnalysis
FSAR FinalSafetyAnalysisReport SAPIUM Systematic APproach for model Input Uncertainty quan-
GSA GlobalSensitivityAnalysis tificationMethodology
GSR GenericSafetyRequirements SET SeparateEffectsTest
IAEA InternationalAtomicEnergyAgency SM SimulationModel
IB IntermediateBreak SRQ SystemResponsesQuantity
IET IntegralEffectsTest SSG SpecificSafetyGuide
IP InversePropagation UA UncertaintyAnalysis
ITF IntegralTestFacility USNRC UnitedStatesNuclearRegulatoryCommission
IUQ InputUncertaintyQuantification UQ UncertaintyQuantification
LOCA LossOfCoolantAccident VV ValidationandVerification
industrial applications and good practices (§4), conclusions and per- andapplication. It isclearthat verification deals withtherelationship
spectives(§5). between the physical model and the simulation model and that vali-
dationaimstoquantifytheaccuracyofthesimulationmodelbasedon
comparisons of physical experiments with calculation outcomes from
2. Technicalstandardsandprocedures
thesimulationmodel.Thesimulationmodelcanonlybeusedtopredict
the behaviour of the intended application after the V&V and the UQ.
The whole process of development and application of a BEPU
Therefore, V&V can be considered as the foundation of the BEPU
methodology for the intended application consists of the following
methodology, as it is essential to answer the critical question: How
mainactivitiesandprocesses,assummarizedinFig.1: confident should be we with the prediction results of the developed
(cid:129) simulationmodelfortheapplication?
Modelling: Development of physical models required for an in-
ParallelandclosetothisV&Vprocess,theuncertaintyineachofthe
tended application (e.g., analysis of a transient or accident in a activitiesshouldbequantifiedintheuncertaintyanalysisorquantifica-
pressurized water reactor using system and sub-channel thermal tion (UA or UQ) process. Uncertainty quantification is a process of im-
hydrauliccodes),
(cid:129) plementation of a set of tools and formalisms for quantifying sources
Simulation Model: Implementation of the physical models in the andsubsequentpropagationofuncertaintyfromallsourcesthataffect
computercodeanddevelopmentofthesimulationmodel(notethat
thereliabilityoftheoutput.
inthispaper,theSimulationModelisreferredtothecomputercode,
The development of technical standards and procedures for code
nodalization, model option and algorithms to approximate the so-
lutionofphysicalequations),
(cid:129)
Scaling and Experimentation: Establishment of the experimental
databaseforsimulationofthephysicalbehaviourfortheintended
application,
(cid:129)
VerificationandValidation(V&V):Assessmentofthecorrectnessof
thecomputercodeandquantificationofaccuracyofthesimulation
model,
(cid:129)
Prediction: Use of the verified and validated simulation model to
predict(orextrapolate)theexpected“bestestimate”(BE)response
of the system for the application, and assessment of adequacy or
predictivecapability,
(cid:129)
Uncertaintyanalysis:Quantificationofuncertaintiesofalltheabove
activities in the whole development process (e.g., due to approx-
imation, model deficiency, scaling distortion, measurement un-
certainty, etc.) and application process (initial and boundary con-
ditions,etc.).
The above modelling, verification and validation process is con-
sistent with the concepts proposed by LANL (Thacker et al., 2004), Fig. 1.The verification and validation process in relation with the BEPU
except that theconcept ofrealityisreplaced byphysical experiments methodology.
2
J.Zhang Nuclear Engineering and Design 355 (2019) 110312
verification, validation and uncertainty quantification (VVUQ) are Thesolutionverificationfocusesonestimatingthenumericalsolu-
underconstantlyevolutionsincethepublicationofthefirstANSstan- tion error, assuring the input and output data for the problem of in-
dardin1987(ANS,1987).OfparticularinteresttoBEPUmethodology terest, which allows quantification of errors introduced during appli-
using thermal hydraulic codes are the AIAA (AIAA, 1998) and ASME cationofthecodetoaparticularsimulation(e.g.,performingagridor
(ASME,2009)guidesforComputationalFluidDynamics(CFD)andheat timeconvergencestudybysuccessivelyrefiningthemeshortimestep
transfer, and the ongoing development of the ASME standard for nu- until a sufficient level of accuracy is obtained). The code verification
clear thermal-fluids software (Harvego et al., 2011). Some slight dif- focusesontheidentificationandremovaloferrorsinthesourcecode
ferencesinthedefinitionofthecomputermodelV&Vprocessinvarious and numerical algorithms, and improving software using software
standards,andthebest-fitoneforourusearethoseintheAIAAguide qualityassurancepractices.
(AIAA,1998): Validationcanalsobefurtherdividedinthreelevels,asshownin
Fig. 3 and proposed by Oberkampf and Barone (Oberkampf and
(cid:129)
Verification: the process of determining that a (computer or simu- Trucano,2007):
lation)modelimplementationaccuratelyrepresentsthedeveloper’s
conceptualdescriptionofthemodel(i.e.,mathematicalmodel)and 1. Quantification of the accuracy of the simulation model results
thesolutiontothemodel. throughcomparingthesystemresponsequantities(SRQs)ofinterest
(cid:129)
Validation:theprocessofdeterminingthedegreetowhicha(com- with experimentally measured SRQs. The simulation model accu-
puterorsimulation)modelisanaccuraterepresentationofthereal racy is quantitatively estimated at the conditions where experi-
worldfromtheperspectiveoftheintendedusesofthemodel. mentaldataisavailable,andthevalidationisaddressedbydefining
andcomputingvalidationmetric(s)associatedtoSRQs;
Brieflyspeaking,verificationistheassessmentoftheaccuracyofthe 2. Useofthesimulationmodel,inthesenseofinterpolationorextra-
solution to a simulation model by comparison with known solutions polationofthesimulationmodel,tomakepredictionsforconditions
(i.e.,howwellaremathematicalformulasrepresentedincomputational correspondingtotheintendeduseofthesimulationmodel;
model?). Validation is the assessment of the accuracy of a simulation 3. Determinationofwhethertheestimatedaccuracyofthesimulation
model through comparison with experimental data (i.e., how well do model results, for the conditions of the intended use, satisfies the
codecalculationoutputsrepresentreality?).Inotherwords,verification accuracyrequirementsspecifiedfortheSRQsofinterest.
isprimarilyamathematicsissue;validationisprimarilyaphysicsissue
(OberkampfandTrucano,2002). The last two levels of validation concern the simulation model
ThesimulationmodelV&Visrelatedto,butfundamentallydifferent predictive capability assessment. It should use the assessed simulation
from software V&V. Code developers of computer programs perform model accuracy (first level) as input, and incorporate the following
software V&V to ensure code correctness, reliability, and robustness. additionalassessment(OberkampfandTrucano,2007):
The code users, however, seek to apply credible predictive models
based on fundamental physics of the problem that are solved in all a) additional uncertainty estimation resulting from interpolation or
applications.Theadequacyofthepredictivemodelsneedtobeassessed extrapolation of the simulation model beyond the existing experi-
according to simulation model V&V guidelines and procedures. The mentaldatabasetofutureapplicationsofinterest;and
expected outcome of the simulation model V&V process is thus the b) comparison of the accuracy requirements needed by a particular
quantified level of agreement between experimental data andsimula- application relative to the estimated accuracy of the simulation
tion model calculation, and most importantly the accuracy of the si- modelforthatspecificextrapolationtotheapplicationsofinterest.
mulationmodel.
Verification can be further divided in two types of activities as Ifthecalculatedvalidationmetricresultmeetstheaccuracyrequire-
shown in Fig. 2, as proposed by Oberkampf et al. (2004): Numerical ments,thesimulationmodelisconsideredasadequatetotheintended
algorithm (orsolution) verification andsoftwarequality engineeringprac- application(i.e.,usedforBEPUmethodologydevelopment).Iftheac-
tices(orcodeverification). curacyrequirementsarenotmet,onemayneedtoeitherupdate(orre-
Fig.2.Thetwotypesofmodelverification(Oberkampfetal.,2004).
3
J.Zhang Nuclear Engineering and Design 355 (2019) 110312
Fig.3.Thethreelevelofvalidationprocess(OberkampfandTrucano,2007).
calibrate)thesimulation modelorimproveoraddexperimentalmea- torealplantconditionsthatarenottotallyavailableinexperimentsisa
surements(OberkampfandBarone,2006).Thismeansanewroundof difficulttask.Somespecificmethods,suchasthepredictivecapability
VVUQ. maturity model (Oberkampf and Trucano, 2007; Oberkampf et al.,
A comprehensive framework for verification, validation, and un- 2007; Rider et al., 2015) may be helpful, but not yet mature and
certainty quantification in scientific computing was proposed by practical enoughtobeappliedtonuclearpowerplantsafety analyses
Oberkampf and Roy (Roy and Oberkampf, 2011). The framework in- usingsystemthermalhydrauliccodes.
cludesthefollowingkeystepsforuncertaintyquantification: Notealsothatforcertainnuclearthermalhydrauliccodes,thecode
developers usually perform extensive developmental code assessment
1) identification of all sources of uncertainty associated to model in- (V&V). However, the code users (or model developers) should also
puts,numericalapproximationandmodelform, perform independent V&V for the selected codes for the intended ap-
2) the characterization of uncertainties including quantification of plications,andagradedapproachofVVUQmaybeappliedinthiscase
modelinputuncertainties, (seeSection4).
3) the estimation of the uncertainty due to numerical approximation
(e.g.,byeliminationorestimationofcodeandsolutionverification
3. Regulatoryrequirementsandrecommendations
errors),
4) propagation of quantified input uncertainties through the simula-
In1988,theUSNRCrevisedtheregulatoryrequirementsforLOCA
tionmodeltoobtainuncertaintiesintheSRQs,
safety analysis, which allows use of a realistic evaluation model with
5) theestimationofthemodelformuncertainty,and
uncertainty quantification (i.e., BEPU analysis methods). Regulatory
6) the determination of the total uncertainty in the SRQs at the ap-
Guide1.157(USNRC,1988)describesacceptablemodels,correlations,
plication conditions of interest (e.g., extrapolation to application
data, modelevaluation procedures, andmethods formeeting thespe-
conditionsofinterest).
cific requirements for a realistic calculation of ECCS performance
during a LOCA. In order to demonstrate the implementation of such
In VVUQ procedure, the validation process can be performed by
BEPUmethodology,theUSNRCtechnicalprogramgrouphasdeveloped
way of comparison of simulated results with available experimental
a Code Scaling, Applicability and Uncertainty evaluation (CSAU) ap-
measurements. It is therefore performed at the conditions where ex-
proach(USNRCtechnicalprogramgroup,1989).
perimental data are available and the validation is addressed by de-
finingandcomputingvalidationmetricsassociatedtoSRQs.Thefinal The CSAU is a structured, traceable and practical approach to
quantify uncertainty. It addresses in a unified and systematic manner
objectiveistoextrapolatethepredictionresultsandtotaluncertaintyto
questionsrelatedto:
theapplicationconditionsofinterestforwherenoexperimentaldatais
available (step (6)). The extrapolated uncertainty is included in the (cid:129)
thescalingapplicabilityofbest-estimatecode,
predictionofthecomputationalmodelattheconditionsofinterest. (cid:129)
theapplicabilityofthecodeandplantmodeltoscenariosofinterest
It should be noted that the above VVUQ concepts and procedures
toNPPsafetystudies,and
are derived from development of software applicable to design of (cid:129)
the evaluation of uncertainties in calculating the SRQs of interest,
missilesandtheirequipmentwhereimpactofempiricandsemi-empiric
whenthecodeisusedtoperformacalculationforaspecifiedsce-
correlationsarenotpronouncedsofarasinnuclearpowerplantsystem
narioandNPPdesign.
thermal hydraulics modelling. Therefore, some of the elements and
stepsshouldbeadaptedforapplicationstothelattercase(see§4).In
It wasrecommended bythe groupofexperts asan acceptable ap-
particular,theextrapolationofthesimulationmodelanditsuncertainty
proachtodevelopaBEPUmethodologyforbestestimateLOCAanalysis
4
J.Zhang Nuclear Engineering and Design 355 (2019) 110312
that complies with the USNRC regulatory guide RG-1.157 (USNRC capabilityandtospecifyrangesofparametersvariationsneededfor
technicalprogramgroup,1989). sensitivitystudies.
(cid:129)
As shown in Fig. 4, the CSAU methodology consists of 14 steps, Element3–SensitivityandUncertaintyAnalysis:theeffectsofin-
organizedinto3majorelements: dividual contributors to total uncertainty are obtained and the
propagation of uncertainty through the transient is properly de-
(cid:129)
Element1–Requirementsandcodecapabilities:scenariomodelling termined.
requirements are identified in a Phenomena Identification and
Ranking Table (PIRT) and compared against code capabilities to The VVUQ process is detailed in Element 2 & 3, which is the key
determinethecodesapplicability totheparticularscenario andto partoftheCSAUmethodology.
identifypotentiallimitations. The CSAU approach described above was recently endorsed as an
(cid:129)
Element 2 – Assessment and Ranging of Parameters: code cap- acceptable structured Evaluation Model Development and Assessment
abilitiestocalculateprocessesimportanttothescenarioareassessed Process(EMDAP)(USNRC,2005).ItdescribesaprocessthattheUSNRC
againstexperimentaldatatodeterminecodeaccuracyandscale-up considers acceptable for use in development and assessment of
Fig.4.Codescaling,applicabilityanduncertainty(CSAU)evaluationmethodology(USNRCtechnicalprogramgroup,1989).
5
J.Zhang Nuclear Engineering and Design 355 (2019) 110312
evaluationmodels(i.e.collectionofcodesandprocedures)thatmaybe purposeistoprovidethebasisfordevelopmentandassessmentof
used to analyse transient and accident behaviour within the design the evaluation methodology. This includes acquiring appropriate
basisofaNPP.TheEMDAPessentiallyfollowsthesamemainprinciples experimental data relevant to the scenario being considered and
of the CSAU methodology described above, however with more em- ensuringthesuitabilityofexperimentalscaling.
(cid:129)
phasis to the evaluation model development, verification and valida- Element 3 – Develop Evaluation Model: In this third element, the
tion,uncertaintyquantification,andapplicationprocess. evaluation model is developed and organized to meet the require-
AsshowninFig.5,theEMDAPconsistsof4elementsand19steps: mentsdefinedinElement1.
(cid:129)
Element4–AssessEvaluationModelAdequacy:Inthisfourthand
(cid:129)
Element 1 – Establish Requirements for Evaluation Model last element, theadequacy andcapability ofthe evaluation model
Capability:Inthisfirstelement,theexactapplicationenvelopefor areassessedanddocumented.Thefinalstepofthislastelement,itis
the evaluation methodology is determined. Furthermore, the im- tobedecidedwhetherthecodeisadequateornot.
portanceofconstituentphenomena,processes,andkeyparameters
withinthisenvelopeareagreedupon. TheVVUQprocessisdetailedinElement4,whichisthekeypartof
(cid:129)
Element2–DevelopAssessmentBase:Inthis secondelement,the theEMDAPapproach.
Fig.5.Evaluationmodeldevelopmentandassessmentprocess(EMDAP)(USNRC,2005).
6
J.Zhang Nuclear Engineering and Design 355 (2019) 110312
Last but not least, USNRC Software quality assurance guides are selectedvalidationmatrices(SETsorcombinedeffecttests);
detailedintwoNUREGreports(USNRC,1993;USNRC,2000).Anex- 4. Toassessthescalability(biasesordistortion)ofthesimulationmodel
ample of application of these guidelines to the USNRC thermal hy- anduncertaintyanalysisbycomparingwiththeIETs;
drauliccodeTRACEcanbefoundin(Bajoreketal.,2015).InFrance, 5. To assess the adequacy of the simulation model and uncertainty
the safety authority ASN recently published recommendations for the analysisforrealplantapplications.
qualification of scientific computing tools used to verify compliance
with the safety criteria associated with the first fuel barrier (ASN, Forpracticalapplicationpurpose,theVVUQwereoftenperformed
2017). In the U.K., the safety authority ONR published recommenda- on a selected or improved code (such as the generic purpose codes
tions for the validation of computer codes and calculation methods RELAP,TRACE,CATHAREorATHLET)oraspecificallydevelopedcode
(ONR, 2016). In Germany, the use of best-estimate codes is allowed, (such as WCOBRA-TRAC, MARS), for a well-defined scenario (e.g.,
combined with conservative initial and boundary conditions, and ef- large-breakLOCA)ofaparticularreactordesign(e.g.,PWR2-,3-or4-
fortsarebeingmadetoincludeuncertaintyevaluationintheregulation loops)andsafetyinjectionsystem(e.g.,Coldleginjectionorcombined
witharevisionintheGermanNuclearRegulation.TheReactorSafety injection, upper plenum injection), and for a particular fuel design
Commission is also recommending a BEPU LOCA licensing analysis (14×14,15×15or17×17,8–14ft.).
(Glaeser,2008).
4.2. Goodpractices
4. Industrialapplicationsandgoodpractices
ThegoodpracticesforVVUQasproposedandusedbytheindustry
4.1. Industrialapplications (Prošek and Mavko, 2007; Martin, 2016) or as developed by the re-
searchorganizations(Rideretal.,2010;Pourgol-Mohamadetal.,2011;
The early USNRC approved BEPU large-break LOCA evaluation Unal et al., 2011; Stoots et al., 2012; Petruzzi and D’Auria, 2016;
methodologies (EMs), such as the Westinghouse’s BELOCA (Young Radaidehetal.,2019b)aresummarizedinthefollowingsub-sections.
et al., 1998) and ASTRUM (Frepoli, 2008) and the Framatome’s
RLBLOCA (Martin and O’Dell, 2005), followed the CSAU approach 4.2.1. Assessmentofcapabilityandlimitationsofthesimulationmodel
(USNRCtechnical programgroup,1989).Similarmethodologies have The first step of VVUQ is to assure that the developed or selected
been developed and approved in other countries, such as ESM-3D in code is capable of representing correctly the process and phenomena
France(Sauvageetal.,2005),KREMinSouthKorea(Banetal.,2004), that have been identified as highly ranked during the PIRT process
statistical methods in Germany (Kozmenkov and Rohde, 2013, 2014; (Element1ofbothCSAUandEMDAP).
Seebergeretal.,2014),andtheBEPULOCAmethodinSpain(Queral Thisassessmentcanbeperformedbyathoroughreviewoftheuser
etal.,2015). manualsofthedevelopedorselectedcode(inparticularthebasicfield
TherecentlyUSNRCapprovedBEPULOCAevaluationmethodolo- equations, code structure and solution method, models and correla-
gies,suchastheWestinghouse’sFSLOCA(FrepoliandOhkawa,2011) tions),bycomparingwiththecapabilityrequirementsasidentifiedinthe
andtheFramatome’sRLBLOCARev.3followedtheEMDAPapproach formersteps.Forexample,ifmultidimensionaleffectsandmulti-fields
(USNRC,2005). arerankedimportantfortransientslikelarge-breakLOCAs,thedevel-
Most recently, the BEPU analysis capability is established and de- oped or selected code should include these models in an acceptable
monstrated using the RELAP5-3D code, in response to the Nuclear way.Itisessentialalsotoverifythatthedevelopedorselectedcodehas
RegulatoryCommission’sproposednew10CFR50.46(c)rulemakingon beenusedinthepastforsimilarapplications,andidentifyitslimitations.
emergency core cooling system/LOCA performance analysis (Zhang
etal.,2016a,b;2017).Theproposednew10CFR50.46(c)rulemaking 4.2.2. Assessmentofapplicabilityofthesimulationmodel
imposes more restrictive cladding embrittlement criteria and new The applicability of the simulation model for the intended applica-
analysis methods (refined core modelling, BEPU) are also required to tion is established by evaluation of the simulation model calculation
provideacompletecharacterizationofthereactorcoremarginsunder resultsagainstrelevantexperimentaldata,asselectedintheso-called
LOCAconditions. validation matrices for the intended application (in Element 2 of both
Since 2000, many BEPU applications have also been made on the CASUandEMDAP).
non-LOCA transientanalyses, although onlyafew hasbeenapproved TheOECD/NEACSNIhaspublishedextensivevalidationmatricesof
for licensing purpose (Kawamura and Hara, 2000; Abdelghany and separateeffect(OECD/NEA,1993;OECD/NEA,1993)andintegralef-
Martin,2010;Avramova,2010;Zhangetal.,2013;daCruzetal.,2014; fect (Aksan et al., 1987) tests (IETs) that provide information on ex-
Pecchia,2015;Martin,2016;Brownetal.,2016;Nguyen,2017;Burns isting experimental data. More recently, a total of 116 thermal hy-
andBrown,2017;Waltersetal.,2018;Radaidehetal.,2019a).Indeed, draulic phenomena have been identified for water cooled reactors
the application of VVUQ and BEPU approaches have been extended including new reactors (Aksan et al., 2018), which cover virtually all
beyond system and subchannel thermal-hydraulics to containment LOCA andnon-LOCAthermal-hydraulic transients. EPRI(EPRI,2014)
thermal hydraulics, nuclear data, reactor core neutronics, fuel perfor- compiled an assessment database and demonstrated a process for
mance,multi-physics,spentfuelcriticalitysafetyanalysis,etc. rankingthedataandmappingittothethermal-hydraulicphenomenaof
InthedevelopmentofBEPUmethodologies,themajoractivitiesfor interestfortransientsoraccidentsofinterest.
theVVUQprocess(namelytheElement2&3ofCSAUorElement4of For developing and qualifying a simulation model (or input deck)
EMDAP)are: forthetestfacilityorplant,theusersofthedevelopedorselectedcode
should follow the detailed guidelines for nodalization and model op-
1. To assess the capability and limitations of the code field equations, tionsprovidedbythedevelopers.Participationincodeusergroupsand
closurelawsandnumericalsolutionstosimulatethephysicalphe- trainingprogramstoshareexperienceswithotherusersisfundamental
nomenaorprocessesinthesystemsorcomponentsofinterest; toreducetheso-calledusereffect.
2. Toassesstheapplicabilityofthesimulationmodel(codeandinput Onekey element of theVVUQ process is to qualify the developed
model, including nodalization and model options) to simulate the nodalizationandmodeloptionsfortheintendedapplication(Bucalossi
physical phenomenaorprocesses inthesystemsorcomponents of and Petruzzi, 2010; Petruzzi and D’Auria, 2016). In this process, one
interest; shouldcheckboththegeometricalfidelityofthenodalizationandthe
3. To quantify the uncertainty of the simulation model for the im- capabilityofthecodenodalizationtoreproducetheexpectedtransient
portant physical phenomena or processes by comparing with the scenario.Acceptabilitycriteriahavetobedefinedandsatisfiedduring
7
J.Zhang Nuclear Engineering and Design 355 (2019) 110312
thenodalization-qualification.Ifneeded,sensitivitystudiesonnodali- uncertaintyquantification(IUQ)methodsbyinversepropagationofthe
zation(i.e.,timestepsize,convergenceandsolutionmethod)shouldbe information based on the discrepancy between code simulation and
performed to improve the quality of code prediction results by de- experimentalresults.
monstrating that a convergent solution has been obtained and pro- TheOECD/NEAPREMIUMbenchmarkhasbeenavaluableexercise
vidinginformationonuncertaintyresultingfromthechoiceofnodali- on methods of uncertainty quantification of physical model input
zationandresolutiontechnique. parameters,andtheirapplicationtothephysicalmodelsinvolvedinthe
To identify and reduce the effects of inadequate nodalization PWR LOCA reflooding prediction. A review and comparison of the
schemes,theincorrectuseofphysicalmodelsoutsidetheirscope,the availableIUQmethodsusedinPREMIUMisgivenin(Reventósetal.,
codeoptionsofdifferentphysicalmodels,andothersourcesofavoid- 2016).Differentmethodsandthermohydrauliccodeswereusedwithin
ableerrorsincalculationsofcode,itisessentialtocheckiftheresults thebenchmark(MendizábalSanzetal.,2017).Resultsweremorede-
areconsistentwiththephysicalmodels.Post-processingtechniqueshave pendent on the quantification methods (Maximum Likelihood in-
been developed to verify this consistency, such as the SCCRED tool ference, Bayesian inference or Coverage method), rather than on the
(PetruzziandD’Auria,2016),makingitpossibletodistinguishbetween codes employed (RELAP, CATHARE, ATHLET, etc.). Furthermore, the
theuncertaintiesinthecalculationandthemodellingerrorsduetouser resultsofquantificationshowedastrongdependencyontopicssuchas:
effects.
(cid:129)
Forthefullyverifiedandvalidatedgenericpurposecodes,although ThesetofselectedSRQsusedinthequantification
(cid:129)
thevalidationmatriceshavebeenpartlyusedbythecodedevelopersto Thesetofselectedinputparameterstobequantified
(cid:129)
evaluateandimprovesystemcodes,theusersshouldalsoperformthe Selectedtestsforquantification
(cid:129)
so-called independent validation, based on one or more integral effect The simulation models, which, in general depend on the thermal
teststhataresimilartotheintendedapplication,inordertoensurethat hydrauliccodesbeingused.
the modelling techniques used for the test facility model adequately
predict the experimental data. A further assessment against separate Based on the experience feedback from the previous OECD/NEA
effectsteststhatdealwithphenomenaidentified asimportantforthe PREMIUMbenchmark,asystematicapproachdevotedtomodelinput
intended application is also suggested. Finally, cross-code verification uncertainty evaluation (i.e. quantification and validation) has been
(orbenchmarking)shouldbemadewhentheexperimentaldataisnot proposedtoimprovethereliabilityoftheanalysisandtheconfidence
available,whichisvaluableandusefultoexplainthedifferenceinthe ontheextrapolationofitsresultstotheNPPcase(Baccouetal.,2017).
predictedbehaviouronthesameproblem. TheproposedsystematicapproachisshowninFig.6.
TheSAPIUMapproachconsistsinthefollowing5keyelements:
(cid:129)
4.2.3. Quantification of the model input uncertainty by comparison with Element1includesthedefinitionoftheobjectivesoftheevaluation
SETsandCETs (e.g.,quantifyandvalidatetheinputuncertaintiesofthereflooding
Another key issue of BEPU methodology is the input uncertainty heattransfermodelsforapplicationtoplantanalysis),theselection
quantification.For anysimulation model, therearetwotypes ofinput ofaNPP(e.g.,a3-loopPWR)andascenario(e.g.,acold-legbreak
parameters:thosearenotsubjectivetocalibration(boundaryandinitial LOCA), as well as the SRQs of interest (e.g., peak cladding tem-
conditions, independent variables, etc.),andthose aresubject tocali- peratureorPCT)associated withtheidentifiedimportantphysical
bration(physicalmodelinputparameters,calibrationparameters,un- phenomena (e.g., quench front propagation, interphase friction,
knownconstants,etc.).Theuncertaintyboundsfortheformercategory dispersed film boiling, etc.). The latter is obtained by applying a
can be obtained from experiments or plant specifications and expert Phenomena Identification and Ranking Table (PIRT) process,
opinion,whilethemodelinputuncertaintycanbeinferredbytheinput
Fig.6.Thesystematicapproachtomodelinputuncertaintyquantification(Baccouetal.,2017).
8
J.Zhang Nuclear Engineering and Design 355 (2019) 110312
initially and primarily based on expert judgement (Wilson and experimentalfacilityandsimilarcomponentsinthenuclearpower
Boyack, 1998), and later quantitatively confirmed by the global plant.Aspecialattentionshouldbealsodevotedtotheconstruction
sensitivity analysis (GSA) method (Luo et al., 2010; Martin, 2011; oferrormetrics(toevaluatetheaccuracycode/experiment)andthe
YurkoandJacopo,2012). definitionofascaleofaccuracy.Finally,importantuncertainmodel
(cid:129)
Element2isrelatedtotheconstructionoftheexperimentaldatabase inputparametershavetobeidentifiedbyusingtheglobalsensitivity
formodelinputuncertaintyquantificationandvalidationthatwill analysis(GSA)methodsinthiselement.
(cid:129)
controlthecapabilityofthemethodtoextrapolateitsresultstoreal Element 4 consists in inferring from the experimental knowledge,
situations.ItshouldbebasedonavailableSETs,CETsandIETsbut the information related to model input uncertainties. The experi-
can also require extra experiments if necessary. It includes the as- mentalknowledgeishereassociatedwithasubsetofthedatabase
sessment ofadequacy ofan experimentand ofcompleteness of an constructed in Element 2 (the remaining subset will be used for
experimental database. At the end of this step, a ranking between modelinputuncertaintyvalidation).Itthenrequiresselectingaset
experiments within the database could be performed using the of differences between code calculation and experimental value.
Multi-CriteriaDecisionMaking(MCDM)outrankingapproaches(as Finally, the inference can be performed. Besides the choice of the
illustrated in (Baccou et al., 2018) or analytic hierarchy process modelinputuncertaintyquantificationmethods(FFT,MonteCarlo,
(AHP)(Saaty,1982). Bayesian, …), an appropriate uncertainty modelling for each un-
(cid:129)
Element3isrelatedtothesimulationmodel.Itconsistsinassessing certaininput(interval,pdf,possibility,…)shouldbedonebytaking
theapplicabilityofthecodeformodellingtheidentifiedimportant intoaccounttherealstateofknowledge(natureofuncertaintyand
phenomena as well as for modelling the considered SETs/CETs/ available information) and by reducing as much as possible extra
IETs.Moreover,thiselementrequirestofollownodalizationstrategy assumptions. Key questions of this element are also related to the
and model option selection that should be consistent between the strategytofollowinpresenceofseveralexperiments(quantification
Fig.7.Generalframeworkforassessingadequacyofathermal-hydraulicanalysisandassociateduncertaintiesinplantsafetyanalysis(NourbakhshandBanerjee,
2013).
9
J.Zhang Nuclear Engineering and Design 355 (2019) 110312
perexperimentorauniquequantificationforallexperimentscon- to safety analysis for real plant. This requires an assessment of the
sideredtogether?)aswellasincaseofseveralquantifications(how adequacy of the simulation model and uncertainty analysis for this
tocombineinputuncertainties,keepinginmindthatseveraloptions application. A general framework for assessing the adequacy of a
exist?). thermalhydraulicanalysisanduncertaintiesassociatedwithitsresults
(cid:129)
Element 5 is based on the propagation of the model input un- inthecontext ofasafetydecisionisgiveninFig.7(Nourbakhshand
certainties obtained in Element 4, together with other uncertain Banerjee,2013).
inputparameters,throughthecomputercode.Itcanbeincludedin InaccordancewiththeCSAUorEMDAPprocess,thefirststepisto
aniterativeprocesswithElement4.Itexploitstheremainingsubset specify the nature of the safety analysis, and the details of the sub-
oftheexperimentaldatabaseidentifiedinElement2andnotusedin sequentevaluationmayvarydependingonthespecificSRQ(orfigure
Element 4. The propagation first implies the selection of an un- ofmerit)requiredforthesafetyanalysis.ThisisfollowedbytheVVUQ
certaintymodelforeachuncertaininput(interval,pdf,possibility, process(identificationandevaluationofthekeyphenomena,selection
…)thatcanbedifferentfromtheuncertaintymodellingassociated ofthecode,assessmentoftheapplicabilityofthecode,scalinganalysis
withElement4.Moreover,theinputsamplingprocedureshouldbe andcodevalidation).
specifiedaswellasthequantitiesofinterestderivedfortheoutput The uncertainty analysis must ultimately be performed during the
sample that will be used for validation (e.g. percentiles in the plant safety analysis. This should begin with identifying sources of
probabilistic framework). Finally, a key point of this step is the uncertaintythatmayaffecttheSRQ.Allrelevantuncertainties,whether
definition and computation of validation metrics. It requires to theycanbeappropriatelyaddressedbycodecalculationsornot,should
reachaconsensusonthedefinitionof“validateduncertaintybands” be taken into account, including the uncertainties of completeness of
(i.e.whichimportantpropertiesanuncertaintybandhastosatisfy experimentaldatabase.However,notallidentifieduncertaintiescanor
to be accepted) and to introduce relevant criteria that mathemati- needtobequantifiedandmanyuncertaintiesmaynotaffecttheresults
callytranslatethisdefinition. ofthesafetyanalysis.Theimportantuncertaintycontributorsthatmust
beconsideredcanbeconfirmedbyusingtheglobalsensitivityanalysis
It should be noted that Elements 1–3 are common to any BEPU method.
methodologybasedonCSAU(USNRCtechnicalprogramgroup,1989) The most widely used uncertainty analysis method is the forward
or EMDAP (USNRC, 2005), focusing on the application of a qualified inputuncertaintypropagationbystatisticalmethod.Theprocedurefor
(i.e., fully verified and validated, with model input uncertainties statistical analysis of the uncertainty chosen to propagate the un-
quantifiedandvalidated)codeforaccidentanalysis.Thegoodpractices certainties must be designed to generate a given sample of code ex-
fromthoseindustrialdevelopmentandapplicationsandnewresearches ecutions,andappropriatestatisticalmethodsmustbeusedtodevelop
will also be taken in the framework of the new OECD/NEA project probabilisticstatementsdemonstratingcompliancewiththeacceptance
SAPIUM(Baccouetal.,2017). criteria.
Ifadetailedstatisticaluncertaintyanalysiscouldnotbeperformed
4.2.4. Assessmentofthescalability(biasesordistortion)ofthesimulation forsomemodelsorinputparameters,theassessmentmustshowthata
modelanduncertaintyanalysisbycomparingwiththeIETs sufficient conservative bias has been retained in the methodology to
IET experimental data is valuable source of information for the justifyitsapplicability.Thealternativeassessmentoftheimpactofsuch
justification of new evaluation model (or methodology) based on the uncertainties can greatly reduce the effort required for the formal
BEPU approach. IETs should be used to provide an appraisal of the propagationofuncertainties.Anexampleofsuchmethodsistousethe
scalability of the simulation model, in specific the interaction among worst case assumptions or a plausible bounding estimate to address
sub-models, compensating errors, predictability of important phe- uncertainty.Thisapproachisparticularlyreasonableiftheworst-case
nomena and the automated simulation model features. The biases or assumptionsdonotaffecttheoutcomeofthesafetyanalysis.
distortion, if any, shall be quantified in order to improve the code Itshould be notedthat evaluating theimpactofuncertainties one
models and correlations, and as well for correcting the compensation aftertheotherusingthealternativeapproachesdescribed abovedoes
errors. not allow for synergistic effects to be taken into account when the
AgoodexamplehasbeenprovidedtohighlighttheuseofIETex- impactofanindividualuncertaintydependsonthevaluesassumedby
perimental data tojustify thevalidity oftheuncertainties ofthephy- the remaining uncertain parameters. Therefore, these uncertainties
sical models of best estimate system code, and more generally of the shouldbeassessedtogetherusingtheGSAmethod.AreviewoftheGSA
consistency of a BEPU methodology based on the approach of propa- methods can be found in (Iooss, 2018). The most commonly used
gating uncertainty in code input (Geiser et al., 2017). In order to de- method is the variance-based methods, such as Sobol indexes (Sobol,
monstratethescalabilityofthesimulationmodelandtheuncertainties 1993), ANOVA (Adetula and Bokov, 2012), Shapley effect (Radaideh
taken into account inthe calculation ofthe BEPU (evaluation model) etal.,2019c),etc.
methodologyattheNPP,itisrecommendedtostudytherepresentative Theeffectsofuncertaintiesthatareappropriatelyaddressedinthe
IET of the transients of interest on ITF (Integral Test Facility) at dif- simulationmodelarereflectedintheresultsoftheuncertaintyanalysis,
ferentscales.TheBEPUmethodologyforIntermediateBreak(IB)LOCA including the probability distribution for the SRQ. For certain un-
based on the propagation of the input uncertainty approach with the certainties that are not properly accounted for by the propagation of
CATHARE best-estimate system code was applied to two IETs re- uncertainties and where their impact cannot be evaluated by alter-
presentingaselectedIBLOCAtransientfromthetwoOECD/NEAROSA nativemethods,compensatorymeasuresmaybeproposedtooffsetthe
tests.Theresultsindicatethattheuncertaintiesofthephysicalmodels impact of such uncertainties. For example, these offsets may include
consideredintheBEPUmethodologyprovideuncertaintybandsofthe elimination of such uncertainties through design and operational
peakcladdingtemperature(PCT)thatboundtheexperimentaldatain changes,aswellasprocedures.
both tests. In particular, the maximum (95/95 percentile) of PCT for Finally,thedecisionontheadequacyoftheBEPUmethodologyis
eachtestisgreaterthanthemeasuredPCT.Theconsistencyandcon- madeonthecomparisonofthecalculatedSRQ(includinguncertainties)
servatismofthemethodologyforIBLOCAarethereforedemonstrated. againsttheacceptancecriteria.
4.2.5. Assessmentoftheadequacyofthesimulationmodelanduncertainty 4.2.6. ApplicationofgradedapproachofVVUQrequirements
analysisforrealplantapplications Insomecases,theuserselectsafullydeveloped,verifiedandvali-
AsshowninSection2,thefinalobjectiveistoapplythesimulation dated generic purpose system thermal hydraulic code (such as the
modelbestestimatepredictionanduncertaintyanalysis(BEPU)method USNRC’s RELAP5 (ISL, 2001) or TRACE code (USNRC, 2008) for the
10