Table Of ContentData
Reconciliation
ross Error
& G
,
Detection
An Intelligent Use
of Process Data
Shankar Narasimhan
and Cornelius Jordache
Publishing Company
Houston, Texas
Data
Reconciliation
& Gross Error
Detection
Copyright 0 2000 by Gulf Publishing Cornparty, Houston. Tr, our grtru Professor Richard S. H. Mak, who played
Texas. All I-ightsr esel-ved. This book, or parts thereof. may tlze roles of an initiator and a catalyst.
not be reproduced in any form without express written
permission of the publisher.
Gulf P~~bl~shCio~migp any
Book Di\,ision
PO. 13ox 260s 1H ou~lnnl.' esas 77252-260s
"Since all measurements and observationb ar-e nothing more
than approximations to the truth. the same must be t!-ue of
-
311 c:ilcuiations resting upon them, and thc: highest am of
all cn:npu!2iior!s matie COiICZi71iEs c0i:Cr::ie ~!:cllOnlsila
i:lcsr t ?!~o approxirnatc, al; ncari)- YLL pr;?c!icab!c. t ~i!li2
tr:!ii~. Rut this can be accomplisi~cdi n ;lo OCIICS \<a?,i !l;?i;
5! ;I .;:litab!e coinhiriatioo of rnorr nhse!-vaiio~liO ::r!? r ! ~
1;umL.er- absi;l~ite;y reyui,ite ti): the determiria~iono f :he
unl:no\%~qi uantiiies."
Contents
Acknowledgments, xi;;
Preface. xv
Chapter I: The Importance of Data Reconciliation
and Cross Error Detection, 1
Piocitss Ihta Conditioning Metl?ods . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Iilduhtriai Esamplcs of Stcidy-State Dats Rccor-iciliaiioil . . . . . . . . . . . . . 5
Cr-udi. Split 0p:il-r~izationin a Preheat 'Train of a Kei~ner-y . . . . . . . 3
?vIin~!l?izinWg ater Consumption i rh~l inera: Beneficiation Circuir\ . . 7
ilara Recdnciliatio;~P roblem Formulztion . . . . . . . . . . . . . . . . . . . . . . . 1.
Exa!nples of Slnlpir iiecoilc:liation I'roblems . . . . . . . . . . . . . . . . . . . . . i 1
S!:stt"ns ?Yith .4li Measured Variables . . . . . . . . . . . . . . . . . . . . . . . . . 1 i
S).srems Wit\, Unmc.usurcd Variabies . . . . . . . . . . . . . . . . . . . . . . . I 1
Systan Cor?tain;ng Cirosi El-:orb . . . . . . . . . . . . . . . . . . . . . . . . . ---- 17
Zitrlzfir.; iroir? Data Recurlciliation and Gross Err-cr Detection . . . . . . . . -7 ; j
.I\ B1-it:' H'istory ofi1a:a Recoriciliation anci Cross I:rr:,r 5etection . . . . . 2 1
Sco!~an d Osgi~nirationo ftlie Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . -? 7i
S~~liiiriar:. ................. .............................. 7- 7
l<efe~-i~~.i c. r. s. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . :D ,,
Chapter 2: hleasurement Errors and
Error Reduction Techniqtles, 32
Cl:t\\if~i-,ition of h.leasurements Error\ . . . . . . . . . . . . . . . . . . . . . . . . . . . :?-
Random Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
Ciross Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
Err-or Reduction hletliods ................................ 38 Solutioi~T echniques for Equality Constrained Problems ............. 122
Exponential Fillers ..................................... 40 Methods Using I. agrange Multipliers .......................... 122
Moving Average 1-'ilters ................................. JS Method of Successive Linear Data Reconciliation ................ 124
I'olynomial Filtel-s ........................................ 5 Nonlinear PI-ogramming (NLP) Methods for Inequality Constrained
Iiybrid Filters ........................................ 54 Reconciliation Problems .................................... 128
Summary .................................................. j6 Sequential Quadratic Programming (SQP) ...................... 129
References ................................................ 57 Generalized Reduced Gradient (GKG) ......................... 132
Vari;~bleC lassification for Nonlinear Data Reconciliation ............ 134
Chapter 3: Linear Steady-State Data Reconciliation. 59 Comparison of Nonlinear Optimization Strategies for
I..inear Systems With All Variahleh Measured ................. 59 Data Reconciliation ........................................ 136
Cieneral Formulation and Solut~ol.~. ........................ 59 Surnrnary .................................................. 138
Statistical Basis of Data Reconciliation ....................... 61 References ................................................. 138
Linear Systerni With Both Meawred and Ul~r~ieaiureVda riables . . 63
The Constructiori of a PI-ojectionh 4atrix . . . . . . . . . . . . . . . . . . 66 Chapter 6: Data Recczc:'iation in Dynamic Systems. 142
Observahility 2nd Redi~ndancy. . . . . . . . . . . . . . . . . . . . . . . . . . 69 The Xeed for Dynamic Data Reconciliation ....................... 142
h4atri.i Decornpo\itio~M~ tihod\ . . . . . . . . . . . . . . . . . . . . . . . . . 70 Linear Discrete Dvnamic System Model ......................... 143
(;rap11 Theoretic \letl~od . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
Optinial State Estimation Using Kalman Filter ..................... 148
Otlitr Ciassificutroii klcthotls ............................ 77
Analogy Between Kal~nanF iltering and Steady-Stare
kj \.t iri?atii12 Mxi\tirsiiii.nr Error Co?. I 1.iank.c hl~itrli . . . . . . . . . 77
I / Data Reconiilistiori ..................................... I53
Si~i?i;iatiorPl Cciln~iiliC!' or ~\~!lli:iliii1gj at;i ii~~~;i~iii:i.ii.~.ii. . . . . hi Optiiil31 Control and Kalrn-n Filterin? ........................ I55
Slilllll.l.~ :l-\ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . >? K;~l:ii;l~Fii l:cr l~llplen?eiit:ition ............................... i57
J: XI.^..., rice\ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . S? 1)ynaiiiic D;:u Reco~iciliationo f Non!incar Syxrenis ................ 1 i)O
h'oii!irtea~S- tate Extini:itior!s ................................ 160
ChBpter 4: Steacf-State Ilata Keconcilistion fcr Nor;linzar Data Reco!-iciliation Mctliods ....................... IOl
Bilinear S\ stern\. 83 Su~:;rnary ................................................ 171
References ............................................... 17!
C!~apter7 : Introduction to Grass Errnr Dt-tectio~i1. 73
Problc.~S:~ta tem~n!.. ........................................ i74
Bas!c Stat;s!ical Tests for Gross En-or Liercction ................... 175
T hz Glohril Test (GT) ...................................... 178
Thc Constraint or Noddl Test .............................. I80
Thc hleasurcmc~ltT est (MT) ................................ 1 X i
The Gclicralired Likc!iliood Ratio (GLR) Test .................. ;., 5
Cornp:ii-ison of thc Powcr of Basic Gross Error Dciection Tests ..... 190
(;SOS\ Error Detectioli IJ\iii$ PI-incipal Cornporicnt (PC) Te\ti . . . . . . 1%
PI-lncipalC omponcilt Tests for Residu:ils of Process C:onstlairirs .... 196
Principal Con~poncn'tl 'csts or] Measurement Adjcstille~its. ....... 147
Relationship Bct\vccn Principal Component Tests and Otlier
St:~tisticalT ebts ......................................... 198
Statistical 'Test5 for General Steady-State Models .................. 290 Chapter 10: Design of Sensor Networks. 300
Techniques for Sinzle Gross Error Identification ................... 203 $
Fcs tirnalion Accuracy of Data Recorlciliatioii ...................... 30 I
SIdeerniatli fEyilningi ian aSti~nogrlelS tGrartoesssy E frorro- rI thle5n; tPifryinincgip a3l SCinOglIeI IGIrIoOssI T~Ee~rsrIcts1I 1~.- ... ..... 21C07l4 Sensol- Network Dcsigr? ...................................... ?(I?
Methods Based on Matrix A1gel)ra ............................ 303
1jetectability and Idzritifiability of Cross El-rors .................... 100
Methods Rased on Graph Theol-y ............................. 3 l i
Detectability of Gross Errors ................................ 2 10
Methods Hased on Optimization Techniques .................... 322
Identifiability oiGross Errors. ............................... 114
Developments in Scrisor Network Ecsiyn . . . . . . . . . . . . . . . . . . . . . 323
PI-oposedP rohlerns .......................................... 2 1 7
Surnrnary .................................................. 323
Sumntary ................................................... 223
Kef$c.cnces ................................................ 325
References ................................................. 224
Chapter 11: Industrial Applicatiorls of Data Reconciliation
Chapter 8: Multiple Gross Error Identification Strategies
and Gross Error Detection Technologies, 327
for Steady-State Processes, 226
Process Unit Baia:lce Rccuuciiialion and Grus; Error Deteciion .......
Stmtegies rnr X,luitipis i;!-oss Emor Identification in Linear Processes . .
Parameter Est1rna:ion and Data !?eco:lci!iciti~~. ....................
Sin~ulr;,i?eouSi trategies ..................................
sc,.1:..1 1' >- ri-;~te~:~:.; ...................................... Plant-W~deM ateria! and 1Jtilit;es Kecoilci!iaiioii ..................
Coir!i>ii;a!ic)i-iA Si;lti'ci~\ ............................... CascStcdies ................ .......
Pcr-fori!ianci. .'il~-a,u;-:.\ for Ij\.al~iatinsG ross fir!-01-! ilcn!i!Ycativil Rccnnciliation of' Sef,ne:.> Crude F'iehcar T:-air; E131:: ............
Keco~iciliatioiio f A-numoniu Pix: Dac;: .....................
si!-,llcgii,s .............................................
Sumiiiar-). .............................................
Coi?ip::i-i,o!~:I: J.i;ri:i;iiz GI-:.,I\E\ i-1-01 !deiitrf'~c;i~ioSri~r a~ezies. ........
f2~:fcrenczs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
; c:I?-<<> l,I \ : ! !I ! 1 . . .' . . . . . . . . . . . . . . . . . .
. E,-:(,c [<:c.;:,fi(:j;;<;!: tr .!<.(>.1;.!!!.j,.>:.ll . p!-:l~,:,$~s t.,Tiiil?
.\r- :r;!i!::,ii GL2. l:cii~:,ci .......................... -
s i r tit i r ;i i : i t i n ........... \<cto!-a ;i;li: Thei:. PI-operrics. ............................. 1.- . - 3
!?-oix!\cS Pi-i>h!e;n\ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . VIar~+.ces and Tiirlr PI-opeitis .............................. 2 -I>--
-
S?!l!ii!1~1." . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Refs~-cnces. ............................................ i//
Kc!'~PLI?c.~.>. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Appendix R: Gralth Theory Fundarr:enfal\. 3-3
Giaphh. PI-cicessG I-,i;;i~sa.: ici Su!;sra;lh\ . . . . . . . . . . . . . . . . . . . 37t.
t'at!i.;. Cycles. aiitl C::nnccri~iry . . . . . . . . . . . . . . . . ,. >:> 1
rj-
I o>h !:r:i Forn1uiati:)n fill- :>e!:<rion of >\leasiiri.lnrnt f3i;ica ......... 2S2 Sp~niiinp'l 'recs. I(~-ai?lheszr.i d Cti,?;.~!... . . . . . . . . . . . . . . . . . . . . . :;SO
Stnrihiical Pi-o;?i.:~izo.i-' TI.IT~~i\ tiii~i;~rril d the Global Tc\t ............ 781 Graph Operations .................................... 7.L>l;,
Gc.ili.i-al~L~iih~i.~ii h<><)i-:i:ii! r,) hI~'[liod .......................... 289 Cutset\. I-undamcniais C~.lrsets.a id F~ir-ida~ni.rltCa il cie.; ............ ?" I
-
!-,iult Lliagiio\l\ 'I~~ih111~.~.i.ii..>. ........................... 3 9 .c, [{efcrence ................................................. 3K3
'I'i1c St;ll< of the ‘Ai~t . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . -7 07
Siilriiiiai-y ............................................ 20s
Kefel-enL.c\ ................................................ ?CIS
Appendix C: Fundamentals of Probability and Statistics. 384
Randorn Variables 2nd PI-ohabilityD ensity Functions ............... 383
Statistical Properties of IZandom Val-iables ........................ 589
Ffypothesis Testing .......................................... 391
References ................................................. 393
Acknowledaments
Thc authors are indebted to several people who have contributed to the
preparation of this book. The main contributions came from Prof.
N;lrasinlhan's students at the Indian Institute of Technology (1lT)-
hladras. 'f. Rcnganatllan and J. Prakash. currently doing their doctc?ra!
prozrams. prepared the solutions for all examples lvith assislance fr-om
Sreerari~M riguluri. Mnrukuria Rajar~~ouslpi ent 110ui.s in :he riigtit psepal--
ins
tlic ~abizsa nd iigurzs in dil.l'erent ch:i~::eu. Thc s~~ccessficilc niltle-
iioii of ii-~i,b ook was clue to their eftoris. ii:--S. ;icl-iln Pat\i.a!-dhan ;1.!1(! S.
i'cslir;~\ an;im. faculty at f Jl' Mdras. pro\~icli.c!c ritical in~)t!isi i?ii: ipro\ c
tile 'li!c.us and c!ari:y of- tni: text.
Thanks are also due to Liii: RAGE softv.-are cie\~zl:)pment tarn at E~gi-
ncers India Lirni!cd, R&D Center. consisii~gi\ f Dr. i\ladhukar Gasg. Dr.
V. !?a\ ikuliiar. and XIIS. Sheoi-aJ Sin?i! fi-ti~;: ~,~hclirP?I -<ti'.N 3sasim!72r:
. .
~ained\a id::bic pi-acrical cx:;cricr:ce in iin!~leixie~iindga is reco~~~:ili:i:i~>ir
to indiistrial ixocesscs. Prof. Ysras~rrrhrtna iss rhafiks l'i-of. Jcse I<:?go:
a:id DL-D. idier Maquiri at CRAN-INPL, in Nancy. I-i.ancc, for- a1-rai:ging a
:;urulne!- visit for hint. during which a significan: part of the iexi rx:;i\
com!Aetcti.
Dr. Jordache thanks all colleagues and ;I-!an:iyr-i; Clic~nsh;l~-c.
ft-om
Coi-p.. Raytheon Process Antcirnation. and cspeciall! Sinlulation Sciences
liic.. for ci~allerigingh im with practical issues that helped hin-i ciarii'!.
niarly implenientatior~( jztails for data I-cconciliarion technology. Dr. Tor~i
Clir~ksc;liesp i-ovidcd valuable input on ?olynonlial filter-s. Drs. Miel!-Jii~g
Lin and Ricardo Duiiia helped with clarifying sonie liteoreticai dcri\,;i-
lions mentioned in :he text.
Special thanks to thc R&L) management of Sirrtulation Sciences Inc.
iSIMSCIm) for the encou:agernent and support given to Dr. Jordachc dur-
ing the writing of this manuscript and for allowing him to use DataconTM
software and the training book for preparing a dctailed ind~lstriale xam-
ple included in this book.
Very special thanks are also due to our respective wives. Jaishree
Preface
Narasimhan, and Doina Jordache. who shielded us from all the problems
on the tlorne front during the past two years.
Moreover, the authors express a heart-felt gratitude to Debbie Markley
of Gulf Publishing Company, who patiently reviewed every detail of the
manuscript and worked long hours to help with finishing the index and to
make sure this book is as accurate as possible.
Fiilaiiy, the authors want to acknowledge and thank Prof. Miguel
Bagajewicz for his excellent review and very useful suggzstions on addi- The quality of process ciata in chemical/petrochemical industries sly-
tionai 123aterial to be included. rlificantly affects the perforniancc a11d the profit gained frorn using v:i!-i-
OLIS software for process monitoring, oiilinc optilr~i/atioii,a rid contro!.
Unfortui1atcly plant ~ncast~re~noeft~e~n tcso ntain errors that invalidate [lit.
prows mode! used for optimitation anti coi~~roDl.i m rectmciliation and
gross error- detection are technique:, deveioped iil the p;~st3 0 years i'oi~
i,n~"uvin?1 11s accuiacy of' data so that the!- s:iti~t?) :he p!ant ninclzl. Dur--
i!;~ the l:!.i decade. thsy ha:.;. br!:n r\idtl~a j~pii~i~r! iire fine!-ie\. pe:l-.:,-
~lierlli~pllld nrs, n~inera!p rc)cessir!c i:idusrries. and so L~rtll.i! l ili-ci~i-
achieve i:iorc ai.cur:kte plant-wide accour?iins :rid superic?r-p rofit~ibiiiiy( 1:
plailt opera:lc~ns.
A!tnough cor~iinercials oftware h r dut;t rcco~lci!i:3rion and grosi cn-or
dercction are ;.,\,ailal.le, t!lc accnir~paiiying~ nsilualsu sual!). gi\-cf ii:tle the-
or-et~cabi ack:_.soar!ti.I n c;r-df:r ro I)c ah!< tc: sc:ecr ihe !IS>;~ nerhnd.;; r!!,i
?air: tilt: iiloi; bci:cfii:; from ilieir iii?p!z1nzn1aiiurl. :)nz nedc a sr;oi!
uridersta:!dil~z of the Fui:damcr:ta! coiiceprs. 'This l~(?~cix.p lai~~:!I- s ??a-ic:
concept, in datd reco~~cili~iti2onnd rross erTor detection usir~?m arly i!lv<-
trativi. exainples. It also contains descriptions of diftsreii! techfiiquei that
have beel: cizveloped for these ~ur-no.;esa nti prescnth :r. unified perspecti\-e
of these di\-crsci nethods. Certaii: crircna for selectin: \,arious tcchniqus
arid guidclincs for their practical i~nplcn~rnratioanre al\o ~ndicztcd.
The fhcus in this entire text i\ 011 classical rllc~t-itiii\t hxt nuhe use of
~)roceisii~ode~l~ 111~tioSnLsI.C ~ c(:I~\<~\.~l~;l:\vosI. Ie quiiib:-iuni I-c!:i-
tions. ziilti equiprner~tp erformance cq~~arion:t,o. rect!r~ciie data arid to
detect and iderltif'y g;-oss errors. In rcczr~ty ears. the usi. of' artificial lieus-
a1 net\vor-ks for- data reconciliatioil and gr-oss error detectior, has bee11
The underlying assumptions, char-acteristics. and relative advarttagcs :and
proposed. These meihods have not been includeti because they have riot
disadvantages of various statistica1,tests arc also discussed. FOI-i dentity-
iittaincd matur-ity,a lthough they ntay become important in the future.
ing nnultiple gross errors, corl~plexs trategies ar-e requil-ed. A plcttior;~o f
This book is organized in such a way that it is useful for both industri-
strategies have been proposed and evaluated in the I-csearch literatur-c. \ ,
al personnel and acade~nia'.P he book will be a valuable tool to enzineers
and managers dealing with the selection and implementation of data rec- special eff'or? has becrr made in Chapter 8 to gi1.e :i ~inificdp erspccti\c h
classifying the different strategies on the basis of their core pr-inciplzs. \t'e
onciliatio~is oftware, or those involved in the development of such soft-
also descr-ibe in detail a typical strategy fronl each of these classes. Chap-
Lvare. The book will also be useful as a supplementary reference for an
ter 9 treats the problem of gross error identification in dynamic systcr~~s.
irndei-g~aduatelgraduatele vel course in chernical process instrt~mentation
The efficacy of data reconciliation and gross error detection dependi
and control in which basic concepts can be taught or as a text for a full
significantly upon the location of measured variables. Recent attempts to
graduate level course in these topics. Unlike this book, the other books
optimally design the sensor network for rnaxinijzing accuracy of data
that are currer~tlya vailable on these topics do not present an in-depth
reconciliation solution are described in Chapter 10.
analysis of the different techniques available, their limitations or their
Several industrial applications and existent software sysienis for data
interrelationships.
reconciliation and gross error detection are also discussed in Uhaptc.1- 11.
The book is organized as follows: The firs, -;lapter motivates the need
Various aspects related to the benefits of offline and on-line data reconciii-
foi- data reconciliation and gross error detection and introduces the major
conccpts in\'olvecl. Chapter ? introduces the statistical cIia1-acterizatio~oi f ation. the methods n-tostly used and their perfo~manccsa re analy~cdh rre.
In order to make this book self-sufficier~tw ith r-espect to the n:xrhi--
liiea>llr.cnlellte rrors mil v:irious filtering techniques used for error rzdtic-
tiori t!l;it can for111 jl;111 of ail eve!-all data processing stratezy. The nest ma tic:^? background required for- a good undzrstandirl~.: tppzridice> iiii.
irlc1udc.d tl~atd escrihe the necessary baic co:iccpts fro111l inex a!gsi?i:a.
fnur ch:lpterc dcsl n,ir11 ihc st~bjecio f data reconciliation in incre:i.;ing
i?: si o!' con:ple~ity.S trad! -sta!e li11ea1d- iita reco~~cilia:ioinis il~sc~ihei~d i %rap11t !:co:v. atid prohabi1i;y ar;d :,ta:isiicai h\;po~hcci.t;c \tiil:.
C'i;~pti-r?- . ~?::iolrlpo\itii~n~ ecilllii!!lcs fill- Iirlear model with borh n:ea-
.;i~i-i-iis rii! ~~nmi.~i;ii.~ i- ~i-ic.i i?~ ibli:ti,r e tiesct-ibc4 here. The techniques rei;~~.-
s.i rcj !!IL.c ia\sii'i~stii)~of: :;ti-iab!cs as obser\.ablz- anj rc.~iundanta rs aI-,o
p! <\e:~ic:i. >fe!hocis (01 c~tiiii:i~itihlc~ ~ie;<surer]zrlCt:- SOI- v2ri:ir1ces fic111
!ilca.urcci dxta arc also cii'sc~ibedi !~th i~ch apter.
C!iap!zr 4 de'~l:, \\ itin htsadv-state data reconciliatroc For- biliricar sys-
:cI:-!\ ~orlsi>ti~~~y. (-~ri:y~lb~:isln;i!t~ ce~~ tild.~ ICI CIT~~~II: i\e<e,i ierz>. h<!i-
~:!CU. 7'11:' ~i~~:i\,;~t9tri n~~i>i ~l\idi-~si-lci~kii gpr occssei is iI1cst1-a:ec! winy
?>Lz~i~I'I~~OIpV le11~1 i1i~~:ll indu\tl-ie~:is ell :I:, utili~yd i~~i-i13u[io~i
ili.r\\ i)?-ks III C~C~IIi~riC~l;t:I\t!r -ie\.C l~:ijit5~ ~tt.- ~iltnj or1lil:ear data rcco~icili-
::tii)r:. N:)niiilear- ~nocizl\a r-c otr~iiu sed to ;;ccur-atcly describe chcnlic;~I
pr~ces.;cs.7 '112 iii<:~~eif t icie~i:3 i1d widel) used so!urion procedur~cf or :he
non!rnear I-econciliariorn 31-ep i-escnied in this chap:er. Ilandling inequalir
i'oil\ti.aints. qircii as hotrridh oil ~ar-iablesi.s also analyzed in this chapter.
I);ira reconciliati~~le~cii iiliqut. for dyniirnic systcllls are discussed iri
C";-i.lprer (7. Both Kalnlar~i iltel-ins nlettiods and gerleral optimization tzch-
iliquei riesigrrcd for- d\ na~iiii.r iordinear problc!ns are pre.;en!ed.
Ct~:ipters 7 throit$~ 9 tical tiit11 the pr-oblem of gross en-or detection.
Chapter 7 intrctd~~cth~es issuz\ irlvolvetf in gross error detectior~a nd
~k\~.i-ihtil.rsc il:l\ic \t;lli\~izalt c.;l\ th;~! call bc wed to dc.tcc1 grciss er-~-or..
The Importance of
Data Reconciliafion and
Gross Error Detection
PROCESS DATA CONDiflONLING METHODS
In aily modern chelnicni p!ant, pctrc~cheiiliiaip rocr3si or refiilel)..
h~ndred5o i- eve:; tiiousai~dso f variabiei---silcfi 2s Eo\v r:;:c.s. terr?pei'a-
tu!-cs, pres.;i!i.c;>, ie\.els. ~ : cdoi iij~o~i:io:~s---31~-e. oL~:~InIizea!s~u red ail,!
auio!na:icaliy I-ccoi-dedi '<ji- tile purFc>c of proc<a.: contro!. online op;i-
mizatio::. <>i. process econo:lii; e.:,tiua;ioi:. h,Toc!crr~c ompurers 2nd dam
l;cc~~iisitiosny stein\ faci!itate the col!ection acd processi~go f a 2re:;t
:.o?i!rne of dai;~o, fteii ;actpled \\ itil a frequency of the order of rnifiutes
or ei7cn secc)~:ds.
The iisz of corilpctcrs no: on!\. ai!i;\~;sd 3t3 tc: be (;b[?,irliC: iit ii si2a:e~
freqaency. hgt bas alio :-ei~!itcti: n thc cli;l~inalic:i:~)f ei~orsp resent in
~nal-iualr ecoi-dilly. This ir? ii!;c!i has srcat:y iii~p!-o\.ecr!i ?e sczuracy ~icd
\:a!iclity of process data. The incre;irii aiEi)Llpt of i!:!iirn;aiio~:. !loweve:..
ciin be cxp!oitc<l for SLIT-theiri ~lp~-o\~itihi:e liccardc! arid cunsistenc). of
.,-,v. ,,.,,,-,.,n v s ,I,,+,, t, t~.~"g!:I: systematic data checkizs and rserttment.
Process 111eastircmentsa re inevit2ibly coi.rup~cd .!I! errors d~iriilgt he
ri:casur-erncnt, processing and trans~niaiono f' the rr?easured si~nalT. he
total en-oi in a ineasurelncnt. which i.; rl~cd ilikrencr hct\veen tile mea-
sured value arid the true value of a v::i-iahle. car1 !7e co~i\.c~lien[rie>p. re-
sented as the sum of the contributions fro111 o types of errors--rzzrzdorll
cl-rnnr a:ld gl-oss errors.
The tel-in r-rrndor~z~ rrori-in plies that lieither tile ~n;ignituden or the sigr:
of ~ h eci- ro:- can be prctlicted \x,~ittl ccr!ainty. In othci- u.nrt!s. if the Inca-
surcment is I-epeatetl uith the banle in:,irurnent under- identical process Research and development i11 the area of sigrlal conditioning have led
conditions, a different value may be obtained depending on the outconle to the design of analog and digital filters which can be ~sedto attenuate
of the rantiom CITOI-. Ttit' (311lyp ossible way these errors can be chal-acter- the effect of high frequency noise in the rneasurerncr~ts.1 ,al-ge gross
izcd is by the use of probabiiity distr-ibntions. en-or<c an be initially detected by using variotls data validation checks.
These errors can be caused by a number of different sources such as These include checking whether the measured data and the I-ate at which
power supply fluctuations, nc~workt ransmission and signal conlersion it is changing is within predefined operational limits. Today, smart sen-
noise, analog input fi1te1-ing.c hanges in ambient conditions, and so on. sors are available which can perfol-m diagnostic checks to determiric
Since these el-rors can arise front different sources (some of which may whether there is any hardware problem in rneasurerncnt and whether the
be beyond the control of the design engineer), they cannot be corllpletely rncasured data is acceptable.
eliminated and are always present in ally ~lieasurementT. hey usually cor- More sophisticated techniques include st~zfisticalq zrali~c ontrol tests
respond to the high frequency components of a measured signal. and are (SQC) which can be used to detect significant errors joutliers) in process
tlsuail~s. rnall in magr~itudee xcept for some occasional spikes. data. These techniques are usually applied tc> each ineasured variable sep-
On the other hand. ~IIJSSP I.~JI-.Sa l-c cauied by nonrandom events such arately. Thus, although these methods improve the accuracy of the mea-
35 instrument malfullitionin_r rdue to inlproper i~istallatio~olf rncasuriny surements, they do not make use of a process rilodel and hence do not
de\ ices), miscrilibratio~i.u Far- (11-c nr-I-oio11o f sensors, and solid deposit<. ensure consistericy of the data with respect to the interrelationships
The ~~oili-ailcion~a1i1i 11-(c1: ij.~i',ir: 'i~oi-i<ii lplies that at any gi\:eii ri!ne the); between different process \~ariablzs.N ever~tieless,t hese teciiniqnrs rrlust
ha\< a certain niagnirude aid 5ign \xhicii nia) be unknown. 'l'hus. if the be used as a first step to reduce the effect of randoin errors in the d;iw
.
n:cast!rtnteiit i\ rzpel;:i'!.! \::ti.? rtii' <;iiiii. iri<tr-umentu rldcr identical iondi- and to eliminate obvious gros:, er-rors.
ti~ri,. t1:e con~r-ibuiioor~r a \\ ~krilatri~ I-(IS> err:.)[- to thc rneahiirzd \aiile It is possible to further reduce the effect of randolii erroi- and ;:]so eliili-
u-iii bc the S:~III<. inate systematic gl-nss er-I-or-i ll the dar:i hy exploirins tiie i-ei:i:inns!iir~.;
31, <<>!j~\~g.ioicigd i,?\!Liii:a:i<>~t; ti:(l II~~;:I:ICII:II!CC !>I.OCCC!I.J~C>. x. i I.!, p<- tiiat arc k;~:)wn to exi~Ci tzr\vce11d iffel-ent \,ariahies <if s process.
iibie ti> eIl$~lI~!hC::! LT!',i\, Cl.ttli\ Ji.2 1101 ?rc\elit 11; I!-&<1 11~~5LIl~~~l~Litl ~!l!S tech~iq;li.so f tlara ~-~cc;tlc.iIiclfioarn:d gi-ox er1-0r d~fe(.rioit:i xt !I;\\.&:
I.>, !'or- s:‘ri11e :r!~it:. (;:-o,< rii-oi\ c;:i!.\cd 17:). hellso:- i~~i\~iilib~-aini?ca>>: i 'Jeer: dcvriop~ii~ :i he field of chcmica! e~igineeririf dtirins tile past 75
ol~curs ~!dtIcnl; at ;I p;ir-tii~li:iit-i r;ii' atid 1!7el-~:ai'tcrr ei~laiil:I : 2 c.-on>tai!t y7;ars fc~trt lis parpose are :iie pi-inci;~alf ocus of this book.
lei i'l o:- r~t:yr-,iri~~0Ite!1. ~2s1i~n )\<~I- I-OI- criu ,eb such as the \.T e21- tc~~~li~~g Data reconci!iation (IIR) is a technique that hai heen deve:op-,d to
o+ \enL;<)!.c>i1 1: ClcCtir ;~-;iiic!:il!>, )vc:- :i 11crio<!o f time arid 50 t!ie i-,l:i~!~i- impre>\e ~hcac cilracy of 1iieasurclnei:~si )y red~~ci~thie? effc>ci cf r;tlictoin
iud? of !:I,' g[o\> erfi!:. :!ic<~:t\,:\\; <I\\ I,,' ~G\'I?sr ~!;i!i\.el>i ~>i!gT ~I~I;KJ,C I-;- err-or. iri ilic dats. The priilcipal dilli.rence he~n,eitnd ata i-c~<>i;~iiia~riiti i~;;
~iti'1. 11~13$. i')><< ::XII\c -~;;i.ii- Ic.\\ !.I-cc~LI?I!>>TQ~I1 !h ei1.I I I ~ ~ I I ~ ~21L<I i>~'[~>iC- \ otlrer fillelin: tcch~?iq:iesi, ihnt da:a recancili:itioii cipiiciti\~r nahe. 1i.e oi'
<2l 11 l:i~;e~ rlia~!.~l~ ohcL )C :-ci~:~Li:Io-I-~OI-~~.~ process niodi.1 constraints and obtains es:inlat.-5 of process :,ariahies hy
Eii-k11-3in nlea\i~!cd: :ata i'iil lc;ici to :,igriitic:~n: dctc!-io:-atio~ii n plarit adju:;ting ~~r(xensicsa suu'-ernznts sc; th:it tlic. eslirnate.; x;it15f?~ :he c(>i~si~~:~iiiix.
13~1-fol-mai~Scnei.~ iilr ;~;idoili, iil,J ~I-G\. ci-i-:)~c-\a n lead to deterioratiori in The r-?i~oncilede siinlates ar-c. ;spetted to be inore accurate tji:in tht,
[!1i' pe~-forriia~iccfc cn!!!i(-l . -!r.!:!r. \?!?i'!-e;fi! :!!-ger gross cr-ror-sc an i~ulli- i;Li;l,~,-~,tiitainid~, more irupclrtantly. ai-e also consiaient \\'it11 the kll~~u'11
6. gains :icliic\al>le rhi-oil,~ip! :{>cfiso prlii~i/arion.1 i1 \omc crseh. err!-,- ~tlatioiisliij~bse twee~tp rocess \lar-iables as defined by the constraints. 111
I~<OLI\ d:iti~ c;ixl a150 d:i\e iilc p~-occ<isn to ;ill u~iecc~~~oor~-t.-leic\s ~i order for data reconciliation to be effecti\~c,t herc. sllonld hc no gros.,
',\or~c-rrii LIIIS:~:o~p ~r~tt1~-erg~1?11 111e .I > t\~c.rcSo~i~-c~ ipoi-t:t~or r~~t ~lucicf . er-ror-s eithcr iri the measurerncnts or ill the pi-ace.; ;iiodei constr;~ints.
ni)t ccjrii~~letclcyl iriiiitdti.. :ic eftcct of hot11 I-antlo111ci ricj yrct:, elxor,. Gro';> erro~d-e tection is 21 companion technique to dats reco1.1 ciliati:in
.
Sc\-er-al ciat; pi~occ.;\i~tgte ihitic]i~ehi ,ail be used together to achie~cth is ha\ beer1 tleveloped to identify and clinlir~iiti:2 1-osse rrors. 1 hur, data rec-
objective. lit tlii re\[. ~ {dci.h cr-i!>i.r i~cthodx~ \-t~icc;h~ rtp lay an iil111oi-t:irlt onciliation and gross error dctcction are applied together to irnpro\.c
rc:le aj p:wt 01 ;ii1 iriiity~.:i:e~iii3 1;t pr-occs\iii~s trategy to reduce errcrri in accuracy of rtieas~rredd ata.
lil?~i~!ll~~l1lC1l1l1~:t5tie 111 ~O!lii!l~l~>{)~I-lll;\i: <\ iil~~U~trif?h.
