Pragmatic Enterprise Architecture Strategies to Transform Information Systems in the Era of Big Data

363 Pages·2014·10.13 MB·English

by  James Luisi

Pragmatic Enterprise
Architecture
Pragmatic Enterprise
Architecture
Strategies to Transform Information
Systems in the Era of Big Data
James V Luisi
AMSTERDAM (cid:129) BOSTON (cid:129) HEIDELBERG (cid:129) LONDON
NEW YORK (cid:129) OXFORD (cid:129) PARIS (cid:129) SAN DIEGO
SAN FRANCISCO (cid:129) SINGAPORE (cid:129) SYDNEY (cid:129) TOKYO
Morgan Kaufmann is an imprint of Elsevier
AcquiringEditor:StevenElliot
EditorialProjectManager:KaitlinHerbert
ProjectManager:MalathiSamayan
Designer:MatthewLimbert
MorganKaufmannisanimprintofElsevier
225WymanStreet,Waltham,MA02451,USA
Copyright©2014ElsevierInc.Allrightsreserved
Nopartofthispublicationmaybereproducedortransmittedinanyformorbyanymeans,electronic
ormechanical,includingphotocopying,recording,oranyinformationstorageandretrievalsystem,
withoutpermissioninwritingfromthepublisher.Detailsonhowtoseekpermission,furtherinformation
aboutthePublisher’spermissionspoliciesandourarrangementswithorganizationssuchasthe
CopyrightClearanceCenterandtheCopyrightLicensingAgency,canbefoundatourwebsite:
www.elsevier.com/permissions.
ThisbookandtheindividualcontributionscontainedinitareprotectedundercopyrightbythePublisher
(otherthanasmaybenotedherein).
Notices
Knowledgeandbestpracticeinthisfieldareconstantlychanging.Asnewresearchandexperiencebroaden
ourunderstanding,changesinresearchmethodsorprofessionalpractices,maybecomenecessary.
Practitionersandresearchersmustalwaysrelyontheirownexperienceandknowledgeinevaluatingand
usinganyinformationormethodsdescribedherein.Inusingsuchinformationormethodstheyshouldbe
mindfuloftheirownsafetyandthesafetyofothers,includingpartiesforwhomtheyhaveaprofessional
responsibility.
Tothefullestextentofthelaw,neitherthePublishernortheauthors,contributors,oreditors,assume
anyliabilityforanyinjuryand/ordamagetopersonsorpropertyasamatterofproductsliability,
negligenceorotherwise,orfromanyuseoroperationofanymethods,products,instructions,orideas
containedinthematerialherein.
LibraryofCongressCataloging-in-PublicationData
Luisi,JamesV.
Pragmaticenterprisearchitecture:strategiestotransforminformationsystemsintheeraofbigdata/
JamesLuisi.
pagescm
Includesbibliographicalreferencesandindex.
ISBN978-0-12-800205-6(paperback)
1.Informationstorageandretrievalsystems–Business.2.Managementinformationsystems.
3.Businessenterprises–Dataprocessing.4.Businessenterprises–Informationtechnology.5.System
design.6.Computerarchitecture.7.Organizationalchange.I.Title.
HF5548.2.L8352014
658.4’038011–dc23
2014003632
BritishLibraryCataloguing-in-PublicationData
AcataloguerecordforthisbookisavailablefromtheBritishLibrary
ISBN:978-0-12-800205-6
PrintedandboundintheUnitedStatesofAmerica
14 15 16 13 12 11 10 9 8 7 6 5 4 3 2 1
ForinformationonallMKpublications,
visitourwebsiteatwww.mkp.comorwww.elsevierdirect.com
Dedication
Thisbookisdedicatedtoeveryoneinterestedinaffectingbeneficialchangeinlarge
organizations by promoting constructive behavior and good outcomes; by clearly
demonstratingmodesty,interestinothers,andagoodworkethic;byshowingawill-
ingnesstomentorandaneagernesstocooperate;andbymakingeveryoneeagerto
get back toworkthe next day.
Evenmoreso,wesalutethemanagementstylethatcontributesallgoodthingsto
their teams and coworkers. It was Dwight Eisenhowerwho said, “Leadership con-
sistsofnothingbuttakingresponsibilityforeverythingthatgoeswrongandgiving
yoursubordinates creditfor everything thatgoeswell.”
IhavetogivespecialthankstoDr.MalcolmChisholmfortheinvaluableadvice
hegavemeforthisbook,andtherepeatedproofreadingwhichrenderedhimmore
nightsof unexpectedly deep andblissfulslumber.
Mostofall,Ithankthelovedonesinmylifeasthatiswhatmakesitallworth-
while, especially my wife and my daughter, whose life story or at least her first
30yearsI will oneday have to write.
Foreword
EnterpriseArchitecture(EA)unitscanbefoundinmostmajorenterprisestoday.This
isencouragingforarelativelynewfieldwhichwasonlyformallyrecognizedinthe
1980s. Indeed, there is now a widespread recognition that EA is a very important
undertakingthatcanhavefar-reachingbenefitsfortheorganizationsthatembraceit.
ThispromiseofEA,however,alltoooftenclasheswithadifferentrealitywhen
weexaminejustwhatEAunitsaredoinginenterprises.Someoftheseunitsaresim-
ply“productpickers”whoselecttechnologyfortheirITdepartments.Perhapsthatis
a little harsh, since managing the technology portfolio is certainly necessary and
arguably a core function of EA. But if technology selection is the only thing an
EAunitdoes,thenitisfallingfarshortoftheacknowledgedgoalsoftheoveralldis-
cipline ofEA.
Inotherorganizations,EAunitscanbefoundthataretheoretical“talkingshops”
staffed by individuals who constantly refer to EA theory and the inevitable argu-
mentsthatexistwithinanybodyoftheorythatisstillbeingworkedout.Again,this
judgmentmaybeoverlyharsh,sincetheoryisvital.Withouttheory,wedonotknow
whywearedoing,whatwearedoing,orwhatweshouldbedoing.Yet,analmost
totalfocusontheoryblindsEApractitionerstotheurgentandpracticalneedsofthe
enterprisestheyworkfor.SuchenterprisesdonotneedEAtospenditstimetryingto
optimize the way that EA works—they want EAtodeliver somethinguseful.
Another failing of EA units is an overly tight coupling with Information Tech-
nology (IT) organizations. Today, attitudes to IT in most large enterprises are
increasinglynegative.Fairlyorunfairly,ITisseenasanexpensive,self-referential,
andincompetentcostcenter.FiftyplusyearsoforganicgrowthinITinfrastructures
have created increasingly unmanageable information management environments
thatnowthreatenthepossibilityofbusinesschange.Thisisoneofthecoreproblems
thatEAmustsolve,butitcannotbedoneifEAfunctionsassubordinateto,ormerely
anextensionof,IT.TheITmindsetofneedingtobetoldeveryrequirement,ofwork-
inginprojects,andofbeingmorealignedtotheITindustrythanthebusinessofthe
enterpriseinwhichITfindsitself,isfataltoEA.EAmustengagewiththebusiness;
infact,itmustbreakdownthedistinctionbetweenITandthebusiness.Regrettably,
toomany EAunitscannot breakoutofthe IT mindset.
It is against this background that Jim Luisi’s Pragmatic Enterprise Architecture
standsasaclearandcomprehensivevisionofhowEAcanbedoneandshouldbedone.
x Foreword
JimhastakenhisexperienceinEAandavarietyoftechnicalITfieldsandsynthesized
anapproachthatcanbeimplemented.AkeyconceptthatJimhasputforwardisthat
EAisnotasinglediscipline,butaconstellationofmanydisciplines.Itisfairtosaythat
EAhastraditionallybeenbrokendownintothefourmainareasofInformationArchi-
tecture, Business Architecture, Technology Architecture, and Application Architec-
ture.However,JimgoesfarbeyondthistoshowhowEAisreallymanydisciplines
which EA brings together to make work in harmony, rather like the conductor of
anorchestra.
Intuitively,thismakessense.Afterall,howcananindividualwhoisanexpertin
dataobfuscationfilltheroleofanexpertinanalyzingbusinesscapabilities?These
architectural disciplines are also bound to come into being as overall information
managementadvances,anddisappearasoldwaysofdoingthingsfallaway.Thus,
the role of an Enterprise Architect is to understand what disciplines need to be
brought to bear on the enterprise, and to ensure that the disciplines relevant to an
enterprise are staffed and functioning well.
Jimalsopointsouttheflawintheoppositeapproach,wherebygeneralistenter-
prise architects try to address specific disciplines. In the first place, all generalists
will carry with them a set of experience, which is likely to be concentrated in just
afewareas.Second,thegeneralistmaysimplynotrecognizethedepthofknowledge
requiredforaparticulardiscipline,perhapsevenfailingtorecognizethatsuchadis-
cipline actuallyexists.
A further set of problems exist because technology and methodologies change
over time. Long ago, when mainframes ruled in enterprises, data was thought of
asanuninterestingby-productofautomation.Inthosedays,theautomationofhith-
ertounautomatedmanualprocesseswasthefocusofIT.Today,automationisprev-
alent and packages for operational systems exist for an incredibly wide array of
enterprise needs. By contrast, the role of data has become increasingly elevated to
thepointwheretodaythevalueofdataisalmostuniversallyrecognized.Overthis
period,theascentofdatahasbeenaccompaniedbynewtechnologies,suchasrela-
tional databases, andnew methodologies such asdata warehousing.
EA must therefore be careful to stay updated with the new architectural disci-
plinesthatareneeded,andbepreparedtode-prioritizedisciplinesthataredeclining
inimportance.ItisthereforeveryimportantthatJimhasincludedanumberofsec-
tionsonBigDataanditsimplicationsforEA.Alreadyweseeenterprisesrushinginto
thisareaandwillingtospendlargesumsonstandinguptechnology.Yetwithoutthe
understandingofthedisciplinesinvolved,thereislikelytobeahighfailurerate,and
anecdotesofthisarealreadycirculating.Jim’ssectionsonBigDataareparticularly
welcomeatthispointintime,becausetheybuildondisciplinesarounddatathatare
necessary to manage traditional relational data, but which again have often been
poorly implemented inmany enterprises.
Foreword xi
Anothermajorarchitecturalchallengethatmanyenterprisesarestrugglingwith
todayiscomplexity.Complexityisdifficulttounderstandbecauseitisveryoftenthe
outcome ofhistorical processes.As notedearlier, 50yearsoforganic growth have
created complexity that is difficult to understand, let alone manage. Innumerable
decisionsmadetogetprojectsdoneintheshorttermhaveledtothegradualemer-
genceofmassiveproblemsinthelongterm.Thisisoutsidetheexperienceofmany
ITstaff.Suchprofessionalsmayhaveexperiencedworkingonmanyprojects,allof
whichweredeployedintimeandmettherequirementstheyweredesignedfor.More
likely, these professionals will have experienced many failed and suboptimal pro-
jects.Yet,evenifalongseriesofprojectshasbeensuccessfulasindividualprojects,
they typically do not work well together for the enterprise. IT professionals fail to
grasphowasetof“successful”projectscanleadtoacrisisofunmanageablecom-
plexity in an enterprise, leading to a situation that prevents business change and
inhibits the adoptionof new technology.
Again,Jimspeakstotheissueofcomplexity.TheroleofEAin“doingtheright
thing”isemphasizedthroughoutthisbook,andthearchitectural“bigpicture”isheld
upasaconstantreferencethatinformstheactivitiesofallthevarieddifferentarchi-
tectural disciplines.
Complexityismademoredifficulttodealwithtodayduetoanotheroutcomeof
anothersetofhistoricalprocesses,bywhichenterpriseshavegraduallylosttheabil-
ityto abstract the business awayfrom the technology. As Jim pointsout,long ago
whenprocesseswereentirelymanual,staffunderstoodwhatthebusinesswasdoing.
Today,stafftypicallyinteractwithautomatedsystems;theyunderstandhowtowork
withthesesystems,butnotnecessarilythebusinessgoalsthesystemsaretryingto
achieve. Further, staff cannot see how all the systems work together to make the
overallenterprisework.Again,longago,wheneverythingwasmanual,staffcould
actuallyseehowthingsweredoneandtalktothepeoplewhoweredoingthem.Ofall
thechallengesthatJimoutlines,thisoneisthemostworryingtome.Relativelyfew
individuals now exist in enterprises who really can really think about the business
independentofthetechnologythatissupportingit.Thisproblemseemstobegetting
worseastheyearsgoby.Onesymptomisthatwhenanapplicationisreplaced,the
idiosyncrasiesoftheoldsystemareperpetuatedintothenewsystemsince nobody
reallyunderstandswhytheyarethere,andareafraidtochangethem.Again,thisisan
issuethat Jim tackles, butto find outmore, you willhave toread the book.
MalcolmChisholm
Inside the Cover
Whendealingwithbigchallenges,successiselusiveonlyuntilyoubringthe
appropriate skills and personalities together.
J.Luisi
Alot ofideas expressed inthisbook,while notodd,are certainlynew.As such, it
maybeappropriatetoshifttheconventionalwisdomofwhatEnterpriseArchitecture
meansandwhatitshoulddotogiveusabetterwaytogetwherelargeorganizations
need togo.
Asweshallsee,thevalueofanideacanonlyberealizedafterithasbeenexe-
cuted,andpriortothatitisonlyanidea,whetherornotaplanexiststoachieveit.
Thisbook,therefore,explainsthislatestphilosophyofwhatEnterpriseArchitecture
isfromasuccessfulexperimentinalargefinancialservicescompany,oneofthetop
10insizeintheUnitedStates.Thephilosophyemployedwasevaluatedbyarepu-
tableresearchcompanytoberadicallydifferentthanwhathasbeenseenpreviously,
butmostimportantly,itssuccessatmeetingitsbusinessobjectiveswasirrefutable.
While this book deals with most every important architecture concept, there is
onenonarchitecturalaspectthatisbestmentionedherethatisalsofoundinthebook,
GoodtoGreat:WhySomeCompaniesMaketheLeap,byJimCollins,whichisthe
need for the right kind of like-minded individuals that can enable the successful
implementation ofany new strategy. Theauthor,inhis professional life, has spent
endlesshoursidentifyingandrecruitingpersonnelwiththerightattitudeandskillset
necessary toparticipate in the journey.
ToborrowtheanalogyofthebusfromJimCollins,youneedtogettherightpeo-
pleonthebusandthewrongpeopleoffthebusbeforeyoudecidewheretotakethe
bus.Specifically,therightpeopleonthebusmusthaveasenseofadventure,agood
senseofhumor,andmoreimportantly,thepeopleonthebusshouldwanttobeonthe
busnotbecauseofwherethebusisgoingasthatcanchange,butinsteadeachindi-
vidual shouldwanttobe on the bus because ofwho elseison the bus.
Alongthis journey, the team hadto overcomesignificant challenges fromboth
businessandIT,includingradicalbudgetaryswingsandpressures,violentriptides
of internal politics, external interference from large vendors, and adversities from
within.Butoncetherightcollectionofindividualsareassembledontheteam,they
automaticallygoouttoaccomplishwhatisbestfortheircompany,andtheinternal
andexternalchallengesbecomemoreliketheweatherforthebustotravelthrough.
xiv Inside the Cover
Additionalfeaturesofthisjourney included asense ofdiscoveryaswe learned
newsynergiesamongvariousarchitecturaldisciplines,aswellashavingattaineda
senseofaccomplishmentbymakingnewideasmaterialize.Intheend,however,the
journeyitselfiswhateveryoneremembersmostfondly,andthebondsoffriendship
that persist inits aftermath.
ItisthereforenobigsurprisewhenSteveJobssharedinsightsintohislifeexplain-
ingthathisbiggestrewardwastheexperienceofhispersonaljourney.Inkind,hope-
fully your journey through the pages of this book will be rewarding, as was the
journey writing it.