APM Best Practices: Realizing Application Performance Management

493 Pages·2010·21.7 MB·English

by  Michael J. Sydor

v
Contents

About the Author.........................................................................................vii
Acknowledgments.........................................................................................ix
Introduction.....................................................................................................xi
Chapter 1: Getting Started with APM.........................................................1
PART I: Planning..........................................................................................37
Chapter 2: Business Justification................................................................39
Chapter 3: Assessments...............................................................................57
Chapter 4: Staffing and Responsibilities..................................................107
Chapter 5: APM Patterns...........................................................................139
Chapter 6: The Pilot Evaluation...............................................................161
PART II: Implementation......................................................................177
Chapter 7: Deployment Strategies..........................................................179
Chapter 8: Essential Processes.................................................................215
Chapter 9: Essential Service Capabilities................................................229
PART III: Practitioners...........................................................................251
Chapter 10: Solution Sizing.......................................................................253
Chapter 11: Load Generation..................................................................285
Chapter 12: Baselines.................................................................................309
Chapter 13: The Application Audit.........................................................329
Chapter 14: Triage with Single Metrics..................................................357
Chapter 15: Triage with Baselines...........................................................395
Chapter 16: Triage with Trends..............................................................405
Chapter 17: Firefighting and Critical Situations....................................439
Index................................................................................................................459
xi
Introduction
Application Performance Management (APM) is the name given to the
use of technology to initiate, deploy, monitor, fix, update and/or
optimize systems within an organization. Application management
software employs measurements of response times, and other
component and resource interactions, to help manage the overall
stability and usability of the software within its purview. This book,
presented in three parts, is intended to take you from the first
discussions about planning and assessing your monitoring needs, through
the details and considerations of an APM technology implementation,
and onto the types of skills, processes and competencies that you will
need to be completely successful in the utilization of APM. And it is not
simply a discussion of ideas for you to consider. The approach used is
largely metrics-based. I have included the appropriate artifacts so that
you may conduct these activities exactly the same as I would if I were
working with you.
My role as an APM practitioner is to guide stakeholders through every
phase of the journey to APM excellence and the discussions in this book
are taken directly from my presentations, engagement artifacts and
training materials. In particular, I lead client teams through a
comprehensive program of best practices that allow them to establish
and staff an APM discipline. What is unique about this collection of best
practices is that they are vendor-neutral, which is an enormous
consideration when you reflect on the reality that your IT tools are
selected from many different vendors. For my work, and especially my
credibility in defining these practices as essential, they have to work for
every vendor technology. I have included specific exercises which you
can use to track your progress in establishing an APM discipline.
For a wide variety of reasons, folks have difficulty getting the full measure
of usefulness from their investments in APM. There is often a disconnect
between the APM technology capabilities and the organizatioal
capabilities in actually understanding and employing the technology. I
developed many of these APM best practices by helping clients recover
xii
from any number of missteps in selecting and employing APM
technology. APM can be complex as much as it is easy to use. While a
good part of what I do might be considered brainstorming (discussing
new techniques and approaches), an equal part is gut-wrenching and
frustration-laden. This is the hard part—figuring out how to get your
organization to reliably adopt APM.
My motivation in writing this book is to show you what many users of
APM miss and also how powerful the technology and processes can be.
When I began in this industry there were no meaningful references nor
texts from which to get some useful ideas. Even today I really don’t find
anything substantial to address APM education across planning,
implementation and best practices – especially with a vendor-neutral
tone. So I’m going to put a stake in the ground with a bold and ambitious
statement: APM starts here.
Vendor Neutrality
Yes, I am employed by a monitoring solution vendor. And you can figure
out who by looking on the book cover. It is not the goal of this book to
highlight or recommend any particular vendor or technology. Instead, we
will focus on the processes that are applicable to any vendor solution,
especially in a multi-vendor or heterogeneous monitoring environment.
We will also pay particular attention to when it makes sense to employ a
particular technology. When it is necessary to illustrate how a tool
presents data, or otherwise how a tool might be employed to support
one of these processes, you may reliably assume that any vendor of the
underlying technology may provide that capability. And if I simply cannot
avoid mentioning a specific type of tool then I will use a freely available
or open source technology as the example.
Any monitoring tool is appropriate—provided you understand its
application and limitations. We will address that as well. In general, any
visibility is better than “no visibility”. However, you cannot simply drop
in a tool without considering some framework of processes to support
and utilize the tool appropriately. This process gap is what many
organizations fail to address and so their monitoring initiatives are less
effective. My goal is simply to take whatever system you have today and
reliably increase its capabilities and value to your organization – as far as
you may want to advance it.
xiii
Conveniently, the guideline of vendor neutrality also fits in with our first
and most important objective for a monitoring initiative: it is not the
time to talk about specifics of a specific vendor technology solution. It is
time to talk about the process of realizing an APM initiative.
APM Starts Here
This book is about how to do APM ‘right’. APM means different things to
different people and this quickly becomes a fantastic gap between reality
and expectation. My professional role is to jump into this gap and get
things back on track. Maybe you want my job? Maybe you want to avoid
seeing me turn up? Maybe you want to get this right the first time.
Nothing available today is going to help you do it right the first time –
save for this book. The single biggest gap in understanding APM is this:
APM is not simply an end-user monitoring application. It is part
of the IT infrastructure and the use of the tool will be divided
over many stakeholders. Fail to accommodate and empower these
stakeholders and your APM initiative will follow.
APM is much more than a monitoring technology. It is a collaboration
technology. Through the techniques in this book you are going to
educate your stakeholders in the language of APM. You are going to
harness their capabilities and tear down the barriers to cooperation. It is
how your organization is going to finally get control of the application life
cycle and improve software quality, without excuse or exception.
Organization of this Book
This book is presented in three parts: Planning, Implementation and
Practitioner’s Guide.
Introduction to Part 1: Planning
Planning for an APM initiative is a course of action that will balance the
capabilities of the IT team, the demands of the business and the
alignment with corporate objectives. It is advantageous to move quickly
from establishing the overall goals of the initiative, to formulate and
xiv
present (sell internally) a tactical plan that will have a high degree of
success. Successful management and execution of this planning activity
sets the stage for a successful deployment of the technology and
implementation of the supporting processes. Failure to plan will result in
an uneven or failed initiative.
We are concerned with five major themes:
• Establishing the business justification (Chapter 2)
• Assessing your current monitoring capabilities (Chapter 3)
• Understanding staffing requirements (Chapter 4)
• Defining the catalog of services that will track the progress of the
initiative (Chapter 5)
• Executing a successful pilot evaluation leading to selection of a
suitable technology (Chapter 6)
Each of these themes will be handled in its own chapter and each needs
to be addressed prior to the deployment phase, which is covered in the
second part of this book: Implementation.
Of these themes, staffing is probably the most controversial as this
exposes the organization’s understanding of impact of APM and how it
should be leveraged. Frankly put, while the greatest potential for APM
initiative failure is inadequate scope, the second greatest has to do with
staffing: who does the work of using APM technology and processes? You
should not really be concerned with total manpower but more about
who will fulfill the various APM roles, where the monitoring initiative will
sit in the organization, and how it will interact with other stakeholders.
An important aspect of the APM best practices presented here is the
focus on an early and detailed assessment of existing skills, processes and
competencies. This is specifically targeted to address the looming
staffing/ownership question but also may be extended to survey the
strengths/weaknesses of existing tools and groups within the IT
organization and how they may be leveraged to help support and
continue the pace of the APM initiative while avoiding any immediate
increase in headcount. Doing more with less is always a theme in
modern IT management and I find that any APM initiative will be under
significant pressure to do exactly this.
xv
Some discussion of the service catalog, and how we use these defined
capabilities as milestones to manage the evolutions of an APM discipline,
is important because this topic is a frequent point of confusion. A catalog
of services is a popular IT feature, and one that is directly descended
from the ITIL descriptions of service management. I find while many
organizations believe that this work is already done; few can quickly
point to precisely what this catalog currently includes. This is another
gap which can significantly impair the progress of the APM initiative. We
focus on this service catalog concept because this is the area where we
will need the greatest degree of tracking, that in fact, these services are
actually being achieved.
Because many organizations have already been initiated to this service
catalog concept, much of my planning response is to make up for the
earlier gaps, so I needed an efficient way to measure exactly what was
missing from the existing service catalog. Very often a client will have a
comprehensive set of services defined but no corresponding
implementation roadmap is specified. And after some months, attention
wanes and other priorities come to the fore, and the service catalog
fades from view. I will show you how to successfully manage the
realization of the APM service catalog by aligning with an evolution of
staff skills and demonstrable competencies. This results in a more
manageable job of tracking and reporting progress towards realizing the
service catalog.
The final discussion topic in this planning part of the book: how to define
a pilot of a new technology, is also a point where the IT team can easily
fail. The IT organization has probably been evaluating a variety of
technologies for many years. However, the current crop of economic
pressures, down-sizing and transition of responsibilities off-shore have
eliminated much of the hard-earned experience in defining and executing
a pilot evaluation. I see the pilot evaluation as an important validation of
the overall objectives of the monitoring initiative. Failing to do a
thorough and timely job here will of course cripple any chance of a
successful deployment and ongoing use of the technology. I will take the
opportunity here to simply address that gap and present a successful
best practice for doing a solid evaluation of a prospective technology.
xvi
Introduction to Part 2: Implementation
In this Implementation part, chapters 7-9, we want to look at the
implementation of APM from the perspective of a project manager
tasked with bringing the initiative to realization. With a small scope, it is
rare that you will have a project manager role assigned, but whomever
has to oversee the progress of the initiative will benefit from the project
management perspective that will be discussed here. For large initiatives,
especially those with multiple and recurring monitoring deployments,
you should considering moving to an APM-specific project manager,
along the lines of what was outlined in Chapter 4 – Planning – Staffing
and Responsibilities.
First, a word of caution. Implementation is often used interchangeably
with deployment. Deployment of software does not include getting the
software to do what you want. This is the largest point of confusion and
dissatisfaction with performance monitoring software—IT thought their
responsibility was to deploy the monitoring solution while the business
expected an implementation that would help them identify performance
problems. Part of this confusion has to do with who “owns” the
software. IT deploys lots of applications on behalf of the application
owners, why is APM different? Often, an initial APM deployment is
limited to a single stakeholder and the reality of APM being more of an
infrastructure choice is lost. Here we mean implementation to cover the
initial physical deployment, developing essential processes for employing
the tool and how these processes are assembled into service
capabilities. These processes are the difference in establishing a
meaningful APM discipline.
Implementation of an APM solution is surprisingly quick, compared to
other IT initiatives: usually on the order of weeks. Sometimes in just a
day or two for a small environment. The overall coordination however
can be large, depending on the scope. Such coordination is required for
any re-testing of applications and also for integration of alerts with
existing Trouble Management systems. The scope (level of effort) will
increase proportionally to the number of applications. For 1-2
applications, all of these implementation considerations can be discussed
in a 1-2 hours planning session. For 20-30 applications, this might take
2-3 months before consensus and approval of the plan is achieved.
xvii
This section will review each of the topics that the implementation plan
needs to consider. Traditionally, this task is managed by a project
manager who will build the project schedule and then work to
coordinate the resources to achieve the deployment. Smaller initiatives
may want to forgo the project manager role, which incurs some risk
(which we expect that an understanding of this section will help you to
avoid). Even the smaller initiatives will benefit from the implementation
considerations that are presented here.
I also know that there will be a number of other goals and activities that
the monitoring initiative needs to accomplish that are outside the
traditional definition of deployment: getting the technology up and
running. For the project manager, these are simply additional objectives
which they will schedule and resource accordingly. You will need to
provide the project manager with a model or template concerning how
these non-traditional requirements will be achieved. I call this template
the Phased Deployment Model.
In the event that your APM initiative planning will involve multiple phased
deployments leading to a dedicated APM team, then you will also want
to consider using the service catalog approach to set goals and track the
progress of your APM Service Bureau or Center of Excellence for the
enterprise. At minimum you will want to document capabilities from the
service catalog that you are not pursuing as part of you implementation.
Often, a smaller initiative does not have the latitude to address these
enterprise considerations but you can do a much better job in managing
expectations if you fully understand and fully communicate to your
stakeholders what you are setting aside.
The bottom line for the APM implementation is to show you how to
scale the available resources in advance of dedicated staff for APM, spread
out the workload and focus on what is important for your organization.
Introduction to Part 3: Practioner’s Guide
This section introduces a variety of techniques for how to employ the
tools that are important in realizing the best possible value from your
APM investment. This has always been a controversial topic because the
motivation for investing in monitoring software is that it “does stuff.”