Software Design X-Rays: Fix Technical Debt with Behavioral Code Analysis

265 Pages·2018·10.45 MB·English

by  Adam Tornhill

Early praise for Software Design X-Rays
Adam has made one of the most significant contributions to software engineering
over the past years. Listen and ye shall learn.
➤ John Lewis
Consultant and Troubleshooter
Adam successfully teaches me why things are the way I have learned that they
are during my 30 years in the software-development world. As if that wasn’t
enough, he also teaches me a whole slew of things I didn’t know about! This is a
must-read!
➤ Jimmy Nilsson
Author of Applying Domain-Driven Design and Patterns
I felt my brain was exploding with ideas and ahas all the way through my reading.
➤ Giovanni Asproni
Principal Consultant
Adam encapsulates the challenges of a technical lead for a product in a large
shared codebase. His social code-analysis techniques turn a dry static codebase
into a living, breathing ecosystem and chart its interactions over its lifetime,
helping you to identify those areas worth refactoring.
➤ Ivan Houston
Principal Software Engineer
Adam takes you behind the scenes of pragmatic software analysis. He’s bridging
the gap between algorithms for mining software repositories and performing
refactorings based on the gained insights. Definitely the right way to go in our
industry!
➤ Markus Harrer
Software Development Analyst
Software systems age and erode like any other human-made structure. Software
Design X-Rays provides immediately useable tools and approaches to spot the
parts in most dire need of improvement and helps you manage your technical
debt. Adam does a great job at explaining that this seemingly complex analysis
is actually not that hard and that you can do it right now.
➤ Michael Hunger
Head of Developer Relations Engineering, Neo4j
This book offers plenty of valuable psychological insights that are likely to surprise
developers and managers alike. Tornhill’s ability to apply heuristics like the
“concept of surprise” to complex code systems reinforces the human element of
software development and connects code to emotion.
➤ Lauri Apple
Open Source Evangelist and Agile Coach, Zalando
An invaluable set of techniques to get a better understanding of your code, your
team and your company.
➤ Vicenç García Altés
IT Consultant, Craft Foster Ltd.
Software Design X-Rays
Fix Technical Debt with Behavioral Code Analysis
Adam Tornhill
The Pragmatic Bookshelf
Raleigh, North Carolina
Many of the designations used by manufacturers and sellers to distinguish their products
are claimed as trademarks. Where those designations appear in this book, and The Pragmatic
Programmers, LLC was aware of a trademark claim, the designations have been printed in
initial capital letters or in all capitals. The Pragmatic Starter Kit, The Pragmatic Programmer,
Pragmatic Programming, Pragmatic Bookshelf, PragProg and the linking g device are trade-
marks of The Pragmatic Programmers, LLC.
Every precaution was taken in the preparation of this book. However, the publisher assumes
no responsibility for errors or omissions, or for damages that may result from the use of
information (including program listings) contained herein.
Our Pragmatic books, screencasts, and audio books can help you and your team create
better software and have more fun. Visit us at https://pragprog.com.
The team that produced this book includes:
Publisher: Andy Hunt
VP of Operations: Janet Furlow
Managing Editor: Brian MacDonald
Supervising Editor: Jacquelyn Carter
Development Editor: Adaobi Obi Tulton
Copy Editor: Candace Cunningham
Indexing: Potomac Indexing, LLC
Layout: Gilson Graphics
For sales, volume licensing, and support, please contact [email protected].
For international rights, please contact [email protected].
Copyright © 2018 The Pragmatic Programmers, LLC.
All rights reserved.
No part of this publication may be reproduced, stored in a retrieval system, or transmitted,
in any form, or by any means, electronic, mechanical, photocopying, recording, or otherwise,
without the prior consent of the publisher.
Printed in the United States of America.
ISBN-13: 978-1-68050-272-5
Encoded using the finest acid-free high-entropy binary digits.
Book version: P1.0—March 2018
Contents
Acknowledgments . . . . . . . . . . . ix
The World of Behavioral Code Analysis . . . . . . xi
Part I — Prioritize and React to Technical Debt
1. Why Technical Debt Isn’t Technical . . . . . . . 3
Questioning Technical Debt 3
The Perils of Quantifying Technical Debt 6
Mine Your Organization’s Collective Intelligence 10
Prioritize Improvements Guided by Data 13
2. Identify Code with High Interest Rates . . . . . . 15
Measure Interest Rates 15
Prioritize Technical Debt with Hotspots 19
Evaluate Hotspots with Complexity Trends 24
Use X-Rays to Get Deep Insights into Code 27
Escape the Technical-Debt Trap 31
Exercises 33
3. Coupling in Time: A Heuristic for the Concept of Surprise . 35
Uncover Expensive Change Patterns 35
Detect Cochanging Files 37
The Dirty Secret of Copy-Paste 44
The Power of Language-Neutral Analyses 46
Exercises 48
4. Pay Off Your Technical Debt . . . . . . . . . 51
Follow the Principle of Proximity 51
Refactor Congested Code with the Splinter Pattern 57
Build Temporary Tests as a Safety Net 64
•
Contents vi
Turn Hotspot Methods into Brain-Friendly Chunks 67
The Curse of a Successful System 70
5. The Principles of Code Age . . . . . . . . . 73
Stabilize Code by Age 73
The Three Generations of Code 76
Refactor Your Package Structure 82
Scale from Files to Systems 87
Exercises 87
Part II — Work with Large Codebases and Organizations
6. Spot Your System’s Tipping Point . . . . . . . 93
Is Software Too Hard? 93
Divide and Conquer with Architectural Hotspots 96
Analyze Subsystems 100
Fight the Normalization of Deviance 107
Toward Team-Oriented Measures 112
Exercises 113
7. Beyond Conway’s Law . . . . . . . . . . 117
Software Architecture Is About Making Choices 117
Measure Coordination Needs 119
Code Ownership and Diffusion of Responsibility 125
Analyze Operational Team Boundaries 129
Social Groups: The Flip Side to Conway’s Law 134
Combine Social and Technical Information 138
8. Toward Modular Monoliths through the Social View
of Code . . . . . . . . . . . 141
Dodge the Silver Bullet 141
Layered Architectures and the Cost of Consistency 144
Monolithic Alternatives: Use Case and Feature-Centric 147
Discover Bounded Contexts Through Change Patterns 151
The Perils of Feature Teams 155
Clean and Dirty Architectures 161
Exercises 162
9. Systems of Systems: Analyzing Multiple Repositories
and Microservices . . . . . . . . 165
Analyze Code in Multiple Repositories 165
•
Contents vii
Compare Hotspots Across Repositories 170
Track Change Patterns in Distributed Systems 173
Measure Technical Sprawl 180
Distribution Won’t Cure the Dependency Blues 185
Exercises 187
10. An Extra Team Member: Predictive and Proactive Analyses . 189
Detect Deviating Evolutionary Patterns 189
Catch the Absence of Change 197
Guide On- and Offboarding with Social Data 200
Know the Biases and Workarounds for Behavioral
Code Analysis 205
Your Code Is Still a Crime Scene 207
Exercises 209
A1. The Hazards of Productivity and Performance Metrics . . 211
Adaptive Behavior and the Destruction of a Data Source 211
The Situation Is Invisible in Code 212
A2. Code Maat: An Open Source Analysis Engine . . . . 215
Run Code Maat 215
Data Mining with Code Maat 215
Visualizations 217
A3. Data Mining with Git, cloc, and CodeScene . . . . . 221
Behavioral Code Analysis with Git 221
A Brief Introduction to cloc 223
Export Analysis Data from CodeScene 224
A4. Hints and Solutions to the Exercises . . . . . . 227
Solutions: Identify Code with High Interest Rates 227
Solutions: Coupling in Time 228
Solutions: The Principles of Code Age 229
Solutions: Spot Your System’s Tipping Point 229
Solutions: Modular Monoliths 230
Solutions: Systems of Systems 232
Solutions: An Extra Team Member 232
Bibliography . . . . . . . . . . . . 235
Index . . . . . . . . . . . . . . 239
Acknowledgments
Writing a book is a lot like programming in the sense that it’s an iterative
process where most of the work is rework. Since Software Design X-Rays is
my fourth book I sort of knew the effort and time it would take. Or at least I
thought I did, as it turned out to be so much harder than I initially expected.
(Again a bit like programming, isn’t it?) At the end it was worth every minute
and I’m really proud of the book you’re reading right now. That wouldn’t have
been possible without all the wonderful people who helped me make this book
much better than what I could have done on my own.
I’d like to thank The Pragmatic Bookshelf for this opportunity. I also want to
thank my editor, Adaobi Obi Tulton, for all her support, motivation, and great
work in shaping the book.
Several people volunteered their time and expertise by reading early drafts of
the book: Jimmy Nilsson, Giovanni Asproni, Ivan Houston, Markus Harrer,
Michael Hunger, Lauri Apple, Per Rovegård, Joseph Fahey, Louis Hansen,
Vicenç García Altés, Nascif Abousalh-Neto, Clare Macrae, Michael Keeling,
Alberto Fernandez Reyes, Javier Collado, and Ian Sleigh. Thanks for all your
helpful advice and constructive criticism.
Over the past years I’ve been contacted by people from all around the globe
who’ve read my previous work or watched my presentations. Thanks to all of
you—your feedback and friendly words are what motivates me to keep doing
this. The same goes for my amazing colleagues at Empear and for my fantastic
parents, Eva and Thorbjörn, who always encourage and support me.
Finally, I want to thank my family—Jenny, Morten, and Ebbe—for their endless
love and support. You mean everything to me and I love you.
report erratum • discuss