Table Of ContentDevelopment testing of
Object-Oriented software
Erkki P(cid:246)yh(cid:246)nen, [email protected]
Nokia, Technology Platforms
Product Creation Services and Tools
Few slides by Roland M(cid:252)ller& TeroLaine, Nokia
1 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP
Introduction
(cid:149) Background
(cid:149)OO Testing takes mostly place in lower abstraction levels
(cid:149)Development testing (Unit, Component, Module, Program
testing) has a big effect but it is badly managed activity
(cid:149)Overlooked by traditional testing approaches, testing
professionals and project management
(cid:149) Objectives
(cid:149)To highlight how different practices support each other
(cid:149)Bring awareness for both development and testing roles
2 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP
Contents: Development testing of OO SW
(cid:147)If there are two or more ways to do something, and one of thoseways
can result in a catastrophic results, then someone will do it.(cid:148)
Edward A. Murphy, Jr.
1. Objects are different - in good and bad
(cid:149) What has changed in OO SW organizations
2. It is still software
(cid:149) Still no silver bullet
3. Models are helping
(cid:149) Being early has its benefits in many ways
4. Focus, focus, focus
(cid:149) Strategy for winning the fight
3 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP
1/4 Objects are different - in good and bad
(cid:149) Testability (visibility and controllability) in low level will
suffer
(cid:149) Because of the information hiding
(cid:149) Because of the active objects
(cid:149) Improved support for managed unit testing in class or
component levels
(cid:149) Test harnesses (drivers and stubs) flexible to implement using
available programming tools and skills
(cid:149) Low level testing parallel with development means fast feedback
(cid:149) Models provide an effective way for preventing software
faults by testing the abstractions
= inspecting SW models at early stage
4 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP
Object benefits
(cid:149) Object-Oriented methods have a great potential in normal
case over older methods
(cid:149) It takes a lot more than using an object-oriented language to
really achieve the potential benefits
(cid:149) Agreed analysis and design practices help to achieve SW asset
(models/documents & code) transferability across organization
(cid:149) Proper OO competence development with agreed modeling methods
(UML with style guide) help to achieve better quality models andcode
(cid:149) Well-managed system and application architecture models is the basis
for SW asset reuse and testing
(cid:149) Remember: large and complex problems arelarge and
complex -no matter which software engineering approach is
used
(cid:149) Software is now bigger and more complex, but quality has not
improved (Les Hatton)
5 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP
Different concerns, different models
Development
process
Use case
Object Sequence Collaboration
Class Statechart Activity
Deployment Component
Structure Behavior
(cid:149)Different model types are used in different project phases and for
different modeling targets (static/structural to dynamic/behavior)
(cid:149)But: different abstraction levels can share model types
(cid:149) Example: class models used in levels from whole system to few
classes
6 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP
One good source
(cid:149) (cid:147)Testing Object-Oriented Systems(cid:148) by Robert V. Binder
(Addison-Wesley 2000, ISBN 0-201-80938-9), rbsc.com
(cid:149) Tells how to test Object-Oriented Software (surprise ☺)
(cid:149) What is special about OO SW for testing?
(cid:149) Collection of tens of OO testing and integration patterns
(cid:149) Binder speaks about:
(cid:149)(cid:149)MMooddeellssthat are necessary for test design
(cid:149)(cid:149)PPaatttteerrnnssexpressing best practices for test design
(cid:149)(cid:149)TToooollssassisting in the implementation of the test design
7 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP
Deriving Tests from UML Diagrams
(cid:149) Use Case
(cid:149) Information from UML use cases is typically not specific enough for
testing:
(cid:149) Therefore testing should be done with scenarios or extended use
cases
(cid:149) State Machines
(cid:149) Sequence of transitions
(cid:149) Class Diagram
(cid:149) Relationships between objects (1:1, 1:n, life cycle)
(cid:149) Sequence Diagram
(cid:149) Sequence of messages and triggered responses
(cid:149) Activity Diagram
(cid:149) Action 1 is followed by Action 2
(cid:149) OCL= Object Constraint Language
(cid:149) Straightforward usage for invariants, preconditions, postconditions
8 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP
Model-based testing
Meta-Model
Consistency
Checking
CompletenessChecking
Required Component
Behavior Representation
Validation RespoTnessitb miliotRyde-esblpaosnesdi bility-based Testing
Implementation-based
Test model
Observed
Component
Behavior
Implementation
Implementation-based Testing
9 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP
An OO Testing manifesto [BIND00, ch. 4.5]
Technical and process issues
(cid:149) Reuse does notreduce the amount of OO testing
(cid:149) Negative impact of OO features on testing
(cid:149)Test case design, testability and coverage analysis all suffer
from Inheritance, polymorphism, late binding and
encapsulation
(cid:149) OO development is iterativeand incremental:
(cid:149)Thus, also test planning, design and execution must be
iterative
(cid:149) Regression testing must be automated and is an essential
part of OO tester toolbox
10 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP
An OO Testing manifesto [BIND00, ch. 4.5]
Effective testing of OO software
(cid:149) Unique bug hazards in OO SW means also using established
test design practices
(cid:149) Application and domain specific test tools
(cid:149)must be object-oriented
(cid:149)and work against the testability problems of OO SW
(cid:149) Testing process must adapt to iterative and incremental
development and small-scale modularity.
(cid:149) Test design must consider method, class and component
levels simultaneously
11 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP
Specific OO Hazards
(cid:149) The use of OO languages may reduce some kinds of errors,
but increase the chance of others
(cid:149) Methods often are small
"control-flow bugs are less likely
(cid:149) OO developers are not more immune to errors
"Coding errors (misspelling, misnaming, wrong syntax) are
as likely as ever
(cid:149) Dynamic binding and complex inheritance structures
"faults due to unanticipated bindings or misinterpretation
of correct usage
(cid:149) Many small components
"more interfaces to have programming and usage errors
(cid:149) Object state control is distributed over the whole program
"state control errors are likely
12 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP
Especially dangerous OO constructs
(cid:149) Classes that send relatively more messages to instance
variable and message parameter objects are more likely to be
buggy
(cid:149) Classes with greater depth (number of superclasses) and
higher specialization (number of new and overridden
methods defined in a class) are more likely to be buggy
[BASILI96]
u t
B
(cid:149) Most literature agrees that biggest threat to programs is not
dangerous coding but ambiguous requirements and sloppy
design
(cid:149)Normally bad programming constructs are a sign of bad
design or delayed code restructuring (compare to
refactoring rules in XP practice)
13 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP
OO Testing at different levels 1/2
(cid:149)Use Cases & Scenarios
(cid:149)Abstraction level similar to functional specification
(cid:149)Possible to create tests on subsystems and components
(cid:149)Useful as functional acceptance tests, complementing
module tests
(cid:149) On SW team level each iteration could end at use case (cid:150)
based tests of team-developed subsystem
(cid:149) Development team could test the subsystem /
component, produced and already module tested by a
partner
(cid:149)Extended Use Case pattern
14 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP
OO Testing at different levels 2/2
(cid:149) Class clusters
(cid:149)PACT (Parallel Architecture for Class Testing) makes test
suites maintainable and modular
(cid:149) Class
(cid:149)State-based (modal) test methods (FREE pattern)
(cid:149)N-switch coverage
(cid:149) Method
(cid:149)Classical Black box techniques
(cid:149)Code-based coverage measured
(cid:149)Category partition pattern
15 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP
OO Testing above component level
(cid:149) Integration
(cid:149) Component interfaces (structural aspect)
(cid:149) Subsystem use-cases (functional aspect)
(cid:149) Architecture
(cid:149) Testing frameworks by developing test applications that exercisethe
framework on a determined way
(cid:149) Many different kind of architecture validation techniques to assess
success in achieving different architecture objectives
(cid:149) OO Middleware
(cid:149) Typically CORBA in Nokia
(cid:149) Also DCOM & .NET elsewhere
(cid:149) Standard interface description languages
(cid:149) Support for 3rdparty tools
(cid:149) Visibility on system-level use-case operation
16 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP
System Testing and Use Cases
(cid:149) System tests are based on use cases
(cid:149) Use cases are normally connected with functional requirements
(cid:149) Use cases, as defined by UML, are not sufficient for system testing test
cases
(cid:149) requires more describing information
(cid:149) You have to also collect information from other sources
(cid:149) Several attributes are needed for prioritization between use cases:
(cid:149) Risk
(cid:149) Frequency = density of enquiries (measurable)
(cid:149) Performance = answering time to an enquiry (measurable)
(cid:149) Load = allowed load generated to system and is the load constantor
varying (measurable)
(cid:149) General technique is to establish three levels:
(cid:149) use case (can be in several layers)
(cid:149) scenarios (instances of the use case)
(cid:149) test cases (based on scenarios with additional details)
17 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP
Lessons learned from current
OO design & testing practice
(cid:149) Caper Jones analyzed 600 projects from 150 organizations depending
on OO methods and found out that
1. OO learning curve is very steep and causes many first-use errors
2. OO analysis and design seem to have higher defect potential thanolder
design methods
3. Defect removal efficiency against OO design problemsseems lower than
against older design methods
(cid:149) OO projects typically produce programs of increased volume and
complexity against more complex requirements
"OO model inspection is even more important than with
older design methods
4. OO programming languages seem to have lower defect potentials than
procedural programming languages
5. Defect removal efficiency against programming errorsis roughly equal to or
somewhat better than removal efficiency against older procedural
language errors
[JONES97]
18 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP
2/4 It is still software
(cid:149) The software must still be tested, regardless of the
programming paradigm used
(cid:149) Testing just the software and not the models is an expensive
way to identify quality problems (cid:150)(cid:147)Quality retrofit(cid:148)
(cid:149) Development testing provides the most important guidance
for lower fault density and also to SW reliability
(cid:149) Low fault density is the essential basis for good reliability
(cid:149) On very buggy software the advanced reliability methods provide only
a marginal effect : user/usage profiling, fault tolerance, parallel
programming, (cid:133)
(cid:149) The promising statistical reliability methods will fail because of too
much noise
(cid:149) Even development testing of OO SW is still first based on
black box methods and focused in higher level risk planning
(cid:149) Product risks and specific component level risks
19 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP
Development Testing (DT):
Process vs. Development Approach
Two different aspects overlap when
speaking about development testing:
1.Organizational aspect / 2. Module Testing as part of
Formal Process developer(cid:146)s daily work, SW
(cid:149)DT as entry criteria for Development Culture
next phase
(cid:149) How work is conducted
(cid:149)Reporting, approving of
in development group or
planning artifacts
by developer
(cid:149)Organizational roles,
collaboration (cid:149) Agile method: MT done
all the time, after each
(cid:149)DT final run as exit
implementation step
criteria is one step, but
prepared over time (TDD, XP)
20 'NOKIA Development testing of OO SW.ppt/ 2004-05-04 / EAP
Description:active,reserve,andretired;senior,junior,andmiddlemanagement;civilservants;uniformed personnel;andcontractors.Theyareforallofustoread,tounderstand,toliveby,andtocherish.
