Seven systems engineering myths and the corresponding realities

Seven systems engineering
myths and the corresponding
realities
Joseph E. Kasser
Visiting Associate Professor
Temasek Defence Systems Institute
National University of Singapore
Email joseph.kasser@incose.org

Copyright Joseph Kasser 2010 1

Topics
 The state of systems
engineering 2010
 Seven systems engineering
myths


Systems engineering
 Means whatever the speaker intends
 Subjective, no anchor points
 Endless pronouncements of positions
 Overlaps with other ways of doing things
 Management, design, problem solving, OR
 Confusing information in text books
 Opinions of authors (mine included)
 Poorly practiced
 Defects in paradigm
 Promises big things
 Reports of successes and failures
 Snake oil salesmen?
INCOSE Fellows Briefing on SEBoK, July 2009

State of Systems Engineering at
INCOSE IS Academic Forum 2009
 Electrical engineering before Ohm’s law was
postulated
 Electrical engineering before Maxwell’s
equations were stated
 engineers built motors by winding coils but had no
theory upon which to predict the behaviour of the
motor before powering it up for the first time
 Chemistry before the periodic table of elements
was discovered
 Medicine in the 1800’s
 before medical science provided a theory of why
some medications work and why some don’t

The discipline
 Systems engineering is in its early stages.
 A discipline in these stages is characterized
by
 debates based on subjective opinions
 participants talking past each other
 a lack of listening
 contradictory and confusing information
 a number of myths


myths
 Myth 1: There are Standards for systems engineering
 Myth 2: The “V” model of the systems engineering
process
 Myth 3: Follow the systems engineering process and all
will be well
 Myth 4: Complexity needs new tools and techniques
 Myth 5: Systems of systems are a different class of
problem and need new tools and techniques
 Myth 6: Changing requirements are a cause of project
failure so get the requirements up front
 Myth 7: The single systems engineering process


Bear with me: helicopter view
“Else” is doing her
systems
engineering here

Systems engineers at work in


The Hitchins-Kasser-Massie Framework
for understanding systems engineering*

* Kasser and Massie, 2000 Copyright Joseph Kasser 2010 8

MIL-STD’s freely available at http://www.everyspec.com

Myths of systems engineering

 Myth
 There are Standards for systems engineering
 Reality
 There are no such Standards
 Standards cover
 Process for engineering systems
 different parts of the process
 Engineering Management
 Moreover, Standards focus on wrong aspect

499 Systems engineering
management

 Purpose to develop a
Systems Engineering
Management Plan
 Not to do systems engineering
 Two templates
 Generally not tailored
 MIL-STD-299A Systems
Engineering Management


EIA-632

 Process for
engineering a
system
 Not process for
systems
engineering


IEEE-1220

 Management of the
systems engineering
process
 Not doing systems
engineering
The systems engineering process provides
a focused approach for product
development that attempts to balance all
factors associated with product life cycle
viability and competitiveness in a global
marketplace.”


ISO-IEC 15288

 Systems Engineering
Process
 Purchase price*
 CHF 168,000
 Current version
15288:2008
 Revised from 2002
version

* http://www.iso.org/iso/catalogue_detail?csnumber=43564

Committed costs vs. Lifecycle

DAU, 1993 quoted in INCOSE Systems Engineering Handbook 3.1 (2nd Printing) modified


Generic perspective:
Common content of standards?

Common content?


Focus of Standards –
chronological perspective
SE Categories MIL-STD- ANSI/ EIA IEEE- CMMI ISO-15288
499C 632 1220
Conceptualizing problem and
alternative solutions No No No No No
Mission/purpose definition No No
Requirements engineering

System architecting
System implementation No No
Technical analysis
Technical management/
leadership
Verification & validation

Based on Table 5 in Honour E.C., Valerdi R., “Advancing an Ontology for Systems
Engineering to Allow Consistent Measurement”, CSER 2006

myths

 Myth 1: There are Standards for systems engineering.
process
will be well

The “V” Model

18

Defects in the V Model

 Lacks ‘prevention of defects*’
 Definition of successful test?
 Design works from requirements
 T&E work from the need
 T&E identify defects and plan to find them
after they have been built into the system
 Why not prevent the defects?
 Does not cope with change
* Kasser, J. E., "Eight deadly defects in systems engineering and how to fix them ", Proceedings of the
17th International Symposium of the INCOSE, San Diego, CA, 2007.
19

Waterfall representation of
series of activities

Requirements Redraw Waterfall
moving these
blocks up
Design

Implement

Test

Operate

20

Waterfall representation in V
format

Shows functional
V is for relationships
View
[not process
Requirements model]
Operate

Design Test

Implement

V is a representation of the Waterfall model,

Does not cope with change
21

myths

process
 Myth 3: Follow the systems engineering process and
all will be well

Focus on systems
engineering process
 The successful implementation of proven,
disciplined systems engineering processes results
in a total system solution that is--
 Robust to changing technical, production, and operating
environments;
 Adaptive to the needs of the user; and
 Balanced among the multiple requirements, design
considerations, design constraints, and program
budgets.*
 A single process, standardizing the scope, purpose
and a set of development actions, has been
traditionally associated with systems engineering.**
* United States Department of Defense 5000 Guidebook 4.1.1
** Arnold, 2000 quoting (MIL-STD-499B, 1993) and (IEEE 1220, 1998)

Which process- 632?
Process Input

Requirements Analysis Systems Analysis
• Analyse Missions & Environments
• Identify Functional Requirements & Control
• Define/Refine Performance & Design
Constraint Requirements • Select Preferred Alternatives
• Trade-off Studies
Requirements Loop • Effectiveness Analysis
Functional Analysis/Allocation • Risk Management
• Decomposition to Lower-Level Functions • Configuration Mgmt
• Allocate Performance & Other Limiting • Interface Management
Requirements to Lower-Level Functions • Data Management
• Define/Refine Functional Interfaces (Internal/External) • Performance Based Progress
• Define/Refine Functional Architecture • Performance Measurement
– SE Master Schedule
Design Loop – Tech Perf Measurement
– Technical Reviews
Synthesis
• Transform Architectures (Functional to Physical)
Verification • Define Alternative Product Concepts
• Define/Refine Physical Interfaces (Internal/External)
• Define Alternative Product & Process Solutions

PROCESS OUTPUT


Which process- 1220?


Which process- DERA?
P ro b le m A p p re c ia tio n

U ser
In s t a lla tio n &
O p e r a tio n a l
r e q u ir e m e n ts
d e fin itio n v a lid a t io n s y s te m

A llo c a t e d P ro p o se d S u p p lie d
S o lu tio n A b s tra c tioqn ir e m e n t s
re u c h a r a c t e r is t ic s
S o ylut e m n R e a lis a tio n
s s tio

Lo cal S y s te m
A r c h ite c tu ra l In te g r a t io n & In te g r a te d
r e q u ir e m e n t s r e q u ir e m e n ts
& c o n s tra in ts d e fin itio n d e s ig n v e r ific a tio n s y s te m

A llo c a t e d P ro p o se d S u p p lie d
r e q u ir e m e n t s c h a r a c t e r is t ic s c o m p o n e n ts

Lo cal Com ponent Com ponent
Com ponent
r e q u ir e m e n t s
r e q u ir e m e n t s d e s ig n b u ild & t e s t C o m p o n e n ts
& c o n s tra in ts
a111

S o lu tio n D e v e lo p m e n t


Blanchard and Fabrycky’s
systems engineering functions
Functional view not a process.
Note as a process time seems to be running
backwards?

Blanchard and Fabrycky, Systems Engineering and Analysis 1981

27

Terry Bahill’s systems
engineering functions

Functional view not a process.
Note as a process time seems to be running backwards?

SIMILAR – acronym from first letter of each box

The Systems Engineering Process from A. T. Bahill and B. Gissing, Re-evaluating systems
engineering concepts using systems thinking, IEEE Transaction on Systems, Man and
Cybernetics, Part C: Applications and Reviews, 28 (4), 516-527, 1998.
http://www.incose.org/practice/fellowsconsensus.aspx

28

Two external perspectives:
The problem solving process
1. Problem Definition* 1. Identify and Select the
2. Problem Analysis. Problem**
3. Generating possible 2. Analyze the Problem
Solutions. 3. Generate Potential Solutions
4. Analyzing the Solutions. 4. Select and Plan the Solution
5. Selecting the best 5. Implement the Solution
Solution(s). 6. Evaluate the Solution
6. Planning the next course of
action (Next Steps)

* http://www.gdrc.org/decision/problem-solve.html (accessed 11 Jan 2009)

** http://www.c-pal.net/course/module3/pdf/Week3_Lesson21.pdf (accessed 11 Jan 2009)

Something’s wrong here
 The systems engineering process
seems to be the problem solving
process
 Semantics - levels of detail in each step
differ
 Similar process steps in other
professions
 Were they doing
 Systems engineering, or problem solving?


What systems engineering
process?
 Each process seems to be different
 Some overlap the problem solving process
 Mar, Hitchins, etc.
 Some cover the whole system lifecycle
 Blanchard and Fabrycky, Bahill and Gissing, etc.
 Some cover the ‘realization of the solution’ part
of the system lifecycle
 MIL-STD 499, EIA-632, IEEE 1220, etc.
 Which one is “the” process?


Standish Report 1994*
Top 10 reasons for …
Where is “process” mentioned?
Project Success on people!
Focus is Project Failure
1. User involvement 1. Incomplete requirements
2. Executive management support 2. Lack of user involvement
3. Clear statement of requirements 3. Lack of resources
4. Proper planning 4. Unrealistic expectations
5. Realistic expectations 5. Lack of executive support
6. Smaller project milestones 6. Changing requirements and
7. Competent staff specifications
8. Ownership 7. Lack of planning
9. Clear vision & objectives 8. Didn’t need it any longer
10. Hard-working, focused staff 9. Lack of IT management
10. Technology illiteracy
* http://www.cs.nmt.edu/~cs328/reading/Standish.pdf

The focus is on people not
process

 Literature
 Is full of advice as to
how to make projects
succeed
 Has little if anything

to say about the
proliferating process
standards

myths

process
will be well

Myths of systems engineering
 “Complexity” and “Systems of Systems”
 Dichotomy of views
 In general: commercial world copes,
Defence has problems
 First view
 Difficult, need new tools and techniques
 Type II?
 Second view
 “What’s the problem, get on with it”
 Type V?


Complex systems

 Quotes Chaos 1995 study suggesting systematic reason for project
failure
 Suggests inherent complexity is reason for difficulties – wrong!
 complexity was not a cause of project failure in Chaos study – poor management was
 Quotes own prior work “for all practical purposes adequate testing of
complex engineered systems is impossible”
 Only applies to the way they are designed today [JEK]
 Suggests evolutionary process for engineering large complex systems –
right! But it applies to engineering any type of system
Published in Systems, Man and Cybernetics, 2003. IEEE International Conference on, 2003
necsi.edu/projects/yaneer/E3-IEEE_final.pdf

Two Types of Complexity *

 Real world complexity - in which elements of the
real world are related in some fashion, and made up
of components.
 We try to abstract out real world complexity
 Complexity is in the eye of the beholder
 Artificial complexity – elements of the real world
that should have been abstracted out when
drawing the internal and external system
boundaries, since they are not relevant to the
system (problem at hand).
 Artificial complexity gives rise to complicatedness
 See cartoons by Rube Goldberg and W. Heath Robinson

Copyright Joseph Kasser 2010
* Kasser J.E., Palmer K., “Reducing and Managing Complexity by Changing the Boundaries of the System", Proceedings of the Conference 37
on Systems Engineering Research, Hoboken NJ , 2005.

Representation of the system

Processes and products
are systems

Complicated example in
Rube Goldberg cartoon
http://www.rubegoldberg.com/gallery_02.php

Building artificial complexity


myths
process
will be well


Focus in on a toolbox of
methodologies*
So why do
we need
OR not SE! complex
systems
engineering
?

 Problem solver needs a methodology for [selecting the appropriate
methodology for] solving a problem
 “Classification of a system as complex or simple will depend on the
user and on the purpose he has for considering the system”
 “Systems engineering has been defined by Jenkins as ‘the science of
designing complex systems in their totality to ensure that the component
subsystems making up the system are designed, fitted together, checked
and operated in the most efficient way’.”**

* M.C. Jackson and P. Keys, 1984, J. Operations Research Society, Copyright Joseph Kasser 2010
Volume 35, Number 6, p 473-486, Published in Great Britain 42
** G.M. Jenkins, (1969) The systems approach. In Systems Behaviour, J. Beishon and G Peters, Ed., 2nd Edn. P 82, Harper & Row, London

System or system of
systems?
Fighter Wing
Red Leader
Aircraft
Ordnance
Airframe
Navigation
Incomplete Propulsion
Guidance
Pilot


System or system of
systems?

 World War II Allied convoy in North Atlantic ocean
 Logistics suppliers for imported equipment

myths
process
will be well
 Myth 6: Changing requirements are a cause of
project failure so get the requirements up front


Incremental Model
The incremental life cycle
User
requirements
Get the requirements up front, still
no consideration of change in needs
System
requirements
Time
Architectural
design
Operational
Component Integration & system
development verification Installation &
Part 1 Part 1 validation
Part 1 Operations

1
Component Integration &
Part 2 Part 2 validation Operations
Part 2

2
Component Integration &
validation
Operations
Part 3
Part 3
Part 3

r135A
3
Students are used to seeing time running horizontally

The evolutionary lifecycle
Evolutionary Development
Any part of the system may change
but project discipline is followed
User
reqs System Operational
reqs Architectural
Time design Component
Integration &
system
development
verification Installation
& validation
Feedback from system 1 Operations
User
reqs System
reqs Architectural
1
design Component Integration &
development verification Installation
& validation
Operations
Feedback from system 2
User
reqs System
2
reqs Architectural
design Component
development
Integration &
verification Operations
Installation

r136A
& validation
3
First consideration of some changes in requirements,
concept of external changes not shown

Myth and reality
 Origins
 The failure to capture the entire problem/need
and create the full set of matching specifications
for the solution system in the early phases of
systems engineering
 Confusion between the original uncaptured
requirements and those requirements that arise
due to changes
 Reality
 Overlooking the fact that requirements change
continuously and failure to manage that change
is a cause of project failureKasser 2010
Copyright Joseph 48

myths
process
will be well


HKMF Area processes
Systems engineering

Start Non-systems engineering End

 Area contains activities
 Process consists of activities
 Parallel processes producing products
 Systems engineering
 Non-systems engineering
 Project management
 Engineering
 Logistics
 Etc.
 Interdependent


Insight
 Each area in the HKMF contains a set of
activities from which a process may be
constructed
 systems engineering (SEP)
 non-systems engineering
 Project management
 Engineering
 Logistics
 Etc.


Why the various versions of
the SEP are different
 “the systems engineer creates a unique process
for his or her particular development effort”
 Biemer and Sage, 2009, page 153,
 Kasser and Palmer, 2005
 Consider each published version of the SEP* as
the unique process created for their particular
development effort
 by someone or some group
 at some point in time,
 at some point in the system lifecycle
* In text books, Standards, web pages, etc.

Two systems engineering
processes
1. Unique systems engineering process
(USEP) to a project
• Constructed from set of appropriate activities
 by someone or some group
 at some point in time,
 at some point in the system lifecycle
• Doing process
• Instantiated in Standards
2. Process that constructs the USEP for
realizing a system
• Planning process


10-Step systems engineering
process to construct the USEP
1. Plan the project
 Review lessons learned, locate industry best practices
2. Explore/survey the what needs to be done.
3. Conceive at least two feasible processes
 Explore industry best practices
4. Identify ideal selection criteria for evaluation of the processes.
 Cost, schedule, resources, technology
5. Perform tradeoffs to determine and select the best process.
6. Fine tune selected process.
 Use best parts of each alternative from Step 3
7. Formulate strategies and plans to create preferred process.
8. Create preferred process using activities as building blocks
 Document them in the PP, SEP, SEMP, TEMP etc.
9. Verify the preferred process can realize the system
 Stakeholder consensus
10. Terminate the project.


Summary
 The state of systems engineering 2010
 Seven systems engineering myths
 Further details
 Read paper
 See CSER, EUSEC, and INCOSE 2010
publications on http://therightrequirement.com
 E.g. Kasser, J. E. and Hitchins, D. K., "Unifying the different
systems engineering processes", proceedings of the
Conference on Systems Engineering Research, Hoboken,
NJ., 2010.


Any questions?
 "It ain't what you don't know that gets you into trouble. It's
what you know for sure that just ain't so."
 Mark Twain 1835-1910
 Myth 2: The “V” model of the systems engineering process
 Myth 3: Follow the systems engineering process and all will be well
 Myth 5: Systems of systems are a different class of problem and
need new tools and techniques
 Myth 6: Changing requirements are a cause of project failure so get
the requirements up front.
 Myth 7: The single systems engineering process.


Seven systems engineering myths and the corresponding realities

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Seven systems engineering myths and the corresponding realities

Similar to Seven systems engineering myths and the corresponding realities (20)

More from Joseph KAsser

More from Joseph KAsser (15)

Recently uploaded

Recently uploaded (20)

Seven systems engineering myths and the corresponding realities