Presentation by Prof. Tom Mens (University of Mons) about the relation between, and guidelines for increasing, the internal and external technical debt and technical health of software. This talk was presented at the Business and Technology Club of the Infopole Cluster TIC in Gosselies (Belgium) on 19 February 2019. The ideas presented are partly based on research conducted in the context of the FRNS-FWO co-financed "Excellence of Science" Research Project SECO-ASSIST (http://secoassist.github.io)

2. Prof. Dr. Tom Mens
Software Engineering Lab
How to increase the
TECHNICAL HEALTH
of your software?
tom.mens@umons.ac.be
@tom_mens
Business & Technology Club – Infopole Cluster TIC – 19
February 2019

3. SECO-ASSIST
"Excellence of Science”
Research Project
2018-2021
seco-assist.github.io@seco-assist

4. What is technical health of software?
Internal point of view
• Focus on internal software quality characteristics
• Measure technical debt
External point of view
• Focus on dependencies to external software components
• Measure technical lag

5. Internal software health
Technical view
Increase technical wealth
by reducing your technical debt
“a concept in programming that reflects the extra development
work that arises when code that is easy to implement in the
short run is used instead of applying the best overall solution”
(Ward Cunningham, 1992)
http://legacycoderocks.libsyn.com/technical-wealth-with-declan-wheelan

6. Internal software health
Technical view
Reduce your technical debt
• By detecting internal quality problems and “bad smells”
• Duplicated code
• Obsolete code
• Poorly structured code
• Inadequate or incomplete tests
• Performance problems
• Potential bugs and security vulnerabilities
• And improving these problems using manual or
automated refactoring and restructuring

7. Implementation of SQALE model in SonarQube

8. Internal software health
Social view
Social debt
“Unforeseen project cost connected to sub-optimal
organizational-social structures”

9. Internal software health
Social view
Reduce social debt
by removing community smells
• Organisational silo
High decoupling and lack of communication between tasks
• Black cloud
Lack of people able to bridge the knowledge and experience gap
• Prima-donnas
Seemingly condescending and egotistical behaviour,
unreceptiveness to collaboration
• Organisational skirmish
Misalignment of organisational culture
• …

10. What is technical health of software?
Internal point of view
• Focus on internal software quality characteristics
• Measure technical debt
External point of view
• Focus on dependencies to external software components
• Measure technical lag

11. External software health
Technical view
• Dependency hell
• Unmaintained or outdated libraries
• Backward incompatibilities
• Incompatible software licenses
• Issues in dependencies may affect your own software
• Bugs
• Security vulnerabilities
• Missing new features
• …
• Technical lag
https://chaoss.community

12. External Software Health
Technical Lag
How outdated is your current software deployment (collection of
software components) w.r.t. the “ideal” situation ?
where “ideal” = “most recent”; “most secure”; ”most stable”; “most compatible”; …
https://chaoss.community

13. External Software Health
Technical Lag
References
• “A formal framework for measuring technical lag in component
repositories – and its application to npm.” A. Zerouali, T. Mens, J.
Gonzalez-Barahona, A. Decan, E. Constantinou, G. Robles. Wiley
Journal on Software Evolution and Process, February 2019
• “On the evolution of technical lag in the npm package
dependency network.” A. Decan, T. Mens, E. Constantinou. IEEE
Int’l Conf. Software Maintenance and Evolution, 2018
• “Technical lag in software compilations: Measuring how outdated
a software deployment is.” J. Gonzalez-Barahona, P. Sherwood, G.
Robles, D. Izquierdo. IFIP Int’l Conf. Open Source Systems, 2017

14. Guidelines based on an
analysis of the JavaScript
packaging ecosystem

15. Guideline 1: Inspect your potential
dependencies
• Is the target software component well-documented?
• Is it well-tested?
• Is its code well-written? (Cf. technical debt)
• Does the component have performance problems?
• Is its community active and responsive? (Cf. social debt)
• Is the component popular?
• Are there known bugs or security issues?
• Is the software licence compatible?

16. Example: leftpad

17. Example: micropackages

18. Example: micropackages

19. Example: transitive dependencies
https://npm.anvaka.com/

20. Example: transitive dependencies
https://npm.anvaka.com/

21. Example: transitive dependencies
https://npm.anvaka.com/

22. Guideline 2: Avoid “unhealthy”
dependency tree
• Strive for few dependencies in your software
• Avoid transitive dependencies: It just takes one transitive
component to break or compromise your software!
• Avoid depending on micro-packages
• Avoid depending on unmaintained/obsolete components

23. Law of increasing complexity

24. Law of increasing complexity
https://npm.anvaka.com/

25. Law of increasing complexity
https://npm.anvaka.com/

26. Law of increasing complexity
https://npm.anvaka.com/

27. Law of increasing complexity
https://npm.anvaka.com/

28. Law of increasing complexity
https://npm.anvaka.com/

29. Guideline 3: Monitor your
dependencies
• Law of increasing complexity:
• Number of transitive dependencies tends to grow over time
• Law of declining quality:
• Quality of dependent component may decrease over time
• To upgrade or not to upgrade?
• Upgrading benefits from bug fixes, security fixes and new
features
• But may introduce breaking changes

30. Guideline 3: Monitor your
dependencies
Use Continuous Integration/Deployement
and automatic dependency monitoring tools

31. Example: breaking changes

32. Guideline 4: Adhere to semantic versioning
and use dependency constraints

33. Guideline 4: Adhere to semantic versioning
and use dependency constraints
• From provider side: Informs your dependents about which
releases are backwards incompatible
• From consumer side: Allows to decide and control when to
upgrade to newer releases of dependencies

34. Security vulnerabilities
security exploit in 2017
“attackers entered its system in mid-May through a web-application
vulnerability that had a patch available in March. In other words, the
credit-reporting giant had more than two months to take precautions
that would have defended the personal data of 143 million people
from being exposed. It didn’t.”
Wired Magazine, “Equifax Has No Excuse”, September 2017

35. Security vulnerabilities
HeartBleed bug
• Vulnerable code introduced in 2012 by simple
programming mistake.
• Allowed anyone on the Internet to read the memory of the
systems protected by OpenSSL
• Vulnerability discovered and traced in April 2014
• 0.5M servers certified by trusted authorities were believed
to be a affected

36. “1 out of 3 dependents never update their
dependency to a vulnerable package”
A. Decan et al. “On the impact of security vulnerabilities in the npm package
dependency network”, MSR 2018.
Improper or too restrictive use of dependency constraints
Dependent package is no longer actively maintained
Maintainers are unaware of the vulnerability or the fix
Fixed package version is incompatible
"37% of websites include a JavaScript library
with a known open source vulnerability.”
T. Lauinger et al. "Thou Shalt Not Depend on Me: Analysing the Use of Outdated
JavaScript Libraries on the Web", NDSS 2017.

37. Guideline 5: Secure your software
• Use security monitoring tools
• To discover vulnerabilities faster
• To update vulnerable dependencies
• Do not depend on unmaintained packages
• Their vulnerabilities may take a long time to get fixed

38. Guideline 5: Secure your software
Use security monitoring tools

39. Guideline 6: Help your consumers
• Inform dependents about
• incompatible upgrades
• planned updates
• deprecated features
• Help your dependents to upgrade more easily
• Provide (automated) migration guidelines
• Provide alpha/beta releases
• Test your changes on dependents before releasing updates
• Backport security fixes to earlier vulnerable releases