SlideShare a Scribd company logo
1 of 32
Download to read offline
Can AppSec Training Really Make a
Smart Developer?
Research From Denim Group
John B. Dickson, CISSP
@johnbdickson

November 20, 2013
About Me

• Application Security
Enthusiast
• Security Professional
• ISSA Distinguished Fellow
• MBA-type and Serial Entrepreneur
• Dad
Hosted by OWASP && the NYC Chapter
Hosted by OWASP the NYC Chapter
About Denim Group

Denim Group Overview
•Professional services firm that builds & secures enterprise
applications
 External application assessments
 Web, mobile, and cloud
 Software development lifecycle development (SDLC) consulting

•Secure development services:
 Secure .NET and Java application development
 Post-assessment remediation

•Classroom and e-Learning for PCI compliance

Hosted by OWASP && the NYC Chapter
Hosted by OWASP the NYC Chapter
“I personally believe that training users in security
is generally a waste of time, and that the money
can be spent better elsewhere.”
Bruce Schneier

Hosted by OWASP && the NYC Chapter
Hosted by OWASP the NYC Chapter
How Developer Training is Different
•Both trying to change behaviors
– Target audience has more power to say “no”
– Deadlines and releases drive training

•For developer, infrequent, but more disruptive
• 15-45 minutes vs. 2-day class

Hosted by OWASP && the NYC Chapter
Hosted by OWASP the NYC Chapter
Yet Training is Mandated
•PCI DSS 3.0
 Train developers in secure coding techniques, including how to
avoid common coding vulnerabilities, and understanding how
sensitive data is handled in memory.
 Testing Procedures: 6.5.a: Examine software development policies
and procedures to verify that secure coding technique training is
required for developers, based on best practices and guidance.
 Testing Procedures: 6.5.b: Interview a sample of developers to
verify that they are knowledgeable in secure coding techniques
 Testing Procedures: 6.5.c : Examine training records to verify that
software developers received training on secure coding techniques,
including how to avoid common coding vulnerabilities, and
understanding how sensitive data is handled in memory.
Hosted by OWASP && the NYC Chapter
Hosted by OWASP the NYC Chapter
But Results Are Not Measured
•Harvard Business Review
– Large-scale organization development is rare
– Measurement of results is even rarer

•Workforce analytics rare
– More than 25% of survey respondents use little or no workforce
analytics
– The vast majority (>61%) report their use as tactical, ad hoc, and
disconnected from other key systems and processes

Hosted by OWASP && the NYC Chapter
Hosted by OWASP the NYC Chapter
Growth & Turnover Spur Sense of
Urgency
– Software development field growing 30%
– Turnover
• Industry – 14-15%
• General IT – ~20%
• Software Development – ~20 – 30%
Sources: Bureau for Labor Statistics and Society of Human
Resources Management

Hosted by OWASP && the NYC Chapter
Hosted by OWASP the NYC Chapter
Research Overview
•Focus: Assess the software developers depth of software
security knowledge
•Purpose: To measure the impact of software security
training on that level of understanding
•Survey size: 600 software developers surveyed in North
America (US and Canada)
•Vertical markets represented: financial, government,
retail, educational, technology, energy and healthcare
segments.

Hosted by OWASP && the NYC Chapter
Hosted by OWASP the NYC Chapter
Respondent Demographics

Hosted by OWASP && the NYC Chapter
Hosted by OWASP the NYC Chapter
Respondent Demographics

Hosted by OWASP && the NYC Chapter
Hosted by OWASP the NYC Chapter
Respondent Demographics

Hosted by OWASP && the NYC Chapter
Hosted by OWASP the NYC Chapter
Methodology
 15 Multiple Choice Quiz-Style Questions
 Targeted at Software Developers
 Varied by years of experience, amounts of previous training,
primary job function, company industry and company size

 Distribution:
 Online and hard-copy questionnaires given to instructor-led
class trainees (before and after)
 Social media networks (sharing and some paid promotion
with incentives)

Hosted by OWASP && the NYC Chapter
Hosted by OWASP the NYC Chapter
Hypotheses
1. Most software developers do not have a basic
understanding of software security concepts.
2. Software security training can improve a developer’s
knowledge of security concepts in the short-term.
3. Certain industries, such as financial services, are
more likely to have software developers that are
already exposed to key software security concepts.

Hosted by OWASP && the NYC Chapter
Hosted by OWASP the NYC Chapter
Sample Questions
4. If an attacker were able to view sensitive customer records they should
not have had access to, this would be a(n)_______breach.
___ Confidentiality
___ Integrity
___ Availability
7. Authentication is...
___ Proving to an application that the user is who they claim to be
___ Confirming that the user is allowed to access a certain page or function
___Verifying that the data displayed on a given page is authentic
___ Thoroughly logging all of a user's important activity

Hosted by OWASP && the NYC Chapter
Hosted by OWASP the NYC Chapter
Key Survey Results
• Enterprises of more than 10,000 personnel had the lowest
secure coding knowledge

Hosted by OWASP && the NYC Chapter
Hosted by OWASP the NYC Chapter
Key Survey Results
• Architects and software developers had a much higher level of
knowledge than QA, yet in many organizations QA has a
material role in application security

Hosted by OWASP && the NYC Chapter
Hosted by OWASP the NYC Chapter
Key Survey Results
•Slightly more than half of the respondents correctly answered
basic awareness questions on application but struggled with ways
to operationalize appsec concepts

Hosted by OWASP && the NYC Chapter
Hosted by OWASP the NYC Chapter
Key Survey Results
•Almost 100 percent could define input validation, demonstrating a
choppy understanding of advanced secure coding knowledge.
•On the other hand, almost 90 percent correctly identified proper
session IDs which is reassuring.

Hosted by OWASP && the NYC Chapter
Hosted by OWASP the NYC Chapter
Key Survey Results
•The majority of the respondents had no prior secure
coding training, which might be surprising

Hosted by OWASP && the NYC Chapter
Hosted by OWASP the NYC Chapter
Key Survey Results
• There was no correlation between years of experience
and knowledge of secure coding highlighting the
continued need for effective security training

Hosted by OWASP && the NYC Chapter
Hosted by OWASP the NYC Chapter
Key Survey Results
•The respondents that had had more than 3 days of
app sec training in the past were able to answer more
than half of the questions correctly.

Hosted by OWASP && the NYC Chapter
Hosted by OWASP the NYC Chapter
Key Survey Results
•100% correctly identified where cross site scripting
executes after completing training, an increase of almost
20 percentage points.

Hosted by OWASP && the NYC Chapter
Hosted by OWASP the NYC Chapter
Key Survey Results
•The number of respondents able to correctly identify
what is application security more than doubled after
training was complete.

Hosted by OWASP && the NYC Chapter
Hosted by OWASP the NYC Chapter
CONCLUSION
• Retention rose by more than 25 percent after completing
secure coding training.
• Other statistics also reported that application vulnerabilities
reduced by over 70 percent after training

Hosted by OWASP && the NYC Chapter
Hosted by OWASP the NYC Chapter
Software Developers Learn Differently
than Companies “Teach”
• Teaching methods are formalized and
structured in order to be repeatable
• Type of structures consist of:
– On-site & off-site classroom training
– E-learning for compliance
– Videos, webinars, etc.

Hosted by OWASP && the NYC Chapter
Hosted by OWASP the NYC Chapter
So How Do Developers Learn?
•Informally and and in an unstructured way
via:
•
•
•
•
•

Blogs & RSS feeds
Social media with emphasis
Developer websites
Influential e-mail lists
Safarionline

Hosted by OWASP && the NYC Chapter
Hosted by OWASP the NYC Chapter
Don’t Ignore Basics of Training
•Refresher training is still needed
•Training must be included in performance
plans
•Managers increasingly want an ROI

Hosted by OWASP && the NYC Chapter
Hosted by OWASP the NYC Chapter
Incentives Matter!

Hosted by OWASP && the NYC Chapter
Hosted by OWASP the NYC Chapter
CONCLUSION
• Software developers largely do not understand
key software security concepts
 73% of respondents “failed” the initial survey
 Average score of 59% before training

• However, software developers’
understanding of key software security
concepts did increase after training

Hosted by OWASP && the NYC Chapter
Hosted by OWASP the NYC Chapter
Where do we Go from Here?

Hosted by OWASP && the NYC Chapter
Hosted by OWASP the NYC Chapter
Questions and Answers?
John B. Dickson
@johnbdickson
john@denimgroup.com

Hosted by OWASP && the NYC Chapter
Hosted by OWASP the NYC Chapter

More Related Content

Similar to Appsec2013 presentation-dickson final-with_all_final_edits

Security Training: Necessary Evil, Waste of Time, or Genius Move?
Security Training: Necessary Evil, Waste of Time, or Genius Move?Security Training: Necessary Evil, Waste of Time, or Genius Move?
Security Training: Necessary Evil, Waste of Time, or Genius Move?Denim Group
 
AppSec Survey 2.0 Fine-Tuning an AppSec Training Program Based on Data
AppSec Survey 2.0 Fine-Tuning an AppSec Training Program Based on DataAppSec Survey 2.0 Fine-Tuning an AppSec Training Program Based on Data
AppSec Survey 2.0 Fine-Tuning an AppSec Training Program Based on DataDenim Group
 
Owasp Summit - Wednesday evening briefing master
Owasp Summit - Wednesday evening briefing masterOwasp Summit - Wednesday evening briefing master
Owasp Summit - Wednesday evening briefing masterDinis Cruz
 
Digital Product Security
Digital Product SecurityDigital Product Security
Digital Product SecuritySoftServe
 
Software Operation Knowledge
Software Operation KnowledgeSoftware Operation Knowledge
Software Operation KnowledgeDevnology
 
DevSecOps - It can change your life (cycle)
DevSecOps - It can change your life (cycle)DevSecOps - It can change your life (cycle)
DevSecOps - It can change your life (cycle)Qualitest
 
Secure DevOPS Implementation Guidance
Secure DevOPS Implementation GuidanceSecure DevOPS Implementation Guidance
Secure DevOPS Implementation GuidanceTej Luthra
 
Succeeding-Marriage-Cybersecurity-DevOps final
Succeeding-Marriage-Cybersecurity-DevOps finalSucceeding-Marriage-Cybersecurity-DevOps final
Succeeding-Marriage-Cybersecurity-DevOps finalrkadayam
 
Getting Executive Support for a Software Security Program
Getting Executive Support for a Software Security ProgramGetting Executive Support for a Software Security Program
Getting Executive Support for a Software Security ProgramCigital
 
Giving your AppSec program the edge - using OpenSAMM for benchmarking and sof...
Giving your AppSec program the edge - using OpenSAMM for benchmarking and sof...Giving your AppSec program the edge - using OpenSAMM for benchmarking and sof...
Giving your AppSec program the edge - using OpenSAMM for benchmarking and sof...Denim Group
 
DevSecCon KeyNote London 2015
DevSecCon KeyNote London 2015DevSecCon KeyNote London 2015
DevSecCon KeyNote London 2015Shannon Lietz
 
IT Application Development - with SDLC.pptx
IT Application Development - with SDLC.pptxIT Application Development - with SDLC.pptx
IT Application Development - with SDLC.pptxdjualaja88
 
CMMI Institute Conference Seattle_Final +.PPT
CMMI Institute Conference Seattle_Final +.PPTCMMI Institute Conference Seattle_Final +.PPT
CMMI Institute Conference Seattle_Final +.PPTMargaret Tanner Glover
 
Software Modernization for the Digital Economy
Software Modernization for the Digital EconomySoftware Modernization for the Digital Economy
Software Modernization for the Digital EconomyZinnov
 
Continuous Security / DevSecOps- Why How and What
Continuous Security /  DevSecOps- Why How and WhatContinuous Security /  DevSecOps- Why How and What
Continuous Security / DevSecOps- Why How and WhatMarc Hornbeek
 

Similar to Appsec2013 presentation-dickson final-with_all_final_edits (20)

Security Training: Necessary Evil, Waste of Time, or Genius Move?
Security Training: Necessary Evil, Waste of Time, or Genius Move?Security Training: Necessary Evil, Waste of Time, or Genius Move?
Security Training: Necessary Evil, Waste of Time, or Genius Move?
 
Web Application Security Strategy
Web Application Security Strategy Web Application Security Strategy
Web Application Security Strategy
 
AppSec Survey 2.0 Fine-Tuning an AppSec Training Program Based on Data
AppSec Survey 2.0 Fine-Tuning an AppSec Training Program Based on DataAppSec Survey 2.0 Fine-Tuning an AppSec Training Program Based on Data
AppSec Survey 2.0 Fine-Tuning an AppSec Training Program Based on Data
 
Owasp Summit - Wednesday evening briefing master
Owasp Summit - Wednesday evening briefing masterOwasp Summit - Wednesday evening briefing master
Owasp Summit - Wednesday evening briefing master
 
Digital Product Security
Digital Product SecurityDigital Product Security
Digital Product Security
 
Software Operation Knowledge
Software Operation KnowledgeSoftware Operation Knowledge
Software Operation Knowledge
 
DevSecOps - It can change your life (cycle)
DevSecOps - It can change your life (cycle)DevSecOps - It can change your life (cycle)
DevSecOps - It can change your life (cycle)
 
Secure DevOPS Implementation Guidance
Secure DevOPS Implementation GuidanceSecure DevOPS Implementation Guidance
Secure DevOPS Implementation Guidance
 
Succeeding-Marriage-Cybersecurity-DevOps final
Succeeding-Marriage-Cybersecurity-DevOps finalSucceeding-Marriage-Cybersecurity-DevOps final
Succeeding-Marriage-Cybersecurity-DevOps final
 
Getting Executive Support for a Software Security Program
Getting Executive Support for a Software Security ProgramGetting Executive Support for a Software Security Program
Getting Executive Support for a Software Security Program
 
Giving your AppSec program the edge - using OpenSAMM for benchmarking and sof...
Giving your AppSec program the edge - using OpenSAMM for benchmarking and sof...Giving your AppSec program the edge - using OpenSAMM for benchmarking and sof...
Giving your AppSec program the edge - using OpenSAMM for benchmarking and sof...
 
Cissp classroom program ievision
Cissp classroom program ievisionCissp classroom program ievision
Cissp classroom program ievision
 
Ey34927932
Ey34927932Ey34927932
Ey34927932
 
DevSecCon KeyNote London 2015
DevSecCon KeyNote London 2015DevSecCon KeyNote London 2015
DevSecCon KeyNote London 2015
 
DevSecCon Keynote
DevSecCon KeynoteDevSecCon Keynote
DevSecCon Keynote
 
IT Application Development - with SDLC.pptx
IT Application Development - with SDLC.pptxIT Application Development - with SDLC.pptx
IT Application Development - with SDLC.pptx
 
CMMI Institute Conference Seattle_Final +.PPT
CMMI Institute Conference Seattle_Final +.PPTCMMI Institute Conference Seattle_Final +.PPT
CMMI Institute Conference Seattle_Final +.PPT
 
Software Modernization for the Digital Economy
Software Modernization for the Digital EconomySoftware Modernization for the Digital Economy
Software Modernization for the Digital Economy
 
Continuous Security / DevSecOps- Why How and What
Continuous Security /  DevSecOps- Why How and WhatContinuous Security /  DevSecOps- Why How and What
Continuous Security / DevSecOps- Why How and What
 
CISSP Training Program
CISSP Training ProgramCISSP Training Program
CISSP Training Program
 

More from drewz lin

Web security-–-everything-we-know-is-wrong-eoin-keary
Web security-–-everything-we-know-is-wrong-eoin-kearyWeb security-–-everything-we-know-is-wrong-eoin-keary
Web security-–-everything-we-know-is-wrong-eoin-kearydrewz lin
 
Via forensics appsecusa-nov-2013
Via forensics appsecusa-nov-2013Via forensics appsecusa-nov-2013
Via forensics appsecusa-nov-2013drewz lin
 
Phu appsec13
Phu appsec13Phu appsec13
Phu appsec13drewz lin
 
Owasp2013 johannesullrich
Owasp2013 johannesullrichOwasp2013 johannesullrich
Owasp2013 johannesullrichdrewz lin
 
Owasp advanced mobile-application-code-review-techniques-v0.2
Owasp advanced mobile-application-code-review-techniques-v0.2Owasp advanced mobile-application-code-review-techniques-v0.2
Owasp advanced mobile-application-code-review-techniques-v0.2drewz lin
 
I mas appsecusa-nov13-v2
I mas appsecusa-nov13-v2I mas appsecusa-nov13-v2
I mas appsecusa-nov13-v2drewz lin
 
Defeating xss-and-xsrf-with-my faces-frameworks-steve-wolf
Defeating xss-and-xsrf-with-my faces-frameworks-steve-wolfDefeating xss-and-xsrf-with-my faces-frameworks-steve-wolf
Defeating xss-and-xsrf-with-my faces-frameworks-steve-wolfdrewz lin
 
Csrf not-all-defenses-are-created-equal
Csrf not-all-defenses-are-created-equalCsrf not-all-defenses-are-created-equal
Csrf not-all-defenses-are-created-equaldrewz lin
 
Chuck willis-owaspbwa-beyond-1.0-app secusa-2013-11-21
Chuck willis-owaspbwa-beyond-1.0-app secusa-2013-11-21Chuck willis-owaspbwa-beyond-1.0-app secusa-2013-11-21
Chuck willis-owaspbwa-beyond-1.0-app secusa-2013-11-21drewz lin
 
Appsec usa roberthansen
Appsec usa roberthansenAppsec usa roberthansen
Appsec usa roberthansendrewz lin
 
Appsec usa2013 js_libinsecurity_stefanodipaola
Appsec usa2013 js_libinsecurity_stefanodipaolaAppsec usa2013 js_libinsecurity_stefanodipaola
Appsec usa2013 js_libinsecurity_stefanodipaoladrewz lin
 
Appsec2013 presentation
Appsec2013 presentationAppsec2013 presentation
Appsec2013 presentationdrewz lin
 
Appsec 2013-krehel-ondrej-forensic-investigations-of-web-exploitations
Appsec 2013-krehel-ondrej-forensic-investigations-of-web-exploitationsAppsec 2013-krehel-ondrej-forensic-investigations-of-web-exploitations
Appsec 2013-krehel-ondrej-forensic-investigations-of-web-exploitationsdrewz lin
 
Appsec2013 assurance tagging-robert martin
Appsec2013 assurance tagging-robert martinAppsec2013 assurance tagging-robert martin
Appsec2013 assurance tagging-robert martindrewz lin
 
Amol scadaowasp
Amol scadaowaspAmol scadaowasp
Amol scadaowaspdrewz lin
 
Agile sdlc-v1.1-owasp-app sec-usa
Agile sdlc-v1.1-owasp-app sec-usaAgile sdlc-v1.1-owasp-app sec-usa
Agile sdlc-v1.1-owasp-app sec-usadrewz lin
 
Vulnex app secusa2013
Vulnex app secusa2013Vulnex app secusa2013
Vulnex app secusa2013drewz lin
 
基于虚拟化技术的分布式软件测试框架
基于虚拟化技术的分布式软件测试框架基于虚拟化技术的分布式软件测试框架
基于虚拟化技术的分布式软件测试框架drewz lin
 
新浪微博稳定性经验谈
新浪微博稳定性经验谈新浪微博稳定性经验谈
新浪微博稳定性经验谈drewz lin
 
无线App的性能分析和监控实践 rickyqiu
无线App的性能分析和监控实践 rickyqiu无线App的性能分析和监控实践 rickyqiu
无线App的性能分析和监控实践 rickyqiudrewz lin
 

More from drewz lin (20)

Web security-–-everything-we-know-is-wrong-eoin-keary
Web security-–-everything-we-know-is-wrong-eoin-kearyWeb security-–-everything-we-know-is-wrong-eoin-keary
Web security-–-everything-we-know-is-wrong-eoin-keary
 
Via forensics appsecusa-nov-2013
Via forensics appsecusa-nov-2013Via forensics appsecusa-nov-2013
Via forensics appsecusa-nov-2013
 
Phu appsec13
Phu appsec13Phu appsec13
Phu appsec13
 
Owasp2013 johannesullrich
Owasp2013 johannesullrichOwasp2013 johannesullrich
Owasp2013 johannesullrich
 
Owasp advanced mobile-application-code-review-techniques-v0.2
Owasp advanced mobile-application-code-review-techniques-v0.2Owasp advanced mobile-application-code-review-techniques-v0.2
Owasp advanced mobile-application-code-review-techniques-v0.2
 
I mas appsecusa-nov13-v2
I mas appsecusa-nov13-v2I mas appsecusa-nov13-v2
I mas appsecusa-nov13-v2
 
Defeating xss-and-xsrf-with-my faces-frameworks-steve-wolf
Defeating xss-and-xsrf-with-my faces-frameworks-steve-wolfDefeating xss-and-xsrf-with-my faces-frameworks-steve-wolf
Defeating xss-and-xsrf-with-my faces-frameworks-steve-wolf
 
Csrf not-all-defenses-are-created-equal
Csrf not-all-defenses-are-created-equalCsrf not-all-defenses-are-created-equal
Csrf not-all-defenses-are-created-equal
 
Chuck willis-owaspbwa-beyond-1.0-app secusa-2013-11-21
Chuck willis-owaspbwa-beyond-1.0-app secusa-2013-11-21Chuck willis-owaspbwa-beyond-1.0-app secusa-2013-11-21
Chuck willis-owaspbwa-beyond-1.0-app secusa-2013-11-21
 
Appsec usa roberthansen
Appsec usa roberthansenAppsec usa roberthansen
Appsec usa roberthansen
 
Appsec usa2013 js_libinsecurity_stefanodipaola
Appsec usa2013 js_libinsecurity_stefanodipaolaAppsec usa2013 js_libinsecurity_stefanodipaola
Appsec usa2013 js_libinsecurity_stefanodipaola
 
Appsec2013 presentation
Appsec2013 presentationAppsec2013 presentation
Appsec2013 presentation
 
Appsec 2013-krehel-ondrej-forensic-investigations-of-web-exploitations
Appsec 2013-krehel-ondrej-forensic-investigations-of-web-exploitationsAppsec 2013-krehel-ondrej-forensic-investigations-of-web-exploitations
Appsec 2013-krehel-ondrej-forensic-investigations-of-web-exploitations
 
Appsec2013 assurance tagging-robert martin
Appsec2013 assurance tagging-robert martinAppsec2013 assurance tagging-robert martin
Appsec2013 assurance tagging-robert martin
 
Amol scadaowasp
Amol scadaowaspAmol scadaowasp
Amol scadaowasp
 
Agile sdlc-v1.1-owasp-app sec-usa
Agile sdlc-v1.1-owasp-app sec-usaAgile sdlc-v1.1-owasp-app sec-usa
Agile sdlc-v1.1-owasp-app sec-usa
 
Vulnex app secusa2013
Vulnex app secusa2013Vulnex app secusa2013
Vulnex app secusa2013
 
基于虚拟化技术的分布式软件测试框架
基于虚拟化技术的分布式软件测试框架基于虚拟化技术的分布式软件测试框架
基于虚拟化技术的分布式软件测试框架
 
新浪微博稳定性经验谈
新浪微博稳定性经验谈新浪微博稳定性经验谈
新浪微博稳定性经验谈
 
无线App的性能分析和监控实践 rickyqiu
无线App的性能分析和监控实践 rickyqiu无线App的性能分析和监控实践 rickyqiu
无线App的性能分析和监控实践 rickyqiu
 

Recently uploaded

Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URLRuncy Oommen
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 

Recently uploaded (20)

Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URL
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
201610817 - edge part1
201610817 - edge part1201610817 - edge part1
201610817 - edge part1
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 

Appsec2013 presentation-dickson final-with_all_final_edits

  • 1. Can AppSec Training Really Make a Smart Developer? Research From Denim Group John B. Dickson, CISSP @johnbdickson November 20, 2013
  • 2. About Me • Application Security Enthusiast • Security Professional • ISSA Distinguished Fellow • MBA-type and Serial Entrepreneur • Dad Hosted by OWASP && the NYC Chapter Hosted by OWASP the NYC Chapter
  • 3. About Denim Group Denim Group Overview •Professional services firm that builds & secures enterprise applications  External application assessments  Web, mobile, and cloud  Software development lifecycle development (SDLC) consulting •Secure development services:  Secure .NET and Java application development  Post-assessment remediation •Classroom and e-Learning for PCI compliance Hosted by OWASP && the NYC Chapter Hosted by OWASP the NYC Chapter
  • 4. “I personally believe that training users in security is generally a waste of time, and that the money can be spent better elsewhere.” Bruce Schneier Hosted by OWASP && the NYC Chapter Hosted by OWASP the NYC Chapter
  • 5. How Developer Training is Different •Both trying to change behaviors – Target audience has more power to say “no” – Deadlines and releases drive training •For developer, infrequent, but more disruptive • 15-45 minutes vs. 2-day class Hosted by OWASP && the NYC Chapter Hosted by OWASP the NYC Chapter
  • 6. Yet Training is Mandated •PCI DSS 3.0  Train developers in secure coding techniques, including how to avoid common coding vulnerabilities, and understanding how sensitive data is handled in memory.  Testing Procedures: 6.5.a: Examine software development policies and procedures to verify that secure coding technique training is required for developers, based on best practices and guidance.  Testing Procedures: 6.5.b: Interview a sample of developers to verify that they are knowledgeable in secure coding techniques  Testing Procedures: 6.5.c : Examine training records to verify that software developers received training on secure coding techniques, including how to avoid common coding vulnerabilities, and understanding how sensitive data is handled in memory. Hosted by OWASP && the NYC Chapter Hosted by OWASP the NYC Chapter
  • 7. But Results Are Not Measured •Harvard Business Review – Large-scale organization development is rare – Measurement of results is even rarer •Workforce analytics rare – More than 25% of survey respondents use little or no workforce analytics – The vast majority (>61%) report their use as tactical, ad hoc, and disconnected from other key systems and processes Hosted by OWASP && the NYC Chapter Hosted by OWASP the NYC Chapter
  • 8. Growth & Turnover Spur Sense of Urgency – Software development field growing 30% – Turnover • Industry – 14-15% • General IT – ~20% • Software Development – ~20 – 30% Sources: Bureau for Labor Statistics and Society of Human Resources Management Hosted by OWASP && the NYC Chapter Hosted by OWASP the NYC Chapter
  • 9. Research Overview •Focus: Assess the software developers depth of software security knowledge •Purpose: To measure the impact of software security training on that level of understanding •Survey size: 600 software developers surveyed in North America (US and Canada) •Vertical markets represented: financial, government, retail, educational, technology, energy and healthcare segments. Hosted by OWASP && the NYC Chapter Hosted by OWASP the NYC Chapter
  • 10. Respondent Demographics Hosted by OWASP && the NYC Chapter Hosted by OWASP the NYC Chapter
  • 11. Respondent Demographics Hosted by OWASP && the NYC Chapter Hosted by OWASP the NYC Chapter
  • 12. Respondent Demographics Hosted by OWASP && the NYC Chapter Hosted by OWASP the NYC Chapter
  • 13. Methodology  15 Multiple Choice Quiz-Style Questions  Targeted at Software Developers  Varied by years of experience, amounts of previous training, primary job function, company industry and company size  Distribution:  Online and hard-copy questionnaires given to instructor-led class trainees (before and after)  Social media networks (sharing and some paid promotion with incentives) Hosted by OWASP && the NYC Chapter Hosted by OWASP the NYC Chapter
  • 14. Hypotheses 1. Most software developers do not have a basic understanding of software security concepts. 2. Software security training can improve a developer’s knowledge of security concepts in the short-term. 3. Certain industries, such as financial services, are more likely to have software developers that are already exposed to key software security concepts. Hosted by OWASP && the NYC Chapter Hosted by OWASP the NYC Chapter
  • 15. Sample Questions 4. If an attacker were able to view sensitive customer records they should not have had access to, this would be a(n)_______breach. ___ Confidentiality ___ Integrity ___ Availability 7. Authentication is... ___ Proving to an application that the user is who they claim to be ___ Confirming that the user is allowed to access a certain page or function ___Verifying that the data displayed on a given page is authentic ___ Thoroughly logging all of a user's important activity Hosted by OWASP && the NYC Chapter Hosted by OWASP the NYC Chapter
  • 16. Key Survey Results • Enterprises of more than 10,000 personnel had the lowest secure coding knowledge Hosted by OWASP && the NYC Chapter Hosted by OWASP the NYC Chapter
  • 17. Key Survey Results • Architects and software developers had a much higher level of knowledge than QA, yet in many organizations QA has a material role in application security Hosted by OWASP && the NYC Chapter Hosted by OWASP the NYC Chapter
  • 18. Key Survey Results •Slightly more than half of the respondents correctly answered basic awareness questions on application but struggled with ways to operationalize appsec concepts Hosted by OWASP && the NYC Chapter Hosted by OWASP the NYC Chapter
  • 19. Key Survey Results •Almost 100 percent could define input validation, demonstrating a choppy understanding of advanced secure coding knowledge. •On the other hand, almost 90 percent correctly identified proper session IDs which is reassuring. Hosted by OWASP && the NYC Chapter Hosted by OWASP the NYC Chapter
  • 20. Key Survey Results •The majority of the respondents had no prior secure coding training, which might be surprising Hosted by OWASP && the NYC Chapter Hosted by OWASP the NYC Chapter
  • 21. Key Survey Results • There was no correlation between years of experience and knowledge of secure coding highlighting the continued need for effective security training Hosted by OWASP && the NYC Chapter Hosted by OWASP the NYC Chapter
  • 22. Key Survey Results •The respondents that had had more than 3 days of app sec training in the past were able to answer more than half of the questions correctly. Hosted by OWASP && the NYC Chapter Hosted by OWASP the NYC Chapter
  • 23. Key Survey Results •100% correctly identified where cross site scripting executes after completing training, an increase of almost 20 percentage points. Hosted by OWASP && the NYC Chapter Hosted by OWASP the NYC Chapter
  • 24. Key Survey Results •The number of respondents able to correctly identify what is application security more than doubled after training was complete. Hosted by OWASP && the NYC Chapter Hosted by OWASP the NYC Chapter
  • 25. CONCLUSION • Retention rose by more than 25 percent after completing secure coding training. • Other statistics also reported that application vulnerabilities reduced by over 70 percent after training Hosted by OWASP && the NYC Chapter Hosted by OWASP the NYC Chapter
  • 26. Software Developers Learn Differently than Companies “Teach” • Teaching methods are formalized and structured in order to be repeatable • Type of structures consist of: – On-site & off-site classroom training – E-learning for compliance – Videos, webinars, etc. Hosted by OWASP && the NYC Chapter Hosted by OWASP the NYC Chapter
  • 27. So How Do Developers Learn? •Informally and and in an unstructured way via: • • • • • Blogs & RSS feeds Social media with emphasis Developer websites Influential e-mail lists Safarionline Hosted by OWASP && the NYC Chapter Hosted by OWASP the NYC Chapter
  • 28. Don’t Ignore Basics of Training •Refresher training is still needed •Training must be included in performance plans •Managers increasingly want an ROI Hosted by OWASP && the NYC Chapter Hosted by OWASP the NYC Chapter
  • 29. Incentives Matter! Hosted by OWASP && the NYC Chapter Hosted by OWASP the NYC Chapter
  • 30. CONCLUSION • Software developers largely do not understand key software security concepts  73% of respondents “failed” the initial survey  Average score of 59% before training • However, software developers’ understanding of key software security concepts did increase after training Hosted by OWASP && the NYC Chapter Hosted by OWASP the NYC Chapter
  • 31. Where do we Go from Here? Hosted by OWASP && the NYC Chapter Hosted by OWASP the NYC Chapter
  • 32. Questions and Answers? John B. Dickson @johnbdickson john@denimgroup.com Hosted by OWASP && the NYC Chapter Hosted by OWASP the NYC Chapter

Editor's Notes

  1. http://gridinternational.com/fm_files/HBR_Articles/HBR_Breakthrough_in_Organization_Development_-_Harvard_Business_Review1.pdf Strangely enough, large-scale organization development is rare, and the measurement of results is even rarer. Even though management has sought for years to grasp and implement the important findings of behavioral science research, the task has proved more difficult than it first seemed. Many findings are subtle and complex. Other findings relate to individual insights or knowledge which is hard to build into the organization’s life stream. Connecting Workforce Analytics to Better Business Results http://talent.sumtotalsystems.com/rs/sumtotalsystems/images/HBR_SumTotal_Report_Aug2013_rev1.pdf
  2. Growth & Turnover Employment of software developers is projected to grow 30 percent from 2010 to 2020, The main reason for the rapid growth is: a large increase in the demand for computer software Mobile technology requires new applications the healthcare industry is greatly increasing its use of computer systems and applications Source: http://www.bls.gov/ooh/Computer-and-Information-Technology/Software-developers.htm#tab-6 http://compforce.typepad.com/compensation_force/2009/03/2008-turnover-rates-by-industry.html 14-15% Source: http://www.shrm.org/research/benchmarks/documents/trends%20in%20turnover_final.pdf
  3. 19% passing rate among companies with over 10,000 employees Observation/Note: Training tactics and frequency in larger organizations?
  4. Yet, only about 20 percent could define the terms application security or threat modeling, the most basic of security skills
  5. 19% passing rate among companies with over 10,000 employees Observation/Note: Training tactics and frequency in larger organizations?
  6. 19% passing rate among companies with over 10,000 employees Observation/Note: Training tactics and frequency in larger organizations?
  7. 19% passing rate among companies with over 10,000 employees Observation/Note: Training tactics and frequency in larger organizations?
  8. 19% passing rate among companies with over 10,000 employees Observation/Note: Training tactics and frequency in larger organizations?
  9. 19% passing rate among companies with over 10,000 employees Observation/Note: Training tactics and frequency in larger organizations?
  10. Average score increased from 59% to 74% “Pass” rate increased from 27% to 67% Observation – 33% of respondents still failed after training
  11. Bryan Beveryly I follow a series of RSS feeds that I keep pretty regularly pruned. I focus on feeds that cover applied technology and development practices over theory. The .NET teams have some okay feeds, but Scott Hu is a real no-shit .NET developer and he has a much better set of content. Usually the good ones lead me to other good ones. The lousy ones or ones with stale content get pruned.  I have moved all of my framework feeds to just following them on Twitter because it's generally less noise. That's how I find out about updates to Grails/Groovy, Spring libraries, .NET MVC, Java, etc.
  12. Bryan Beveryly I follow a series of RSS feeds that I keep pretty regularly pruned. I focus on feeds that cover applied technology and development practices over theory. The .NET teams have some okay feeds, but Scott Hu is a real no-shit .NET developer and he has a much better set of content. Usually the good ones lead me to other good ones. The lousy ones or ones with stale content get pruned.  I have moved all of my framework feeds to just following them on Twitter because it's generally less noise. That's how I find out about updates to Grails/Groovy, Spring libraries, .NET MVC, Java, etc.
  13. Bryan Beveryly I follow a series of RSS feeds that I keep pretty regularly pruned. I focus on feeds that cover applied technology and development practices over theory. The .NET teams have some okay feeds, but Scott Hu is a real no-shit .NET developer and he has a much better set of content. Usually the good ones lead me to other good ones. The lousy ones or ones with stale content get pruned.  I have moved all of my framework feeds to just following them on Twitter because it's generally less noise. That's how I find out about updates to Grails/Groovy, Spring libraries, .NET MVC, Java, etc.
  14. http://www.entrepreneur.com/article/220569 http://blogs.hbr.org/2010/06/how-to-translate-training-into/ Luckily the “fix” for these kinds of programs is really quite simple: Require that participants come to the program with a specific business challenge (either individually or as a team); build time into the program to create a plan for addressing that challenge based on the content that is presented; and then insist that managers execute against these plans after the program http://gridinternational.com/fm_files/HBR_Articles/HBR_Breakthrough_in_Organization_Development_-_Harvard_Business_Review1.pdf