Continuous Integration, Continuous Delivery, workflow automation, more and more tools rely on a vibrant marketplace to share extensions that add new functionality to the systems.
Unfortunately, this means you’re putting trust into a LOT of different people and many of these extensions are built by professionals and enthusiasts in their limited spare time.
GitHub has such a marketplace for Actions and Azure DevOps one for Pipelines. But how do you know whether you can trust these extensions, how do you stay up to date and if you’re building these extensions yourself, how do you keep them secure and your users on the latest version?
Jesse takes us through some of the best practices from the perspective of an extension author, user as well as the administrator of the system that loads these extensions.
13. Extensions in the context of Azure Pipelines
Manifests
Marketplace pritties
Tasks
And potentially tabs and context menu actions…
14. So what is a task?
Loads of dependencies
At least azure-pipelines-task-lib
Frozen at time of publish
Another Manifest
The actual custom code
More pretties
15. Or… using PowerShell
A lot fewer dependencies
at least VstsTaskSdk
Another Manifest
The actual custom code
More pretties
17. Where do these tasks come from?
• They’re “built-in” tasks
• Automatically updated by Microsoft in Azure DevOps
• Manually updated through Microsoft Update packs in Azure DevOps Server
• Manually updated by administrator through sideloading (see below)
• They’re from an extension from the public marketplace
• Installed through marketplace.visualstudio.com
• Automatically updated in Azure DevOps and Azure DevOps Server
• They’re from an extension from the private marketplace
• Manually uploaded to Azure DevOps Server
• They’re sideloaded directly
• Manually uploaded to Azure DevOps or Azure DevOps Server
18. 2 problems with task updates
By default, the agent:
1. Pins the major version to the one you selected
2. Updates to the latest minor version available
20. So how do we know we can trust these tasks?
You don’t.
21. How do you know?
Well…
I downloaded 700GB of vsix files from the Azure DevOps marketplace
Extracted them and scanned them
22. So how do we know we can trust them?
Trusted publisher?
Verification?
Rating?
Number of issues and pull-requests?
Metadata provided by the author?
Recently updated?
Looks nice?
27. So what’s the risk we’re running
• Some of these extensions turn off TLS security
• Some might overwrite arbitrary files
• Some might be used to send data to the outside world
• Some install things directly from npmjs.
• Others download 100’s of MB, extract it and execute without
verification
• Often on a host with admin permissions
There is a lot of potential
28.
29. Why is PowerShell so much better?
It might not be but…
• Many of the dependencies are automatically updated through
Windows Update
• PowerShell relies on the .NET framework for most functionality
• There are a lot fewer dependencies to worry about
30. What is Microsoft doing?
• They’ve introduced a new Node 16 handler for Azure Pipelines
• They’re deprecating Node 6 and Node 10 for Azure Pipelines tasks.
• They’re updating all the built-in tasks for Azure DevOps.
• They’re updating all the built-in tasks for Azure DevOps Server 2022?
• They’ve added the ability to turn off Node 6 on the Pipelines Agent.
• They’re adding warnings to every workflow that runs uses Node 6
(and later 10).
31. What am I doing / have I done
• I actively update all my extensions regularly
• I automatically update my dependencies using Snyk, Dependabot and
RenovateBot
• I scan my code for vulnerabilities using Snyk Code and GitHub
advanced Security
• I maintain the Azure DevOps Extension Tasks to implement CI / CD for
Azure DevOps extensions
• I recently contributed to RenovateBot to add support for the Azure
Pipeline Marketplace. Now doing the same for dependabot-core.
32. Guidance for deprecating an extension
• Add a warning to all versions of your tasks
• Timebomb the task to fail after today+X
• Mark all tasks deprecated:true in the task.json
• Add [DEPRECATED] to your task friendly name in the task.json
• Add [DEPRECATED] to your extension name in the vss-extension.json
• Publish this last version
• Unpublish the extension
• Archive the GitHub repository
33. Guidance for deprecating task version
• Add both the old and the new version of the task to your extension
• Update the old version’s friendlyName and add [DEPRECATED] in
task.json
• Add a warning / error to the implementation to warn you users
34. What should Extension authors do?
• Employ secure password practices and use 2FA
• Deprecate extensions you’re no longer maintaining
• Update your existing extensions to Node 16 and latest dependencies
• Enable DependaBot to automatically keep your dependencies up-to-
date
• Enable GitHub Advanced Security to detect issues in *your* code
• Implement CI / CD to ship new versions with ease
• Add repo metadata to their extension manifests
35. What should Azure DevOps admins do
• Disable insecure extensions
• Implement RenovateBot to automatically
suggest updates to installed Azure
Pipelines tasks
• Disable Node 6 support on the Azure
Pipelines Agent
• Do not install every extension in the
marketplace without thought
• Provide Hosted Agents or ephemeral Scale-
set agents
36. What should Azure DevOps admins do
• Run Endpoint Security on your Azure Pipelines Agents
• For DevOps Server
• Upgrade to Azure DevOps Server 2022 and stay current
• Migrate to Azure DevOps Service
• Manually upgrade “built-in” tasks
• Be careful with internal marketplace
39. What should Azure Pipelines authors do?
• Consider whether you actually need an extension.
• Keep your pipelines current. Update your major task versions and
resolve any issues.
• Use Azure Pipelines YAML
• Pin the exact version of your tasks
• Use RenovateBot to automatically keep your pipelines up to date
• Use Hosted Agents / Ephemeral Scale-set agents whenever possible
40. What should the community do?
• Sponsor the authors of extensions you rely on.
• Submit pull requests to improve extensions.
• Submit pull requests to upgrade extensions to Node 16.
• Leave reviews on the marketplace and update them.