Code Signing Using Azure Pipelines and Azure Key Vault
There are numerous cloud services available for software developers, and some of them can be almost forgotten until we need them. And when we need them, they turn out to be invaluable.
Nowadays, code signing is more critical than ever for software engineers. Apple and Microsoft both demand that software is signed before being deployed on their operating systems. Apple will point blank refuse to run a code that isn’t signed, and Microsoft’s Windows shows large warning signs that may scare customers away.
So, to comply with the OS giants and provide more trust to users, developers have to take this mandatory step. There are various ways of code signing your own software. We'll be going over the options further on.
The first option is getting a digital certificate onto a USB key and using it when prompted to sign your code. The downside of it is that someone on the team has to have it plugged into their computer.
So, if you are a software engineer working with CI/CD pipelines, this is not the right option for you.
The way to get around it is to send the key to your cloud provider and get them to plug it in to their hyper-secure data centre. Don’t hold your breath. Next you have to get a certificate that runs on an HSM (hardware security module) - we'll go more into further on.
It’s necessary to understand the difference between Extended Validation (EV) certificates used for code signing and standard SSL certificates. The standard ones can be exported as many times as you need, which means you can upload it to Azure Pipelines or any other CI/CD system. From there, you can use it with the standard Microsoft Sign Tool, or any other tool of your choice.
EV certificates are different when they are held in an HSM, which is the standard for code signing certificates. The difference is that EV certificates are usually kept on a USB key, and therefore it is up to the owner to maintain good security. But HSM certificates are kept in the cloud, and so any maintainer of them has to have high standards to keep the owners of the certificates happy.
In essence, it’s the difference between losing the keys to your front door vs. an online firm divulging your details to hackers. The first is your mistake, the second is someone else’s and it is far more public. Consequently, USB certificates can’t be transferred to HSM ones. The decision must be taken when the certificate is generated.
You can use any certificate authority to generate the security certificate, as long as that authority understands that you'll need an EV certificate, and specifically one that has a hardware security module (HSM) - not one with a physical USB key.
Any cloud-based system like Key Vault will need an HSM version of the EV certificate.
Therefore, you need the certificate authority to upload your new code signing certificate to Azure Key Vault. How is this achieved depends on your certificate authority.
Now you need to pull the certificate from Azure Key Vault into your Azure Pipelines. You can use the excellent Azure Sign Tool to do this.
You need to get the permissions to access this certificate externally, and this page gives a very good overview on how to do that. However, they missed a step and to set up the Key Vault permissions, I compiled this comprehensive list of steps.
How to set up Azure Key Vault Permissions
- 1. Open the Azure portal, go to the Azure Active Directory area, and create an App registration: enter a memorable name, ignore the Redirect URI, and save it.
- 2. Go to your Key Vault, then Access control (IAM), then Add role assignment. Enter the name of the app that you just created into the select input box. Also, choose a Role, such as Reader, and then save.
- 3. The Missing Step: While in the Key Vault, click the Access policies menu item. Click Add Access Policy and add your application. The Certificate Permissions need to have the Get option ticked. And, the Key Permissions need to have the Get and Sign options ticked. You would have thought these two would be in the certificate perms...but no.
- 4. Go back to the application you just created. Select the Certificates & secrets, and either choose to upload a certificate (a new one purely for accessing the Key Vault remotely) or create a client secret. If you choose the latter, keep a copy of the password - you won't see it again!
- 5. In the Overview section of the app will be the Application (client) ID. You will use this, plus the password or certificate, to feed into the Azure Sign Tool later on in a Azure Pipelines task.
Now, your Key Vault is ready to accept requests from your Pipelines account to access the certificate. At this point, you can either use Advanced Installer to pull in the certificate and sign the code, or do it yourself.
Signing directly from Azure
Code signing right from Azure requires a number of steps. The tasks below apply to Microsoft hosted agents, although very similar issues will affect any private agents that you have.
1. The Azure Sign Tool needs the .NET Core SDK to be installed, at least version 2.x, and since the latest update, .NET Core SDK is always used in Windows.
This means that as long as the version of Windows is current, there is no need to install it yourself. You can find which version of the SDK is shipped with which Windows agent.
The current Hosted OS version in Azure Pipelines, also called Default Hosted, is (at the time of writing) Windows Server 2012 R2. Which isn't up to date. Installing a newer .NET Core SDK to overcome this is a drag on every build, and although the installation works, calling the Azure Sign Tool may not work. It seems to find only older versions of the SDK, and throws this error: Unable to find an entry point named 'SignerSignEx3' in DLL 'mssign32'.
So, the easiest thing to do is change your build to use a later OS image. In my experience, Windows 2019 works very well, with no need to install any newer version of .NET Core.
2. Installing the Azure Sign Tool requires a command line task. You can use a .NET Core CLI task instead, but it's not mandatory. In the task, type this:
set DOTNET_SKIP_FIRST_TIME_EXPERIENCE=true dotnet tool install --global AzureSignTool --version 2.0.17
Of course you can use whatever version you want.
The DOTNET_SKIP_FIRST_TIME_EXPERIENCE environment variable isn't absolutely necessary, but it can accelerate things a bit (see an explanation here).
3. Finally, create another command line task and enter the Azure Sign Tool command that you wish to run with. On Windows, this would be something like the one listed below - note that it includes a ^ and not / as a line continuation marker. You can have a look here for more parameter information:
AzureSignTool.exe sign -du "MY-URL" ^ -kvu https://MY-VAULT-NAME.vault.azure.net ^ -kvi CLIENT-ID-BIG-GUID ^ -kvs CLIENT-PASSWORD ^ -kvc MY-CERTIFICATE-NAME ^ -tr http://timestamp.digicert.com ^ -v ^ $(System.DefaultWorkingDirectory)/Path/To/My/Setup/Exe
And, you're set - your pipeline will build and your code will be signed! If you have any issues, the output of the sign tool is rather helpful. The Key Vault is as dependable as you’d expect, and works well. One of the best things is that moving to another CI/CD won’t mean losing the certificate.