Using Azure Service Principal in CI
Table of contents:
The purpose of this article is to showcase several authentication options for utilizing Azure resources in Continuous Integration (CI) pipelines. I’m going to use GitLab CI (any other DevOps platform would work as well). This guide demonstrates multiple authentication methods for integrating GitLab CI with Azure Cloud. These techniques can be applied to various scenarios, such as deploying infrastructure using Terraform or Azure Bicep. By exploring these authentication options, you’ll be better equipped to securely manage your Azure resources within your GitLab CI/CD pipelines.
This article is focused on the following three:
- Certificates
- Client secrets
- Federated credentials
Getting started #
With Azure CLI (az) creating the following two:
1# Create Azure AD Application
2
3az ad app create --display-name gitlab-ci --query appId -otsv
4
5# Create Azure SPN
6
7az ad sp create --id appId --query appId -otsv
8
9# Assign SPN with Owner (or any other) scopes to MG or Sub level to allow interacting with resources
10
11az role assignment create --assignee {appObjectID} \
12--role "Owner" \
13--scope "/subscriptions/{subscriptionId}"
For a visual approach, you can follow Microsoft’s official documentation to create the necessary resources through the Azure portal. This step-by-step guide provides a user-friendly interface for those who prefer a graphical method over command-line operations.
Client secrets #
We are going to make secret. By default, this command clears all passwords and keys, and let graph service generate a password credential.
1
2# Create new secret
3az ad app credential reset --id AppId
4
5The output includes credentials that you must protect. Be sure that you do not include these credentials in your code or check the credentials into your source control. For more information, see https://aka.ms/azadsp-cli
6{
7 "appId": "<REDACTED>",
8 "password": "<REDACTED>",
9 "tenant": "<REDACTED>"
10}
To validate this, in Azure’s Portal: Microsoft Entra ID => App registrations => All applications => gitlab-ci => Certificates & secrets => Client secrets (1):
I’m going to use the following example for my CI:
1param location string = resourceGroup().location
2param name string = uniqueString(resourceGroup().id)
3
4resource storageaccount 'Microsoft.Storage/storageAccounts@2023-05-01' = {
5 name: name
6 location: location
7 kind: 'StorageV2'
8 sku: {
9 name: 'Premium_LRS'
10 }
11}
As this doc suggests we are going to use:
1# Login
2
3az login --service-principal --username APP_ID --password CLIENT_SECRET --tenant TENANT_ID
4
5# Set correct Sub (in case of multiple Sub)
6
7az account set -s $AZURE_SUBSCRIPTION_ID
Here is what I put inside the .gitlab-ci.yml file to create and remove couple of resources:
1image: mcr.microsoft.com/azure-cli
2
3stages:
4 - deploy
5
6deploy_job_with_client_secret:
7 stage: deploy
8 script:
9 - echo "This job is using the secret ${AZURE_APP_ID} and ${AZURE_PASSWORD} to login to Azure."
10 - az login --service-principal -u $AZURE_APP_ID -p $AZURE_PASSWORD -t $AZURE_TENANT_ID
11 - az account set -s $AZURE_SUBSCRIPTION_ID
12 - az group create --name myResourceGroup --location eastus
13 - az deployment group create --resource-group myResourceGroup --template-file storage.bicep
14 - az group delete --name myResourceGroup --yes
15 - echo "All resources have been deleted! Quitting the job... "
Note that AZURE_APP_ID, AZURE_PASSWORD, AZURE_SUBSCRIPTION_ID and AZURE_TENANT_ID are stored as CICD variables in my case.
Here is the job’s log.
Certificates are considered better than secrets for authentication due to several reasons. Firstly, certificates provide stronger security as they are less susceptible to phishing attacks compared to passwords or secrets. Secondly, certificates are harder to steal or guess, reducing the risk of unauthorized access. Thirdly, they offer a better user experience by eliminating the need for users to remember and manage complex passwords. Additionally, certificates can be automatically renewed and managed, reducing administrative overhead. They also support multi-factor authentication, adding an extra layer of security. Lastly, certificates can be used in various scenarios, such as device authentication and secure communications, making them versatile for different security needs. I am going to advocate and use certificates over secrets. To learn more about why certificate-based authentication is more secure, see Microsoft Entra certificate-based authentication.
Certificates #
Use the --append parameter in az ad sp credential reset to append a certificate to an existing service principal. By default, this command clears all passwords and keys so use carefully.
1# Create new certificate (append)
2
3az ad sp credential reset --id myServicePrincipalID --create-cert
4
5The output includes credentials that you must protect. Be sure that you do not include these credentials in your code or check the credentials into your source control. For more information, see https://aka.ms/azadsp-cli
6{
7 "appId": "<REDACTED>",
8 "fileWithCertAndPrivateKey": "/Users/evgeny/tmpifvrfuzt.pem",
9 "password": null,
10 "tenant": "<REDACTED>"
11}
12
13# Append certificate
14az ad sp credential reset --id myServicePrincipalID \
15--append \
16--cert @/Users/evgeny/tmpifvrfuzt.pem
17Certificate expires 2025-01-01 13:44:06+00:00. Adjusting key credential end date to match.
18The output includes credentials that you must protect. Be sure that you do not include these credentials in your code or check the credentials into your source control. For more information, see https://aka.ms/azadsp-cli
19{
20 "appId": "<REDACTED>",
21 "password": null,
22 "tenant": "<REDACTED>"
23}
The CI file in this case:
1get_certificate:
2 needs: []
3 image: ubuntu
4 stage: deploy-with-certificate
5 variables:
6 SECURE_FILES_DOWNLOAD_PATH: ./
7 AZURE_CERTIFICATE_PATH: ./tmpifvrfuzt.pem
8 before_script:
9 - apt-get update
10 - apt-get install -y curl
11 script:
12 # https://docs.gitlab.com/ee/ci/secure_files/#use-secure-files-in-cicd-jobs
13 - curl --silent "https://gitlab.com/gitlab-org/incubation-engineering/mobile-devops/download-secure-files/-/raw/main/installer" | bash
14 artifacts:
15 paths:
16 - $AZURE_CERTIFICATE_PATH
17
18deploy_job_with_a_client_certificate:
19 variables:
20 AZURE_CERTIFICATE_PATH: ./tmpifvrfuzt.pem
21 needs:
22 - get_certificate
23 stage: deploy-with-certificate
24 script:
25 - echo "This job is using the secret ${AZURE_APP_ID} and ${AZURE_PASSWORD} to login to Azure."
26 - az login --service-principal -u $AZURE_APP_ID --certificate $AZURE_CERTIFICATE_PATH -t $AZURE_TENANT_ID
27 - az account set -s $AZURE_SUBSCRIPTION_ID
28 - az group create --name myResourceGroupCert --location eastus
29 - az deployment group create --resource-group myResourceGroupCert --template-file storage.bicep
30 - az group delete --name myResourceGroupCert --yes
31 - echo "All resources have been deleted! Quitting the job... "
Note that AZURE_APP_ID, AZURE_SUBSCRIPTION_ID and AZURE_TENANT_ID are stored as CICD variables in my case. I’m storing pem file (which is AZURE_CERTIFICATE_PATH) in secure file and consume it in job like this. Note, that the part with secure files download can be implemented differently.
Here is the job’s log.
Federated credentials #
There is GitLab doc that suggests to use OpenID Connect in Azure to retrieve temporary credentials. Let’s get the federated identity created per this tutorial.
1# Pull the objectID of App
2az ad app show --id appId --query id -otsv
3
4# Generate payload
5cat <<EOF > body.json
6{
7 "name": "gitlab-ci-identity",
8 "issuer": "https://gitlab.com",
9 "subject": "project_path:erudinsky/azure-spn-for-automation:ref_type:branch:ref:main",
10 "description": "GitLab service account federated identity",
11 "audiences": [
12 "https://gitlab.com"
13 ]
14}
15EOF
16
17# Created federated identity
18az rest --method POST --uri "https://graph.microsoft.com/beta/applications/$objectId/federatedIdentityCredentials" --body @body.json
19
20{
21 "@odata.context": "https://graph.microsoft.com/beta/$metadata#applications('6e511a73-8d1c-4726-8cc7-bbc32f8b4256')/federatedIdentityCredentials/$entity",
22 "audiences": [
23 "https://gitlab.com"
24 ],
25 "claimsMatchingExpression": null,
26 "description": "GitLab service account federated identity",
27 "id": "f62a044b-a6b4-45a9-ae9a-d1d8f50fe5e8",
28 "issuer": "https://gitlab.com",
29 "name": "gitlab-ci-identity",
30 "subject": "project_path:erudinsky/azure-spn-for-automation:ref_type:branch:ref:main"
31}
32
33#
To validate this, in Azure’s Portal: Microsoft Entra ID => App registrations => All applications => gitlab-ci => Certificates & secrets => Federated credentials (1):
The ci file in this case:
1deploy_job_with_federated_identity:
2 id_tokens:
3 GITLAB_OIDC_TOKEN:
4 aud: https://gitlab.com
5 stage: deploy-with-federated-identity
6 needs: []
7 script:
8 - echo "This job is using the ${AZURE_APP_ID} and ${AZURE_TENANT_ID} to login to Azure using federated identity."
9 - az login --service-principal -u $AZURE_APP_ID -t $AZURE_TENANT_ID --federated-token $GITLAB_OIDC_TOKEN
10 - az account set -s $AZURE_SUBSCRIPTION_ID
11 - az group create --name myResourceGroupFed --location eastus
12 - az deployment group create --resource-group myResourceGroupFed --template-file storage.bicep
13 - az group delete --name myResourceGroupFed --yes
14 - echo "All resources have been deleted! Quitting the job... "
Here is the job’s log.
However, this approach limits us (see more details). You might have noticed that "subject": "project_path:erudinsky/azure-spn-for-automation:ref_type:branch:ref:main" in our payload requires explicit matching (doc). However, there is Claims matching expression (Preview) feature, that we are going to try.
Claims matching expression (Preview) #
According to this issue, let’s make another App with SPN and create separate federated identity so that we can pass flexible claims based on regex (i.e. pass wildcard).
1{
2 "name": "gitlab-ci-identity-flex",
3 "issuer": "https://gitlab.com",
4 "subject": null,
5 "claimsMatchingExpression": {
6 "value": "claims['sub'] matches 'project_path:erudinsky/azure-spn-for-automation:ref_type:branch:ref:*'",
7 "languageVersion": 1
8 },
9 "description": "GitLab service account federated identity",
10 "audiences": [
11 "https://gitlab.com"
12 ]
13}
Now we see that this job has failed (because ref does not match, i.e. deploy-with-federated-identity-flex is not main). But here, since we use wildcard it works! The sub’s claim that used used: project_path:erudinsky/azure-spn-for-automation:ref_type:branch:ref:*!
Here is the example of federated credential in UI:
Clean up #
Make sure to remove:
1# Remove application
2
3az ad app delete --id AppId
That’s it! Thanks.