OIDC Authentication for Atlantis on GKE to Azure

Goal

This article summarizes how to perform OIDC authentication to manage Azure resources with Atlantis running on GKE.

Introduction

Atlantis is one of the CD tools available for Terraform.
https://www.runatlantis.io/

By using this tool, you can execute Terraform from sources like GitHub PRs.
Unlike other tools, by executing Terraform before merging a PR, the code on the main branch is always maintained in a healthy state.
Additionally, Atlantis features its own locking mechanism and policy checking functionality, making it well-suited for team development.

https://www.runatlantis.io/

While Atlantis is convenient, it tends to possess powerful permissions since it manages various cloud resources.
Therefore, using a secure authentication method like OIDC authentication is recommended.

In this article, I will explain the steps for Atlantis running on GKE to acquire the necessary permissions to manage Azure resources via OIDC authentication.

Prerequisites

Atlantis version v0.27.1 is used.

GitHub connection is established using a GitHub App.
https://www.runatlantis.io/docs/access-credentials.html#github-app

Since the goal is limited to reaching the point of authentication with Azure, the actual usage of the Azure provider is not covered here.

Steps

Registering an Azure App

Register an application via the Azure portal.
https://portal.azure.com/#view/Microsoft_AAD_RegisteredApps/ApplicationsListBlade

Set an appropriate name for the Name field.
For Supported account types, select Accounts in this organizational directory only (XXX only - Single tenant).

After creation, copy the Application (client) ID and Directory (tenant) ID from the Overview screen.

Configuring OIDC for the Azure App

First, gather the OIDC information from the GKE cluster where Atlantis is running.
The Issuer URL for GKE cluster OIDC is composed as follows:

https://container.googleapis.com/v1/projects/< GCP Project Name >/locations/< Region or Zone where the GKE cluster is running >/clusters/< GKE Cluster Name >

Return to the detail screen of the Azure App you just created.

Register the OIDC information from Add a certificate or secret under Client credentials.
Enter each item accordingly.

The Audience needs to be edited via Edit(Optional).

Click Add credential from the Federated credentials tab.
Enter the following information into the form:

• Federated credential scenario: Kubernetes accessing Azure resources
• Cluster issuer URL: OIDC Issuer URL of the GKE cluster
• Namespace: GKE Namespace where Atlantis is running
• Service account name: GKE ServiceAccount used by Atlantis
• Audience: OIDC Issuer URL of the GKE cluster

Configuring the Azure provider used by Atlantis

In this setup, we will use the following provider:
https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs

Environment Variables for the Provider

First, copy the following required environment variables for the provider from the Azure portal:
ARM_CLIENT_ID / ARM_SUBSCRIPTION_ID / ARM_TENANT_ID

Environment Variables for OIDC

Set the file path for the ServiceAccount token used by the Pod running on GKE using the following environment variable:
ARM_OIDC_TOKEN_FILE_PATH

A specific adjustment is required for the file path set in this environment variable.

By default, GKE stores the ServiceAccount token at /var/run/secrets/kubernetes.io/serviceaccount/token.
However, this token cannot be used for OIDC authentication to Azure.
Tokens used for Azure OIDC authentication must have an expiration time of 1 hour or less.
Since the ServiceAccount token issued by GKE defaults to an expiration of 1 year, you will encounter an error like the following:

External OIDC Provider token must have a lifetime of less than or equal to 1.01:00:00.

Therefore, use a token with a shorter expiration time, separate from the default one.

By using a projected volume as shown below, you can mount the ServiceAccount token while specifying an expiration time.
In the example below, a volume named azure-token is created with a token that has an expiration of 50 minutes (3000 seconds).

volumes:
- name: azure-token
  projected:
    defaultMode: 420
    sources:
    - serviceAccountToken:
        expirationSeconds: 3000
        path: token

Mount the volume created above to the Pod using volumeMounts.

volumeMounts:
- mountPath: /var/run/secrets/kubernetes.io/azure-token
  name: azure-token

Set ARM_OIDC_TOKEN_FILE_PATH to the path mounted here.
In the case above, you would set /var/run/secrets/kubernetes.io/azure-token/token.

By following these settings, you can access Azure from Atlantis running on GKE using OIDC authentication.