Enhancing Microsoft Teams: Developing a Secure RAG Bot with Azure

Introduction

Microsoft Teams is used daily by many companies and has become a core platform for business operations. However, information is often scattered across channels and files, leading to the challenge of "not being able to quickly find necessary materials or decisions."

In this article, I will introduce the process of developing a RAG (Retrieval-Augmented Generation) enabled Teams bot that answers questions about internal documents by leveraging various Azure services. In particular, I will focus on a configuration that uses Managed Identities for secure authentication without holding any secrets.

Development Environment

The main technologies used in this development are as follows:

Item	Version / Remarks
Cloud	Microsoft Azure
Language	Python 3.11+
Web Framework	FastAPI
Bot Framework	Microsoft Bot Framework SDK for Python
Main Azure Services	Bot Service, App Service, ACR, AI Search, OpenAI
Container	Docker

I decided to implement the application itself using Python and FastAPI, which I am familiar with.

---

Setup Steps

Here are the actual construction steps. The general flow is as follows:

1. Creating Azure resources
2. Implementing the authentication part
3. Containerizing and deploying the application
4. Creating an index for RAG documents
5. Registering as a Teams app

1. Creating Azure Resources

First, prepare the foundation infrastructure using the Azure CLI. While you can create them one by one via the GUI, using the CLI is convenient for reproducibility.

Please replace the placeholders indicated by <...> in the following commands with arbitrary names suited to your environment.

We will create the App Service to host the Bot application body, Azure Container Registry (ACR) to store container images, Azure OpenAI and Azure AI Search as the brain of the RAG, and Azure Storage to store documents.

The Bot Service plays a central role in connecting these. When creating the Bot Service, you can choose the authentication method. This time, we adopted the User-Assigned Managed Identity method, which does not use secrets. To do this, we first create a Managed Identity resource.

# Variable settings
RESOURCE_GROUP="<your-resource-group-name>"
LOCATION="japaneast"
MANAGED_IDENTITY_NAME="<your-managed-identity-name>"
BOT_NAME="<your-bot-name>"
APP_SERVICE_NAME="<your-app-service-name>"

# Create resource group
az group create --name $RESOURCE_GROUP --location $LOCATION

# Create identity for authentication
az identity create --name $MANAGED_IDENTITY_NAME --resource-group $RESOURCE_GROUP

# Get identity information
IDENTITY_RESOURCE_ID=$(az identity show --name $MANAGED_IDENTITY_NAME --resource-group $RESOURCE_GROUP --query id -o tsv)
CLIENT_ID=$(az identity show --name $MANAGED_IDENTITY_NAME --resource-group $RESOURCE_GROUP --query clientId -o tsv)

# Specify Managed Identity as the authentication method when creating the Bot Service
az bot create \
    --resource-group $RESOURCE_GROUP \
    --name $BOT_NAME \
    --app-type UserAssignedMSI \
    --appid $CLIENT_ID \
    --tenant-id $(az account show --query tenantId -o tsv) \
    --msi-resource-id $IDENTITY_RESOURCE_ID \
    --endpoint "https://$APP_SERVICE_NAME.azurewebsites.net/api/messages" \
    --location "global"

The point is to specify the authentication method at the time of Bot Service creation.

2. Implementing the Authentication Part

This is the code for the authentication part when accessing other Azure services from the Bot application. Since we decided to use Managed Identity for authentication this time, implementation tailored to that was necessary.

The Bot Framework Python SDK did not seem to support this authentication method directly by default. Therefore, I wrote a small amount of additional code using the azure-identity library.

Specifically, I implemented a custom class that inherits from AppCredentials and calls ManagedIdentityCredential internally to retrieve the access token.

app/auth/managed_identity_credentials.py:

from botframework.connector.auth import AppCredentials
from azure.identity import ManagedIdentityCredential

class ManagedIdentityAppCredentials(AppCredentials):
    def __init__(self, app_id: str, **kwargs):
        super().__init__(app_id=app_id, **kwargs)
        self.credential = ManagedIdentityCredential(client_id=app_id)

    def get_access_token(self, force_refresh: bool = False) -> str:
        token = self.credential.get_token("https://api.botframework.com/.default")
        return token.token

Then, during application initialization, set this custom class to the Bot adapter.

app/main.py (excerpt):

adapter_settings = BotFrameworkAdapterSettings(app_id=settings.MicrosoftAppId, app_password="")
adapter = BotFrameworkAdapter(adapter_settings)

# Overwrite the authentication part with the custom class
adapter._credentials = ManagedIdentityAppCredentials(app_id=settings.MicrosoftAppId)

If you were to authenticate using a traditional App ID and password, this implementation would not be necessary, and you would write the secret in the configuration file instead.

3. Containerizing and Deploying the Application

The application is deployed to App Service as a Docker container. The Dockerfile is a standard one for Python applications.

After building the container image and pushing it to Azure Container Registry (ACR), you specify that image in the App Service configuration.

4. Creating an Index for RAG Documents

As a knowledge source for the Bot to answer from, internal documents (such as PDFs) are registered in Azure AI Search. This process was also automated using a Python script.

1. Load the documents and split them into appropriate sizes (chunking).
2. Vectorize each chunk using an Azure OpenAI embedding model.
3. Register the original text and vector data into the Azure AI Search index.

By searching this index, the Bot can quickly find document sections relevant to the question.

5. Registering as a Teams App

Finally, describe the Bot's information in a file called manifest.json, package it into a ZIP file, and upload it via the Teams Admin Center. Now, users within the organization can use the Bot on Teams.

Verification Methods

After building the Bot, verify whether the conversation with users actually works correctly. There are two main ways to verify this.

1. Local Testing with Bot Emulator

During the development phase, you can verify the Bot's behavior in a local environment using the Bot Framework Emulator.

• With the Bot app running locally, you can test conversations by specifying an endpoint like http://localhost:3978/api/messages from the Emulator.
• Since you can verify the behavior before deploying to Teams, debugging efficiency is significantly improved.
• This is useful for checking fine-grained behaviors during development, such as authentication logic and RAG integration responses.

2. Test in Web Chat on Azure Portal

After deploying the Bot to Azure, you can directly test the Bot deployed in the cloud environment using the Bot Service → Test in Web Chat feature on the Azure Portal.

• You can confirm whether it can respond correctly in the cloud before actually registering it with Teams.
• Authentication settings and Managed Identity connections that cannot be reproduced locally can also be verified in this environment.
• Since you can verify operation in an environment close to production, it helps in detecting issues beforehand.

Verification and Impressions

Through the above steps, a RAG bot that operates on Teams was successfully completed. When you actually ask a question, AI Search searches for relevant documents, and based on that content, OpenAI generates and returns an answer.

In this project, I realized that by combining various Azure services, it is possible to build a high-performance chatbot in a relatively short period of time. In particular, I feel that a great advantage of leveraging cloud features for authentication is that it allows for a secure configuration by eliminating secrets from the code.

On the other hand, the accuracy of RAG responses depends heavily on document chunking methods and prompt engineering. I felt once again that this is a deep area that requires continuous improvement through actual operation.

Summary

In this article, I introduced a case study of building a Teams-compatible RAG chatbot using a suite of Azure services.

Management of authentication information is a universal challenge in application development, but I believe that by utilizing cloud-native mechanisms effectively, more secure and efficient development becomes possible. I hope the insights shared here will be helpful to those facing similar challenges.

References

• Azure Bot Service documentation
• Configure a bot with a user-assigned managed identity
• Retrieval-Augmented Generation (RAG) in Azure AI Search