Copilot in GitHub Support is Now GA! Resolve Questions About GitHub Specifications Faster Than Ever

On February 9, 2024, GitHub Support introduced the ability to interact with GitHub Copilot trained on official GitHub documentation^[1].

When I actually tried it, it seemed very convenient for looking up GitHub specifications, so I'd like to introduce it along with some concrete examples and observations.

https://github.blog/2024-02-09-copilot-in-github-support-is-now-available/

What exactly is it?

The announced blog post says the following:

Copilot in GitHub Support is trained on the official GitHub documentation and offers a responsive conversational experience to deliver reliable advice on a wide range of GitHub-related topics.

(Free translation) Copilot in GitHub Support is trained based on official GitHub documentation and provides reliable advice on a wide range of GitHub-related topics through a responsive conversational experience.

In short, it means something like ChatGPT or GitHub Copilot Chat, which has learned the official GitHub documentation, is now available on the support site.

What are the benefits?

The official blog says:

Copilot in GitHub Support reduces the need to manually search for the right context across different pages in the official GitHub documentation. The assistant efficiently distills relevant information from multiple GitHub documents at once into a concise, tailored response—significantly reducing the time required to research a topic and often removing the need for a traditional round-trip response from our Support team.

(Translation) Copilot in GitHub Support reduces the need to manually search for the appropriate context across various pages of the official GitHub documentation. This assistant efficiently extracts relevant information from multiple GitHub documents at once and returns a concise, customized response. This significantly reduces the time required for investigation and, in many cases, eliminates the need for traditional back-and-forth responses from the support team.

In essence, there are the following benefits:

• Reduced need to manually search official GitHub documentation
• No more waiting for interactions with the support team
• The above significantly reduces the time required for investigation

Official GitHub documentation is massive, and frankly, it's hard to find what you're looking for.

Also, while you could always contact support if you couldn't find the answer yourself, previously the GitHub support team had to answer everything, which inevitably took time. Furthermore, support interactions are basically^[2] in English, which can be a high hurdle for those who aren't comfortable with the language.

Of course, since it is generative AI, please note that you won't always get the correct answer.

Feature Introduction

Please refer to the official blog post and the official documentation for usage instructions.
Here, I will provide a brief overview of the features.

When seeking support from Copilot in GitHub Support, select "Chat with Copilot in GitHub Support" on the ticket creation screen (this is the default selection). You can ask follow-up questions to Copilot in GitHub Support's responses right then and there.

For each response, a link indicating "which documentation was referenced" is output. It is a good idea to check the primary source to verify that Copilot has not provided incorrect information.

If the issue is finally resolved, that's great.
If it isn't resolved, you can proceed to submit a support ticket as is.

Be sure to give a thumbs up 👍 or thumbs down 👎 to responses as appropriate.

When using it, please also read the official documentation carefully. The official documentation covers the mechanics of Copilot in GitHub Support, its limitations, best practices, use cases, and how to share feedback.

https://docs.github.com/en/support/learning-about-github-support/about-copilot-in-github-support

Disclaimer

Additionally, a disclaimer is displayed on the usage screen.

Disclaimer: Copilot in GitHub Support
Please note that 'Copilot in GitHub Support' is an AI engine, not a human. For the most effective support, we recommend communicating in English. If you prefer assistance from a human, please click the button below the chat to submit your ticket. While we strive to keep Copilot in GitHub Support current, there may be occasions when the official documentation is updated before these changes are reflected in Copilot in GitHub Support. We advise visiting the provided links to ensure you have the most accurate and up-to-date information.

In summary, it states the following:

• Be aware that it is not a human.
• English is recommended for the most effective support.
• Copilot in GitHub Support is not always up-to-date, so please also refer to the provided official documentation links.

Putting it to the test

I gave it a try immediately.

Previously, I struggled with permissions when trying to update GitHub Actions variables from within GitHub Actions, so let's ask about that.

Initial Question

Question/Answer 1

Question 1

Can I update GitHub Actions variables using the ${{ secrets.GITHUB_TOKEN }} permissions?

Answer 1

To update GitHub Actions variables, you need to use the REST API. However, you cannot update variables using GITHUB_TOKEN. This is because GITHUB_TOKEN has limited permissions.

To update variables, you need an access token with the admin:org scope. This token can be obtained via GitHub Apps or a Personal Access Token.

Below is an example of updating an organization variable using the REST API:

curl \
  -X PATCH \
  -H "Accept: application/vnd.github+json" \
  -H "Authorization: Bearer <YOUR-TOKEN>" \
  https://api.github.com/orgs/ORG/actions/variables/VARIABLE_NAME \
  -d '{
    "value": "NEW_VALUE"
  }'

In this example, replace <YOUR-TOKEN> with an access token that has the admin:org scope, ORG with the organization name, VARIABLE_NAME with the name of the variable you want to update, and NEW_VALUE with the new value.

Please ensure you manage access tokens with the necessary permissions securely before performing this operation.

That's correct! You cannot update variables with GITHUB_TOKEN.

It's also very helpful that it provides a curl example for the API execution.

However, since the X-GitHub-Api-Version header isn't specified, this command example might stop working in the future (currently, if omitted, the 2022-11-28 version is applied by default).

Further Question

But then I remembered I actually wanted to update repository variables. Let's ask a follow-up.

Question/Answer 2

Question 2

What about for repository variables?

Answer 2

When updating repository variables, you also need an access token with the repo scope instead of GITHUB_TOKEN. This is because GITHUB_TOKEN has limited permissions.

Below is an example of updating a repository variable using the REST API:

curl \
  -X PATCH \
  -H "Accept: application/vnd.github+json" \
  -H "Authorization: Bearer <YOUR-TOKEN>" \
  https://api.github.com/repos/OWNER/REPO/actions/variables/VARIABLE_NAME \
  -d '{
    "value": "NEW_VALUE"
  }'

In this example, replace <YOUR-TOKEN> with an access token that has the repo scope, OWNER with the repository owner's name, REPO with the repository name, VARIABLE_NAME with the name of the variable you want to update, and NEW_VALUE with the new value.

Please ensure you manage access tokens with the necessary permissions securely before performing this operation.

Correct again! The scope changes for repositories.

Yet Another Question

However, the answers provided assume the use of API tokens. These days, it's all about GitHub Apps (though opinions vary). Let's ask how to handle it with GitHub Apps.

Question/Answer 3

Question 3

What if I use GitHub Apps instead of an access token?

Answer 3

If you use a GitHub App to update repository variables, that App must have the actions_variables:write repository permission.

Below is an example of updating a repository variable using the REST API:

curl \
  -X PATCH \
  -H "Accept: application/vnd.github+json" \
  -H "Authorization: Bearer <APP-TOKEN>" \
  https://api.github.com/repos/OWNER/REPO/actions/variables/VARIABLE_NAME \
  -d '{
    "value": "NEW_VALUE"
  }'

In this example, replace <APP-TOKEN> with the GitHub App installation access token, OWNER with the repository owner's name, REPO with the repository name, VARIABLE_NAME with the name of the variable you want to update, and NEW_VALUE with the new value.

Please ensure the GitHub App has the necessary permissions before performing this operation.

Excellent! It's absolutely right that actions_variables:write permission is required for GitHub Apps.

Permissions around GitHub Actions are often quite tedious to look up, so it's great to be able to get answers easily and quickly with Copilot in GitHub Support.

However, after trying it out, I noticed a few points regarding the user experience:

• Content disappears if you navigate away from the page
  • Since the ticket isn't created until you click "Submit a Ticket."
    • It won't remain in your ticket list either.
  • If you get a good answer, save it with a screenshot or similar.
• Pressing the Enter key to confirm text conversion sends the message
  • When chatting in Japanese, it might be better to write the question elsewhere and paste it.
• Confetti flies when you click "My problem is solved!"
  • Let's click it if your issue is resolved.

The fact that content is lost when moving pages is a particularly significant pitfall. Keep this in mind when using it.

Impressions

My impression after trying it out is that it's great to be able to reach an answer without having to search through documentation.
To be honest, I was surprised because I didn't expect it to give such a good response.

Of course, you can't take the answers as absolute truth, so you need to carefully verify if it's saying anything incorrect. However, since links to the documentation are provided, verification is relatively easy.
Since I'm not great with English, I'd like to use a workflow where I first ask Copilot in GitHub Support, check the documents, and if I'm still stuck, I'll submit a support ticket.

By the way, I asked the same questions to other AI assistants, but Copilot in GitHub Support was the only one that provided a decent answer. (For more details, see the Extra section below.)
As expected, the fact that it is trained on GitHub Docs really makes a difference.
I want to actively use Copilot in GitHub Support for GitHub-related questions.

Copilot in GitHub Support is a quite useful feature. It's less intimidating than asking a person, and since you can ask in Japanese, I want to keep using it more and more.

Extra: Asking other AI assistants

Right. To see if Copilot in GitHub Support really has an advantage, I asked the same questions to other AI assistants.

ChatGPT 3.5 Response

Question 1

Can I update GitHub Actions variables using the ${ secrets.GITHUB_TOKEN } permissions?

Answer 1

${{ secrets.GITHUB_TOKEN }} in GitHub Actions is a token that provides access to a specific repository, but using that token to update variables is generally not supported.

${{ secrets.GITHUB_TOKEN }} is typically associated with jobs running within a workflow and is used to access the GitHub repository within that job. However, variable updates may occur outside of the workflow's control flow, and for security reasons, updating variables using ${{ secrets.GITHUB_TOKEN }} may not be permitted.

To update variables, you might instead use a Personal Access Token (PAT) or a service account token with the necessary permissions. However, this requires carefully managed permissions and security measures.

More specific information would be needed to provide advice for a particular case based on detailed requirements.

Since I don't pay for ChatGPT, this is the response from ChatGPT 3.5.

The answer seems mostly correct, but the source is unclear. Also, it didn't tell me about other ways to achieve it, which Copilot in GitHub Support did.
Let's ask while we're at it.

Question 2

Then how can I update GitHub Actions variables via API?

Answer 2

To update GitHub Actions variables via the API, you need to use the GitHub REST API. Updating variables requires permissions to manage repository secrets. Below are the general steps for updating variables.

1. Create a Personal Access Token (PAT):

   • Create a PAT in your GitHub account. This token needs the necessary permissions. At minimum, the repo scope is required, but also check permissions related to variable management.

2. Update variables using the REST API:

   • Use the secrets API to update variables. Below is an example for creating or updating a variable.

REPO_OWNER=your_username
REPO_NAME=your_repository
SECRET_NAME=your_secret_name
PAT=your_personal_access_token

# Create or update a secret
curl -X PUT \
  -H "Authorization: token $PAT" \
  -H "Accept: application/vnd.github.v3+json" \
  https://api.github.com/repos/$REPO_OWNER/$REPO_NAME/actions/secrets/$SECRET_NAME \
  -d '{"encrypted_value":"your_encrypted_value","key_id":"your_key_id"}'

In this command, you need to appropriately replace $REPO_OWNER, $REPO_NAME, $SECRET_NAME, and $PAT with your own repository and account information. encrypted_value and key_id must be replaced with the encrypted value of the variable and its key ID.

Note that GitHub Actions variables are encrypted for security reasons. Therefore, when setting a new value, you must use GitHub's API to encrypt the variable. Otherwise, the variable's value might not function correctly.

Hmm. Unfortunately, it seems to have mistaken variables for secrets.

Let's ask again.

Question 3

That method is for updating secrets. Please tell me how to update variables.

Answer 3

Variables in GitHub Actions are typically set dynamically within a workflow and used afterwards, but there is no provided method to directly update the variables themselves using an API.

GitHub Actions variables are set dynamically during job execution and are only valid within that workflow. Updating variables directly using an API is not supported, and no API endpoint exists for such a purpose.

If you want to update a variable, you need to update it using standard programming techniques within the job or script where the variable is defined. It is common for a variable to be set when a workflow is executed and then used in subsequent steps or jobs.

It's saying something quite off the mark. It doesn't seem to know about GitHub Actions variables.

Screenshots

Microsoft Copilot Response

Question

Can I update GitHub Actions variables using the ${ secrets.GITHUB_TOKEN } permissions?

Answer

In GitHub Actions, you can use secrets to store sensitive information for organizations, repositories, or repository environments. Secrets are variables available within workflows, and they can only be read by GitHub Actions when explicitly included in a workflow.

secrets.GITHUB_TOKEN is a token automatically provided within GitHub Actions workflows and has access permissions for the repository. Using this token, you can perform operations on the repository within the workflow (e.g., building, testing, or deploying code). However, some operations may result in permission errors.

Specifically, you can update repository variables using secrets.GITHUB_TOKEN, but please note that there are limitations for some operations. For detailed operations, please check the action's README file.

Simply put, it is possible to update repository variables using secrets.GITHUB_TOKEN, but please be mindful of permission limitations. .

Just like ChatGPT 3.5, it was judged as secrets. I asked more questions after that, but unfortunately, I couldn't get a response that hit the mark.

Screenshots

Gemini's Response

Question

Can I update GitHub Actions variables using the ${ secrets.GITHUB_TOKEN } permissions?

Answer

Yes, it is possible to update GitHub Actions variables using ${ secrets.GITHUB_TOKEN } permissions. However, there are several points to note.

Method

1. Set GITHUB_TOKEN as a secret in repository settings

   • Go to your repository settings.
   • From the left menu, select Security > Secrets and variables.
   • Click New repository secret.
   • Enter GITHUB_TOKEN for the Name.
   • For the Value, enter a GitHub Personal Access Token (PAT).
   • In Permissions, configure it to only give access to actions workflows.
   • Click Save.

2. Set variables in the workflow file

   • Edit your workflow file (.github/workflows/*.yml).
   • In a job step, use the env key to set variables.
   • For the variable value, use ${ secrets.GITHUB_TOKEN }.

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - name: Checkout code
      uses: actions/checkout@v2

    - name: Install dependencies
      run: npm install

    - name: Build and test
      run: npm run build && npm test

    - name: Deploy to GitHub Pages
      uses: actions/deploy-pages@v1
      with:
        github_token: ${{ secrets.GITHUB_TOKEN }}

Notes

• ${ secrets.GITHUB_TOKEN } can only be used within the workflow file.
• Be careful not to output ${ secrets.GITHUB_TOKEN } as plain text.
• Workflows using ${ secrets.GITHUB_TOKEN } must be configured to only grant access to actions workflows.
• When using ${ secrets.GITHUB_TOKEN }, security risks must be considered.

References

• Security hardening for GitHub Actions: https://docs.github.com/en/actions/security-guides/security-hardening-for-github-actions
• Automatic token authentication: https://docs.github.com/en/actions/security-guides/automatic-token-authentication
• Using secrets in GitHub Actions: https://docs.github.com/en/actions/security-guides/using-secrets-in-github-actions

This one also judged them as secrets. It seems it doesn't know about the existence of variables after all.

Screenshots

脚注

1. It seems some GitHub Enterprise users could use it in a closed beta before then. With this general availability, GitHub Free users can now also use it.

2. Depending on your plan, support may be available in Japanese.