iTranslated by AI
Released a Custom GitHub Action to Comment Flutter Static Analysis Results on PRs
I have created a GitHub Action that writes issues detected during Flutter static analysis in GitHub Actions CI to Pull Requests as review comments.
Comments will be displayed on the Pull Request screen as follows:
Internally, it receives the output of flutter analyze, parses it, and uses the GitHub REST API to write comments to the PR.
To use it, simply add the above action to a step in your workflow that executes flutter analyze.
The step uses: yorifuji/flutter-analyze-commenter@v1 is the one using the action I created (refer to the README for details).
jobs:
flutter-analyze:
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write # required to add comment on PR
steps:
# checkout repository and setup flutter environment
...
# run flutter analyze with --write option
- run: flutter analyze --write=flutter_analyze.log
# use flutter-analyze-commenter
- uses: yorifuji/flutter-analyze-commenter@v1
if: ${{ !cancelled() }} # required to run this step even if flutter analyze fails
with:
analyze_log: flutter_analyze.log
I will introduce the motivation behind the development, how to create custom actions, and the implementation details.
Motivation for Development
In daily Flutter development, I often set up CI/CD using GitHub Actions, performing static analysis and testing on Pull Requests during CI. For static analysis, the flutter analyze command is used, and if there are issues in the code, a report like the following is obtained (note that the output depends on the project's lint settings).
Example of code with issues (this is just an example)
Example of flutter analyze output:
% flutter analyze
Analyzing flutter_application_9...
info • Use 'const' for final variables initialized to a constant value • lib/main.dart:4:3 • prefer_const_declarations
warning • The value of the local variable 'y1' isn't used • lib/main.dart:4:16 • unused_local_variable
error • A value of type 'int' can't be assigned to a variable of type 'String' • lib/main.dart:4:21 • invalid_assignment
3 issues found. (ran in 0.9s)
There are three levels of alerts: info, warning, and error.
If you have the Dart Extension installed in VSCode, the analysis is performed by the Dart Analysis Server and displayed in the "Problems" tab. For reference, this uses dart analyze, and the output format is slightly different from flutter analyze.
When static analysis is executed on GitHub Actions, the results appear in the CI logs. However, since digging through CI logs to check the report content is inconvenient, it is best to be able to check them directly on the PR screen.
You can confirm results on the PR screen by combining Problem Matchers for warnings and errors, and Danger with danger-flutter_lint for info (detailed explanations of each tool are omitted).
While each tool's functionality meets the objectives, in actual use, I was bothered by subtle differences such as:
- Problem Matchers content appears in the PR's Files Changed tab.
- Danger content appears in the Conversation tab.
Furthermore, while Danger is highly functional and versatile, its technical stack is Ruby, requiring the installation of dependent libraries. It felt a bit overkill for the sole purpose of displaying static analysis results. To clarify, the tools mentioned above are all relatively mature and are good choices for use in CI where stability is required.
Thinking that handling static analysis content on a PR shouldn't require a very difficult implementation, I decided I could make it myself. I set out to create a "Custom Action" that achieves the functionality covered by the two tools above in a single action.
Creating a Custom Action
By the way, the "custom action" I'm referring to here is not a workflow itself, but an action referenced by uses: within a workflow. For example, actions/checkout@v4, which performs a git checkout, is one of the publicly available custom actions.
The documentation describes how to create custom actions.
There are three types of actions:
- Docker container actions
- JavaScript actions
- Composite actions
While Docker can only be used on Linux (Ubuntu), the other two are also available on macOS and Windows runners. Since I wanted the action I'm creating to be usable on macOS and Windows as well, I had to choose between JavaScript or Composite actions.
JavaScript actions are actions that execute JavaScript using Node.js. Since they can leverage ecosystems like npm, they seem suitable for full-scale development. However, I am not very familiar with development using npm, so I decided to use the other option: Composite actions.
Composite actions are actions that combine steps just like a regular workflow. They can also be used to extract common, reusable processes from a workflow into a separate file (which is probably the more common usage).
In Composite actions, you need to describe the actual logic using shell scripts or uses:. While implementation using shell scripts is possible, it is cumbersome for complex tasks like log parsing or using the GitHub API, so I decided to use actions/github-script.
actions/github-script allows you to write logic using JavaScript within a workflow (job). It also supports access to CI contexts and the use of the GitHub API via octokit/rest.js. Please check the link for what it can do.
In the actual code, it is used as follows; the actual logic, such as log analysis and API usage, is moved to a separate file (index.js).
runs:
using: composite
steps:
- uses: actions/github-script@v7
if: ${{ github.event_name == 'pull_request' }}
env:
ANALYZE_LOG: ${{ inputs.analyze_log }}
VERBOSE: ${{ inputs.verbose }}
with:
retries: 3
script: |
const analyzeLog = process.env.ANALYZE_LOG;
const verboseLogging = process.env.VERBOSE === 'true';
const workingDir = process.env.GITHUB_WORKSPACE;
const actionPath = process.env.GITHUB_ACTION_PATH;
const path = require('path');
const script = require(path.join(actionPath, 'index.js'));
await script({ core, github, context, workingDir, analyzeLog, verboseLogging })
(The full code for action.yml can be found here.)
This covers the general flow of implementing a Composite action. The pros and cons of Composite actions are summarized below.
Pros
- You can write custom actions with a syntax similar to the workflows you use regularly.
- You can use
uses:, which allows you to leverage existing actions.
Cons
- Not suitable for implementing complex or large-scale actions. Depending on the case, the following two are recommended:
- Docker container actions for managing dependencies of execution environments, tools, and libraries.
- JavaScript actions for development using npm and related tools.
Implementation
The actual logic is implemented in index.js. You can find the code here.
Parsing flutter analyze logs
The action retrieves warnings from the log file output by flutter analyze --write=filename. Using the --write option produces the following format. Since the error type, message, filename, and line number are obtained, this information can be used to create comments for the Pull Request.
[error] A value of type 'int' can't be assigned to a variable of type 'String' (/Users/yorifuji/git/flutter_analyze_commenter_ci/lib/main.dart:4:20)
[warning] The value of the local variable 'x' isn't used (/Users/yorifuji/git/flutter_analyze_commenter_ci/lib/main.dart:4:16)
[info] Use 'const' for final variables initialized to a constant value (/Users/yorifuji/git/flutter_analyze_commenter_ci/lib/main.dart:4:3)
Writing static analysis results to the PR
The warnings are written as comments to the PR. However, since flutter analyze performs static analysis on all files in the repository, it also detects issues in lines not included in the PR diff, so it was necessary to distinguish between them.
By using the GitHub REST API to retrieve and parse the PR diff, we can identify the files and line numbers added or modified in the PR. By cross-referencing this with the analysis results, we can separate the warnings into those contained within the PR changes and those that existed in pre-existing code. Warnings within the PR are posted as comments specifying the file and line number. Since warnings in existing code are not part of the PR diff, they are posted as regular comments without specific file or line assignments.
Furthermore, the results of the static analysis change when additional commits are made to the PR. For instance, if a problematic line is fixed, the corresponding comment is no longer needed. In such cases, the action is designed to delete the comment. It uses the REST API to fetch all PR comments, compares them with the static analysis results of the latest commit, and removes any unnecessary comments via the API.
The state management for comments is currently being refined through trial and error while referencing the behavior of Danger, and it is subject to further adjustments in the future.
Release/Registration on GitHub Marketplace
Once the implementation of the custom action is complete, you proceed with the release process. If action.yml exists at the root level of the repository, an item related to the action will be added to GitHub's Releases.
This screen checks if the yml content has no issues and if a README is present. If there are no problems, you can publish it to the Marketplace.
Note that while a public repository is required to publish to the Marketplace, custom actions can also be created in private repositories. In that case, the action can be used by other repositories that have access to the private repository—for example, from other public or private repositories under the same account. I haven't tested it, but I believe it should also be accessible within the same Organization.
Following these steps, you can create and publish a custom action.
Conclusion
Being able to create custom actions expands the possibilities of what you can achieve with GitHub Actions. It could be interesting to incorporate trending technologies like AI as well.
Since the action created this time hasn't been used extensively yet, there might be some bugs. If you use it, I recommend trying it out in side projects first.
Discussion