iTranslated by AI
Productivity Weekly (2021-12-08 Issue)
Hello. I'm Hirakiba from the Cybozu Inc. Productivity Improvement Team.
In the Productivity Improvement Team, we hold a weekly meeting every Wednesday called "Productivity Weekly," where we share topics related to improving developer productivity discovered over the past week within the company.
This article is a summary of the topics from that meeting.
This is the 54th issue. Past articles can be found here.
news 📺
Track CircleCI Updates - CircleCI
A RESOURCES tab has been added to the CircleCI job screen, allowing you to monitor the CPU and memory utilization of the Docker Executor.
Now that this information is visualized, it has become easier to set the resource class appropriately. This is a great feature.
RESOURCES tab
There is also a commentary by someone from CircleCI (in Japanese), so please check that out as well.
Legacy Convenience Image Deprecation - Announcements - CircleCI Discuss
CircleCI provides various "convenience" Docker images (which come pre-installed with tools needed for execution on CircleCI). There are two types of namespaces: the circleci namespace and the cimg namespace. The cimg namespace images are the next-generation convenience images.
It was announced that most of these circleci/* images will reach EOL (End of Life) by the end of this year. To my shame, I was completely unaware of this. The discussion linked above details the deprecation, migration methods, and the deprecation schedule.
Support for all circleci/* images except for Clojure, MariaDB, MySQL, Redis, and MongoDB will end on 2021/12/31, after which no bug fixes or security updates will be provided. Most images can be migrated by simply replacing circleci with cimg, but some images have different names or specifications, so care is needed during migration.
If you are using older images, let's do our best to migrate them.
There is also an explanation by someone from CircleCI (in Japanese). It also introduces articles regarding migration, so please check it out.
Use the new Amazon EC2 M1 Mac instances to build and test applications for iPhone, iPad, Mac, Apple Watch, and Apple TV | Amazon Web Services Blog
Last year, Mac instances became available on Amazon EC2, but now M1 Mac instances are also available (in preview).
The currently available regions are us-east-1 and us-west-2. Billing remains the same as before, charged for the host reservation time (minimum 24 hours) rather than the instance execution time. If you use it for testing, be prepared for unexpected costs.
Sign-up for the preview is required. Those who wish to use it should apply.
Safeguard your containers with new container signing capability in GitHub Actions | The GitHub Blog
This is an example of signing Docker images using sigstore[1] with GitHub Actions. To sign using sigstore, you use sigstore/cosign, and since it now supports OIDC, it has become easy to sign images.
Previously, you had to manage private keys yourself, but (since OIDC support was recently added to GitHub Actions,) using OIDC tokens means you no longer need to manage private keys yourself.
An example can be found in actions/starter-workflows. However, the COSIGN_EXPERIMENTAL environment variable is required, and it still feels quite experimental.
Anything under the
COSIGN_EXPERIMENTALenvironment variable
- Integration with the Rekor transparency log
- Keyless signatures using the Fulcio CA
GitHub Enterprise Server 3.3 is generally available | The GitHub Blog
Version 3.3 of GitHub Enterprise Server, the on-premises version of GitHub, has been officially released.
There are many welcome changes for those using GitHub Actions self-hosted runners. With support for ephemeral runners and the workflow_job event for Webhooks, it's now possible to build scalable and secure self-hosted runners. Additionally, composite actions can now be used[2].
There are various other changes, so if you're interested in the details, please check the release notes.
Enrolling all npm publishers in enhanced login verification and next steps for two-factor authentication enforcement | The GitHub Blog
This article outlines the future schedule for making npm authentication more secure.
The plans are as follows:
- Accounts that have previously published packages but do not have 2FA (two-factor authentication) enabled will be required to use a one-time password (from 2021/12/7 to 2022/1/4).
- Maintainers of high-impact packages will gradually be required to enable 2FA (starting 2022/2/1).
They are also working on enhancements such as increasing 2FA options and allowing the registration of multiple devices.
If you don't have a specific reason not to, you should enable 2FA immediately to enhance your security.
know-how 🎓
GitHub Actions workflow not triggering at scheduled time | Upptime
This article summarizes issues where GitHub Actions scheduled workflows are not triggered as scheduled.
When a user inquired with GitHub support, it was clarified that there is no guarantee of timely execution; it may be delayed by more than an hour, or it might not even run at all. If you want to guarantee periodic execution, it seems the only way is to trigger it by calling workflow_dispatch from an external system.
We often see Actions scheduled executions being delayed, but I never imagined they might not run at all. It's good to keep this in mind.
Atlantis for those who want to automate Terraform - Qiita
This is an introductory article about Atlantis, an open-source software (OSS) that provides an automated execution environment for Terraform.
While Terraform automation can be achieved using any CI/CD service, you would need to implement various mechanisms such as locking (exclusivity) or pasting plan results into pull requests. You would also need to maintain those workflows. Atlantis handles these tasks by default, making the management of the automated execution environment much easier (though you do need to host Atlantis yourself).
This article includes a brief explanation of Atlantis, along with what the author finds beneficial or challenging. I recommend this article to both those looking to start automating and those who are already doing so but aren't familiar with Atlantis. I didn't know much about Atlantis either, so it was very informative.
Incorporating Terraform refactoring features into CI/CD with tfmigrate + Atlantis - Qiita
This is also an article about Atlantis.
This article explains how to integrate tfmigrate, a tool that allows for declarative management of tfstate operations, into Atlantis and how to run tfmigrate when Atlantis executes. It also covers basic explanations of tfmigrate and Atlantis, server setup, and configuration, making it a comprehensive read even for those unfamiliar with these tools.
As you automate Terraform, managing state operations becomes increasingly difficult, so you'll likely want to use tfmigrate alongside Atlantis. This is a great companion piece to the previous article.
Regarding OpenID Connect support in GitHub Actions
This article explains the research findings regarding OIDC support in GitHub Actions. It covers an overview of OIDC, configuration examples for AWS (including Terraform examples!), explanations of settings to authorize authentication, and many other details related to OIDC.
It's a must-read for anyone who wants to try OIDC authentication but isn't quite sure what it actually is.
Knowledge and Know-how for Running Docker on M1 Macs
This article explains the knowledge and know-how for running Docker on M1 Macs. It includes basic knowledge of M1 Macs and Docker, troubles encountered when running Docker images on M1 Macs, and how to deal with them. It even includes a flowchart to check when running images on M1 Macs.
I would recommend this to anyone who has only used Docker on x86_64 or amd64 and hasn't experienced issues caused by platform differences.
Procedures and Journey for Removing Sensitive Data from GitHub | Mercari Engineering
This article covers the cleanup after the customer information leak incident[3] caused by Codecov's Bash Uploader[4] at Mercari. It discusses purging sensitive data (sensitive data) from all Git repositories, both remote and local. It covers the overall workflow, identifying sensitive data to be deleted, removing it from repositories, and deleting sensitive data found outside repositories (such as in GitHub PRs).
Even if you wipe data from a Git repository, if a commit SHA is referenced from a GitHub PR, that commit remains on GitHub. The article explains how they worked with the GitHub support team to address this and how to search for relevant data throughout the entire history, providing many valuable lessons.
This is an article you'll want to remember immediately if you ever encounter a similar situation (though we hope to avoid it at all costs). It must have been an incredibly difficult task...
1st Anniversary of Productivity Weekly Articles 🎉 - Thoughts on Continuing -
This is an article written by the person behind Productivity Weekly about how it was to continue for a year. Well, actually, it's an article I wrote. It covers what was good and what was challenging about running a weekly technical series.
Writing it every week is tough, but there are definitely benefits. I hope it serves as a reference for anyone interested in doing something similar.
History 📚
AWS Through History and Timelines
This is an article by NRI Netcom about AWS seen through history and timelines. There are two articles: one covering all services and one specifically for S3. Investigating almost all announcement dates and GA (General Availability) must have been quite a feat. Highly recommended for those wondering what services AWS has recently or those who haven't kept up with S3 updates.
- List of all AWS services seen through history/timeline - Summary of announcement dates, General Availability (GA), and AWS service overviews - NRI Netcom Design and Tech Blog
- AWS services seen through history/timeline (Amazon S3 edition) - Transitions of functions, roles, and pricing that are more than just storage - NRI Netcom Design and Tech Blog
Looking back at CircleCI from the 2021 Changelog
An article by someone from CircleCI looking back at the 2021 Changelog. CircleCI had many changes in 2021, and quite a few people might not have kept up with them. Since this summarizes and explains the changes from this year, you can easily catch up. Recommended for such people.
- Looking back at CircleCI from the 2021 Changelog - Part 1 - Qiita
- Looking back at CircleCI from the 2021 Changelog - Part 2 - Qiita
- Looking back at CircleCI from the 2021 Changelog - Part 3 (Final) - Qiita
History of GitHub Actions
An article summarizing the changes in GitHub Actions based on its Changelog. I wrote this one. Actually, only about two and a half years have passed since the GA of Actions, but there have been an incredible number of changes. Recommended for those who haven't been able to keep track. It's summarized by quarter.
Tidbits 🍘
This is a section where I briefly mention topics from Productivity Weekly that I wanted to share but didn't have the energy to cover in detail, or that didn't have much to say about.
-
Codespaces can install dotfiles from any repo | GitHub Changelog
- GitHub Codespaces can now load dotfiles from a different repository.
- Codespaces automatically executes files like
setup.shin a<account-name>/dotfilesrepository, and this update means you can now specify which repository to use.
Afterword
Since the Advent Calendar started this month, there are so many interesting articles being published that it's hard to keep up. I never get bored. Thanks to that, today's topics became quite numerous. On a side note, I recently tried using a matching app. Even when I get a match, keeping the conversation going is so difficult. What on earth am I supposed to talk about?
At Cybozu's Productivity Improvement Team, we engage in activities to boost the development productivity of our internal engineers. If you're interested in our team, click the link below!
Bonus 🃏
This week's bonus.
Home Kubernetes feat. cybozu-go/neco-apps - Poyo Memo
A case study of the "Home Kubernetes" that everyone loves.
While we often see Home Kubernetes built with Raspberry Pis, this is a unique article that starts with hardware selection and builds a cluster using small Intel PCs. What's even more interesting is that they decided which components to deploy by referring to neco (cybozu-go/neco-apps) developed by Cybozu.
This article explains hardware selection, cluster bootstrapping, descriptions of each component, and the development environment (VMs).
You should also use this article as a reference to run Cybozu's neco (or at least a reproduction of it) in your Home Kubernetes cluster!
-
It seems to be a free signing service developed by the Linux Foundation. ↩︎
-
I'm surprised they weren't available for self-hosted runners until now! ↩︎
-
A story where source code and customer information were leaked via a compromised bash script because customer information had been included in the source code. https://about.mercari.com/press/news/articles/20210521_incident_report/ ↩︎
-
It feels like cases of countermeasures against supply chain attacks have been increasing since around this time. https://zenn.dev/korosuke613/articles/productivity-weekly-20210421#bash-uploader-security-update---codecov ↩︎
Discussion