iTranslated by AI
Productivity Weekly (2021-11-17 Issue)
Hello. I'm Hirakiba from the Productivity Improvement Team at Cybozu, Inc.
Our Productivity Improvement Team holds an internal session every Wednesday called "Productivity Weekly," where we share topics related to improving developer productivity discovered during the week.
This article summarizes the topics from that session.
This is the 51st issue. Past articles can be found here.
news 📺
Datadog Delivers Industry-First Observability for Software Build and Testing Pipelines | Datadog
Datadog's CI Visibility feature was officially released at the end of October.
I briefly introduced Datadog's CI Visibility feature before. At that time, it was still in beta, and the pricing was not yet known.
Datadog states that CI Visibility helps maintain efficient and reliable pipelines by providing insights to identify why problems occurred in the CI pipeline and how to resolve them.
CI Visibility includes the following features:
- Pipeline Visibility: Visualization of pipelines
- Testing Visibility: Visualization of tests
- Advanced performance analysis
- Identify performance degradation and display the commit that introduced faulty tests
- Correlate test results with logs and network performance data
- etc.
- Support for various CI providers and languages
- CircleCI, GitHub Actions, Jenkins, ...
- Java, JS, Python, Swift, ...
With the official release, the usage pricing has been clarified. It appears to be per-user billing. (I did find it to be a bit pricey.)
- Pipeline Visibility: from $20/user
- Testing Visibility: from $20/user
- Pipeline + Testing Visibility: from $30/user
Since it was in beta until now, it might have been difficult to use for business tasks, but now we can actively utilize it at work. By using rich analysis features, we should be able to improve test and workflow performance.
GitHub Actions: Input types for manual workflows | GitHub Changelog
The variety of input parameters that can be specified when manually running a workflow in GitHub Actions (workflow_dispatch) has increased. Previously, only string (pure text) was available, but now you can also choose choice, boolean, and environment as well.
choice provides a select box, allowing users to choose from a fixed set of options. boolean provides a checkbox, which returns true when checked and false when not. environment makes it easy to select an environment from pre-configured GitHub Actions Environments and switch between Secrets. If you want to accept free-form input from users or inputs that cannot be prepared in advance, you can continue to use string as before.
The input type is specified with on.workflow_dispatch.inputs.<inputs_name>.type. If type is missing, it defaults to string (likely to maintain compatibility).
Until now, these options or true/false values were all accepted as free-form input (string), which was tedious for users and required validation checks within the workflow. This update makes workflow_dispatch much more convenient.
Also, an article verifying this update has been posted on Zenn, which you might find helpful.
Require pull requests without requiring reviews | GitHub Changelog
In GitHub's branch protection, it is now possible to enforce pull requests without requiring a review.
Previously, enforcing a pull request (enabling Require a pull request before merging) also forced Require approvals (requiring one or more reviews), so it couldn't accommodate use cases where you wanted to enforce pull requests but didn't necessarily need a review.
With this change, you can now enforce pull requests without enforcing reviews, making it possible to enforce pull requests even in repositories developed by mobs or single individuals. This is a subtle but very welcome change. It should make personal development much smoother.
The GitHub Enterprise Server 3.3 Release Candidate is available | GitHub Changelog
GitHub Enterprise Server 3.3 has been released as a Release Candidate (RC).
In 3.3, the following features are primarily coming to GitHub Enterprise Server:
- High contrast dark theme
- [GitHub Actions] ephemeral self-hosted runners
- [GitHub Actions]
workflow_jobwebhook - [GitHub Teams] security manager role
It's great that ephemeral self-hosted runners (disposable runners) will be available in GHES. I'm looking forward to the official release.
know-how 🎓
RIP Copy and Paste from Stackoverflow! 🚨 (+Trojan Source Solution) - DEV Community 👩💻👨💻
This article explains why you should stop copy-pasting (RIP) code found on the internet, as it may contain invisible malicious code that cleverly utilizes character encodings.
For example, the following code in the article doesn't look like it enters the if statement, so Console.WriteLine("You are an admin."); should not be executed.
Code from the article (intentionally pasted as an image)
However, when actually executed, Console.WriteLine("You are an admin."); is executed.
This is because invisible special characters are hidden in if (access_level != "user") //Check if admin.
Pasting the above code into VS Code. In my environment, it highlighted the invisible characters.
Looking at an editor that displays special characters, we can see that a Unicode character U+2066[1] is hidden.
By using such special characters, malicious code can be embedded into source code. The author states that editors should warn about or highlight these hidden characters.
The article introduces this method, the underlying mechanism, and how to find them (specifically for C#). It also mentions that GitHub has a feature to warn about such hidden characters.
It's quite scary that code you thought was safe after reading it could actually be something entirely different once copy-pasted. We should be careful.
GitHub's commitment to npm ecosystem security | The GitHub Blog
An article about GitHub's efforts to improve security in the npm ecosystem.
GitHub is focusing on ensuring the security of the npm registry to counter account takeovers and malicious packages. Specifically, they are carrying out the following activities:
- Improving automated malware detection
- In recent cases, they have been able to detect and respond to takeovers of popular packages within about 10 minutes.
- Enforcing 2FA (two-factor authentication) for maintainers and administrators of popular packages.
- Planning WebAuthn support.
- Addressing other recent security issues.
The article provides specific details for each of these activities.
Regarding 2FA, they have recently been sending emails to general users asking them to enable it. I received an email as well, so I have enabled 2FA.
Since we often hear about various supply chain attack examples lately, it's reassuring that the registry side is doing a lot to ensure security. Let's all cooperate by strengthening our authentication.
Make your monorepo feel small with Git’s sparse index | The GitHub Blog
An explanation of how Git's sparse index can now suppress the size of the index during sparse checkouts.
git sparse-checkout[2] is a mechanism for checking out only a portion of the files within a Git repository. By checking out only a subset, you can reduce the size of the repository. This feature is useful in large-scale monorepos.
While git sparse-checkout allows for partial checkouts of blobs, the Git index still holds information for all files, which leaves the index size large. This has been an ongoing issue.
To address this, the sparse-index mechanism was added to keep the index to a minimum when performing a sparse checkout (it seems sparse-index itself was added around June 2021). Using sparse-index has reportedly improved performance for commands like git status and git commit in large-scale monorepos.
The article covers the Git index, the mechanism of sparse-index, and compatibility with other subcommands.
While many major subcommands are already supported, support for git diff, git blame, and others is planned for the future.
Monorepos offer many benefits, but repository bloat is an inevitable issue as they grow. It's a good idea to use these features effectively to create an environment that is easy to develop in.
tool 🔨
command-not-found.com
This website allows you to look up how to install specific packages across various Linux distributions (+ Mac).
It provides installation commands for the appropriate package manager for each distribution (such as brew, apt-get, yum, or pacman). Even if the command name and package name are different, it provides the correct package name. It also displays the package's homepage.
When you're working with an OS you don't typically use, you often find yourself wondering how to install a particular package, so being able to look it up easily is very helpful. However, since there are commands and distributions that are not supported, and official installation procedures should ideally be checked on the homepage, it's best used when you just want to install a package quickly.
Okteto - The Kubernetes development platform | Okteto
Okteto is a service for rapidly developing and previewing cloud-native applications (Kubernetes and docker-compose).
It provides a Kubernetes cluster for application development. Additionally, it features tools to streamline development, such as automatically building previews of your app from pull requests and allowing you to easily enter containers from your local machine.
What's particularly interesting is the provision of a Kubernetes cluster. Surprisingly, you can use up to 10 pods for free (with some other limitations).
Setting up a cluster for learning Kubernetes is often quite tedious (the local machine becomes sluggish, and setting up a cluster for verification on GKE or similar costs quite a bit), so being able to skip that and use a cluster is very welcome. Of course, you can't use it to its full extent, but it should be sufficient for things like studying various resources.
It also seems useful for developing your own applications. I'd like to give it a try.
Lens | The Kubernetes IDE
This is an IDE for Kubernetes that supports macOS, Linux, and Windows. It allows you to manage multiple contexts together and check or operate on the status of resources and nodes.
While it might be better to learn and use kubectl commands regularly, being able to easily view statistical information like CPU usage in graphs is a definite plus.
Small Topics 🍘
This is a section for topics that came up in Productivity Weekly that I wanted to introduce but didn't have the stamina for, or topics where I didn't have much to say beyond a quick sentence.
-
"Twitter API v2" becomes the official primary Twitter API ~ Free access levels added without requiring application - Mado-no-Mori
- It seems Twitter API v2 has finally become the official primary Twitter API.
- Additionally, for those using a limited plan called "Essential access," applications are no longer required, making it easier to access the API.
- Going forward, the existing v1.1 will only receive security updates.
- Side note: Previously, you couldn't send Tweets with v2, but an endpoint for tweeting was added recently.
Afterword
The end of the year is approaching. I can't forget about Furusato Nozei (hometown tax donation). I donated to Fukui Prefecture and got some crab. This year went by so fast...
The Productivity Improvement Team holds sessions to share these kinds of topics every week. If you're interested in the Productivity Improvement Team, click the link below!
Bonus
This week's bonus.
Is this now an essential technology for researchers? Translation service DeepL can now "translate PDF files as they are" - Togetter
DeepL, everyone's favorite service known for its natural translations, has finally added support for PDF translation.
You might wonder if there are many opportunities to translate PDFs, but researchers, including students, read an enormous number of English papers. I can't survive without reading them. Of course, I used to read English papers when I was a student, but since I was bad at English, it was always a struggle.
While web pages are fine, PDFs often have line breaks following the layout, which makes copy-pasting a pain. The process of copying, removing line breaks, pasting into a translation app, understanding the result, and copying again... is quite a lot of work.
DeepL's new feature makes translating these PDFs easy. It seems it can even do it without breaking the layout. It's amazing that such a thing is possible with PDFs.
Of course, DeepL's translation isn't perfect, so you shouldn't take the results at face value, and I can almost hear voices saying "don't run away from English," but it's good to use it effectively while understanding its limitations.
By the way, the Free plan only allows translating 3 files per month, so a paid plan seems essential. Since paid plans also ensure the confidentiality of input data, I recommend subscribing to a paid plan for serious use anyway.
I wish I had this when I was a student. It's great for people who aren't good at English to be able to skim through papers easily.
-
Upon investigation, this seems to be a bidirectional character literal used for languages like Arabic where text flows from right to left. To be honest, I don't quite understand it well. ↩︎
-
For details on sparse-checkout, see the article here: https://github.blog/2020-01-17-bring-your-monorepo-down-to-size-with-sparse-checkout/ ↩︎
Discussion