Zenn
Closed42

Homebrew Formulaを作ってみる (licensed)

Homebrew
#
Formula
tasshitasshi

isn’t already in Homebrew (check brew search <formula>)

すでに存在しないか確認する。

$ brew search licensed
==> Formulae
licensor

==> Casks
licensed

Caskのlicensedは別物っぽかった。

https://formulae.brew.sh/cask/licensed

tasshitasshi

passes all brew audit --new-formula <formula> tests

brew auditコマンドを通すこと

これはFormula作ってから確認

tasshitasshi

Grab the URL

以下のコマンドを実行。
URLには最新バージョンのソースコードのZIPを指定

brew create https://github.com/github/licensed/archive/refs/tags/3.7.3.zip

以下の内容が自動生成される

/opt/homebrew/Library/Taps/homebrew/homebrew-core/Formula/licensed.rb
# Documentation: https://docs.brew.sh/Formula-Cookbook
#                https://rubydoc.brew.sh/Formula
# PLEASE REMOVE ALL GENERATED COMMENTS BEFORE SUBMITTING YOUR PULL REQUEST!
class Licensed < Formula
  desc "A Ruby gem to cache and verify the licenses of dependencies"
  homepage ""
  url "https://github.com/github/licensed/archive/refs/tags/3.7.3.zip"
  sha256 "609e7aa514f12769e5700b0cb104c92d24c91b5883590b76082f81f630b13c08"
  license "MIT"

  # depends_on "cmake" => :build

  def install
    # ENV.deparallelize  # if your formula fails when building in parallel
    # Remove unrecognized options if warned by configure
    # https://rubydoc.brew.sh/Formula.html#std_configure_args-instance_method
    system "./configure", *std_configure_args, "--disable-silent-rules"
    # system "cmake", "-S", ".", "-B", "build", *std_cmake_args
  end

  test do
    # `test do` will create, run in and delete a temporary directory.
    #
    # This test will fail and we won't accept that! For Homebrew/homebrew-core
    # this will need to be a test that verifies the functionality of the
    # software. Run the test with `brew test licensed`. Options passed
    # to `brew install` such as `--HEAD` also need to be provided to `brew test`.
    #
    # The installed folder is not in the path, so use the entire path to any
    # executables being tested: `system "#{bin}/program", "do", "something"`.
    system "false"
  end
end
tasshitasshi

Check the build system

インストールをインタラクティブに実施できるらしい。
しかし、指定されたコマンドではエラーが出た。

brew install --interactive licensed

Error: licensed: no bottle available!
You can try to install from source with:
  brew install --build-from-source licensed
Please note building from source is unsupported. You will encounter build
failures with some formulae. If you experience any issues please create pull
requests instead of asking for help on Homebrew's GitHub, Twitter or any other
official channels.

以下のコマンドで成功した

$ brew install --build-from-source --interactive licensed
Warning: Treating licensed as a formula. For the cask, use homebrew/cask/licensed
==> Downloading https://github.com/github/licensed/archive/refs/tags/3.7.3.zip
Already downloaded: /Users/tasshi/Library/Caches/Homebrew/downloads/e2c593df8f2595d3b8e0b4f067d453f3fab1c37afc5792e9ecb2ccf3e29bee2e--licensed-3.7.3.zip
==> Entering interactive mode...
Type `exit` to return and finalize the installation.
Install to this prefix: /opt/homebrew/Cellar/licensed/3.7.3
tasshitasshi

Check the build system

ビルドスクリプトを書く。

/opt/homebrew/Library/Taps/homebrew/homebrew-core/Formula/licensed.rb
  def install
    # ENV.deparallelize  # if your formula fails when building in parallel
    ENV["GEM_HOME"] = libexec
    system "gem", "build", "licensed.gemspec"
    system "gem", "install", "licensed-#{version}.gem"
    bin.install libexec/"bin/licensed"
    bin.env_script_all_files(libexec/"bin", GEM_HOME: ENV["GEM_HOME"])
  end

MEMO: LicenseFinderやlolcatなどRuby製のツールも大体同じようなことをしてる

tasshitasshi

Check for dependencies

依存関係を調べる。
licensedのREADME.mdに書いてある

brew install cmake pkg-config

Specifying other formulae as dependencies

formulaに依存関係を指定する

/opt/homebrew/Library/Taps/homebrew/homebrew-core/Formula/licensed.rb
  depends_on "cmake" => :build
  depends_on "pkg-config" => :build

Specifying conflicts with other formulae

コンフリクトは大丈夫そう

tasshitasshi

ビルドが上手くいかないのでインタラクティブモードで調べる。

ダウンロードしてるのがGitリポジトリじゃないとダメっぽい。

/private/tmp/licensed-20221004-62084-1whs028/licensed-3.7.3
❯ gem build licensed.gemspec
fatal: not a git repository (or any of the parent directories): .git
WARNING:  no files specified
WARNING:  open-ended dependency on thor (>= 0.19) is not recommended
  if thor is semantically versioned, use:
    add_runtime_dependency 'thor', '~> 0.19'
WARNING:  open-ended dependency on bundler (>= 1.10) is not recommended
  if bundler is semantically versioned, use:
    add_runtime_dependency 'bundler', '~> 1.10'
WARNING:  open-ended dependency on parallel (>= 0.18.0) is not recommended
  if parallel is semantically versioned, use:
    add_runtime_dependency 'parallel', '~> 0.18', '>= 0.18.0'
WARNING:  open-ended dependency on json (>= 2.6.2) is not recommended
  if json is semantically versioned, use:
    add_runtime_dependency 'json', '~> 2.6', '>= 2.6.2'
WARNING:  open-ended dependency on rake (>= 12.3.3, development) is not recommended
  if rake is semantically versioned, use:
    add_development_dependency 'rake', '~> 12.3', '>= 12.3.3'
WARNING:  See http://guides.rubygems.org/specification-reference/ for help
  Successfully built RubyGem
  Name: licensed
  Version: 3.7.3
  File: licensed-3.7.3.gem
tasshitasshi

URLをGitで置き換える

/opt/homebrew/Library/Taps/homebrew/homebrew-cask/Casks/licensed.rb
  url "https://github.com/github/licensed.git",
      tag:      "3.7.3",
      revision: "76727f75d486a24758890a030e540ebf87bba78b"
tasshitasshi

インストールできた

$ brew install --build-from-source licensed
Warning: Treating licensed as a formula. For the cask, use homebrew/cask/licensed
==> Cloning https://github.com/github/licensed.git
Updating /Users/tasshi/Library/Caches/Homebrew/licensed--git
==> Checking out tag 3.7.3
HEAD is now at 76727f7 Merge pull request #537 from github/release-3.7.3
HEAD is now at 76727f7 Merge pull request #537 from github/release-3.7.3
==> gem build licensed.gemspec
==> gem install licensed-3.7.3.gem
🍺  /opt/homebrew/Cellar/licensed/3.7.3: 5,902 files, 31.0MB, built in 44 seconds
==> Running `brew cleanup licensed`...
Disable this behaviour by setting HOMEBREW_NO_INSTALL_CLEANUP.
Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).
tasshitasshi

cmake, pkg-configは本当にbuild時だけ必要？

Licensed uses the libgit2 bindings for Ruby provided by rugged

ruggedのREADMEを読むと、ビルド時だけ必要そう。

You need to have CMake and pkg-config installed on your system to be able to build the included version of libgit2.

Please follow the above in case installation of the gem fails with ERROR: CMake is required to build Rugged..

tasshitasshi

Install the formula

改めてインストールできるか確認

brew reinstall --build-from-source --verbose --debug licensed
tasshitasshi

Add a test to the formula

テストを追加する。

ヘルプやバージョンを表示するのは悪いパターン。
できるだけ実際の機能をテストできるような試験を書くべき。

If the software cannot function without credentials or requires a virtual machine, docker instance, etc. to run, a test could be to try to connect with invalid credentials (or without credentials) and confirm that it fails as expected. This is preferred over mocking a dependency.

ソフトウェアが何にかしらの外的要件に依存している場合は、それをモックするよりもエラーを表示させるほうがよい。

licensedは対象のリポジトリがないと正常に動作させるの難しいからエラー出すようにしようかなぁ、、、？

tasshitasshi

結局一旦versionを表示する感じにした。

    assert_equal "#{version}", shell_output("#{bin}/licensed version").strip
tasshitasshi

大体できたのでauditチェックする

brew audit --new-formula licensed

落ちた

licensed:
  * 1: col 1: Please remove default template comments
  * 4: col 9: Description shouldn't start with an article.
  * 17: col 5: Please remove default template comments
  * 26: col 18: Prefer `to_s` over string interpolation.
  * Files were found with references to the Homebrew shims directory.
    The offending files are:
      libexec/extensions/universal-darwin-21/2.6.0/rugged-1.5.0.1/gem_make.out
      libexec/extensions/universal-darwin-21/2.6.0/rugged-1.5.0.1/mkmf.log
      libexec/gems/rugged-1.5.0.1/vendor/libgit2/build/CMakeFiles/CMakeOutput.log
      libexec/gems/rugged-1.5.0.1/vendor/libgit2/build/CMakeFiles/3.24.2/CMakeCCompiler.cmake
      libexec/gems/rugged-1.5.0.1/vendor/libgit2/build/CMakeFiles/CMakeError.log
      libexec/gems/rugged-1.5.0.1/vendor/libgit2/build/deps/http-parser/CMakeFiles/http-parser.dir/build.make
      libexec/gems/rugged-1.5.0.1/vendor/libgit2/build/deps/http-parser/CMakeFiles/http-parser.dir/flags.make
      libexec/gems/rugged-1.5.0.1/vendor/libgit2/build/deps/ntlmclient/CMakeFiles/ntlmclient.dir/build.make
      libexec/gems/rugged-1.5.0.1/vendor/libgit2/build/deps/ntlmclient/CMakeFiles/ntlmclient.dir/flags.make
      libexec/gems/rugged-1.5.0.1/vendor/libgit2/build/deps/pcre/CMakeFiles/pcre.dir/build.make
      libexec/gems/rugged-1.5.0.1/vendor/libgit2/build/deps/pcre/CMakeFiles/pcre.dir/flags.make
      libexec/gems/rugged-1.5.0.1/vendor/libgit2/build/CMakeCache.txt
      libexec/gems/rugged-1.5.0.1/vendor/libgit2/build/src/util/CMakeFiles/util.dir/build.make
      libexec/gems/rugged-1.5.0.1/vendor/libgit2/build/src/util/CMakeFiles/util.dir/flags.make
      libexec/gems/rugged-1.5.0.1/vendor/libgit2/build/src/libgit2/CMakeFiles/libgit2.dir/build.make
      libexec/gems/rugged-1.5.0.1/vendor/libgit2/build/src/libgit2/CMakeFiles/libgit2.dir/flags.make
      libexec/gems/rugged-1.5.0.1/vendor/libgit2/build/src/libgit2/CMakeFiles/libgit2package.dir/flags.make
Error: 5 problems in 1 formula detected
tasshitasshi
  • 1: col 1: Please remove default template comments

テンプレートのコメントは全部消さないとダメ。
リファレンスっぽいのは残しといてもいいと思うんだけどなぁ。

  • 4: col 9: Description shouldn't start with an article.

descriptionが冠詞で始まるのはダメらしい

  • 2: col 9: Description shouldn't start with the formula name.

書き換えたら今度はFormula nameで始めるなって怒られた

  • 26: col 18: Prefer to_s over string interpolation.

テストの変数展開が良くないらしい（Ruby書かないから全然分からなかった）

https://www.rubydoc.info/gems/rubocop/RuboCop/Cop/Style/RedundantInterpolation

tasshitasshi

エラーが1つ残ってる。
rugged, libgit2がHomebrew shims directoryを参照しているらしいが、
どうやったら解消されるんだ、、、？

$ brew audit --new-formula licensed
Warning: Treating licensed as a formula. For the cask, use homebrew/cask/licensed
licensed:
  * Files were found with references to the Homebrew shims directory.
    The offending files are:
      libexec/extensions/arm64-darwin-21/2.7.0/rugged-1.5.0.1/gem_make.out
      libexec/extensions/arm64-darwin-21/2.7.0/rugged-1.5.0.1/mkmf.log
      libexec/gems/rugged-1.5.0.1/vendor/libgit2/build/CMakeFiles/CMakeOutput.log
      libexec/gems/rugged-1.5.0.1/vendor/libgit2/build/CMakeFiles/3.24.2/CMakeCCompiler.cmake
      libexec/gems/rugged-1.5.0.1/vendor/libgit2/build/CMakeFiles/CMakeError.log
      libexec/gems/rugged-1.5.0.1/vendor/libgit2/build/deps/http-parser/CMakeFiles/http-parser.dir/build.make
      libexec/gems/rugged-1.5.0.1/vendor/libgit2/build/deps/http-parser/CMakeFiles/http-parser.dir/flags.make
      libexec/gems/rugged-1.5.0.1/vendor/libgit2/build/deps/ntlmclient/CMakeFiles/ntlmclient.dir/build.make
      libexec/gems/rugged-1.5.0.1/vendor/libgit2/build/deps/ntlmclient/CMakeFiles/ntlmclient.dir/flags.make
      libexec/gems/rugged-1.5.0.1/vendor/libgit2/build/deps/pcre/CMakeFiles/pcre.dir/build.make
      libexec/gems/rugged-1.5.0.1/vendor/libgit2/build/deps/pcre/CMakeFiles/pcre.dir/flags.make
      libexec/gems/rugged-1.5.0.1/vendor/libgit2/build/CMakeCache.txt
      libexec/gems/rugged-1.5.0.1/vendor/libgit2/build/src/util/CMakeFiles/util.dir/build.make
      libexec/gems/rugged-1.5.0.1/vendor/libgit2/build/src/util/CMakeFiles/util.dir/flags.make
      libexec/gems/rugged-1.5.0.1/vendor/libgit2/build/src/libgit2/CMakeFiles/libgit2.dir/build.make
      libexec/gems/rugged-1.5.0.1/vendor/libgit2/build/src/libgit2/CMakeFiles/libgit2.dir/flags.make
      libexec/gems/rugged-1.5.0.1/vendor/libgit2/build/src/libgit2/CMakeFiles/libgit2package.dir/flags.make
Error: 1 problem in 1 formula detected
tasshitasshi

shimsのパス(/opt/homebrew/Library/Homebrew/shims)を含むファイルがあるとエラーになるっぽい。
他のFormulaではinreplaceで対象の文字列を置き換えたり、ファイルごと削除したりしている。

inreplace: https://docs.brew.sh/Formula-Cookbook#inreplace

https://github.com/Homebrew/homebrew-core/blob/master/Formula/pari.rb#L44

https://github.com/Homebrew/homebrew-core/blob/master/Formula/hwloc.rb#L46

ただエラーが起きているファイルが見つからない。
インストール中だけ存在する？

tasshitasshi

エラーを起こしているファイルが見つからなくて困っていたが解消。
/usr/libexecではなく、Celler内のFormulaのディレクトリにあるlibexecだった。

/opt/homebrew/Cellar/licensed/3.7.3/libexec

対象のファイルはすべてログっぽいので置き換えでも削除でも良さそう

  • libexec/extensions/arm64-darwin-21/2.7.0/rugged-1.5.0.1/
    • gem_make.out
    • mkmf.log
    • どちらもログファイルだった
  • libexec/gems/rugged-1.5.0.1/vendor/libgit2/build/
    • この配下はすべてCMakeFilesというディレクトリ内のファイル
    • 中間生成物 or CMakeの設定ファイル系なのですべて対応できそう
tasshitasshi

--fixをつけるとrubocopが自動で修正できるエラーは修正してもらえる（コードスタイルなど）

 brew audit --new-formula --fix licensed
tasshitasshi

愚直にファイル指定してshimsの参照を消したらエラー消えた。
あとはこれをどう綺麗にするか。

/opt/homebrew/Library/Taps/homebrew/homebrew-core/Formula/licensed.rb
    # Avoid references to Homebrew shims directory
    references_to_shims = [
      libexec/"extensions/arm64-darwin-21/2.7.0/rugged-1.5.0.1/gem_make.out",
      libexec/"extensions/arm64-darwin-21/2.7.0/rugged-1.5.0.1/mkmf.log",
      libexec/"gems/rugged-1.5.0.1/vendor/libgit2/build/CMakeFiles/CMakeOutput.log",
      libexec/"gems/rugged-1.5.0.1/vendor/libgit2/build/CMakeFiles/3.24.2/CMakeCCompiler.cmake",
      libexec/"gems/rugged-1.5.0.1/vendor/libgit2/build/CMakeFiles/CMakeError.log",
      libexec/"gems/rugged-1.5.0.1/vendor/libgit2/build/deps/http-parser/CMakeFiles/http-parser.dir/build.make",
      libexec/"gems/rugged-1.5.0.1/vendor/libgit2/build/deps/http-parser/CMakeFiles/http-parser.dir/flags.make",
      libexec/"gems/rugged-1.5.0.1/vendor/libgit2/build/deps/ntlmclient/CMakeFiles/ntlmclient.dir/build.make",
      libexec/"gems/rugged-1.5.0.1/vendor/libgit2/build/deps/ntlmclient/CMakeFiles/ntlmclient.dir/flags.make",
      libexec/"gems/rugged-1.5.0.1/vendor/libgit2/build/deps/pcre/CMakeFiles/pcre.dir/build.make",
      libexec/"gems/rugged-1.5.0.1/vendor/libgit2/build/deps/pcre/CMakeFiles/pcre.dir/flags.make",
      libexec/"gems/rugged-1.5.0.1/vendor/libgit2/build/CMakeCache.txt",
      libexec/"gems/rugged-1.5.0.1/vendor/libgit2/build/src/util/CMakeFiles/util.dir/build.make",
      libexec/"gems/rugged-1.5.0.1/vendor/libgit2/build/src/util/CMakeFiles/util.dir/flags.make",
      libexec/"gems/rugged-1.5.0.1/vendor/libgit2/build/src/libgit2/CMakeFiles/libgit2.dir/build.make",
      libexec/"gems/rugged-1.5.0.1/vendor/libgit2/build/src/libgit2/CMakeFiles/libgit2.dir/flags.make",
      libexec/"gems/rugged-1.5.0.1/vendor/libgit2/build/src/libgit2/CMakeFiles/libgit2package.dir/flags.make",
    ]
    inreplace references_to_shims, "#{Superenv.shims_path}/", "<Homebrew shims directory>"
tasshitasshi

inreplaceはデフォルトだと置換が発生しなかった場合にエラーを投げる。

これを抑制するにはaudit_resultにfalseを指定する。

https://github.com/Homebrew/brew/blob/master/Library/Homebrew/utils/inreplace.rb#L48

https://github.com/Homebrew/brew/blob/9c03493774500cf16ced8938e1eb4eeae8216b20/Library/Homebrew/utils/string_inreplace_extension.rb#L47

Error: inreplace failed
/opt/homebrew/Cellar/licensed/3.7.3/libexec/gems/rugged-1.5.0.1/vendor/libgit2/build/CMakeFiles/cmake.check_cache:
  expected replacement of "/opt/homebrew/Library/Homebrew/shims/mac/super/" with "<Homebrew shims directory>"
tasshitasshi

置換できた。

  1. 対象パスをglobで指定、ファイルのみに絞り込み
  2. inreplaceでshimsへの参照を置き換える
/opt/homebrew/Library/Taps/homebrew/homebrew-core/Formula/licensed.rb
    # Avoid references to the Homebrew shims directory
    shims_references = Dir[
      libexec/"extensions/**/rugged-*/gem_make.out",
      libexec/"extensions/**/rugged-*/mkmf.log",
      libexec/"gems/rugged-*/vendor/libgit2/build/CMakeCache.txt",
      libexec/"gems/rugged-*/vendor/libgit2/build/**/CMakeFiles/**/*",
    ].select { |f| File.file? f }
    inreplace shims_references, Superenv.shims_path.to_s, "<**Reference to the Homebrew shims directory**>", false

置き換え済みの箇所に<**Reference to the Homebrew shims directory**>と表示されている。

/opt/homebrew/Cellar/licensed/3.7.3/libexec/extensions/arm64-darwin-21/2.7.0/rugged-1.5.0.1/gem_make.out
...
-- Detecting C compiler ABI info - done
-- Check for working C compiler: <**Reference to the Homebrew shims directory**>/clang - skipped
-- Detecting C compile features
...
tasshitasshi

Bottleを作ろうと思ったけどhomebrew/coreにあるFormulaは勝手にやってくれるらしい。

Bottles for homebrew/core formulae are created by Brew Test Bot when a pull request is submitted.

https://docs.brew.sh/Bottles#creation

bottleとはそのまま「ボトル」で，ビルド済みのバイナリを表します． keg（後述）と同じファイル構成をtar.gzにしたもので，環境に合うbottleがformulaに含まれていれば，ビルドせずにインストール可能です．

https://blog.ottijp.com/2020/05/23/homebrew/

tasshitasshi

MEMO: 今後Formulaを更新したい場合はbrew bump-formula-prコマンドを使うと良さそう。

Create a pull request to update <formula> with a new URL or a new tag.

おそらくversionだけ指定したらよしなにしてくれそう

If a <version> is specified, a best effort to determine the <URL> and <SHA-256> or the <tag> and <revision> will be made if both values are not supplied by the user.

https://github.com/Homebrew/brew/blob/master/Library/Homebrew/dev-cmd/bump-formula-pr.rb

tasshitasshi

レビュー対応。

  • ruby@3に変更
  • テストをちゃんとしたのを追加（やっぱりlicensed versionではダメだった）
tasshitasshi

再レビューが来たので対応してた。
vendorできない？ということだったので色々試してたけど結局うまく行かなくて質問した。
https://github.com/Homebrew/homebrew-core/pull/112414#discussion_r1007251868

やろうとしたこと

ビルド中のディレクトリにvendorディレクトリを作成して、依存関係をそこに入れる

調べたこと

bundler

https://bundler.io/man/bundle-install.1.html

--pathオプションでインストール先を指定できるらしい。
実際に実行すると、bundle configで指定しろって言われた。

bundle config path vendor/bundle

また、nokogiri周りでエラーが出たが、以下のコマンドで回避できた

bundle config set force_ruby_platform true

https://koic.hatenablog.com/entry/nokogiri-1-11-0-will-be-precompiled
https://abillyz.com/watanabe/studies/244

gem

--ignore-dependenciesでいけそうだけど分からなかった

https://guides.rubygems.org/command-reference/#gem-install

このスクラップは2023/04/24にクローズされました