Closed8

パッケージマネージャ

Kazuhiro MimakiKazuhiro Mimaki

やること

  • package.json から dependencies / devDependencies を読み取る
  • production オプションを指定すると devDependencies は無視される
  • パッケージを指定してインストールコマンドを実行するとそのパッケージをpackage.jsonに追加する
  • lockファイルを作成する
  • セマンティックバージョンのインストール
  • 依存パッケージの再帰的な解決
  • consoleにログを出力
Kazuhiro MimakiKazuhiro Mimaki

npmのmanifestをダウンロードして解凍するところで詰まった。
npmのtarballを解凍すると、root階層が package ディレクトリになっているっぽい。
が、/node_modules に展開する時 package は含めず、その配下にあるモジュールを配置する。
この対応は strip_prefix というメソッドを使うことで解決できた。
https://github.com/rust-lang-nursery/rust-cookbook/blob/master/src/compression/tar/tar-strip-prefix.md

もう1つ色々試行錯誤したのがダウンロードしたフォルダの解凍。
archive の unpack をする時、対象のディレクトリが存在しないと解凍ファイルを展開してくれない。
そのため、先に対象となるディレクトリを作成しておく必要がある。
当たり前だけどnpmのパッケージはそれぞれフォルダ構成が異なるので、階層が深い場合はその深さまでディレクトリを作成する必要がある。
と、いうので結構汚くなってしまった。。。

async fn install_npm_manifest(
    package_name: &str,
    version: &Version,
) -> Result<(), Box<dyn std::error::Error>> {
    let client = reqwest::Client::new();
    const REGISTRY: &str = "https://registry.npmjs.org/";
    let url = Url::parse(&format!("{}/{}", REGISTRY, package_name))?;
    let manifest: NpmManifest = client.get(url).send().await?.json().await?;

    let tarball_url = &manifest.versions[version].dist.tarball;
    let res = client.get(tarball_url).send().await?.bytes().await?;
    let mut archive = Archive::new(GzDecoder::new(res.as_ref()));
    const PREFIX: &str = "package";

    fs::create_dir_all(format!("./node_modules/{}", package_name))?;

    archive
        .entries()?
        .filter_map(|e| e.ok())
        .map(|mut entry| -> Result<PathBuf, Box<dyn std::error::Error>> {
            let path_buf = entry.path()?.strip_prefix(&PREFIX)?.to_owned();

            // 先に解凍したファイルを展開するためのディレクトリを作っておく必要がある
            let binding = path_buf.display().to_string();
            let mut splitted_paths = binding.split("/").collect::<Vec<&str>>();
            if splitted_paths.len() > 1 {
                splitted_paths.pop();
                let path = format!(
                    "./node_modules/{}/{}",
                    package_name,
                    splitted_paths.join("/")
                );
                fs::create_dir(&path)?;
            }
            fs::create_dir_all(format!(
                "./node_modules/{}/{}",
                package_name,
                path_buf.display()
            ))?;
            // ここまで

            entry.unpack(format!(
                "./node_modules/{}/{}",
                package_name,
                path_buf.display()
            ))?;
            Ok(path_buf)
        })
        .filter_map(|e| e.ok())
        .for_each(|x| println!("> {}", x.display()));

    Ok(())
}

PathBuf struct に生えている parent というメソッドを使えば、リーフノードから見て親に当たるディレクトリを全て取得できることが分かったので、スッキリした。

let prefix = path_buf.parent().unwrap();
fs::create_dir_all(format!(
    "./node_modules/{}/{}",
    package_name,
    prefix.display()
))?;
Kazuhiro MimakiKazuhiro Mimaki

とりあえずトップレベルのパッケージを1つだけインストールすることはできるようになった。
依存先のパッケージとかは無理。少しコード整理。

Kazuhiro MimakiKazuhiro Mimaki

semantic versionからインストール対象のバージョンをどうやって決定するか。
nodejs_semver というクレートがあったので利用する。これ自作するのは結構めんどくさいからありがたい...！

pub type Version = nodejs_semver::Version;
pub type Range = nodejs_semver::Range;
pub type Constraint = String;

pub fn max_satisfying(versions: &Vec<Version>, constraint: Constraint) -> &Version {
    let range: Range = constraint.parse().unwrap();
    range.max_satisfying(versions).unwrap()
}
Kazuhiro MimakiKazuhiro Mimaki

依存パッケージを再帰的に取れるようにした。

pub async fn resolve(root_manifest: PackageJson) -> Result<Info, Box<dyn std::error::Error>> {
    let mut top_level: TopLevel = HashMap::new();
    let mut npm_manifest_cache: HashMap<String, Manifest> = HashMap::new();

    if !root_manifest.dependencies.is_empty() {
        for (package_name, version) in root_manifest.dependencies {
            resolve_package_recursively(
                package_name,
                version,
                &mut top_level,
                &mut npm_manifest_cache,
            )
            .await?;
        }
    }

    // TODO: dev_dependencies

    return Ok(Info { top_level });
}

#[async_recursion]
async fn resolve_package_recursively(
    name: String,
    constraint: String,
    top_level: &mut TopLevel,
    npm_manifest_cache: &mut HashMap<String, Manifest>,
) -> Result<(), Box<dyn std::error::Error>> {
    let manifest = installer::fetch_manifest(name.clone(), npm_manifest_cache).await?;
    let version = semver_max_satisfying(manifest.clone(), constraint.clone()).await?;
    println!(
        "[Root Resolve by manifest] {}@{} to {}",
        &name, constraint, &version
    );
    let matched_manifest = manifest.get(&version.to_string()).unwrap().clone();

    if top_level.get(&name).is_none() {
        top_level.insert(
            name,
            TopLevelDetail {
                url: matched_manifest.dist.tarball,
                version: version.to_string(),
            },
        );
    }

    if let Some(dependencies) = matched_manifest.dependencies {
        for (name, constraint) in dependencies {
            resolve_package_recursively(name, constraint, top_level, npm_manifest_cache).await?;
        }
    }

    Ok(())
}

async fn semver_max_satisfying(
    manifest: Manifest,
    constraint: String,
) -> Result<npm_semver::Version, Box<dyn std::error::Error>> {
    let versions: Vec<nodejs_semver::Version> = manifest
        .iter()
        .map(|(version, _)| version.parse().unwrap())
        .collect();
    let max_satisfying = npm_semver::max_satisfying(versions, constraint);
    Ok(max_satisfying)
}

async function を再帰で使うとエラーで怒られる。async_recursion クレートを使うことでこれを解消できるっぽい、この辺は後でちゃんと読みたいな。

https://maguro.dev/async-recursion/

viteインストールして起動したら動いた...！

node_modules/vite/bin/vite.js
Kazuhiro MimakiKazuhiro Mimaki

キャッシュを取る。

Fetch manifest https://registry.npmjs.org/vue
[Root Resolve by manifest] vue@^3.2.37 to 3.5.13
Fetch manifest https://registry.npmjs.org/@vue/shared
[Resolve by manifest] @vue/shared@3.5.13 to 3.5.13
Fetch manifest https://registry.npmjs.org/@vue/compiler-dom
[Resolve by manifest] @vue/compiler-dom@3.5.13 to 3.5.13
Fetch manifest https://registry.npmjs.org/@vue/server-renderer
[Resolve by manifest] @vue/server-renderer@3.5.13 to 3.5.13
Fetch manifest https://registry.npmjs.org/@vue/runtime-dom
[Resolve by manifest] @vue/runtime-dom@3.5.13 to 3.5.13
Fetch manifest https://registry.npmjs.org/@vue/compiler-sfc
[Resolve by manifest] @vue/compiler-sfc@3.5.13 to 3.5.13
Fetch manifest https://registry.npmjs.org/postcss
[Resolve by manifest] postcss@^8.4.48 to 8.4.49
Fetch manifest https://registry.npmjs.org/@vue/compiler-core
[Resolve by manifest] @vue/compiler-core@3.5.13 to 3.5.13
[Resolve by manifest] @vue/compiler-dom@3.5.13 to 3.5.13
Fetch manifest https://registry.npmjs.org/estree-walker
[Resolve by manifest] estree-walker@^2.0.2 to 2.0.2
Fetch manifest https://registry.npmjs.org/magic-string
[Resolve by manifest] magic-string@^0.30.11 to 0.30.14
Fetch manifest https://registry.npmjs.org/@babel/parser
[Resolve by manifest] @babel/parser@^7.25.3 to 7.26.2
[Resolve by manifest] @vue/shared@3.5.13 to 3.5.13
Fetch manifest https://registry.npmjs.org/@vue/compiler-ssr
[Resolve by manifest] @vue/compiler-ssr@3.5.13 to 3.5.13
Fetch manifest https://registry.npmjs.org/source-map-js
[Resolve by manifest] source-map-js@^1.2.0 to 1.2.1
[Resolve by manifest] @vue/compiler-dom@3.5.13 to 3.5.13
[Resolve by manifest] @vue/shared@3.5.13 to 3.5.13
[Resolve by manifest] @vue/shared@3.5.13 to 3.5.13
[Resolve by manifest] @vue/compiler-core@3.5.13 to 3.5.13
[Resolve by manifest] @babel/parser@^7.25.3 to 7.26.2
[Resolve by manifest] source-map-js@^1.2.0 to 1.2.1
[Resolve by manifest] estree-walker@^2.0.2 to 2.0.2
Fetch manifest https://registry.npmjs.org/entities
[Resolve by manifest] entities@^4.5.0 to 4.5.0
[Resolve by manifest] @vue/shared@3.5.13 to 3.5.13
Fetch manifest https://registry.npmjs.org/@babel/types
[Resolve by manifest] @babel/types@^7.26.0 to 7.26.0
Fetch manifest https://registry.npmjs.org/@babel/helper-string-parser
[Resolve by manifest] @babel/helper-string-parser@^7.25.9 to 7.25.9
Fetch manifest https://registry.npmjs.org/@babel/helper-validator-identifier
[Resolve by manifest] @babel/helper-validator-identifier@^7.25.9 to 7.25.9
[Resolve by manifest] @babel/types@^7.26.0 to 7.26.0
[Resolve by manifest] @babel/helper-string-parser@^7.25.9 to 7.25.9
[Resolve by manifest] @babel/helper-validator-identifier@^7.25.9 to 7.25.9
Fetch manifest https://registry.npmjs.org/@jridgewell/sourcemap-codec
[Resolve by manifest] @jridgewell/sourcemap-codec@^1.5.0 to 1.5.0
[Resolve by manifest] @vue/shared@3.5.13 to 3.5.13
[Resolve by manifest] @vue/compiler-core@3.5.13 to 3.5.13
[Resolve by manifest] @babel/parser@^7.25.3 to 7.26.2
[Resolve by manifest] source-map-js@^1.2.0 to 1.2.1
[Resolve by manifest] estree-walker@^2.0.2 to 2.0.2
[Resolve by manifest] entities@^4.5.0 to 4.5.0
[Resolve by manifest] @vue/shared@3.5.13 to 3.5.13
[Resolve by manifest] @babel/types@^7.26.0 to 7.26.0
[Resolve by manifest] @babel/helper-string-parser@^7.25.9 to 7.25.9
[Resolve by manifest] @babel/helper-validator-identifier@^7.25.9 to 7.25.9
[Resolve by manifest] @babel/parser@^7.25.3 to 7.26.2
[Resolve by manifest] source-map-js@^1.2.0 to 1.2.1
[Resolve by manifest] estree-walker@^2.0.2 to 2.0.2
[Resolve by manifest] entities@^4.5.0 to 4.5.0
[Resolve by manifest] @vue/shared@3.5.13 to 3.5.13
[Resolve by manifest] @babel/types@^7.26.0 to 7.26.0
[Resolve by manifest] @babel/helper-string-parser@^7.25.9 to 7.25.9
[Resolve by manifest] @babel/helper-validator-identifier@^7.25.9 to 7.25.9
Fetch manifest https://registry.npmjs.org/picocolors
[Resolve by manifest] picocolors@^1.1.1 to 1.1.1
[Resolve by manifest] source-map-js@^1.2.1 to 1.2.1
Fetch manifest https://registry.npmjs.org/nanoid
[Resolve by manifest] nanoid@^3.3.7 to 3.3.8
Fetch manifest https://registry.npmjs.org/csstype
[Resolve by manifest] csstype@^3.1.3 to 3.1.3
[Resolve by manifest] @vue/shared@3.5.13 to 3.5.13
Fetch manifest https://registry.npmjs.org/@vue/reactivity
[Resolve by manifest] @vue/reactivity@3.5.13 to 3.5.13
Fetch manifest https://registry.npmjs.org/@vue/runtime-core
[Resolve by manifest] @vue/runtime-core@3.5.13 to 3.5.13
[Resolve by manifest] @vue/reactivity@3.5.13 to 3.5.13
[Resolve by manifest] @vue/shared@3.5.13 to 3.5.13
[Resolve by manifest] @vue/shared@3.5.13 to 3.5.13
[Resolve by manifest] @vue/shared@3.5.13 to 3.5.13
[Resolve by manifest] @vue/compiler-ssr@3.5.13 to 3.5.13
[Resolve by manifest] @vue/shared@3.5.13 to 3.5.13
[Resolve by manifest] @vue/compiler-dom@3.5.13 to 3.5.13
[Resolve by manifest] @vue/shared@3.5.13 to 3.5.13
[Resolve by manifest] @vue/shared@3.5.13 to 3.5.13
[Resolve by manifest] @vue/compiler-core@3.5.13 to 3.5.13
[Resolve by manifest] @babel/parser@^7.25.3 to 7.26.2
[Resolve by manifest] source-map-js@^1.2.0 to 1.2.1
[Resolve by manifest] estree-walker@^2.0.2 to 2.0.2
[Resolve by manifest] entities@^4.5.0 to 4.5.0
[Resolve by manifest] @vue/shared@3.5.13 to 3.5.13
[Resolve by manifest] @babel/types@^7.26.0 to 7.26.0
[Resolve by manifest] @babel/helper-string-parser@^7.25.9 to 7.25.9
[Resolve by manifest] @babel/helper-validator-identifier@^7.25.9 to 7.25.9
[Resolve by manifest] @vue/shared@3.5.13 to 3.5.13
[Resolve by manifest] @vue/compiler-core@3.5.13 to 3.5.13
[Resolve by manifest] @babel/parser@^7.25.3 to 7.26.2
[Resolve by manifest] source-map-js@^1.2.0 to 1.2.1
[Resolve by manifest] estree-walker@^2.0.2 to 2.0.2
[Resolve by manifest] entities@^4.5.0 to 4.5.0
[Resolve by manifest] @vue/shared@3.5.13 to 3.5.13
[Resolve by manifest] @babel/types@^7.26.0 to 7.26.0
[Resolve by manifest] @babel/helper-string-parser@^7.25.9 to 7.25.9
[Resolve by manifest] @babel/helper-validator-identifier@^7.25.9 to 7.25.9

キャッシュできてそうやけど、コードが汚くなってきた。。。

Kazuhiro MimakiKazuhiro Mimaki

async function 直列で実行してるので遅いんだけど、ここの並列化はかなり重そうなので今回は端折りたい。
別の機会に勉強する。

このスクラップは2024/12/04にクローズされました