やること

優先度: 高(必須)

type チェック
依存関係グラフ作る
一つにバンドル

優先度:中

幅優先探索を応用した依存関係グラフの走査を利用してなんかしたい
巡回の回避
pathの省略に対応
node_modulesのインポート
CLIツール追加

優先度:小

htmlテンプレートの追加
dev-server追加
グラフ理論をもうちょいつめたい
unitテストかきつつ

togami 2021/01/05に更新

build.ts

import fs from "fs";
import path from "path";
import { absolutePath, relativePath } from "./types/common";

type IO = {
  entryFile: absolutePath;
  outputDir: absolutePath;
};

type Graph = {
  id: number;
  filePath: relativePath;
  dependency: relativePath[];
  code: string;
}[];

type Bundle = {
  name: string;
  code: string;
}[];

const build = ({ entryFile, outputDir }: IO) => {
  const graph: Graph = createDependencyGraph(entryFile);
  const outputFiles: Bundle = createBundle(graph);
  for (const outputFile of outputFiles) {
    fs.writeFileSync(
      path.join(outputDir, outputFile.name),
      outputFile.code,
      "utf-8"
    );
  }
};

ざっくり外観を作成。
コード中でrelative pathかabsolute pathかで迷う部分があるので前もってrelativePath とabsolutePathという名前のstringの型を作っておいた。
型定義は後々修正がはいるかも

Graphを構築するcreateDependencyGraph関数
Graphを元にbundleを作るcreateBundle関数
上記２点の作成に移る。

togami 2021/01/05に更新

typeChckerを実装

typeChecker.ts

import * as ts from "typescript";
import { absolutePath } from "../types/common";

const TYPE_ERROR = 1;

export const typeChecker = (entryFile: absolutePath): void => {
  let diagnostics = ts.getPreEmitDiagnostics(
    ts.createProgram([entryFile], {
      strict: true,
      target: ts.ScriptTarget.Latest,
      moduleResolution: ts.ModuleResolutionKind.NodeJs,
    })
  );
  if (diagnostics.length) {
    diagnostics.forEach((d) => console.log(d.messageText));
    console.error("Error Occurred");
    process.exit(TYPE_ERROR);
  } else {
    console.log("There are no Type Error");
  }
};

togami 2021/01/09に更新

typeCheckerの単体テスト。
process.exit(1)のmockにjest-mock-processというパッケージを用いた。

typeChecker.unittest.js

import { typeChecker } from "../lib/typeChecker/typeChecker";
import { mockProcessExit } from "jest-mock-process";

describe("typeChecker", () => {
  describe("No typeError", () => {
    it("should be compile successfully", () => {
      const consoleMock = jest.spyOn(console, "log");
      const result = typeChecker("test/example/success.ts");
      expect(consoleMock).toHaveBeenCalledWith("There are no Type Error");
    });
  });
  describe("typeError", () => {
    it("should be occur error and get an error message", () => {
      const consoleMock = jest.spyOn(console, "error");
      const exitMock = mockProcessExit();
      typeChecker("test/example/error.ts");
      expect(consoleMock).toHaveBeenCalledWith("Error Occurred");
      expect(exitMock).toHaveBeenCalledWith(1);
    });
  });
});

togami 2021/01/06に更新

moduleを定義および、graphの構築。
ファイルの分割が悩みどころ

graph.ts

import { readFileSync } from "fs";
import path = require("path");
import { parse } from "@babel/parser";
import traverse from "@babel/traverse";
import { transformSync } from "@babel/core";
import { absolutePath, relativePath, Graph } from "../types/common";
import { typeChecker } from "../typeChecker/typeChecker";

let ID = 0;

export const createDependencyGraph = (entryFile: absolutePath) => {
  typeChecker(entryFile);
  const rootModule = createModule(entryFile);
  const queue = [rootModule];
  for (const asset of queue) {
    const dirName = path.dirname(asset.filePath);
    asset.dependencies.forEach((relativePath) => {
      const absolutePath = path.join(dirName, relativePath);
      typeChecker(absolutePath);
      const childModule = createModule(absolutePath);
      asset.mapping[relativePath] = childModule.id;
      queue.push(childModule);
    });
  }
  return queue;
};

export const createModule = (filePath: absolutePath) => {
  const content = readFileSync(filePath, "utf-8");
  return new Module(filePath, content);
};

export class Module {
  filePath: absolutePath;
  id: number;
  dependencies: relativePath[];
  code: string;
  mapping: object;
  constructor(filePath: absolutePath, content) {
    this.filePath = filePath;
    this.id = ID++;
    this.dependencies = this.findDependency(content);
    this.code = this.transpileCode(content);
    this.mapping = {};
  }
  findDependency(content) {
    const dependencies: relativePath[] = [];
    const ast = parse(content, {
      sourceType: "module",
    });
    traverse(ast, {
      ImportDeclaration: ({ node }) => {
        dependencies.push(node.source.value);
      },
    });
    return dependencies;
  }
  transpileCode(content) {
    const { code } = transformSync(content, {
      filename: this.filePath,
      presets: ["@babel/preset-env", "@babel/preset-typescript"],
    });
    return code;
  }
}

togami 2021/01/06

class Moduleの単体テスト。
まだ途中

module.test.ts

import { Module } from "../graph";
import { readFileSync } from "fs";

describe("Module Class", () => {
  const content = readFileSync("example/index.ts", "utf-8");
  const filePath = "example/__test__/index.ts";
  let module;
  beforeEach(() => {
    module = new Module(filePath, content);
  });
  it("test", () => {
    expect(module.dependencies).toHaveLength(1);
  });
});

togami 2021/01/09に更新

ファイルの構成変更　& 色々改名

.
├── README.md
├── example
│   ├── __test__
│   ├── add.ts
│   └── index.ts
├── jest.config.js
├── lib
│   ├── asset
│   ├── build.ts
│   ├── graph
│   ├── typeChecker
│   └── types
├── package.json
├── test
│   ├── assetGenerator.test.ts
│   ├── createAsset.test.ts
│   ├── example
│   └── typeChecker.test.ts
└── yarn.lock

togami 2021/01/09

classをassetGeneratorに改名

asset/assetgenerator.ts

import { readFileSync } from "fs";
import path = require("path");
import { parse } from "@babel/parser";
import traverse from "@babel/traverse";
import { transformSync } from "@babel/core";
import { absolutePath, relativePath } from "../types/common";
let ID = 0;

export class assetGenerator {
  filePath: absolutePath;
  id: number;
  dependencies: relativePath[];
  code: string;
  mapping: object;
  constructor(filePath: absolutePath, content) {
    this.filePath = filePath;
    this.id = ID++;
    this.dependencies = this.findDependency(content);
    this.code = this.transpileCode(content);
    this.mapping = {};
  }
  findDependency(content) {
    const dependencies: relativePath[] = [];
    const ast = parse(content, {
      sourceType: "module",
    });
    traverse(ast, {
      ImportDeclaration: ({ node }) => {
        dependencies.push(node.source.value);
      },
    });
    return dependencies;
  }
  transpileCode(content) {
    const { code } = transformSync(content, {
      filename: this.filePath,
      presets: ["@babel/preset-env", "@babel/preset-typescript"],
    });
    return code;
  }
}

togami 2021/01/09

問題発生
importするときは拡張子を省略できる

import { add } from "./add";　<--OK
import { add } from "./add.ts"; <-- ダメ

しかし省略すると、dependenciesに["./add"]として収められる。
よってcreateAsset関数内でreadFileSyncがエラーを起こす
(絶対pathは拡張子まで込)

正確なファイル名を取得するために、nodeのrequire関数の擬似コードを参考にして解決を図る
document

togami 2021/01/10

トランスパイルにはBabelTypeScript APIを使用するように変更した。

そもそも型チェックはtsにしていたので揃えたほうが良いかなってなった。
babelのドキュメントの情報が少ない・一部明らかに間違ってる

型チェックの際、同時にESModuleのjsファイルに変換するように変更。
依存性を調べたいので、CommonではなくESModuleで止めている

togami 2021/01/10

依存関係グラフの作成まで完了。
現状.tsファイルにしか対応していない。

graph.ts

import path from "path";
import { createAsset } from "../asset/createAsset";
import { absolutePath, relativePath } from "../types/common";

export const createDependencyGraph = (entryFile: absolutePath) => {
  const rootAsset = createAsset(entryFile);
  const queue = [rootAsset];
  for (const asset of queue) {
    const dirName = path.dirname(asset.filePath);
    asset.dependencies.forEach((relativePath) => {
      const absolutePath = path.join(dirName, relativePath);
      const childModule = createAsset(absolutePath);
      asset.mapping[relativePath] = childModule.id;
      queue.push(childModule);
    });
  }
  return queue;
};

createAsset.ts

import fs from "fs";
import path from "path";
import { absolutePath } from "../types/common";
import { assetGenerator } from "./assetGenerator";
import { resolveExt } from "../path/path";
import { typeChecker } from "../typeChecker/typeChecker";

export const createAsset = (filePath: absolutePath) => {
  filePath = resolveExt(filePath);
  const code = typeChecker(filePath);
  return new assetGenerator(filePath, code);
};

assetGenerator.ts

import { parse } from "@babel/parser";
import traverse from "@babel/traverse";
import { transformSync, BabelFileResult } from "@babel/core";
import { absolutePath, relativePath } from "../types/common";
let ID = 0;

export class assetGenerator {
  filePath: absolutePath;
  id: number;
  dependencies: relativePath[];
  code: string;
  mapping: object;
  constructor(filePath: absolutePath, code: string) {
    this.filePath = filePath;
    this.id = ID++;
    this.dependencies = this.findDependency(code);
    this.code = this.transpileToCJS(code);
    this.mapping = {};
  }
  findDependency(code: string) {
    const dependencies: relativePath[] = [];
    const ast = parse(code, {
      sourceType: "module",
    });
    traverse(ast, {
      ImportDeclaration: ({ node }) => {
        dependencies.push(node.source.value);
      },
    });
    return dependencies;
  }
  transpileToCJS(code) {
    code = transformSync(code, {
      filename: this.filePath,
      presets: ["@babel/preset-env"],
    });
    return code;
  }
}

typeChecker.ts

import * as ts from "typescript";
import { absolutePath } from "../types/common";
import { transpileToJS } from "./transpileToJS";

const TYPE_ERROR = 1;

export const typeChecker = (filePath: absolutePath): string => {
  let diagnostics = ts.getPreEmitDiagnostics(
    ts.createProgram([filePath], {
      strict: true,
      target: ts.ScriptTarget.Latest,
      moduleResolution: ts.ModuleResolutionKind.NodeJs,
    })
  );
  if (diagnostics.length) {
    diagnostics.forEach((d) => console.error(d.messageText));
    process.exit(TYPE_ERROR);
  } else {
    return transpileToJS(filePath);
  }
};

transpileToJS

import fs from "fs";
import * as ts from "typescript";

export const transpileToJS = (filePath): string => {
  const file = fs.readFileSync(filePath, "utf-8");
  const code = ts.transpileModule(file, {
    compilerOptions: {
      target: ts.ScriptTarget.ES2015,
      module: ts.ModuleKind.ESNext,
      noImplicitUseStrict: true,
      pretty: true,
    },
  }).outputText;
  return code;
};

path.ts

export const resolveExt = (filePath) => {
  return filePath + ".ts";
};

createDependencyGraph関数実行結果

// console.log(createDependencyGraph("test/example/valid"))
[
  assetGenerator {
    filePath: 'test/example/valid.ts',
    id: 0,
    dependencies: [ './add' ],
    code: {
      metadata: {},
      options: [Object],
      ast: null,
      code: '"use strict";\n' +
        '\n' +
        'var _add = require("./add");\n' +
        '\n' +
        'console.log((0, _add.add)(1, 1));',
      map: null,
      sourceType: 'script'
    },
    mapping: { './add': 1 }
  },
  assetGenerator {
    filePath: 'test/example/add.ts',
    id: 1,
    dependencies: [],
    code: {
      metadata: {},
      options: [Object],
      ast: null,
      code: '"use strict";\n' +
        '\n' +
        'Object.defineProperty(exports, "__esModule", {\n' +
        '  value: true\n' +
        '});\n' +
        'exports.add = void 0;\n' +
        '\n' +
        'var add = function add(x, y) {\n' +
        '  return x + y;\n' +
        '};\n' +
        '\n' +
        'exports.add = add;',
      map: null,
      sourceType: 'script'
    },
    mapping: {}
  }
]

togami 2021/02/23

バンドル作成まで完了

createBundle.ts

export const createBundle = (graph) => {
  let modules = "";
  graph.forEach((mod) => {
    modules += `${mod.id}:[
      function(require,module,exports){
        ${mod.code.code}
      },
      ${JSON.stringify(mod.mapping)},
    ],`;
  });
  const result = `
  (function(modules){
    function require(id){
      const [fn,mapping] = modules[id]
      function localRequire(relativePath){
        return require(mapping[relativePath])
      }
      const module = {exports:{}};
      fn(localRequire,module,module.exports);
      return module.exports;
    }
    require(0)
  })({${modules}})
  `;
  return [{ name: "bundle.js", code: result }];
};

結果

bundle.js


  (function(modules){
    function require(id){
      const [fn,mapping] = modules[id]
      function localRequire(relativePath){
        return require(mapping[relativePath])
      }
      const module = {exports:{}};
      fn(localRequire,module,module.exports);
      return module.exports;
    }
    require(0)
  })({0:[
      function(require,module,exports){
        "use strict";

var _add = require("./add");

console.log("Hello");
console.log((0, _add.add)(1, 1));
      },
      {"./add":1},
    ],1:[
      function(require,module,exports){
        "use strict";

Object.defineProperty(exports, "__esModule", {
  value: true
});
exports.add = void 0;

var add = function add(x, y) {
  return x + y;
};

exports.add = add;
      },
      {},
    ],})

togami 2021/02/23

htmlテンプレートを作れるようになった。

generateHTMLTemplate

import fs from "fs";

export const generateHTMLTemplate = (
  htmlPath: string,
  outputFiles: {
    name: string;
    code: string;
  }[]
) => {
  let HTMLTemplate = fs.readFileSync(htmlPath, "utf-8");
  HTMLTemplate = HTMLTemplate.replace(
    "</body>",
    outputFiles.map(({ name }) => `<script src="/${name}"></script>`).join("") +
      "</body>"
  );
  return { name: "index.html", code: HTMLTemplate };
};

togami 2021/02/23

dev-serverを追加
細かいリクエストは後

build.ts

const dev = ({ entryFile, htmlPath }) => {
  const { outputFiles } = _build({ entryFile, htmlPath });
  const outputFileMap = {};
  for (const outputFile of outputFiles) {
    outputFileMap[outputFile.name] = outputFile.code;
  }
  const indexHTML = outputFileMap["index.html"];
  const app = express();
  app.use("/", (req, res) => {
    // const requestFile = res.path.slice(1);
    // if (outputFileMap[requestFile]) {
    //   return res.send(outputFileMap[requestFile]);
    // }
    res.send(indexHTML);
  });
  app.listen(8080, () =>
    console.log(`Dev server starts at http://localhost:8080`)
  );
};

togami 2021/02/23

CLIツールを追加したい。
jsで作ったこと無いので手始めにやってみる。