Module Structure and Workflow Optimized for Generative AI Code Generation

tl;dr

• I thought about the directory structure and operations first.
• I wrote a CLI tool to perform code generation accordingly.

Proposed Directory Structure

I came up with the following directory structure.

xxx-module/
  deps.ts
  mod.ts
  mod.test.ts
  lib.ts
  v0.ts
  v1.ts
  v*.ts
  README.md
  PROMPT.md

The versioning approach is inspired by vercel/v0.

deps.ts

A file for declaring dependencies.

Example

export { tpl } from "jsr:@mizchi/tpl";

Assume that all external dependencies must be declared here to be used. By narrowing the scope, accuracy is improved.

v*.ts

Versioned files. A new v*.ts is generated every time code is generated. I opted for this approach because I didn't want to save every command execution to git history, although one could pull them from history by committing each time.

It imports dependencies from lib.ts and deps.ts.

When generating complex code, it can sometimes be treated as a black box, so I want to proceed step-by-step while checking at which point it was functional.

Once a certain stage is confirmed to work, versions other than the latest are deleted.

lib.ts

A file for defining internal functions that are not exported.

It is provided as a prompt but is not a target for generation.

While v*.ts is always subject to full-text generation, refactoring and moving code here can significantly reduce generation time.

README.md

Since this section is intended to explain how to use the module to users, it is fed directly into the prompt as is.

PROMPT.md

Write user prompts here. Separate them with --- and instruct the AI to refer to the latest version.

Instruction 1

---

Instruction 2

Since it is fed directly into the AI, it can also be used casually as a notepad or scratchpad.

@mizchi/codegen

So I created a CLI tool. Since I used Claude 3, the ANTHROPIC_API_KEY environment variable is required.

$ deno install -Afg jsr:@mizchi/codegen --name codegen

$ codegen new mymod
$ cd mymod
# Edit PROMPT.md
$ codegen # generate v1.ts
$ codegen fix # Run tests and attempt auto-fixing if they fail
$ codegen dedupe # Rename the latest version to v0.ts and move inline test code to mod.test.ts

Intended workflow:

• Roughly describe the specifications in PROMPT.md.
• Generate v1.ts with codegen.
• Refactor by moving parts that don't need changes to lib.ts.
• Describe additional change intentions in PROMPT.md, separated by ---.
• Run codegen fix to execute tests and perform auto-fixes.
• Once functionality is confirmed, run codegen dedupe to generate a new v0.ts and discard others.

In the codegen dedupe command, I've instructed it to move inline tests to mod.test.ts while summarizing the current change intentions at the beginning of the file.

Impressions

It works in a very rough way, but it's based on the premise that humans will make fine adjustments.

As the amount of code in v*.ts increases, the regeneration speed slows down noticeably. Therefore, frequent refactoring into lib.ts is expected. It would probably be good to provide a dedicated command for that as well.

It seems usable when specifications are documented to some extent and external dependencies are few, but whether it can handle complex cases is still unknown.

More research is needed.