Building Testable CLI Applications with Cobra

I was just thinking that v1.1.0 of the spf13/cobra package had finally been released, when I noticed that a site called Cobra.Dev had also appeared. To celebrate, I'll introduce Cobra by actually building a CLI (Command-Line Interface) using it.

By the way, Cobra is a framework/package for building CLIs in Go. Compared to the standard flag package, it has the following main features:

• Multi-level subcommands can be composed relatively easily.
• Can be combined with the spf13/pflag package to implement POSIX/GNU style flags. Flag cascading is also possible.
• Can be combined with the spf13/viper package to bind flags and configuration file contents.
• Automatically generates and displays Usage information (customizable).

Today's Project

The specifications for the CLI application we will build today are as follows:

• The command hash has an encode subcommand.
• The encode subcommand encodes input text into a hash value using the SHA256 algorithm.

UNIX Philosophy

Before actually building the CLI application, I'd like to introduce the "UNIX Philosophy" as a design guideline. It says:

1. Small is beautiful.
2. Make each program do one thing well.
3. Build a prototype as soon as possible.
4. Choose portability over efficiency.
5. Store data in flat text files.
6. Use software leverage to your advantage.
7. Use shell scripts to increase leverage and portability.
8. Avoid captive user interfaces.
9. Make every program a Filter.

(Quoted from the Japanese Wikipedia). Keeping these in mind, we should focus on the following points when designing a CLI in Go:

• Create filter programs that utilize standard input/output so they can collaborate with other tools via the shell.
• Support formats like JSON and YAML for output to make it easier to integrate with external tools.
• If possible, unify input/output using UTF-8 encoding.
• Consider code portability and try to avoid platform dependencies.

Please note that these guidelines do not apply to so-called "interactive modes" in a CLI.

Subcommands and the Facade Pattern

The subcommand approach might seem to contradict the "UNIX Philosophy" at first glance. However, since Go combines all packages into a single binary, incorporating related functions as subcommands is not a bad approach.

When structuring subcommands, it is helpful to think in terms of the "facade pattern."

The term "facade" means the "front of a building," and it functions as a gateway to various subsystems within a system. The facade itself does not need to know the details of the subsystems; it simply passes context information and kicks off the process. The key is to make the subsystems independent of the facade, capable of processing as long as they have the context information.

Since this is a CLI, the subsystems can return their processing results to the facade as string, []byte, io.Reader, or types that can output these.

Using the Cobra command to generate a template

That was a long introduction, but let's jump right in and build a simple CLI using Cobra.

Installing the Cobra command

The Cobra package provides a tool to output code templates. Since it is not provided as a binary, build and install it using the go get command.

$ go get -u github.com/spf13/cobra/cobra

Generating main and cmd packages

First, set up the environment. Something like this should work.

$ mkdir hash && cd hash
$ go mod init sample/hash

For convenience, I'll set the package or module path to sample/hash. In a real-world scenario, it would likely be a path like github.com/user/hash.

Next, use the cobra command installed in the previous section to create a main package template. Like this:

$ cobra init --pkg-name sample/hash --viper=false
$ tree .
.
├── LICENSE
├── cmd
│   └── root.go
├── go.mod
└── main.go

Since we won't be using spf13/viper to manage configuration files this time, I've added the --viper=false option.

You can delete the LICENSE file or replace it with an appropriate one. The contents of main.go look like this (excluding comments):

main.go

package main

import "sample/hash/cmd"

func main() {
    cmd.Execute()
}

The contents of cmd/root.go look like this (excluding comments):

cmd/root.go

package cmd

import (
    "fmt"
    "github.com/spf13/cobra"
    "os"
)

var rootCmd = &cobra.Command{
    Use:   "hash",
    Short: "A brief description of your application",
    Long: `A longer description that spans multiple lines and likely contains
examples and usage of using your application. For example:

Cobra is a CLI library for Go that empowers applications.
This application is a tool to generate the needed files
to quickly create a Cobra application.`,
}

func Execute() {
    if err := rootCmd.Execute(); err != nil {
        fmt.Println(err)
        os.Exit(1)
    }
}

func init() {
    rootCmd.Flags().BoolP("toggle", "t", false, "Help message for toggle")
}

You can actually run it in this state.

$ go run main.go -h
A longer description that spans multiple lines and likely contains
examples and usage of using your application. For example:

Cobra is a CLI library for Go that empowers applications.
This application is a tool to generate the needed files
to quickly create a Cobra application.

I'd like to start making changes, but let's be patient and move on to the next step first.

Adding a Subcommand

Let's add the encode subcommand. Like this:

$ cobra add encode
$ tree .
.
├── LICENSE
├── cmd
│   ├── encode.go
│   └── root.go
├── go.mod
└── main.go

Can you see that cmd/encode.go has been added? Note that other files like main.go and cmd/root.go have not been modified.

The contents of cmd/encode.go are as follows (excluding comments):

cmd/encode.go

package cmd

import (
    "fmt"

    "github.com/spf13/cobra"
)

var encodeCmd = &cobra.Command{
    Use:   "encode",
    Short: "A brief description of your command",
    Long: `A longer description that spans multiple lines and likely contains examples
and usage of using your command. For example:

Cobra is a CLI library for Go that empowers applications.
This application is a tool to generate the needed files
to quickly create a Cobra application.`,
    Run: func(cmd *cobra.Command, args []string) {
        fmt.Println("encode called")
    },
}

func init() {
    rootCmd.AddCommand(encodeCmd)
}

Running it in this state results in the following:

$ go run main.go
A longer description that spans multiple lines and likely contains
examples and usage of using your application. For example:

Cobra is a CLI library for Go that empowers applications.
This application is a tool to generate the needed files
to quickly create a Cobra application.

Usage:
  hash [command]

Available Commands:
  encode      A brief description of your command
  help        Help about any command

Flags:
  -h, --help     help for hash
  -t, --toggle   Help message for toggle

Use "hash [command] --help" for more information about a command.

$ go run main.go encode -h
A longer description that spans multiple lines and likely contains examples
and usage of using your command. For example:

Cobra is a CLI library for Go that empowers applications.
This application is a tool to generate the needed files
to quickly create a Cobra application.

Usage:
  hash encode [flags]

Flags:
  -h, --help   help for encode

$ go run main.go encode
encode called

So, we can see that the encode subcommand has been incorporated.

Rewriting the Template Code

You could continue adding features directly to the template code generated by the cobra command, but the template code has the following issues:

• It uses standard input/output and command-line arguments directly inside the cmd package template.
• The cobra.Command instance is defined as a static variable within the cmd package.
• There are locations within the cmd.Execute() function where the program is terminated forcibly using the os.Exit() function.

This makes it difficult to test the cmd package itself, so let's refactor it to make it easier to test.

Introducing the spiegel-im-spiegel/gocli Package

I hope you'll excuse the self-promotion, but here I'll introduce the spiegel-im-spiegel/gocli package.

The spiegel-im-spiegel/gocli package allows input and output to be passed as context information, and can be used as follows:

package main

import (
    "os"

    "github.com/spiegel-im-spiegel/gocli/exitcode"
    "github.com/spiegel-im-spiegel/gocli/rwi"
)

func run(ui *rwi.RWI) exitcode.ExitCode {
    ui.Outputln("Hello world")
    return exitcode.Normal
}

func main() {
    run(rwi.New(
        rwi.WithReader(os.Stdin),
        rwi.WithWriter(os.Stdout),
        rwi.WithErrorWriter(os.Stderr),
    )).Exit()
}

Using this, let's rewrite the template code for the cmd package.

Redefining the Subcommand

First, let's look at the encode subcommand. This is what I've done:

cmd/encode.go

func newEncodeCmd(ui *rwi.RWI) *cobra.Command {
    encodeCmd := &cobra.Command{
        Use:     "encode",
        Aliases: []string{"enc", "e"},
        Short:   "hash input data",
        Long:    "hash input data (detail)",
        RunE: func(cmd *cobra.Command, args []string) error {
            if err := ui.Outputln("encode called"); err != nil {
                return err
            }
            return nil
        },
    }
    return encodeCmd
}

Now we can dynamically generate the cobra.Command instance for the encode subcommand using an internal function.

Redefining the Command

In the same way, we rewrite the contents of root.go.

cmd/root.go

func newRootCmd(ui *rwi.RWI, args []string) *cobra.Command {
    rootCmd := &cobra.Command{
        Use:   "hash",
        Short: "Hash functions",
        Long:  "Hash functions (detail)",
    }
    rootCmd.SilenceUsage = true
    rootCmd.SetArgs(args)            //arguments of command-line
    rootCmd.SetIn(ui.Reader())       //Stdin
    rootCmd.SetOut(ui.ErrorWriter()) //Stdout -> Stderr
    rootCmd.SetErr(ui.ErrorWriter()) //Stderr
    rootCmd.AddCommand(
        newEncodeCmd(ui),
    )
    return rootCmd
}

func Execute(ui *rwi.RWI, args []string) exitcode.ExitCode {
    if err := newRootCmd(ui, args).Execute(); err != nil {
        return exitcode.Abnormal
    }
    return exitcode.Normal
}

By the way, in the following part of the newRootCmd() function:

rootCmd.SetArgs(args)            //arguments of command-line
rootCmd.SetIn(ui.Reader())       //Stdin
rootCmd.SetOut(ui.ErrorWriter()) //Stdout -> Stderr
rootCmd.SetErr(ui.ErrorWriter()) //Stderr

the command-line arguments and input/output are being set. Also, while it may look like the error instance returned from cobra.Command.Execute() isn't being evaluated within the cmd.Execute() function, the error content has actually already been evaluated within the cobra.Command.Execute() method itself. Therefore, the cmd.Execute() function only checks the truth value of err != nil.

Finally, let's update the main() function to match the changes above.

main.go

func main() {
    cmd.Execute(
        rwi.New(
            rwi.WithReader(os.Stdin),
            rwi.WithWriter(os.Stdout),
            rwi.WithErrorWriter(os.Stderr),
        ),
        os.Args[1:],
    ).Exit()
}

That's it.

Testing the Command

We need to write tests for the code we've written. Something like this?

cmd/encode_test.go

func TestEncode(t *testing.T) {
    testCases := []struct {
        inp  string
        outp string
        ext  exitcode.ExitCode
    }{
        {inp: "hello world\n", outp: "a948904f2f0f479b8f8197694b30184b0d2ed1c1cd2a1ec0fb85d299a192a447\n", ext: exitcode.Normal},
    }

    for _, tc := range testCases {
        r := strings.NewReader(tc.inp)
        wbuf := &bytes.Buffer{}
        ebuf := &bytes.Buffer{}
        ext := Execute(
            rwi.New(
                rwi.WithReader(r),
                rwi.WithWriter(wbuf),
                rwi.WithErrorWriter(ebuf),
            ),
            []string{"encode"},
        )
        if ext != tc.ext {
            t.Errorf("Execute() is \"%v\", want \"%v\".", ext, tc.ext)
            fmt.Println(ebuf.String())
        }
        str := wbuf.String()
        if str != tc.outp {
            t.Errorf("Execute() -2 \"%v\", want \"%v\".", str, tc.outp)
        }
    }
}

Note the arguments for the Execute() function. The execution result is:

$ go test ./...
?       sample/hash    [no test files]
--- FAIL: TestEncode (0.00s)
    encode_test.go:36: Execute() -2 "encode called
        ", want "a948904f2f0f479b8f8197694b30184b0d2ed1c1cd2a1ec0fb85d299a192a447
        ".
FAIL
FAIL    sample/hash/cmd    0.002s
FAIL

Oops. I forgot to write the actual logic (lol). Well then, let's create the encode/encode.go file:

encode/encode.go

package encode

import (
    "crypto"
    "errors"
    "io"
)

var (
    ErrNoImplement = errors.New("no implementation")
)

//Value returns hash value string from io.Reader
func Value(r io.Reader, alg crypto.Hash) ([]byte, error) {
    if !alg.Available() {
        return nil, ErrNoImplement
    }
    h := alg.New()
    if _, err := io.Copy(h, r); err != nil {
        return nil, err
    }
    return h.Sum(nil), nil
}

And let's incorporate this into the cmd.newEncodeCmd() function:

cmd/encode.go

func newEncodeCmd(ui *rwi.RWI) *cobra.Command {
    encodeCmd := &cobra.Command{
        Use:     "encode",
        Aliases: []string{"enc", "e"},
        Short:   "hash input data",
        Long:    "hash input data (detail)",
        RunE: func(cmd *cobra.Command, args []string) error {
            v, err := encode.Value(ui.Reader(), crypto.SHA256)
            if err != nil {
                return err
            }
            fmt.Fprintf(ui.Writer(), "%x\n", v)
            return nil
        },
    }
    return encodeCmd
}

Resuming tests:

$ go test ./...
?       sample/hash    [no test files]
ok      sample/hash/cmd    0.003s
?       sample/hash/encode    [no test files]

All right, yes, all right.

Finally, let's actually run the command.

$ go run main.go encode -h
hash input data (detail)

Usage:
  hash encode [flags]

Aliases:
  encode, enc, e

Flags:
  -h, --help   help for encode

$ echo hello world | go run main.go encode
a948904f2f0f479b8f8197694b30184b0d2ed1c1cd2a1ec0fb85d299a192a447

It executed successfully.

By the way, the code written this time is located at github.com/spiegel-im-spiegel/zenn-docs/code/cobra/hash. Please refer to it for your convenience.