Trying out the goccy/go-json package

I recently saw a post on Twitter about a JSON handling package called github.com/goccy/go-json. It claims to be a drop-in replacement for the standard encoding/json and boasts that it is faster than both encoding/json and other compatible packages.

via “github.com/goccy/go-json”

First, I'll try to see if it can really replace the standard encoding/json package.

Since I recently created a package called spiegel-im-spiegel/emojis that organizes emoji information in JSON format, I'll write a process to load it. Here is how it looks using the standard encoding/json package.

sample1.go

// +build run

package main

import (
    "bytes"
    "encoding/json"
    "fmt"
    "os"
    "strings"

    emoji "github.com/spiegel-im-spiegel/emojis/json"
    "github.com/spiegel-im-spiegel/fetch"
)

func getEmojiSequenceJSON() ([]byte, error) {
    u, err := fetch.URL("https://raw.githubusercontent.com/spiegel-im-spiegel/emojis/main/json/emoji-sequences.json")
    if err != nil {
        return nil, err
    }
    resp, err := fetch.New().Get(u)
    if err != nil {
        return nil, err
    }
    return resp.DumpBodyAndClose()
}

func main() {
    b, err := getEmojiSequenceJSON()
    if err != nil {
        fmt.Fprintln(os.Stderr, err)
        return
    }
    list := []emoji.EmojiSequence{}
    if err := json.NewDecoder(bytes.NewReader(b)).Decode(&list); err != nil {
        fmt.Fprintln(os.Stderr, err)
        return
    }

    fmt.Println("| Sequence | Shortcodes |")
    fmt.Println("| :------: | ---------- |")
    for _, ec := range list {
        var bldr strings.Builder
        for _, c := range ec.Shortcodes {
            bldr.WriteString(fmt.Sprintf(" `%s`", c))
        }
        fmt.Printf("| %v |%s |\n", ec.Sequence, bldr.String())
    }
}

When running this, the output is:

$ go run sample1.go
| Sequence | Shortcodes |
| :------: | ---------- |
| #️⃣ | `:hash:` `:keycap_#:` |
| *️⃣ | `:asterisk:` `:keycap_*:` `:keycap_star:` |
| 0️⃣ | `:zero:` `:keycap_0:` |
| 1️⃣ | `:one:` `:keycap_1:` |
...

Indeed. I see.

Now, let's replace it with the github.com/goccy/go-json package.

sample2.go

import (
    "bytes"
-   "encoding/json"
    "fmt"
    "os"
    "strings"

+   "github.com/goccy/go-json"
    emoji "github.com/spiegel-im-spiegel/emojis/json"
    "github.com/spiegel-im-spiegel/fetch"
)

Running this code results in:

$ go run sample2.go 
| Sequence | Shortcodes |
| :------: | ---------- |
| #️⃣ | `:hash:` `:keycap_#:` |
| *️⃣ | `:asterisk:` `:keycap_*:` `:keycap_star:` |
| 0️⃣ | `:zero:` `:keycap_0:` |
| 1️⃣ | `:one:` `:keycap_1:` |
...

The output is exactly the same.

Next, let's use this code to run a benchmark. This should do it.

json2_test.go

package jsonbench

import (
    "bytes"
    "encoding/json"
    "testing"

    another "github.com/goccy/go-json"
    emoji "github.com/spiegel-im-spiegel/emojis/json"
    "github.com/spiegel-im-spiegel/fetch"
)

func getMustEmojiSequenceJSON() []byte {
    u, err := fetch.URL("https://raw.githubusercontent.com/spiegel-im-spiegel/emojis/main/json/emoji-sequences.json")
    if err != nil {
        panic(err)
    }
    resp, err := fetch.New().Get(u)
    if err != nil {
        panic(err)
    }
    b, err := resp.DumpBodyAndClose()
    if err != nil {
        panic(err)
    }
    return b
}

var jsonText = getMustEmojiSequenceJSON()

func BenchmarkDecodeOrgPkg(b *testing.B) {
    for i := 0; i < b.N; i++ {
        _ = json.NewDecoder(bytes.NewReader(jsonText)).Decode(&([]emoji.EmojiSequence{}))
    }
}

func BenchmarkDecodeAnotherPkg(b *testing.B) {
    for i := 0; i < b.N; i++ {
        _ = another.NewDecoder(bytes.NewReader(jsonText)).Decode(&([]emoji.EmojiSequence{}))
    }
}

Executing this yielded the following results on my local environment.

$ go test -bench Decode -benchmem
goos: linux
goarch: amd64
pkg: json2
cpu: Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz
BenchmarkDecodeOrgPkg-4                 99      11095017 ns/op     2257059 B/op       17398 allocs/op
BenchmarkDecodeAnotherPkg-4            352       3386700 ns/op      907342 B/op        6403 allocs/op
PASS
ok      json2    3.172s

Here is what it looks like in a table.

Package used	Execution time	Alloc size	Alloc count
encoding/json	11,095,017 ns/op	2,257,059 B/op	17,398 allocs/op
goccy/go-json	3,386,700 ns/op	907,342 B/op	6,403 allocs/op

Wait. So it's different starting from the number of allocations. No wonder there's a difference. The total execution time is reduced to about 30% of the standard package.

This seems worth considering, doesn't it?

Slides by the author, goccy

https://docs.google.com/presentation/d/1VCUPh21hrZRN-2kneQ07dxJPjgScSo3gsKESWlEBzIQ/

References

https://zenn.dev/spiegel/articles/20210113-fetch
https://zenn.dev/spiegel/articles/20210322-emoji-shortcode-for-markdown
https://text.baldanders.info/remark/2021/04/emoji-list/