Correlating Logs and Traces with Google Cloud Logging and Cloud Trace in Go

I always forget this, so here is a note.

Introduction

As stated in the title, this article describes how to implement log and trace correlation in Go using Google Cloud Logging and Cloud Trace.

Prerequisites

There are several reserved fields in Cloud Logging.

https://cloud.google.com/logging/docs/structured-logging#structured_logging_special_fields

By setting the necessary information in the following fields among them, you can correlate logs and traces between Google Cloud's Cloud Logging and Cloud Trace.

• logging.googleapis.com/spanId
  • Example: "logging.googleapis.com/spanId":"000000000000004a"
• logging.googleapis.com/trace
  • Example: "logging.googleapis.com/trace":"projects/my-projectid/traces/0679686673a"
• logging.googleapis.com/trace_sampled
  • Example: "logging.googleapis.com/trace_sampled":"true"

We will add implementation to each logger to include the necessary information in the fields above.

Implementation

Example implementations for several loggers are listed below.

Zap

https://github.com/uber-go/zap

import (
	"go.uber.org/zap"
	"go.opentelemetry.io/otel"
)

func main() {
	l := zap.NewProduction() // Initialize the logger appropriately
	_, span := otel.Tracer("main").Start(context.Background(), "main")
	l.Info(
		"main",
		// Replace PROJECT_ID with your own project ID
		zap.String("logging.googleapis.com/trace", fmt.Sprintf("projects/%s/traces/%s", "PROJECT_ID", span.SpanContext().TraceID().String())),
		zap.String("logging.googleapis.com/spanId", span.SpanContext().SpanID().String()),
		zap.Bool("logging.googleapis.com/trace_sampled", span.SpanContext().IsSampled()),
	)
}

slog

WIP

Verification

In Cloud Logging, a link to Cloud Trace is displayed.

In Cloud Trace, selecting the target trace displays the corresponding logs in "Logs and Events".