Repository: blendle/zapdriver Branch: master Commit: 9200777f8a3d Files: 27 Total size: 59.2 KB Directory structure: gitextract_ukk2e0f3/ ├── .gitignore ├── LICENSE ├── README.md ├── common_test.go ├── config.go ├── core.go ├── core_test.go ├── encoder.go ├── encoder_test.go ├── go.mod ├── go.sum ├── http.go ├── http_test.go ├── label.go ├── label_test.go ├── logger.go ├── logger_test.go ├── operation.go ├── operation_test.go ├── report.go ├── report_test.go ├── service.go ├── service_test.go ├── source.go ├── source_test.go ├── trace.go └── trace_test.go ================================================ FILE CONTENTS ================================================ ================================================ FILE: .gitignore ================================================ vendor/ ================================================ FILE: LICENSE ================================================ ISC License Copyright (c) Blendle Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies. THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. ================================================ FILE: README.md ================================================ # :zap: Zapdriver Blazing fast, [Zap][zap]-based [Stackdriver][stackdriver] logging. [zap]: https://github.com/uber-go/zap [stackdriver]: https://cloud.google.com/stackdriver/ ## Usage This package provides three building blocks to support the full array of structured logging capabilities of Stackdriver: * [Special purpose logging fields](#special-purpose-logging-fields) * [Pre-configured Stackdriver-optimized encoder](#pre-configured-stackdriver-optimized-encoder) * [Custom Stackdriver Zap core](#custom-stackdriver-zap-core) * [Using Error Reporting](#using-error-reporting) The above components can be used separately, but to start, you can create a new Zap logger with all of the above included: ```golang logger, err := zapdriver.NewProduction() // with sampling logger, err := zapdriver.NewDevelopment() // with `development` set to `true` ``` The above functions give back a pointer to a `zap.Logger` object, so you can use [Zap][zap] like you've always done, except that it now logs in the proper [Stackdriver][stackdriver] format. You can also create a configuration struct, and build your logger from there: ```golang config := zapdriver.NewProductionConfig() config := zapdriver.NewDevelopmentConfig() ``` Or, get the Zapdriver encoder, and build your own configuration struct from that: ```golang encoder := zapdriver.NewProductionEncoderConfig() encoder := zapdriver.NewDevelopmentEncoderConfig() ``` Read on to learn more about the available Stackdriver-specific log fields, and how to use the above-mentioned components. ### Special purpose logging fields You can use the following fields to add extra information to your log entries. These fields are parsed by Stackdriver to make it easier to query your logs or to use the log details in the Stackdriver monitoring interface. * [`HTTP`](#http) * [`Label`](#label) * [`SourceLocation`](#sourcelocation) * [`Operation`](#operation) * [`TraceContext`](#tracecontext) #### HTTP You can log HTTP request/response cycles using the following field: ```golang HTTP(req *HTTPPayload) zap.Field ``` You can either manually build the request payload: ```golang req := &HTTPPayload{ RequestMethod: "GET", RequestURL: "/", Status: 200, } ``` Or, you can auto generate the struct, based on the available request and response objects: ```golang NewHTTP(req *http.Request, res *http.Response) *HTTPPayload ``` You are free to pass in `nil` for either the request or response object, if one of them is unavailable to you at the point of logging. Any field depending on one or the other will be omitted if `nil` is passed in. Note that there are some fields that are not populated by either the request or response object, and need to be set manually: * `ServerIP string` * `Latency string` * `CacheLookup bool` * `CacheHit bool` * `CacheValidatedWithOriginServer bool` * `CacheFillBytes string` If you have no need for those fields, the quickest way to get started is like so: ```golang logger.Info("Request Received.", zapdriver.HTTP(zapdriver.NewHTTP(req, res))) ``` #### Label You can add a "label" to your payload as follows: ```golang Label(key, value string) zap.Field ``` Note that underwater, this sets the key to `labels.`. You need to be using the `zapdriver.Core` core for this to be converted to the proper format for Stackdriver to recognize the labels. See "Custom Stackdriver Zap core" for more details. If you have a reason not to use the provided Core, you can still wrap labels in the right `labels` namespace by using the available function: ```golang Labels(fields ...zap.Field) zap.Field ``` Like so: ```golang logger.Info( "Did something.", zapdriver.Labels( zapdriver.Label("hello", "world"), zapdriver.Label("hi", "universe"), ), ) ``` Again, wrapping the `Label` calls in `Labels` is not required if you use the supplied Zap Core. #### SourceLocation You can add a source code location to your log lines to be picked up by Stackdriver. Note that you can set this manually, or use `zapdriver.Core` to automatically add this. If you set it manually, _and_ use `zapdriver.Core`, the manual call stack will be preserved over the automated one. ```golang SourceLocation(pc uintptr, file string, line int, ok bool) zap.Field ``` Note that the function signature equals that of the return values of `runtime.Caller()`. This allows you to catch the stack frame at one location, while logging it at a different location, like so: ```golang pc, file, line, ok := runtime.Caller(0) // do other stuff... logger.Error("Something happened!", zapdriver.SourceLocation(pc, file, line, ok)) ``` If you use `zapdriver.Core`, the above use-case is the only use-case where you would want to manually set the source location. In all other situations, you can simply omit this field, and it will be added automatically, using the stack frame at the location where the log line is triggered. If you don't use `zapdriver.Core`, and still want to add the source location at the frame of the triggered log line, you'd do it like this: ```golang logger.Error("Something happened!", zapdriver.SourceLocation(runtime.Caller(0))) ``` #### Operation The `Operation` log field allows you to group log lines into a single "operation" performed by the application: ```golang Operation(id, producer string, first, last bool) zap.Field ``` For a pair of logs that belong to the same operation, you should use the same `id` between them. The `producer` is an arbitrary identifier that should be globally unique amongst all the logs of all your applications (meaning it should probably be the unique name of the current application). You should set `first` to true for the first log in the operation, and `last` to true for the final log of the operation. ```golang logger.Info("Started.", zapdriver.Operation("3g4d3g", "my-app", true, false)) logger.Debug("Progressing.", zapdriver.Operation("3g4d3g", "my-app", false, false)) logger.Info("Done.", zapdriver.Operation("3g4d3g", "my-app", false, true)) ``` Instead of defining the "start" and "end" booleans, you can also use these three convenience functions: ```golang OperationStart(id, producer string) zap.Field OperationCont(id, producer string) zap.Field OperationEnd(id, producer string) zap.Field ``` #### TraceContext You can add trace context information to your log lines to be picked up by Stackdriver. ```golang TraceContext(trace string, spanId string, sampled bool, projectName string) []zap.Field ``` Like so: ```golang logger.Error("Something happened!", zapdriver.TraceContext("105445aa7843bc8bf206b120001000", "0", true, "my-project-name")...) ``` ### Pre-configured Stackdriver-optimized encoder The Stackdriver encoder maps all Zap log levels to the appropriate [Stackdriver-supported levels][levels]: > DEBUG (100) Debug or trace information. > > INFO (200) Routine information, such as ongoing status or performance. > > WARNING (400) Warning events might cause problems. > > ERROR (500) Error events are likely to cause problems. > > CRITICAL (600) Critical events cause more severe problems or outages. > > ALERT (700) A person must take an action immediately. > > EMERGENCY (800) One or more systems are unusable. [levels]: https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry#LogSeverity It also sets some of the default keys to use [the right names][names], such as `timestamp`, `severity`, and `message`. [names]: https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry You can use this encoder if you want to build your Zap logger configuration manually: ```golang zapdriver.NewProductionEncoderConfig() ``` For parity-sake, there's also `zapdriver.NewDevelopmentEncoderConfig()`, but it returns the exact same encoder right now. ### Custom Stackdriver Zap core A custom Zap core is included in this package to support some special use-cases. First of all, if you use `zapdriver.NewProduction()` (or `NewDevelopment`) , you already have this core enabled, so everything _just works_ ™. There are two use-cases which require this core: 1. If you use `zapdriver.Label("hello", "world")`, it will initially end up in your log with the key `labels.hello` and value `world`. Now if you have two labels, you could also have `labels.hi` with value `universe`. This works as- is, but for this to be correctly parsed by Stackdriver as true "labels", you need to use the Zapdriver core, so that both of these fields get rewritten, to use the namespace `labels`, and use the keys `hello` and `hi` within that namespace. This is done automatically. 2. If you don't want to use `zapdriver.SourceLocation()` on every log call, you can use this core for the source location to be automatically added to each log entry. When building a logger, you can inject the Zapdriver core as follows: ```golang config := &zap.Config{} logger, err := config.Build(zapdriver.WrapCore()) ``` ### Using Error Reporting To report errors using StackDriver's Error Reporting tool, a log line needs to follow a separate log format described in the [Error Reporting][errorreporting] documentation. [errorreporting]: https://cloud.google.com/error-reporting/docs/formatting-error-messages The simplest way to do this is by using `NewProductionWithCore`: ```golang logger, err := zapdriver.NewProductionWithCore(zapdriver.WrapCore( zapdriver.ReportAllErrors(true), zapdriver.ServiceName("my service"), )) ``` For parity-sake, there's also `zapdriver.NewDevelopmentWithCore()` If you are building a custom logger, you can use `WrapCore()` to configure the driver core: ```golang config := &zap.Config{} logger, err := config.Build(zapdriver.WrapCore( zapdriver.ReportAllErrors(true), zapdriver.ServiceName("my service"), )) ``` Configuring this way, every error log entry will be reported to Stackdriver's Error Reporting tool. #### Reporting errors manually If you do not want every error to be reported, you can attach `ErrorReport()` to log call manually: ```golang logger.Error("An error to be reported!", zapdriver.ErrorReport(runtime.Caller(0))) // Or get Caller details pc, file, line, ok := runtime.Caller(0) // do other stuff... and log elsewhere logger.Error("Another error to be reported!", zapdriver.ErrorReport(pc, file, line, ok)) ``` Please keep in mind that ErrorReport needs a ServiceContext attached to the log entry. If you did not configure this using `WrapCore`, error reports will get attached using service name as `unknown`. To prevent this from happeneing, either configure your core or attach service context before (or when) using the logger: ```golang logger.Error( "An error to be reported!", zapdriver.ErrorReport(runtime.Caller(0)), zapdriver.ServiceContext("my service"), ) // Or permanently attach it to your logger logger = logger.With(zapdriver.ServiceContext("my service")) // and then use it logger.Error("An error to be reported!", zapdriver.ErrorReport(runtime.Caller(0))) ``` ================================================ FILE: common_test.go ================================================ package zapdriver_test import ( "time" "go.uber.org/zap/zapcore" ) // sliceArrayEncoder is an ArrayEncoder backed by a simple []interface{}. Like // the MapObjectEncoder, it's not designed for production use. type sliceArrayEncoder struct { elems []interface{} } func (s *sliceArrayEncoder) AppendArray(v zapcore.ArrayMarshaler) error { enc := &sliceArrayEncoder{} err := v.MarshalLogArray(enc) s.elems = append(s.elems, enc.elems) return err } func (s *sliceArrayEncoder) AppendObject(v zapcore.ObjectMarshaler) error { m := zapcore.NewMapObjectEncoder() err := v.MarshalLogObject(m) s.elems = append(s.elems, m.Fields) return err } func (s *sliceArrayEncoder) AppendReflected(v interface{}) error { s.elems = append(s.elems, v) return nil } func (s *sliceArrayEncoder) AppendBool(v bool) { s.elems = append(s.elems, v) } func (s *sliceArrayEncoder) AppendByteString(v []byte) { s.elems = append(s.elems, v) } func (s *sliceArrayEncoder) AppendComplex128(v complex128) { s.elems = append(s.elems, v) } func (s *sliceArrayEncoder) AppendComplex64(v complex64) { s.elems = append(s.elems, v) } func (s *sliceArrayEncoder) AppendDuration(v time.Duration) { s.elems = append(s.elems, v) } func (s *sliceArrayEncoder) AppendFloat64(v float64) { s.elems = append(s.elems, v) } func (s *sliceArrayEncoder) AppendFloat32(v float32) { s.elems = append(s.elems, v) } func (s *sliceArrayEncoder) AppendInt(v int) { s.elems = append(s.elems, v) } func (s *sliceArrayEncoder) AppendInt64(v int64) { s.elems = append(s.elems, v) } func (s *sliceArrayEncoder) AppendInt32(v int32) { s.elems = append(s.elems, v) } func (s *sliceArrayEncoder) AppendInt16(v int16) { s.elems = append(s.elems, v) } func (s *sliceArrayEncoder) AppendInt8(v int8) { s.elems = append(s.elems, v) } func (s *sliceArrayEncoder) AppendString(v string) { s.elems = append(s.elems, v) } func (s *sliceArrayEncoder) AppendTime(v time.Time) { s.elems = append(s.elems, v) } func (s *sliceArrayEncoder) AppendUint(v uint) { s.elems = append(s.elems, v) } func (s *sliceArrayEncoder) AppendUint64(v uint64) { s.elems = append(s.elems, v) } func (s *sliceArrayEncoder) AppendUint32(v uint32) { s.elems = append(s.elems, v) } func (s *sliceArrayEncoder) AppendUint16(v uint16) { s.elems = append(s.elems, v) } func (s *sliceArrayEncoder) AppendUint8(v uint8) { s.elems = append(s.elems, v) } func (s *sliceArrayEncoder) AppendUintptr(v uintptr) { s.elems = append(s.elems, v) } ================================================ FILE: config.go ================================================ package zapdriver import ( "go.uber.org/zap" "go.uber.org/zap/zapcore" ) // NewProductionEncoderConfig returns an opinionated EncoderConfig for // production environments. func NewProductionEncoderConfig() zapcore.EncoderConfig { return encoderConfig } // NewDevelopmentEncoderConfig returns an opinionated EncoderConfig for // development environments. func NewDevelopmentEncoderConfig() zapcore.EncoderConfig { return encoderConfig } // NewProductionConfig is a reasonable production logging configuration. // Logging is enabled at InfoLevel and above. // // It uses a JSON encoder, writes to standard error, and enables sampling. // Stacktraces are automatically included on logs of ErrorLevel and above. func NewProductionConfig() zap.Config { return zap.Config{ Level: zap.NewAtomicLevelAt(zap.InfoLevel), Development: false, Sampling: &zap.SamplingConfig{ Initial: 100, Thereafter: 100, }, Encoding: "json", EncoderConfig: NewProductionEncoderConfig(), OutputPaths: []string{"stderr"}, ErrorOutputPaths: []string{"stderr"}, } } // NewDevelopmentConfig is a reasonable development logging configuration. // Logging is enabled at DebugLevel and above. // // It enables development mode (which makes DPanicLevel logs panic), uses a // console encoder, writes to standard error, and disables sampling. // Stacktraces are automatically included on logs of WarnLevel and above. func NewDevelopmentConfig() zap.Config { return zap.Config{ Level: zap.NewAtomicLevelAt(zap.DebugLevel), Development: true, Encoding: "json", EncoderConfig: NewDevelopmentEncoderConfig(), OutputPaths: []string{"stderr"}, ErrorOutputPaths: []string{"stderr"}, } } ================================================ FILE: core.go ================================================ package zapdriver import ( "strings" "go.uber.org/zap" "go.uber.org/zap/zapcore" ) // driverConfig is used to configure core. type driverConfig struct { // Report all logs with level error or above to stackdriver using // `ErrorReport()` when set to true ReportAllErrors bool // ServiceName is added as `ServiceContext()` to all logs when set ServiceName string } // Core is a zapdriver specific core wrapped around the default zap core. It // allows to merge all defined labels type core struct { zapcore.Core // permLabels is a collection of labels that have been added to the logger // through the use of `With()`. These labels should never be cleared after // logging a single entry, unlike `tempLabel`. permLabels *labels // tempLabels keeps a record of all the labels that need to be applied to the // current log entry. Zap serializes log fields at different parts of the // stack, one such location is when calling `core.With` and the other one is // when calling `core.Write`. This makes it impossible to (for example) take // all `labels.xxx` fields, and wrap them in the `labels` namespace in one go. // // Instead, we have to filter out these labels at both locations, and then add // them back in the proper format right before we call `Write` on the original // Zap core. tempLabels *labels // Configuration for the zapdriver core config driverConfig } // zapdriver core option to report all logs with level error or above to stackdriver // using `ErrorReport()` when set to true func ReportAllErrors(report bool) func(*core) { return func(c *core) { c.config.ReportAllErrors = report } } // zapdriver core option to add `ServiceContext()` to all logs with `name` as // service name func ServiceName(name string) func(*core) { return func(c *core) { c.config.ServiceName = name } } // WrapCore returns a `zap.Option` that wraps the default core with the // zapdriver one. func WrapCore(options ...func(*core)) zap.Option { return zap.WrapCore(func(c zapcore.Core) zapcore.Core { newcore := &core{ Core: c, permLabels: newLabels(), tempLabels: newLabels(), } for _, option := range options { option(newcore) } return newcore }) } // With adds structured context to the Core. func (c *core) With(fields []zap.Field) zapcore.Core { var lbls *labels lbls, fields = c.extractLabels(fields) lbls.mutex.RLock() c.permLabels.mutex.Lock() for k, v := range lbls.store { c.permLabels.store[k] = v } c.permLabels.mutex.Unlock() lbls.mutex.RUnlock() return &core{ Core: c.Core.With(fields), permLabels: c.permLabels, tempLabels: newLabels(), config: c.config, } } // Check determines whether the supplied Entry should be logged (using the // embedded LevelEnabler and possibly some extra logic). If the entry // should be logged, the Core adds itself to the CheckedEntry and returns // the result. // // Callers must use Check before calling Write. func (c *core) Check(ent zapcore.Entry, ce *zapcore.CheckedEntry) *zapcore.CheckedEntry { if c.Enabled(ent.Level) { return ce.AddCore(ent, c) } return ce } func (c *core) Write(ent zapcore.Entry, fields []zapcore.Field) error { var lbls *labels lbls, fields = c.extractLabels(fields) lbls.mutex.RLock() c.tempLabels.mutex.Lock() for k, v := range lbls.store { c.tempLabels.store[k] = v } c.tempLabels.mutex.Unlock() lbls.mutex.RUnlock() fields = append(fields, labelsField(c.allLabels())) fields = c.withSourceLocation(ent, fields) if c.config.ServiceName != "" { fields = c.withServiceContext(c.config.ServiceName, fields) } if c.config.ReportAllErrors && zapcore.ErrorLevel.Enabled(ent.Level) { fields = c.withErrorReport(ent, fields) if c.config.ServiceName == "" { // A service name was not set but error report needs it // So attempt to add a generic service name fields = c.withServiceContext("unknown", fields) } } c.tempLabels.reset() return c.Core.Write(ent, fields) } // Sync flushes buffered logs (if any). func (c *core) Sync() error { return c.Core.Sync() } func (c *core) allLabels() *labels { lbls := newLabels() lbls.mutex.Lock() c.permLabels.mutex.RLock() for k, v := range c.permLabels.store { lbls.store[k] = v } c.permLabels.mutex.RUnlock() c.tempLabels.mutex.RLock() for k, v := range c.tempLabels.store { lbls.store[k] = v } c.tempLabels.mutex.RUnlock() lbls.mutex.Unlock() return lbls } func (c *core) extractLabels(fields []zapcore.Field) (*labels, []zapcore.Field) { lbls := newLabels() out := []zapcore.Field{} lbls.mutex.Lock() for i := range fields { if !isLabelField(fields[i]) { out = append(out, fields[i]) continue } lbls.store[strings.Replace(fields[i].Key, "labels.", "", 1)] = fields[i].String } lbls.mutex.Unlock() return lbls, out } func (c *core) withLabels(fields []zapcore.Field) []zapcore.Field { lbls := newLabels() out := []zapcore.Field{} lbls.mutex.Lock() for i := range fields { if isLabelField(fields[i]) { lbls.store[strings.Replace(fields[i].Key, "labels.", "", 1)] = fields[i].String continue } out = append(out, fields[i]) } lbls.mutex.Unlock() return append(out, labelsField(lbls)) } func (c *core) withSourceLocation(ent zapcore.Entry, fields []zapcore.Field) []zapcore.Field { // If the source location was manually set, don't overwrite it for i := range fields { if fields[i].Key == sourceKey { return fields } } if !ent.Caller.Defined { return fields } return append(fields, SourceLocation(ent.Caller.PC, ent.Caller.File, ent.Caller.Line, true)) } func (c *core) withServiceContext(name string, fields []zapcore.Field) []zapcore.Field { // If the service context was manually set, don't overwrite it for i := range fields { if fields[i].Key == serviceContextKey { return fields } } return append(fields, ServiceContext(name)) } func (c *core) withErrorReport(ent zapcore.Entry, fields []zapcore.Field) []zapcore.Field { // If the error report was manually set, don't overwrite it for i := range fields { if fields[i].Key == contextKey { return fields } } if !ent.Caller.Defined { return fields } return append(fields, ErrorReport(ent.Caller.PC, ent.Caller.File, ent.Caller.Line, true)) } ================================================ FILE: core_test.go ================================================ package zapdriver import ( "runtime" "strconv" "sync" "sync/atomic" "testing" "github.com/stretchr/testify/require" "github.com/stretchr/testify/assert" "go.uber.org/zap" "go.uber.org/zap/zapcore" "go.uber.org/zap/zaptest/observer" ) func TestWithLabels(t *testing.T) { fields := []zap.Field{ zap.String("hello", "world"), Label("one", "value"), Label("two", "value"), } labels := newLabels() labels.store = map[string]string{"one": "value", "two": "value"} want := []zap.Field{ zap.String("hello", "world"), zap.Object("logging.googleapis.com/labels", labels), } assert.Equal(t, want, (&core{}).withLabels(fields)) } func TestExtractLabels(t *testing.T) { var lbls *labels c := &core{ Core: zapcore.NewNopCore(), permLabels: newLabels(), tempLabels: newLabels(), } fields := []zap.Field{ zap.String("hello", "world"), Label("one", "world"), Label("two", "worlds"), } lbls, fields = c.extractLabels(fields) require.Len(t, lbls.store, 2) lbls.mutex.RLock() assert.Equal(t, "world", lbls.store["one"]) assert.Equal(t, "worlds", lbls.store["two"]) lbls.mutex.RUnlock() require.Len(t, fields, 1) assert.Equal(t, zap.String("hello", "world"), fields[0]) } func TestWithSourceLocation(t *testing.T) { fields := []zap.Field{zap.String("hello", "world")} pc, file, line, ok := runtime.Caller(0) ent := zapcore.Entry{Caller: zapcore.NewEntryCaller(pc, file, line, ok)} want := []zap.Field{ zap.String("hello", "world"), zap.Object(sourceKey, newSource(pc, file, line, ok)), } assert.Equal(t, want, (&core{}).withSourceLocation(ent, fields)) } func TestWithSourceLocation_DoesNotOverwrite(t *testing.T) { fields := []zap.Field{zap.String(sourceKey, "world")} pc, file, line, ok := runtime.Caller(0) ent := zapcore.Entry{Caller: zapcore.NewEntryCaller(pc, file, line, ok)} want := []zap.Field{ zap.String(sourceKey, "world"), } assert.Equal(t, want, (&core{}).withSourceLocation(ent, fields)) } func TestWithSourceLocation_OnlyWhenDefined(t *testing.T) { fields := []zap.Field{zap.String("hello", "world")} pc, file, line, ok := runtime.Caller(0) ent := zapcore.Entry{Caller: zapcore.NewEntryCaller(pc, file, line, ok)} ent.Caller.Defined = false want := []zap.Field{ zap.String("hello", "world"), } assert.Equal(t, want, (&core{}).withSourceLocation(ent, fields)) } func TestWithErrorReport(t *testing.T) { fields := []zap.Field{zap.String("hello", "world")} pc, file, line, ok := runtime.Caller(0) ent := zapcore.Entry{Caller: zapcore.NewEntryCaller(pc, file, line, ok)} want := []zap.Field{ zap.String("hello", "world"), zap.Object(contextKey, newReportContext(pc, file, line, ok)), } assert.Equal(t, want, (&core{}).withErrorReport(ent, fields)) } func TestWithErrorReport_DoesNotOverwrite(t *testing.T) { fields := []zap.Field{zap.String(contextKey, "world")} pc, file, line, ok := runtime.Caller(0) ent := zapcore.Entry{Caller: zapcore.NewEntryCaller(pc, file, line, ok)} want := []zap.Field{ zap.String(contextKey, "world"), } assert.Equal(t, want, (&core{}).withErrorReport(ent, fields)) } func TestWithErrorReport_OnlyWhenDefined(t *testing.T) { fields := []zap.Field{zap.String("hello", "world")} pc, file, line, ok := runtime.Caller(0) ent := zapcore.Entry{Caller: zapcore.NewEntryCaller(pc, file, line, ok)} ent.Caller.Defined = false want := []zap.Field{ zap.String("hello", "world"), } assert.Equal(t, want, (&core{}).withErrorReport(ent, fields)) } func TestWithServiceContext(t *testing.T) { fields := []zap.Field{zap.String("hello", "world")} want := []zap.Field{ zap.String("hello", "world"), zap.Object(serviceContextKey, newServiceContext("test service")), } assert.Equal(t, want, (&core{}).withServiceContext("test service", fields)) } func TestWithServiceContext_DoesNotOverwrite(t *testing.T) { fields := []zap.Field{zap.String(serviceContextKey, "world")} want := []zap.Field{ zap.String(serviceContextKey, "world"), } assert.Equal(t, want, (&core{}).withServiceContext("test service", fields)) } func TestWrite(t *testing.T) { temp := newLabels() temp.store = map[string]string{"one": "1", "two": "2"} debugcore, logs := observer.New(zapcore.DebugLevel) core := &core{ Core: debugcore, permLabels: newLabels(), tempLabels: temp, } fields := []zap.Field{ zap.String("hello", "world"), Label("one", "value"), Label("two", "value"), } err := core.Write(zapcore.Entry{}, fields) require.NoError(t, err) assert.NotNil(t, logs.All()[0].ContextMap()[labelsKey]) } func TestWriteConcurrent(t *testing.T) { temp := newLabels() temp.store = map[string]string{"one": "1", "two": "2"} goRoutines := 8 counter := int32(10000) debugcore, logs := observer.New(zapcore.DebugLevel) core := &core{ Core: debugcore, permLabels: newLabels(), tempLabels: temp, } fields := []zap.Field{ zap.String("hello", "world"), Label("one", "value"), Label("two", "value"), } var wg sync.WaitGroup wg.Add(goRoutines) for i := 0; i < goRoutines; i++ { go func() { defer wg.Done() for atomic.AddInt32(&counter, -1) > 0 { err := core.Write(zapcore.Entry{}, fields) require.NoError(t, err) } }() } wg.Wait() assert.NotNil(t, logs.All()[0].ContextMap()[labelsKey]) } func TestWithAndWrite(t *testing.T) { debugcore, logs := observer.New(zapcore.DebugLevel) core := zapcore.Core(&core{ Core: debugcore, permLabels: newLabels(), tempLabels: newLabels(), }) core = core.With([]zapcore.Field{Label("one", "world")}) err := core.Write(zapcore.Entry{}, []zapcore.Field{Label("two", "worlds")}) require.NoError(t, err) labels := logs.All()[0].ContextMap()[labelsKey].(map[string]interface{}) assert.Equal(t, "world", labels["one"]) assert.Equal(t, "worlds", labels["two"]) } func TestWithAndWrite_MultipleEntries(t *testing.T) { debugcore, logs := observer.New(zapcore.DebugLevel) core := zapcore.Core(&core{ Core: debugcore, permLabels: newLabels(), tempLabels: newLabels(), }) core = core.With([]zapcore.Field{Label("one", "world")}) err := core.Write(zapcore.Entry{}, []zapcore.Field{Label("two", "worlds")}) require.NoError(t, err) labels := logs.All()[0].ContextMap()[labelsKey].(map[string]interface{}) require.Len(t, labels, 2) assert.Equal(t, "world", labels["one"]) assert.Equal(t, "worlds", labels["two"]) err = core.Write(zapcore.Entry{}, []zapcore.Field{Label("three", "worlds")}) require.NoError(t, err) labels = logs.All()[1].ContextMap()[labelsKey].(map[string]interface{}) require.Len(t, labels, 2) assert.Equal(t, "world", labels["one"]) assert.Equal(t, "worlds", labels["three"]) } func TestWriteReportAllErrors(t *testing.T) { debugcore, logs := observer.New(zapcore.DebugLevel) core := zapcore.Core(&core{ Core: debugcore, permLabels: newLabels(), tempLabels: newLabels(), config: driverConfig{ ReportAllErrors: true, }, }) pc, file, line, ok := runtime.Caller(0) // core.With should return with correct config core = core.With([]zapcore.Field{Label("one", "world")}) err := core.Write(zapcore.Entry{ Level: zapcore.ErrorLevel, Caller: zapcore.NewEntryCaller(pc, file, line, ok), }, []zapcore.Field{Label("two", "worlds")}) require.NoError(t, err) context := logs.All()[0].ContextMap()[contextKey].(map[string]interface{}) rLocation := context["reportLocation"].(map[string]interface{}) assert.Contains(t, rLocation["filePath"], "zapdriver/core_test.go") assert.Equal(t, strconv.Itoa(line), rLocation["lineNumber"]) assert.Contains(t, rLocation["functionName"], "zapdriver.TestWriteReportAllErrors") // Assert that a service context was attached even though service name was not set serviceContext := logs.All()[0].ContextMap()[serviceContextKey].(map[string]interface{}) assert.Equal(t, "unknown", serviceContext["service"]) } func TestWriteServiceContext(t *testing.T) { debugcore, logs := observer.New(zapcore.DebugLevel) core := zapcore.Core(&core{ Core: debugcore, permLabels: newLabels(), tempLabels: newLabels(), config: driverConfig{ ServiceName: "test service", }, }) err := core.Write(zapcore.Entry{}, []zapcore.Field{}) require.NoError(t, err) // Assert that a service context was attached even though service name was not set serviceContext := logs.All()[0].ContextMap()[serviceContextKey].(map[string]interface{}) assert.Equal(t, "test service", serviceContext["service"]) } func TestWriteReportAllErrors_WithServiceContext(t *testing.T) { debugcore, logs := observer.New(zapcore.DebugLevel) core := zapcore.Core(&core{ Core: debugcore, permLabels: newLabels(), tempLabels: newLabels(), config: driverConfig{ ReportAllErrors: true, ServiceName: "test service", }, }) pc, file, line, ok := runtime.Caller(0) err := core.Write(zapcore.Entry{ Level: zapcore.ErrorLevel, Caller: zapcore.NewEntryCaller(pc, file, line, ok), }, []zapcore.Field{}) require.NoError(t, err) assert.Contains(t, logs.All()[0].ContextMap(), contextKey) // Assert that a service context was attached even though service name was not set serviceContext := logs.All()[0].ContextMap()[serviceContextKey].(map[string]interface{}) assert.Equal(t, "test service", serviceContext["service"]) } func TestWriteReportAllErrors_InfoLog(t *testing.T) { debugcore, logs := observer.New(zapcore.DebugLevel) core := zapcore.Core(&core{ Core: debugcore, permLabels: newLabels(), tempLabels: newLabels(), config: driverConfig{ ReportAllErrors: true, }, }) pc, file, line, ok := runtime.Caller(0) err := core.Write(zapcore.Entry{ Level: zapcore.InfoLevel, Caller: zapcore.NewEntryCaller(pc, file, line, ok), }, []zapcore.Field{}) require.NoError(t, err) assert.NotContains(t, logs.All()[0].ContextMap(), contextKey) assert.NotContains(t, logs.All()[0].ContextMap(), serviceContextKey) } func TestAllLabels(t *testing.T) { perm := newLabels() perm.store = map[string]string{"one": "1", "two": "2", "three": "3"} temp := newLabels() temp.store = map[string]string{"one": "ONE", "three": "THREE"} core := &core{ Core: zapcore.NewNopCore(), permLabels: perm, tempLabels: temp, } out := core.allLabels() require.Len(t, out.store, 3) out.mutex.RLock() assert.Equal(t, out.store["one"], "ONE") assert.Equal(t, out.store["two"], "2") assert.Equal(t, out.store["three"], "THREE") out.mutex.RUnlock() } ================================================ FILE: encoder.go ================================================ package zapdriver import ( "time" "go.uber.org/zap/zapcore" ) // logLevelSeverity maps the Zap log levels to the correct level names as // defined by Stackdriver. // // DEFAULT (0) The log entry has no assigned severity level. // DEBUG (100) Debug or trace information. // INFO (200) Routine information, such as ongoing status or performance. // NOTICE (300) Normal but significant events, such as start up, shut down, or a configuration change. // WARNING (400) Warning events might cause problems. // ERROR (500) Error events are likely to cause problems. // CRITICAL (600) Critical events cause more severe problems or outages. // ALERT (700) A person must take an action immediately. // EMERGENCY (800) One or more systems are unusable. // // See: https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry#LogSeverity var logLevelSeverity = map[zapcore.Level]string{ zapcore.DebugLevel: "DEBUG", zapcore.InfoLevel: "INFO", zapcore.WarnLevel: "WARNING", zapcore.ErrorLevel: "ERROR", zapcore.DPanicLevel: "CRITICAL", zapcore.PanicLevel: "ALERT", zapcore.FatalLevel: "EMERGENCY", } // encoderConfig is the default encoder configuration, slightly tweaked to use // the correct fields for Stackdriver to parse them. var encoderConfig = zapcore.EncoderConfig{ TimeKey: "timestamp", LevelKey: "severity", NameKey: "logger", CallerKey: "caller", MessageKey: "message", StacktraceKey: "stacktrace", LineEnding: zapcore.DefaultLineEnding, EncodeLevel: EncodeLevel, EncodeTime: RFC3339NanoTimeEncoder, EncodeDuration: zapcore.SecondsDurationEncoder, EncodeCaller: zapcore.ShortCallerEncoder, } // EncodeLevel maps the internal Zap log level to the appropriate Stackdriver // level. func EncodeLevel(l zapcore.Level, enc zapcore.PrimitiveArrayEncoder) { enc.AppendString(logLevelSeverity[l]) } // RFC3339NanoTimeEncoder serializes a time.Time to an RFC3339Nano-formatted // string with nanoseconds precision. func RFC3339NanoTimeEncoder(t time.Time, enc zapcore.PrimitiveArrayEncoder) { enc.AppendString(t.Format(time.RFC3339Nano)) } ================================================ FILE: encoder_test.go ================================================ package zapdriver_test import ( "testing" "time" "github.com/blendle/zapdriver" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" "go.uber.org/zap/zapcore" ) func TestEncodeLevel(t *testing.T) { t.Parallel() var tests = []struct { lvl zapcore.Level want string }{ {zapcore.DebugLevel, "DEBUG"}, {zapcore.InfoLevel, "INFO"}, {zapcore.WarnLevel, "WARNING"}, {zapcore.ErrorLevel, "ERROR"}, {zapcore.DPanicLevel, "CRITICAL"}, {zapcore.PanicLevel, "ALERT"}, {zapcore.FatalLevel, "EMERGENCY"}, } for _, tt := range tests { t.Run(tt.want, func(t *testing.T) { enc := &sliceArrayEncoder{} zapdriver.EncodeLevel(tt.lvl, enc) require.Len(t, enc.elems, 1) assert.Equal(t, enc.elems[0].(string), tt.want) }) } } func TestRFC3339NanoTimeEncoder(t *testing.T) { t.Parallel() ts := time.Date(2018, 4, 9, 12, 43, 12, 678359, time.UTC) enc := &sliceArrayEncoder{} zapdriver.RFC3339NanoTimeEncoder(ts, enc) require.Len(t, enc.elems, 1) assert.Equal(t, ts.Format(time.RFC3339Nano), enc.elems[0].(string)) } ================================================ FILE: go.mod ================================================ module github.com/blendle/zapdriver go 1.13 require ( github.com/davecgh/go-spew v1.1.1 // indirect github.com/pkg/errors v0.8.1 // indirect github.com/stretchr/testify v1.3.0 go.uber.org/atomic v1.4.0 // indirect go.uber.org/multierr v1.1.0 // indirect go.uber.org/zap v1.10.0 ) ================================================ FILE: go.sum ================================================ github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/pkg/errors v0.8.1 h1:iURUrRGxPUNPdy5/HRSm+Yj6okJ6UtLINN0Q9M4+h3I= github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/testify v1.3.0 h1:TivCn/peBQ7UY8ooIcPgZFpTNSz0Q2U6UrFlUfqbe0Q= github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= go.uber.org/atomic v1.4.0 h1:cxzIVoETapQEqDhQu3QfnvXAV4AlzcvUCxkVUFw3+EU= go.uber.org/atomic v1.4.0/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE= go.uber.org/multierr v1.1.0 h1:HoEmRHQPVSqub6w2z2d2EOVs2fjyFRGyofhKuyDq0QI= go.uber.org/multierr v1.1.0/go.mod h1:wR5kodmAFQ0UK8QlbwjlSNy0Z68gJhDJUG5sjR94q/0= go.uber.org/zap v1.10.0 h1:ORx85nbTijNz8ljznvCMR1ZBIPKFn3jQrag10X2AsuM= go.uber.org/zap v1.10.0/go.mod h1:vwi/ZaCAaUcBkycHslxD9B2zi4UTXhF60s6SWpuDF0Q= ================================================ FILE: http.go ================================================ package zapdriver // "Broker: Request timed out" // https://console.cloud.google.com/logs/viewer?project=bnl-blendle&minLogLevel= // 0&expandAll=false×tamp=2018-05-23T22:21:56.142000000Z&customFacets=&limi // tCustomFacetWidth=true&dateRangeEnd=2018-05-23T22:21:52.545Z&interval=PT1H&re // source=container%2Fcluster_name%2Fblendle-2%2Fnamespace_id%2Fstream- // composition-analytic-events- // backfill&scrollTimestamp=2018-05-23T05:29:33.000000000Z&logName=projects // %2Fbnl-blendle%2Flogs%2Fstream-composition-analytic-events- // pipe-1&dateRangeUnbound=backwardInTime import ( "bytes" "io" "net/http" "strconv" "go.uber.org/zap" "go.uber.org/zap/zapcore" ) // HTTP adds the correct Stackdriver "HTTP" field. // // see: https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry#HttpRequest func HTTP(req *HTTPPayload) zap.Field { return zap.Object("httpRequest", req) } // HTTPPayload is the complete payload that can be interpreted by // Stackdriver as a HTTP request. type HTTPPayload struct { // The request method. Examples: "GET", "HEAD", "PUT", "POST". RequestMethod string `json:"requestMethod"` // The scheme (http, https), the host name, the path and the query portion of // the URL that was requested. // // Example: "http://example.com/some/info?color=red". RequestURL string `json:"requestUrl"` // The size of the HTTP request message in bytes, including the request // headers and the request body. RequestSize string `json:"requestSize"` // The response code indicating the status of response. // // Examples: 200, 404. Status int `json:"status"` // The size of the HTTP response message sent back to the client, in bytes, // including the response headers and the response body. ResponseSize string `json:"responseSize"` // The user agent sent by the client. // // Example: "Mozilla/4.0 (compatible; MSIE 6.0; Windows 98; Q312461; .NET CLR 1.0.3705)". UserAgent string `json:"userAgent"` // The IP address (IPv4 or IPv6) of the client that issued the HTTP request. // // Examples: "192.168.1.1", "FE80::0202:B3FF:FE1E:8329". RemoteIP string `json:"remoteIp"` // The IP address (IPv4 or IPv6) of the origin server that the request was // sent to. ServerIP string `json:"serverIp"` // The referrer URL of the request, as defined in HTTP/1.1 Header Field // Definitions. Referer string `json:"referer"` // The request processing latency on the server, from the time the request was // received until the response was sent. // // A duration in seconds with up to nine fractional digits, terminated by 's'. // // Example: "3.5s". Latency string `json:"latency"` // Whether or not a cache lookup was attempted. CacheLookup bool `json:"cacheLookup"` // Whether or not an entity was served from cache (with or without // validation). CacheHit bool `json:"cacheHit"` // Whether or not the response was validated with the origin server before // being served from cache. This field is only meaningful if cacheHit is True. CacheValidatedWithOriginServer bool `json:"cacheValidatedWithOriginServer"` // The number of HTTP response bytes inserted into cache. Set only when a // cache fill was attempted. CacheFillBytes string `json:"cacheFillBytes"` // Protocol used for the request. // // Examples: "HTTP/1.1", "HTTP/2", "websocket" Protocol string `json:"protocol"` } // NewHTTP returns a new HTTPPayload struct, based on the passed // in http.Request and http.Response objects. func NewHTTP(req *http.Request, res *http.Response) *HTTPPayload { if req == nil { req = &http.Request{} } if res == nil { res = &http.Response{} } sdreq := &HTTPPayload{ RequestMethod: req.Method, Status: res.StatusCode, UserAgent: req.UserAgent(), RemoteIP: req.RemoteAddr, Referer: req.Referer(), Protocol: req.Proto, } if req.URL != nil { sdreq.RequestURL = req.URL.String() } buf := &bytes.Buffer{} if req.Body != nil { n, _ := io.Copy(buf, req.Body) // nolint: gas sdreq.RequestSize = strconv.FormatInt(n, 10) } if res.Body != nil { buf.Reset() n, _ := io.Copy(buf, res.Body) // nolint: gas sdreq.ResponseSize = strconv.FormatInt(n, 10) } return sdreq } // MarshalLogObject implements zapcore.ObjectMarshaller interface. func (req HTTPPayload) MarshalLogObject(enc zapcore.ObjectEncoder) error { enc.AddString("requestMethod", req.RequestMethod) enc.AddString("requestUrl", req.RequestURL) enc.AddString("requestSize", req.RequestSize) enc.AddInt("status", req.Status) enc.AddString("responseSize", req.ResponseSize) enc.AddString("userAgent", req.UserAgent) enc.AddString("remoteIp", req.RemoteIP) enc.AddString("serverIp", req.ServerIP) enc.AddString("referer", req.Referer) enc.AddString("latency", req.Latency) enc.AddBool("cacheLookup", req.CacheLookup) enc.AddBool("cacheHit", req.CacheHit) enc.AddBool("cacheValidatedWithOriginServer", req.CacheValidatedWithOriginServer) enc.AddString("cacheFillBytes", req.CacheFillBytes) enc.AddString("protocol", req.Protocol) return nil } ================================================ FILE: http_test.go ================================================ package zapdriver_test import ( "io/ioutil" "net/http" "net/http/httptest" "net/url" "strings" "testing" "github.com/blendle/zapdriver" "github.com/stretchr/testify/assert" "go.uber.org/zap" ) func TestHTTP(t *testing.T) { t.Parallel() req := &zapdriver.HTTPPayload{} field := zapdriver.HTTP(req) assert.Equal(t, zap.Object("httpRequest", req), field) } func TestNewHTTP(t *testing.T) { t.Parallel() var tests = map[string]struct { req *http.Request res *http.Response want *zapdriver.HTTPPayload }{ "empty": { nil, nil, &zapdriver.HTTPPayload{}, }, "RequestMethod": { &http.Request{Method: "GET"}, nil, &zapdriver.HTTPPayload{RequestMethod: "GET"}, }, "Status": { nil, &http.Response{StatusCode: 404}, &zapdriver.HTTPPayload{Status: 404}, }, "UserAgent": { &http.Request{Header: http.Header{"User-Agent": []string{"hello world"}}}, nil, &zapdriver.HTTPPayload{UserAgent: "hello world"}, }, "RemoteIP": { &http.Request{RemoteAddr: "127.0.0.1"}, nil, &zapdriver.HTTPPayload{RemoteIP: "127.0.0.1"}, }, "Referrer": { &http.Request{Header: http.Header{"Referer": []string{"hello universe"}}}, nil, &zapdriver.HTTPPayload{Referer: "hello universe"}, }, "Protocol": { &http.Request{Proto: "HTTP/1.1"}, nil, &zapdriver.HTTPPayload{Protocol: "HTTP/1.1"}, }, "RequestURL": { &http.Request{URL: &url.URL{Host: "example.com", Scheme: "https"}}, nil, &zapdriver.HTTPPayload{RequestURL: "https://example.com"}, }, "RequestSize": { &http.Request{Body: ioutil.NopCloser(strings.NewReader("12345"))}, nil, &zapdriver.HTTPPayload{RequestSize: "5"}, }, "ResponseSize": { nil, &http.Response{Body: ioutil.NopCloser(strings.NewReader("12345"))}, &zapdriver.HTTPPayload{ResponseSize: "5"}, }, "simple request": { httptest.NewRequest("POST", "/", strings.NewReader("12345")), nil, &zapdriver.HTTPPayload{ RequestSize: "5", RequestMethod: "POST", RemoteIP: "192.0.2.1:1234", Protocol: "HTTP/1.1", RequestURL: "/", }, }, "simple response": { nil, &http.Response{Body: ioutil.NopCloser(strings.NewReader("12345")), StatusCode: 404}, &zapdriver.HTTPPayload{ResponseSize: "5", Status: 404}, }, "request & response": { &http.Request{Method: "POST", Proto: "HTTP/1.1"}, &http.Response{StatusCode: 200}, &zapdriver.HTTPPayload{RequestMethod: "POST", Protocol: "HTTP/1.1", Status: 200}, }, } for name, tt := range tests { t.Run(name, func(t *testing.T) { assert.Equal(t, tt.want, zapdriver.NewHTTP(tt.req, tt.res)) }) } } ================================================ FILE: label.go ================================================ package zapdriver import ( "strings" "sync" "go.uber.org/zap" "go.uber.org/zap/zapcore" ) const labelsKey = "logging.googleapis.com/labels" // Label adds an optional label to the payload. // // Labels are a set of user-defined (key, value) data that provides additional // information about the log entry. // // Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }. func Label(key, value string) zap.Field { return zap.String("labels."+key, value) } // Labels takes Zap fields, filters the ones that have their key start with the // string `labels.` and their value type set to StringType. It then wraps those // key/value pairs in a top-level `labels` namespace. func Labels(fields ...zap.Field) zap.Field { lbls := newLabels() lbls.mutex.Lock() for i := range fields { if isLabelField(fields[i]) { lbls.store[strings.Replace(fields[i].Key, "labels.", "", 1)] = fields[i].String } } lbls.mutex.Unlock() return labelsField(lbls) } func isLabelField(field zap.Field) bool { return strings.HasPrefix(field.Key, "labels.") && field.Type == zapcore.StringType } func labelsField(l *labels) zap.Field { return zap.Object(labelsKey, l) } type labels struct { store map[string]string mutex *sync.RWMutex } func newLabels() *labels { return &labels{store: map[string]string{}, mutex: &sync.RWMutex{}} } func (l *labels) Add(key, value string) { l.mutex.Lock() l.store[key] = value l.mutex.Unlock() } func (l *labels) reset() { l.mutex.Lock() l.store = map[string]string{} l.mutex.Unlock() } func (l labels) MarshalLogObject(enc zapcore.ObjectEncoder) error { l.mutex.RLock() for k, v := range l.store { enc.AddString(k, v) } l.mutex.RUnlock() return nil } ================================================ FILE: label_test.go ================================================ package zapdriver import ( "testing" "github.com/stretchr/testify/assert" "go.uber.org/zap" ) func TestLabel(t *testing.T) { t.Parallel() field := Label("key", "value") assert.Equal(t, zap.String("labels.key", "value"), field) } func TestLabels(t *testing.T) { t.Parallel() field := Labels( Label("hello", "world"), Label("hi", "universe"), ) labels := newLabels() labels.store = map[string]string{"hello": "world", "hi": "universe"} assert.Equal(t, zap.Object(labelsKey, labels), field) } ================================================ FILE: logger.go ================================================ package zapdriver import ( "go.uber.org/zap" ) // NewProduction builds a sensible production Logger that writes InfoLevel and // above logs to standard error as JSON. // // It's a shortcut for NewProductionConfig().Build(...Option). func NewProduction(options ...zap.Option) (*zap.Logger, error) { options = append(options, WrapCore()) return NewProductionConfig().Build(options...) } // NewProductionWithCore is same as NewProduction but accepts a custom configured core func NewProductionWithCore(core zap.Option, options ...zap.Option) (*zap.Logger, error) { options = append(options, core) return NewProductionConfig().Build(options...) } // NewDevelopment builds a development Logger that writes DebugLevel and above // logs to standard error in a human-friendly format. // // It's a shortcut for NewDevelopmentConfig().Build(...Option). func NewDevelopment(options ...zap.Option) (*zap.Logger, error) { options = append(options, WrapCore()) return NewDevelopmentConfig().Build(options...) } // NewDevelopmentWithCore is same as NewDevelopment but accepts a custom configured core func NewDevelopmentWithCore(core zap.Option, options ...zap.Option) (*zap.Logger, error) { options = append(options, core) return NewDevelopmentConfig().Build(options...) } ================================================ FILE: logger_test.go ================================================ package zapdriver import ( "testing" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" "go.uber.org/zap" ) func TestNewProduction(t *testing.T) { logger, err := NewProduction(zap.Fields(zap.String("hello", "world"))) require.NoError(t, err) assert.IsType(t, &zap.Logger{}, logger) } func TestNewProductionWithCore(t *testing.T) { logger, err := NewProductionWithCore( WrapCore(ReportAllErrors(true)), zap.Fields(zap.String("hello", "world")), ) require.NoError(t, err) assert.IsType(t, &zap.Logger{}, logger) } func TestNewDevelopment(t *testing.T) { logger, err := NewDevelopment(zap.Fields(zap.String("hello", "world"))) require.NoError(t, err) assert.IsType(t, &zap.Logger{}, logger) } func TestNewDevelopmentWithCore(t *testing.T) { logger, err := NewDevelopmentWithCore( WrapCore(ReportAllErrors(true)), zap.Fields(zap.String("hello", "world")), ) require.NoError(t, err) assert.IsType(t, &zap.Logger{}, logger) } ================================================ FILE: operation.go ================================================ package zapdriver import ( "go.uber.org/zap" "go.uber.org/zap/zapcore" ) const operationKey = "logging.googleapis.com/operation" // Operation adds the correct Stackdriver "operation" field. // // Additional information about a potentially long-running operation with which // a log entry is associated. // // see: https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry#LogEntryOperation func Operation(id, producer string, first, last bool) zap.Field { op := &operation{ ID: id, Producer: producer, First: first, Last: last, } return zap.Object(operationKey, op) } // OperationStart is a convenience function for `Operation`. It should be called // for the first operation log. func OperationStart(id, producer string) zap.Field { return Operation(id, producer, true, false) } // OperationCont is a convenience function for `Operation`. It should be called // for any non-start/end operation log. func OperationCont(id, producer string) zap.Field { return Operation(id, producer, false, false) } // OperationEnd is a convenience function for `Operation`. It should be called // for the last operation log. func OperationEnd(id, producer string) zap.Field { return Operation(id, producer, false, true) } // operation is the complete payload that can be interpreted by Stackdriver as // an operation. type operation struct { // Optional. An arbitrary operation identifier. Log entries with the same // identifier are assumed to be part of the same operation. ID string `json:"id"` // Optional. An arbitrary producer identifier. The combination of id and // producer must be globally unique. Examples for producer: // "MyDivision.MyBigCompany.com", "github.com/MyProject/MyApplication". Producer string `json:"producer"` // Optional. Set this to True if this is the first log entry in the operation. First bool `json:"first"` // Optional. Set this to True if this is the last log entry in the operation. Last bool `json:"last"` } // MarshalLogObject implements zapcore.ObjectMarshaller interface. func (op operation) MarshalLogObject(enc zapcore.ObjectEncoder) error { enc.AddString("id", op.ID) enc.AddString("producer", op.Producer) enc.AddBool("first", op.First) enc.AddBool("last", op.Last) return nil } ================================================ FILE: operation_test.go ================================================ package zapdriver import ( "testing" "github.com/stretchr/testify/assert" "go.uber.org/zap" ) func TestOperation(t *testing.T) { t.Parallel() op := &operation{ID: "id", Producer: "producer", First: true, Last: false} field := Operation("id", "producer", true, false) assert.Equal(t, zap.Object(operationKey, op), field) } func TestOperationStart(t *testing.T) { t.Parallel() op := &operation{ID: "id", Producer: "producer", First: true, Last: false} field := OperationStart("id", "producer") assert.Equal(t, zap.Object(operationKey, op), field) } func TestOperationCont(t *testing.T) { t.Parallel() op := &operation{ID: "id", Producer: "producer", First: false, Last: false} field := OperationCont("id", "producer") assert.Equal(t, zap.Object(operationKey, op), field) } func TestOperationEnd(t *testing.T) { t.Parallel() op := &operation{ID: "id", Producer: "producer", First: false, Last: true} field := OperationEnd("id", "producer") assert.Equal(t, zap.Object(operationKey, op), field) } ================================================ FILE: report.go ================================================ package zapdriver import ( "runtime" "strconv" "go.uber.org/zap" "go.uber.org/zap/zapcore" ) const contextKey = "context" // ErrorReport adds the correct Stackdriver "context" field for getting the log line // reported as error. // // see: https://cloud.google.com/error-reporting/docs/formatting-error-messages func ErrorReport(pc uintptr, file string, line int, ok bool) zap.Field { return zap.Object(contextKey, newReportContext(pc, file, line, ok)) } // reportLocation is the source code location information associated with the log entry // for the purpose of reporting an error, // if any. type reportLocation struct { File string `json:"filePath"` Line string `json:"lineNumber"` Function string `json:"functionName"` } // MarshalLogObject implements zapcore.ObjectMarshaller interface. func (location reportLocation) MarshalLogObject(enc zapcore.ObjectEncoder) error { enc.AddString("filePath", location.File) enc.AddString("lineNumber", location.Line) enc.AddString("functionName", location.Function) return nil } // reportContext is the context information attached to a log for reporting errors type reportContext struct { ReportLocation reportLocation `json:"reportLocation"` } // MarshalLogObject implements zapcore.ObjectMarshaller interface. func (context reportContext) MarshalLogObject(enc zapcore.ObjectEncoder) error { enc.AddObject("reportLocation", context.ReportLocation) return nil } func newReportContext(pc uintptr, file string, line int, ok bool) *reportContext { if !ok { return nil } var function string if fn := runtime.FuncForPC(pc); fn != nil { function = fn.Name() } context := &reportContext{ ReportLocation: reportLocation{ File: file, Line: strconv.Itoa(line), Function: function, }, } return context } ================================================ FILE: report_test.go ================================================ package zapdriver import ( "runtime" "testing" "github.com/stretchr/testify/assert" ) func TestErrorReport(t *testing.T) { t.Parallel() got := ErrorReport(runtime.Caller(0)).Interface.(*reportContext) assert.Contains(t, got.ReportLocation.File, "zapdriver/report_test.go") assert.Equal(t, "13", got.ReportLocation.Line) assert.Contains(t, got.ReportLocation.Function, "zapdriver.TestErrorReport") } func TestNewReportContext(t *testing.T) { t.Parallel() got := newReportContext(runtime.Caller(0)) assert.Contains(t, got.ReportLocation.File, "zapdriver/report_test.go") assert.Equal(t, "23", got.ReportLocation.Line) assert.Contains(t, got.ReportLocation.Function, "zapdriver.TestNewReportContext") } ================================================ FILE: service.go ================================================ package zapdriver import ( "go.uber.org/zap" "go.uber.org/zap/zapcore" ) const serviceContextKey = "serviceContext" // ServiceContext adds the correct service information adding the log line // It is a required field if an error needs to be reported. // // see: https://cloud.google.com/error-reporting/reference/rest/v1beta1/ServiceContext // see: https://cloud.google.com/error-reporting/docs/formatting-error-messages func ServiceContext(name string) zap.Field { return zap.Object(serviceContextKey, newServiceContext(name)) } // serviceContext describes a running service that sends errors. // Currently it only describes a service name. type serviceContext struct { Name string `json:"service"` } // MarshalLogObject implements zapcore.ObjectMarshaller interface. func (service_context serviceContext) MarshalLogObject(enc zapcore.ObjectEncoder) error { enc.AddString("service", service_context.Name) return nil } func newServiceContext(name string) *serviceContext { return &serviceContext{ Name: name, } } ================================================ FILE: service_test.go ================================================ package zapdriver import ( "testing" "github.com/stretchr/testify/assert" ) func TestServiceContext(t *testing.T) { t.Parallel() got := ServiceContext("test service name").Interface.(*serviceContext) assert.Equal(t, "test service name", got.Name) } func TestNewServiceContext(t *testing.T) { t.Parallel() got := newServiceContext("test service name") assert.Equal(t, "test service name", got.Name) } ================================================ FILE: source.go ================================================ package zapdriver import ( "runtime" "strconv" "go.uber.org/zap" "go.uber.org/zap/zapcore" ) const sourceKey = "logging.googleapis.com/sourceLocation" // SourceLocation adds the correct Stackdriver "SourceLocation" field. // // see: https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry#LogEntrySourceLocation func SourceLocation(pc uintptr, file string, line int, ok bool) zap.Field { return zap.Object(sourceKey, newSource(pc, file, line, ok)) } // source is the source code location information associated with the log entry, // if any. type source struct { // Optional. Source file name. Depending on the runtime environment, this // might be a simple name or a fully-qualified name. File string `json:"file"` // Optional. Line within the source file. 1-based; 0 indicates no line number // available. Line string `json:"line"` // Optional. Human-readable name of the function or method being invoked, with // optional context such as the class or package name. This information may be // used in contexts such as the logs viewer, where a file and line number are // less meaningful. // // The format should be dir/package.func. Function string `json:"function"` } // MarshalLogObject implements zapcore.ObjectMarshaller interface. func (source source) MarshalLogObject(enc zapcore.ObjectEncoder) error { enc.AddString("file", source.File) enc.AddString("line", source.Line) enc.AddString("function", source.Function) return nil } func newSource(pc uintptr, file string, line int, ok bool) *source { if !ok { return nil } var function string if fn := runtime.FuncForPC(pc); fn != nil { function = fn.Name() } source := &source{ File: file, Line: strconv.Itoa(line), Function: function, } return source } ================================================ FILE: source_test.go ================================================ package zapdriver import ( "runtime" "testing" "github.com/stretchr/testify/assert" ) func TestSourceLocation(t *testing.T) { t.Parallel() got := SourceLocation(runtime.Caller(0)).Interface.(*source) assert.Contains(t, got.File, "zapdriver/source_test.go") assert.Equal(t, "13", got.Line) assert.Contains(t, got.Function, "zapdriver.TestSourceLocation") } func TestNewSource(t *testing.T) { t.Parallel() got := newSource(runtime.Caller(0)) assert.Contains(t, got.File, "zapdriver/source_test.go") assert.Equal(t, "23", got.Line) assert.Contains(t, got.Function, "zapdriver.TestNewSource") } ================================================ FILE: trace.go ================================================ package zapdriver import ( "fmt" "go.uber.org/zap" ) const ( traceKey = "logging.googleapis.com/trace" spanKey = "logging.googleapis.com/spanId" traceSampledKey = "logging.googleapis.com/trace_sampled" ) // TraceContext adds the correct Stackdriver "trace", "span", "trace_sampled fields // // see: https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry func TraceContext(trace string, spanId string, sampled bool, projectName string) []zap.Field { return []zap.Field{ zap.String(traceKey, fmt.Sprintf("projects/%s/traces/%s", projectName, trace)), zap.String(spanKey, spanId), zap.Bool(traceSampledKey, sampled), } } ================================================ FILE: trace_test.go ================================================ package zapdriver import ( "testing" "github.com/stretchr/testify/assert" "go.uber.org/zap" ) func TestTraceContext(t *testing.T) { t.Parallel() fields := TraceContext("105445aa7843bc8bf206b120001000", "0", true, "my-project-name") assert.Equal(t, fields, []zap.Field{ zap.String(traceKey, "projects/my-project-name/traces/105445aa7843bc8bf206b120001000"), zap.String(spanKey, "0"), zap.Bool(traceSampledKey, true), }) }