Getting Started

Installation

You can download and install msgp using the standard go toolchain.

$ go get -u -d -t github.com/tinylib/msgp
$ cd $GOPATH/src/github.com/tinylib/msgp
$ go install ./...

Serialization Strategy: Tool + Library

msgp has two parts: a tool that generates code, and a library that is used by the generated code.

The primary difference between msgp and other serialization libraries for go (such as those found in the standard library) is that msgp doesn't perform runtime reflection. Instead, the msgp tool reads .go source files and generates code that binds methods to your existing type declarations.

Hello World

First, create a new directory in your GOPATH, and create main.go.

$ mkdir -p $GOPATH/src/msgp-demo
$ cd $GOPATH/src/msgp-demo
$ touch main.go

Then open main.go in your editor of choice and add the following:

package main

import (
    "fmt"
)

//go:generate msgp

type Foo struct {
    Bar string  `msg:"bar"`
    Baz float64 `msg:"baz'`
}

func main() {
    fmt.Println("Nothing to see here yet!")
}

(You can verify that this builds and runs with $ go build && ./msgp-demo.)

Now let's bind some methods to Foo by running go generate:

$ go generate
======== MessagePack Code Generator =======
>>> Input: "main.go"...
parsing Foo...
>>> Wrote and formatted "main_gen_test.go"
>>> Wrote and formatted "main_gen.go"
>>> Done.
$ ls
main.go			main_gen.go		main_gen_test.go
$ go test -v -bench .
=== RUN TestFooMarshalUnmarshal
--- PASS: TestFooMarshalUnmarshal (0.00s)
=== RUN TestFooEncodeDecode
--- PASS: TestFooEncodeDecode (0.00s)
PASS
BenchmarkFooMarshalMsg	10000000	       179 ns/op	      32 B/op	       1 allocs/op
BenchmarkFooAppendMsg	30000000	        54.8 ns/op	 346.67 MB/s	       0 B/op	       0 allocs/op
BenchmarkFooUnmarshal	20000000	       116 ns/op	 163.26 MB/s	       0 B/op	       0 allocs/op
BenchmarkFooEncode	20000000	        66.5 ns/op	 285.54 MB/s	       0 B/op	       0 allocs/op
BenchmarkFooDecode	10000000	       131 ns/op	 143.98 MB/s	       0 B/op	       0 allocs/op
ok  	msgp-demo	9.460s

Let's break down what happened here:

• go generate scanned each file in msgp-demo for a go:generate directive.
• //go:generate msgp was found in main.go, which caused $GOFILE to be set to main.go
• msgp was invoked by go generate, and it parsed $GOFILE and extracted type declarations.
• msgp created main_gen.go, which contains all of the generated methods, and main_gen_test.go, which has tests and benchmarks for each generated method.

The key takeaway here is that msgp works on a per-file, not a per-package basis. (You can, however, invoke the code generator on an entire directory at once by passing a directory path using the -file flag.) Our suggestion is that users put types requiring code generation in their own file (say, wiretypes.go), and put //go:generate msgp at the top. However, other workflows are possible.

Let's look at the generated code in main_gen.go:

package main

// NOTE: THIS FILE WAS PRODUCED BY THE
// MSGP CODE GENERATION TOOL (github.com/tinylib/msgp)
// DO NOT EDIT

import (
	"github.com/tinylib/msgp/msgp"
)

// DecodeMsg implements msgp.Decodable
func (z *Foo) DecodeMsg(dc *msgp.Reader) (err error) {
	var field []byte
	_ = field
	var isz uint32
	isz, err = dc.ReadMapHeader()
	if err != nil {
		return
	}
	for isz > 0 {
		isz--
		field, err = dc.ReadMapKeyPtr()
		if err != nil {
			return
		}
		switch msgp.UnsafeString(field) {
		case "bar":
			z.Bar, err = dc.ReadString()
			if err != nil {
				return
			}
		case "baz":
			z.Baz, err = dc.ReadFloat64()
			if err != nil {
				return
			}
		default:
			err = dc.Skip()
			if err != nil {
				return
			}
		}
	}
	return
}

// EncodeMsg implements msgp.Encodable
func (z Foo) EncodeMsg(en *msgp.Writer) (err error) {
	// map header, size 2
	err = en.Append(0x82)
	if err != nil {
		return err
	}
	// write "bar"
	err = en.Append(0xa3, 0x62, 0x61, 0x72)
	if err != nil {
		return err
	}
	err = en.WriteString(z.Bar)
	if err != nil {
		return
	}
	// write "baz"
	err = en.Append(0xa3, 0x62, 0x61, 0x7a)
	if err != nil {
		return err
	}
	err = en.WriteFloat64(z.Baz)
	if err != nil {
		return
	}
	return
}

// MarshalMsg implements msgp.Marshaler
func (z Foo) MarshalMsg(b []byte) (o []byte, err error) {
	o = msgp.Require(b, z.Msgsize())
	// map header, size 2
	o = append(o, 0x82)
	// string "bar"
	o = append(o, 0xa3, 0x62, 0x61, 0x72)
	o = msgp.AppendString(o, z.Bar)
	// string "baz"
	o = append(o, 0xa3, 0x62, 0x61, 0x7a)
	o = msgp.AppendFloat64(o, z.Baz)
	return
}

// UnmarshalMsg implements msgp.Unmarshaler
func (z *Foo) UnmarshalMsg(bts []byte) (o []byte, err error) {
	var field []byte
	_ = field
	var isz uint32
	isz, bts, err = msgp.ReadMapHeaderBytes(bts)
	if err != nil {
		return
	}
	for isz > 0 {
		isz--
		field, bts, err = msgp.ReadMapKeyZC(bts)
		if err != nil {
			return
		}
		switch msgp.UnsafeString(field) {
		case "bar":
			z.Bar, bts, err = msgp.ReadStringBytes(bts)
			if err != nil {
				return
			}
		case "baz":
			z.Baz, bts, err = msgp.ReadFloat64Bytes(bts)
			if err != nil {
				return
			}
		default:
			bts, err = msgp.Skip(bts)
			if err != nil {
				return
			}
		}
	}
	o = bts
	return
}

func (z Foo) Msgsize() (s int) {
	s = 1 + 4 + msgp.StringPrefixSize + len(z.Bar) + 4 + msgp.Float64Size
	return
}

There are 5 methods implemented by the code generator:

• MarshalMsg([]byte) ([]byte, error)
• UnmarshalMsg([]byte) ([]byte, error)
• EncodeMsg(*msgp.Writer) error
• DecodeMsg(*msgp.Reader) error
• Msgsize() int

Each of those methods is actually an implementation of an interface defined in the msgp library. In effect, the library at github.com/tinylib/msgp/msgp contains everything we need to encode and decode MessagePack, and the code generator exists simply to write boilerplate code using that library. We could, of course, implement all of these interfaces ourselves, but that would be unnecessarily laborious and error-prone. (Plus, the code generator can perform optimizations like pre-encoding static strings, like the example above. This would be especially cumbersome to write by hand!)