Developers Club geek daily blog

2 years, 11 months ago
Transfer of article of Rob Payk from the official blog Go about an automatic kodogeneration by means of go generate. Article became a little outdated (it was written before an output of Go 1.4 in which there was go generate), but well explains an essence of work of go generate.

One of properties of computability theory — Turing completeness — is that the program can write other program. It is strong idea which is not so evaluated as deserves that, though meets rather often. It is rather powerful part of determination of what is done by compilers, for example. Also, the go test command works by the same principle too: it scans packets which need to be tested, creates new Go the program in which the necessary weather-cloth for tests is added, then compiles and starts it. Modern computers are so fast that such, apparently, expensive sequence of actions is performed for fractions of a second.

Around there is a mass of other examples when programs write programs. Yacc, for example, reads the description of grammar and issues the program which parsit this grammar. "Compiler" Protocol Buffers reads interface description and issues determinations of structures, methods and an other code. Various utilities of a configuration work similarly too, retrieving metadata from the environment of an environment and forming custom teams of start.

Thus, the programs writing programs are an important element in software development, but programs like Yacc which create the source code, have to be integrated into build process that their output could be transferred to the compiler. When the external system of assembly, like Make is used, usually just to make it. But in Go in which the utility of go obtains all necessary information on a bilda from source codes it is a problem. In it just there is no mechanism to start Yacc by means of go tool.

Up to this point, in sense.

The last release of Go, 1.4, includes new command, go generate which allows to start similar utilities. It is called go generate, and at start scans a code on existence of special comments which specify what commands need to be started. It is important to understand that go generate is not part of go build. She does not analyze dependence and has to be started to go build. It is intended for the author of Go of a packet, but not for its users.

The go generate command is very simple in use. For warm-up, here is how to use it to generate Yacc grammar. Let's tell, you have an input Yacc-file which is called by gopher.y which defines grammar of your modern language. To generate a code on Go which will parsit this grammar you usually would start the standard yacc Go-version, somehow so:
go tool yacc -o gopher.go -p parser gopher.y

The option - o enters a name of the resulting file here, and - p — a packet name.

To shift this process to go generate, it is necessary in any normal (not autogenerated) .go the file in this directory to add here such comment:
//go:generate go tool yacc -o gopher.go -p parser gopher.y

This text is the same command, but with the comment added in the beginning which will recognize go generate. The comment has to begin at the beginning of a line and not have spaces between//and go:generate. After this marker, the rest specifies what go generate command has to start.

And now start it. Pass into an initial directory and start go generate, then go build and so on:
$ cd $GOPATH/myrepo/gopher
$ go generate
$ go build
$ go test

And all this that is necessary. If there are no errors, then go generate will cause yacc which will create gopher.go, on this moment the directory will contain all necessary go-files which we can collect, test and normally work with them. Every time when gopher.y changes, just restart go generate to recreate the parser.

If more parts how go generate works inside, including parameters, variable environments and so on are interesting to you, you watch the document with the description of design.

Go generate does not do anything that could not be made by means of Make or other mechanism of assembly, but it goes from a box in the go command — it is not necessary to set anything in addition — and it well fits into Go ecosystem. The main thing, you remember that it for authors of a packet, not for users, at least for reasons of the fact that the program which will be caused can be absent by the user's machine. Also, if the packet is supposed to be used with go get, do not forget to enter the generated files into the monitoring system of versions, having made available to users.

Now, let's look how it is possible to use it for something new. As considerably other example where go generate can help, in a repository of there is a new stringer program. It automatically generates the line String methods () for sets of numerical constants. It does not enter the standard Go set, but it is easy to set it:
$ go get

Here an example from documentation to stringer. Provide that we have some code, with a set of the numerical constants defining different types of drugs:
package painkiller

type Pill int

const (
    Placebo Pill = iota
    Acetaminophen = Paracetamol

For the debug purposes, we would like that these constants could give beautifully the name, in other words, we want a method with the following signature:
func (p Pill) String() string

It is easy to write it manually, for example somehow so:
func (p Pill) String() string {
    switch p {
    case Placebo:
        return "Placebo"
    case Aspirin:
        return "Aspirin"
    case Ibuprofen:
        return "Ibuprofen"
    case Paracetamol: // == Acetaminophen
        return "Paracetamol"
    return fmt.Sprintf("Pill(%d)", p)

There are several methods to write this function, certainly. We can use a slice of lines, indexed on Pill, either map, or some other equipment. Anyway, we have to support its every time when we change a set of drugs, and we have to check that a code correct. (Two different names for paracetamol, for example, do this code to a little more sophisticated, than it could be). Plus, a question of the choice of a method of implementation depends on types of values: sign or unsigned, dense and scattered, starting from scratch or not and so on.

The stringer program undertakes these cares. Though it can be started and manually, but it is intended for start through go generate. To use it, add the comment to the source code, most likely, in a code with determination of type:
//go:generate stringer -type=Pill
This rule specifies that go generate has to start the stringer command to generate the String method for the Pill type. The output will be automatically written in the pill_string.go file (the output can be redefined by means of a flag - output).

Let's start it:
$ go generate
$ cat pill_string.go
// generated by stringer -type Pill pill.go; DO NOT EDIT

package pill

import "fmt"

const _Pill_name = "PlaceboAspirinIbuprofenParacetamol"

var _Pill_index = [...]uint8{0, 7, 14, 23, 34}

func (i Pill) String() string {
    if i < 0 || i+1 >= Pill(len(_Pill_index)) {
        return fmt.Sprintf("Pill(%d)", i)
    return _Pill_name[_Pill_index[i]:_Pill_index[i+1]]

Every time when we change determination of Pill or constants, everything that we have to make, it to start
$ go generate

to update String a method. And of course, if we have several types in one packet which need to be updated, go generate will update all of them.

Needless to say that the generated code is ugly. It is OK, however, as people will not work with this code; the autogenerated code is very often ugly. He tries to be the most effective. All names are united together in one line which saves memory (only one line on all names, even them infinite quantity here). Then the array, _Pill_index, finds compliance of type with a name using simple and very effective equipment. Pay attention that _Pill_index is an array (not a slice; one heading it is less) uint8 values, the naimeyshy possible integral type capable to contain in themselves the necessary values. If it will be more values, or will be negative, the type of the generated array of _Pill_index can exchange on uint16 or int8, will depending on what work better.

The approach used in the methods generated by means of stringer changes, depending on properties of a set of constants. For example, if the constants discharged, he can use map. There is a simple example based on a set of the constants representing powers of two:

const _Power_name = "p0p1p2p3p4p5..."

var _Power_map = map[Power]string{
    1:    _Power_name[0:2],
    2:    _Power_name[2:4],
    4:    _Power_name[4:6],
    8:    _Power_name[6:8],
    16:   _Power_name[8:10],
    32:   _Power_name[10:12],

func (i Power) String() string {
    if str, ok := _Power_map[i]; ok {
        return str
    return fmt.Sprintf("Power(%d)", i)

Summarizing, automatic generation of a method allows us to solve a problem better, than it would be made by the person.

In the source codes Go there is a mass of other examples of use of go generate. Here generation of the tables Unicode in unicode packet, creation of effective methods for coding and decoding of arrays creations of a data set taymzon in time packet enter encoding/gob, and so forth.

Please, use go generate creatively. It here to encourage experiments.

And even if is not present, use stringer to add String methods to your numerical constants. Allow the computer to do work for you.

This article is a translation of the original post at
If you have any questions regarding the material covered in the article above, please, contact the original author of the post.
If you have any complaints about this article or you want this article to be deleted, please, drop an email here:

We believe that the knowledge, which is available at the most popular Russian IT blog, should be accessed by everyone, even though it is poorly translated.
Shared knowledge makes the world better.
Best wishes.

comments powered by Disqus