Essential Steps for your GO CI build pipeline.
Add to bookmarksFri Nov 13 2020
Intro
So, full disclosure; This article has little to do with CI and more to do with go tools. The title was written that way because most of these build tools should be used when setting up a CI pipeline in order to make sure buggy/messy code doesn't make its way into the production source code.
Race Condition Detection.
Before we dive into the available tools, one of the most important parts of your build pipeline as we all know is go test
and go build
. Seeing how go is heavily used for concurrent programming, it can get very easy to have racy code. What is a Race condition?
A race condition is when two or more routines have access to the same resource, such as a variable or data structure and attempt to read and write to that resource without any regard to the other routines. Source: ArdanLabs.
The go compiler accepts a -race
flag that performs a race check accross your code, it can be specified when building with go build -race
or testing with go test -race
. ArdanLabs has an excellent article on it to get more information on it.
Syntax Check
The first and most important step is to check the syntax for any syntax or compilation errors that might occur. For this, there are a few tools to touch:
Go Vet
What is go vet
? Well, the official documentation explains it pretty well:
Vet examines Go source code and reports suspicious constructs, such as
Printf calls whose arguments do not align with the format string. Vet uses
heuristics that do not guarantee all reports are genuine problems, but it
can find errors not caught by the compilers.
Basically, go vet
finds problems/possible optimizations with your code that may not be caught at compiler time (although those are found too) e.g unreachable code, passing non-pointers to JSON unmarshals. For a full list of checks you can read the documentation, but here is a quick break down:
asmdecl report mismatches between assembly files and Go declarations
assign check for useless assignments
atomic check for common mistakes using the sync/atomic package
bools check for common mistakes involving boolean operators
buildtag check that +build tags are well-formed and correctly located
cgocall detect some violations of the cgo pointer passing rules
composites check for unkeyed composite literals
copylocks check for locks erroneously passed by value
httpresponse check for mistakes using HTTP responses
loopclosure check references to loop variables from within nested functions
lostcancel check cancel func returned by context.WithCancel is called
nilfunc check for useless comparisons between functions and nil
printf check consistency of Printf format strings and arguments
shift check for shifts that equal or exceed the width of the integer
stdmethods check signature of methods of well-known interfaces
structtag check that struct field tags conform to reflect.StructTag.Get
tests check for common mistaken usages of tests and examples
unmarshal report passing non-pointer or non-interface values to unmarshal
unreachable check for unreachable code
unsafeptr check for invalid conversions of uintptr to unsafe.Pointer
unusedresult check for unused results of calls to some functions
How would you add it to your CI pipeline? Running go vet ./...
(read up on the official documentation on extra options) returns a non-zero error if issues are found in your code, so that should cause your CI pipeline to fail when there are issues.
Staticcheck
Staticcheck is a monster of a linter (sorry, I might be fanboying a bit). It performs static analysis on the modules specified and checks for bugs, possible optimizations, style enforcements etc.
It performs an immense amount of checks ranging from little things like copying a slice with copy
to running defers in an infinite loop (will not be reached). A full list of the checks can be found on the documentation.
You can simply run it by running
$ staticheck -f stylish/text/json ./...
Syntax Formating
Not necessarily a must-have, but doesn't hurt to do a little bit of syntax tidying here and there in your code. For that, go comes with the gofmt
tool.
Take this main.go
file for example
package main
import "fmt"
func main() {
mehStruct := struct {
name string
number int
}{
name:"kofo",
number: 1,
}
fmt.Printf("struct value is %#v",mehStruct)
}
If you run cat main.go | gofmt
(pipe the contents of the main.go into gofmpt), you should get this output
package main
import "fmt"
func main() {
mehStruct := struct {
name string
number int
}{
name: "kofo",
number: 1,
}
fmt.Printf("struct value is %#v", mehStruct)
}
The difference? Notice the indentation change on line 10 and 11. Also, an empty space was added after the comma in this line fmt.Printf("struct value is %#v", mehStruct)
. Gofmt accepts a path pointing to a go file or a directory of go files, if there is no parameter, it reads the stdin like our example above.
It's important to note that gofmt does not work on modules, rather it works on individual gofiles, so you could have some compilation errors in your code and gofmt wouldn't point it out... e.g Calling a property of a package that doesn't exist, gofmt would not check for that. Since it only formats the syntax, it would spot syntax errors though.
Verifying dependencies
If you're writing go code in 2020, then you are most likely using go moduels as your go-to way of dependency management. You definitely have to make sure your downloaded dependencies are valid as well as your go.mod
and go.sum
files. Go has a few tools for that
Go mod tidy
Go mod tidy
does exactly what it sounds like, it tidies up your module dependencies file. It goes through your source code and adds any missing dependencies to the go.mod
. It also removes dependencies not used in code.
A good way to prevent a module from being removed is importing them as underscores.
The go mod tidy
command almost always exits with a success status, since it modifies the go.mod
file in place. It is up to you to check the state of the modified go.mod
file in your build pipeline and fail the build step depending on that.
Go mod verify
go mod verify
is not necessarily needed, but it can be useful in the rare case of your build-cache being corrupted. It is a bit more complex than tidy. Instead of checking for used/unused dependencies, Verify checks the locally downloaded dependencies to ensure they have not been modified since download. The documentation goes into a bit more detail about how it works, but essentially you can use it to ensure your CI build cache is still intact.
Conclusion.
Hopefully, this article showed ways you can improve the quality of your go code, it's a beginner-oriented article, but experienced devs can find it useful as a source of reference or whatnot.
Hope it helped. Enjoy!