Memory management under the hood

Memory for a programming language can be allocated via the stack or the heap. Allocation on the stack happens in contiguous blocks of memory. When variables go out of scope these are deallocated automatically. Stack allocation is faster but stack size is limited.

Heap does not have a single block of memory but a set of free regions. Hence it can be expanded as needed. Since memory with the heap is fragmented, access is slower.

There is also the fixed sized segments such as data segment and code segment. Data segment normally contains global variables. Code segments contain the code instructions and constant values.

Traditionally, function parameters and local variables are located on the stack. and dynamic memory allocation is done on the heap.

Memory management in Go

Memory allocation in Go is a bit ambiguous. Go manages its own memory allocation and garbage collection. The language specification does not clearly draw a line about what will be stored where. If more space is needed it will accommodate that.

The initial stack size for a goroutine is only 2kB. Additional memory is allocated as needed. The Go compiler sets the default maximum stack to 1GB on 64-bit systems and 250 MB on 32-bit systems

To its credit, Go prefers allocation on the stack. But stack memory is limited and stack allocation has to be decided at compile time. If the size of the data could change at runtime, heap allocation will happen.

Go uses an optimization technique called escape analysis. The main idea is:

  • If a lifetime of the variable can be determined at compile time, it will be allocated on the stack.
  • If a value of a variable is shared or can escape outside a function scope, it will be allocated on the heap.

To find out what will escape to the heap in your code, use gcflags option:

go build  -gcflags '-m'
go tool compile -m test.go

Note that passing -m multiple times will give you a more verbose response.

There are some obvious escape patterns:


func foo() *string{
a := "hello world"
return &a
go tool compile -m hello.go
./hello.go:9:6: can inline foo
./hello.go:6:17: inlining call to foo
./hello.go:6:17: foo() escapes to heap
./hello.go:6:17: &a escapes to heap
./hello.go:6:17: moved to heap: a
./hello.go:11:9: &a escapes to heap
./hello.go:10:2: moved to heap: a

This is a not so obvious example where memory escapes to heap

func main() {
    a := "hello"

$go tool compile -m hello.go
hello.go:7:13: a escapes to heap
hello.go:7:13: main … argument does not escape

$go tool compile -m -m hello.go
hello.go:5:6: cannot inline main: function too complex: cost 89 exceeds budget 80
hello.go:7:13: a escapes to heap
hello.go:7:13: from … argument (arg to …) at hello.go:7:13
hello.go:7:13: from *(… argument) (indirection) at hello.go:7:13
hello.go:7:13: from … argument (passed to call[argument content escapes]) at hello.go:7:13
hello.go:7:13: main … argument does not escape

This showed that vara escapes to the heap because it is passed as a function argument to a function that takes a variadic argument. Had I passed a to a function that took just a string it would stay on the stack.

Per example below, if I pass a reference to a function, it does not escape to the heap but returning a reference will cause it to escape to the heap.

func main() {
    x := 2
    a, b := hello.AddTwo(&x)

    a = a + 2
    *b = *b + 2

func AddTwo(a *int) (int, *int) {
    b := 2
    return *a + 2, &b

./test.go:12:6: can inline main as: func() { x := 2; a, b = hello.AddTwo(&x); a = a + 2; *b = *b + 2 }
./test.go:14:22: inlining call to hello.AddTwo func(*int) (int, *int) { var hello.b·4 int; hello.b·4 = ; hello.b·4 = int(2); return hello.a + int(2), &hello.b·4 } ./test.go:20:6: can inline AddTwo as: func(int) (int, *int) { b := 2; return *a + 2, &b }
./test.go:14:23: main &x does not escape
./test.go:14:22: main &hello.b·4 does not escape
./test.go:22:17: &b escapes to heap
./test.go:22:17: from ~r2 (return) at ./test.go:22:2
./test.go:21:2: moved to heap: b
./test.go:20:13: AddTwo a does not escape
~/sources/src/github.com/mariadesouza/scratch $

There is no easy way to determine all the memory allocation within your code. For optimization, here are some guidelines:

  • Re-use variables where possible.
  • Run the compile tool with -m to inspect and if time permits, rewrite the code when the variable is detected as escaped to heap.
  • Preallocate memory for arrays if boundary is known rather than use a slice.


Writing tests in Go

Unit tests in Go are run using the go test command. To create unit tests in go to check functions, add a file with the same name as the original and add a suffix “_test”


|- cache.go
|- cache_test.go

To test function Foo your test name should be of the form:

func TestFoo(*testing.T)

Even if a function is not exported, and starts with a lowercase, you need to start it with an uppercase in the test function or it is not run by the go test command. When I first started go, I wrote a small piece of code that have one exported function and when I wrote the test for it. I ran it and I got a “no tests to run” warning. Then I noticed the Golang doc states:

func TestXxx(*testing.T) 
where Xxx does not start with a lowercase letter. 

E.g. Test for function findNumber

func TestFindNumber(t *testing.T) {
    result := findNumber([]int{5, 3, 1})
    expected := 2
    if result != expected {
        t.Error("Incorrect result Expected ", expected, "Got:", result)


Test tables

We can use anonymous structs to write very clear table tests that have multiple inputs and expected results without relying on any external package. E.g.

var testTable = []struct { 
isdev bool
expected string
{true, "/Users/"},
{false, "/home/httpd/"},
for _, test := range testTable {
config.IsDev = test.isdev
actual := getUsersHomeDir()
if !strings.Contains(actual, test.expected) { t.Errorf("getUsersHomeDir: Expected %s, Got %s",
test.expected, actual)

Test options

Here are some very useful go test options

//Run Verbose 
go test -v
//run tests with race detector
go test -race

This is a neat trick, I used to test multiple packages in a repo and exclude one or more folders. go list will only include folders with .go files in them. I had functional tests in a folder called test-client written in Go that I wanted to exclude.

go test `go list ./... | grep -v test-client`

Also check out the post Mocking with Golang so you could write unit tests that rely on external dependencies like servers. Using interfaces these can be simulated to write tests without actually accessing the external resource.

Reference Links


Interfaces in Go

An interface type is a method set. If a type contains methods of the interface, it implements the interface. 

A type can implement multiple interfaces. For instance all type implement the empty interface.

An interface may use a  interface type name  in place of a method specification. This is called embedding interface.

type ReadWriter interface {	
Read(b Buffer) bool
Write(b Buffer) bool
type File interface {
ReadWriter // same as adding the methods of ReadWriter

The empty interface

A type of empty interface can be used to declare a generic data type. 

E.g. Implementing a linked list in Golang.

linked list is a linear data structure where each element is a separate object. Linked list elements or nodes are not stored at contiguous location; the elements are linked using pointers.

We could declare the linked list node struct as follows

type Node struct {
    Next *Node
    Data interface{}

//creates a New Node
func NewNode(d interface{}) *Node {
    nn := Node{Data: d, Next: nil}
    return &nn

//Append : Adds a node to linked list
func (n *Node) Append(d interface{}) {
    nn := NewNode(d)

    if n == nil {
        n = nn
    } else {
        m := n
        for m.Next != nil {
            m = m.Next
        m.Next = nn

 This allows us to use the same struct to hold data of different types.

Here I use int as the data type for Node.Data

n := NewNode(0) // int Data
for m := n; m != nil; m = m.Next {

I could also use string as the data type using the same struct that I defined above.

n1 := NewNode("a") //string Data

Here is the full example:


I also have examples of using the empty interface when Unmarshalling JSON in my other post: Custom Unmarshal JSON in Go

Useful interfaces in Go stdlib

Error Interface

The error type is an interface that has a method Error.

type error interface {    
Error() string

The most commonly used implementation of the error interface is the errorString type in the errors package.

// errorString is a trivial implementation of error.type errorString struct {    
s string
func (e *errorString) Error() string {
return e.s

Handler Interface

The Handler Interface in the net/http package requires one method ServerHTTP

type Handler interface {	
  ServeHTTP(ResponseWriter, *Request)

Within that same package you will find HandlerFunc implements the Handler interface. 

type HandlerFunc func(ResponseWriter, *Request)     

// ServeHTTP calls f(w, r).  
func (f HandlerFunc) ServeHTTP(w ResponseWriter, r *Request) {  
f(w, r)  

The HandlerFunc makes it possible for us to pass in any function to make it a Handler. All we would have to do is wrap it in  HandlerFunc. 

http.Handle("/", http.HandlerFunc(indexHandlerHelloWorld))null

We can have also have our own struct that has fields and methods and implements the Handler Interface by defining the ServeHTTP method as a member of the struct. 

Stringer Interface

The fmt package has a stringer interface. It can be implemented by a type that declares the String method. 

type Stringer interface {        
String() string

If a type implements the stringer interface, a call to fmt.Println or fmt.Printf of the variable of type will use that method. 

E.g. If you want  to print a struct in a formatted fashion with key names and values, the struct needs to implement the Stringer interface

type Family struct {
    Name string
    Age int

func (f *Family) String() string {
    return fmt.Sprintf("Name: %s\t Age: %d", f.Name, f.Age)
func main() {
family := []Family{
        {"Alice", 23},
        {"David", 6},
        {"Erik", 2},
        {"Mary", 32},

    for _, i := range family {


The fmt package has other interfaces like Scanner, Formatter and State.




Auto-generate code using Go templates

Go has two template packages: a HTML template package that can be used to templatize HTML files and a more general text template.

The Go text template package can be used easily to generate custom code like Javascript, Swift, React etc.


This package can be very useful for building products for varied customers that are similar but require their own company logo, names, images and text branding.

In fact, this can be implemented in any codebase that is very similar but needs a few tweaks to work for different platforms or products.

I have used Go text templates for code generation in the following scenarios:

  1. Generating multiple Chromium, Firefox, Edge extensions for varied products.
  2. Interfacing with different API’s to get data into our main systems

I’m also working on using it to auto generate custom mobile apps.

Here, I will explain in detail how I build new Chrome and Firefox extensions in seconds using Go templates. This can be expanded to include Microsoft Edge extensions as well.

Start with a basic chrome extension

There is a detailed description on how to build a chrome extension. 


Once you have a skeletal chrome extension, it is easy to duplicate and create multiple custom extensions if needed or even build Firefox extension using the same set of extension files.

All you need is a set of template javascript files for your extension and your config json values.

Templatize your extension

I start out by taking my extension file and saving with a .template extension. This way, I can recognize it needs to be parsed to replace values.

Files that do not need have any customizations can be left as is in the folder.

E.g. The chrome extension manifest.json.template file will look like this with template fields:

"name": "{{.Name}}",
"version": "{{.Version}}",
"manifest_version": 2,
"default_locale": "en",
 "description": "{{.Description}}", 
"background": { 
"page": "background.html"   
"browser_action": {     
"default_title": "{{.Tooltip}}",   
"default_icon": "icon.png"
"content_scripts": [
"js": ["contentscript.js"]
"icons": {
"16": "icon16.png",            
"48": "icon48.png",           
"128": "icon128.png"    
"homepage_url": "{{.Protocol}}://{{.Domain}}", 
"permissions": [ "tabs",     

Similarly we write templates file for all the extension files like popup.js etc.

To build a basic chrome extension, I also define the following in a global.json file. This is external to your extension files and will be used by the Go build system.

"production": "true",
"author": "Maria De Souza",
"author_home": "https://mariadesouza.com/",
"protocol": "https",
"version" : "",
"domain" : "www.mysite.com",
"name" : "testExtension",
"description" : "This is a test extension",
"tooltip":"Click here to view settings",
"title" : "Test extension",

Gather Customized Settings

I create a folder per customer product where I keep the images related to the customer. This source folder is used later when creating the extension.

The global settings can be overridden by a product specific json file:

"title" : "My cool new extension",
"version" : "",
"domain" : "www.myotherwebsite.com",

The customized product.json can be a subset of the original global.json.

Write the build Script

Now we can get to the fun part, building a script to generate the extensions.

Templates are executed by applying them to a data structure. We first define a struct to Unmarshal our config JSON and use it in our build script. Notice the JSON tags correspond to the JSON field names in the global.json and product.json file.

type Config struct {  
Production  string `json:"production,omitempty"`
Author      string `json:"author,omitempty"`
AuthorHome  string `json:"author-home,omitempty"`
Version     string `json:"version,omitempty"`
Domain      string `json:"domain,omitempty"`
Protocol    string `json:"protocol,omitempty"`
Name        string `json:"name,omitempty"`
Description string `json:"description,omitempty"`
Tooltip     string `json:"tooltip,omitempty"`
Title       string `json:"title,omitempty"`
Browser     string `json:"browser,omitempty"`
ProductDir  string `json:"product_dir,omitempty"`
HomePage    string `json:"home_page,omitempty"`

The script starts by unmarshalling the global file in a struct value as below. Note that I have left out error handling to reduce noise. The second part will Unmarshal the custom product values.

var globalConfig Config 
configFile, _ := ioutil.ReadFile("global.json")
json.Unmarshal(configFile, &globalConfig)

var productConfig Config
productconfigFile,_ := ioutil.ReadFile("product.json") json.Unmarshal(productconfigFile, &productConfig)

Using reflect, I override the custom product values:

func mergeWithGlobal(globalConfig, productConfig *Config){  

st := reflect.TypeOf(*globalConfig)

for i := 0; i < st.NumField(); i++ {
tag := strings.Split(field.Tag.Get("json"), ",")
v2 :=

if tag[0] != "" && v2.String() != "" {
v := reflect.ValueOf(productConfig).Elem().FieldByName(st.Field(i).Name)


Using this Config struct, I can now populate the template files. To do this I read all files with extension .template in the source diectory, execute the template using the populated Config struct and save the result in the destination directory.

Here is what the code would like. Again, I removed some error handling so the main flow is understandable. But error handling should definitely be part of your code.

func populateTemplateFiles(source, dest string, config *Config) error {
//Make destination directory
    os.MkdirAll(destination, 0755)
    re := regexp.MustCompile(`(.*)\.template$`)
    files, _ := ioutil.ReadDir(source)

    for _, file := range files {

        //if it is a template file read and populate tags
        filename := file.Name()
        if re.MatchString(filename) == true {

            buf, _ := ioutil.ReadFile(filepath.Join(source, filename))
            tmpl, _ := template.New("extensions").Parse(string(buf))
           // final file will drop the .template file ext
            targetfilename := strings.Split(filename, ".template")
            destFile := targetfilename[0]

            targetfile := filepath.Join(dest, destFile)
            f, err := os.OpenFile(targetfile, os.O_WRONLY|os.O_CREATE, 0755)
            if err != nil {
                fmt.Println("Failed to create file", targetfile, err)
            w := bufio.NewWriter(f)

            tmpl.Execute(w, config)
        } else {

// not a template file - copy as is
          copyFile(filepath.Join(dest, filename), filepath.Join(source, filename))
    return nil

The customized images in the product directory are copied into the destination directory.

The destination directory will then contain all the customized javascript, other javascript files and custom images.

We can then upload a zipped version of the destination directory to the Chrome Webstore. I have a make file that also generates a pem file and zips the directory.

Extending build system for Firefox

I build Firefox extensions from the same set of Chrome extension files.

This is the link to developing a Firefox browser extension:


Firefox and Chrome extensions have very subtle differences, in the way they are implemented. They are all based largely on the WebExtensions API for cross browser compatibility.

Notice that my earlier declaration of Config struct had the following field

 Browser     string `json:"browser,omitempty"`

I dynamically change this in my Go script to instruct the system to generate a new set of extension files for Firefox.

I set the value for Browser in my code and then build for Firefox.

productConfig.Browser = "Firefox"

In fact, you can parameterize the Go script to accept a command line argument that will control which browser you want to build for. The default could be all.

The differences between Firefox and Chrome will be conditionals in your template files. It depends on what functionality you are using within your extensions.

E.g. to send a message back to the background script from the content script, I use the template system to conditionally generate code for Firefox vs Chrome using the config value.

{{if eq .Browser "Firefox"}}
browser.runtime.sendMessage({ name: "options", value: event.target.value });
chrome.extension.sendRequest({ name: "options", value: event.target.value });

Another example of this would be, add FirefoxID in my config struct and in my manifest.json.template, I add

{{if eq .Browser "Firefox"}}
"applications": {
"gecko": {
"id": "{{.FirefoxID}}",
"strict_min_version": "42.0",

Microsoft Edge extensions are also similar and the same set of template files can be used to auto-generate Edge with a few tweaks.


Launch a Golang web server using Docker

If you want to create a web server using Go the simplest way to deploy is using Docker.  Golang code is compiled to a binary and does not need a special environment to run.

Here is the simplest web server code in Go to get started. Save this as webserver.go

package main 
import (

func main() {
http.HandleFunc("/", indexHandlerHelloWorld)
log.Fatal(http.ListenAndServe(":8080", nil))

func indexHandlerHelloWorld(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "Hello world, I'm running search on %s with an %s CPU ", runtime.GOOS, runtime.GOARCH)

We can use the simplest docker image scratch and add a directive to copy the binary to the server. Save as Dockerfile.

FROM scratch 
MAINTAINER Maria De Souza <maria.g.desouza@gmail.com>

ADD go-webserver go-webserver
ENTRYPOINT ["/go-webserver"]

We setup a start shell script to build and kick up the docker

GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -a -o go-webserver webserver.go || exit 1

if [ "$(docker ps -q -f name=go-web-server)" ]; then
docker stop $(docker ps -a -q --filter name=go-web-server --format="{{.ID}}")

docker build -t go-web-server . || exit 1

docker run -p 8000:8080 go-web-server || exit 1

You can use the -h option to add the hostname to the webserver

Voila! Now your webserver is running. Navigate to http://localhost:8000/ to test.

Known Issue

If you make any SSL requests from your webserver, you will see the following error when running the webserver using Docker:

x509: failed to load system roots and no roots provided

This is because the /etc/ssl/certs/ca-certificates.crt, is missing from the scratch image that is read by the golang tls package.

To avoid this copy the cert to your docker image from your local system. I normally add this to my bash script that will copy it based on OS.

if [ ! -e ca-certificates.crt ]; then  
if [[ $(uname) = "Darwin" ]]; then
cp /usr/local/etc/openssl/cert.pem ca-certificates.crt || exit 1
cp /etc/ssl/certs/ca-certificates.crt ca-certificates.crt || exit 1

This is what your Dockerfile should then look like:

FROM scratch  
MAINTAINER Maria De Souza <maria.g.desouza@gmail.com>

COPY ca-certificates.crt /etc/ssl/certs/ca-certificates.crt

ADD go-webserver go-webserver
ENTRYPOINT ["/go-webserver"]

String manipulation in Go

Basic Operations

  • Get char array of a String
greeting := "Comment allez-vous" 
greeingCharacterArr := []rune(greeting)
  • Get a char at the specific index
fmt.Printf("%c", greeting[5])

The result would be  “n“. However, without the character formatter, the result would be the Unicode decimal code 110.

  • Get string length
  • Substrings
func substring(s string, beginIndex int) string{
return s[beginIndex:]

func substring2(s string, beginIdx int, endIdx int) string {
return s[beginIdx:endIdx]
  • String to Integer
testN,_ := strconv.Atoi("1234")  
testN += 1

strconv package

Above examples : https://play.golang.org/p/kds_Lu9HyTJ

  • Comparing Strings

To compare strings, use the comparison operators ==, < and >

The strings package has a Compare function that comes with a warning that no one should use strings.Compare


Sorting and Searching Strings in Golang

The sort package in Go has some very useful methods for sorting and searching slices and strings.

The sort.StringSlice attaches the methods of Interface to []string, sorting in increasing order. https://golang.org/pkg/sort/#StringSlice


package main 
import (

func main() {
countries := sort.StringSlice{"USA", "India", "South Africa"}
n := countries.Search( "USA")
fmt.Println(""Result: ", n, countries[n])


Result: 3 USA

Sorting Characters in a string

To sort characters in a string, we use the fact that strings in Go can be represented as a slice of runes. We could extend the generic sort function in Go to sort the slice of runes. To use the generic sort, we need to implement sort.Interface – Len, Swap, Less.

type sortRunes []rune 

func (s sortRunes) Less(i, j int) bool {
return s[i] < s[j]

func (s sortRunes) Len() int{
return len(s)

func (s sortRunes) Swap(i, j int) {
s[i], s[j] = s[j], s[i]

str1 := "sortthisstring"
s := []rune(str1) sort.Sort(sortRunes(s))

See the full working example here:


Note the sort package comments, Sort makes one call to data.Len to determine n, and O(n*log(n)) calls to   data.Less and data.Swap. 


Embedding in Go

Go supports defining is-a relationships using an embedded type.

Fields and methods in a struct have a has-a relationship

E.g.  Person has-a name and email.

type Person struct { 
Name string
Email string

Embedding to define an is-a relationships  is declared with a type but no explicit field name. The unqualified type name acts as the field name.

type Employee struct {        
EmployeeID string

This way the Person struct can be accessed using the type name:

accountant := new(Employee) 

We can also invoke any Person methods directly on the Employee object

accountant := new(Employee) 

We cannot embed a slice or a map

type Employee struct {   
EmployeeID string

This will give  a syntax error: unexpected [, expecting field name or embedded type.

A workaround will be defining a type and then embedding :

type SpecialPeople []Person 
type Employee struct {
EmployeeID string

The spec in Go describes embedding as below:

EmbeddedField = [ "*" ] TypeName .

Embed by-pointer

The advantage of embedding by reference is that  you are embedding all the functionality of a type without needing to know when it is instantiated. The major application of this would be to have thousands of instances sharing a single underlying data structure. This can significantly reduce memory consumption.

type Image struct {      
data [5][5]
type Block struct {
show bool

Because, the unqualified type name acts as the field name for an embedded struct, we can’t have an embedded struct and its pointer in the same struct.

type Image struct { 
T // conflicts with embedded field *T and *P.T
*T // conflicts with embedded field T and *P.T
*P.T // conflicts with embedded field T and *T

Promotion of fields or methods

All field or method calls for embedded type objects are resolved at compile-time without the use of a virtual table. A field or method of an embedded field in a struct is called promoted.

type Person struct {        
Name   string
Email  string
func (p *Person) FreeGift() int64{
return 100
type Employee struct {      
EmployeeID string 

In this example person.FreeGift()  and employee.FreeGift() is the same so the method is promoted. Promotion  occurs only at the first level.


However, if we have a method with the same name defined on the parent struct as below, then employee.FreeGift() will invoke the method on the Employee struct.

func (e *Employee) FreeGift() int64{        
return 200

In the above example, calling person.FreeGift() and employee.Person.FreeGift() will invoke the method on the Person struct


One very important design aspect to remember is, if you plan to use an embedded struct and the embedded type has non-exported fields or methods,  those are completely inaccessible to you in a separate package.

Embedding Interfaces

Embedding an interface will add all (exported and non-exported) methods of the embedded interface to the enclosing interface.

type ImageWriter interface {        
Read(b Buffer) bool
Write(b Buffer) bool
type ImageFile interface {
ImageWriter  // same as adding methods of ImageWriter

Check out this example to add logging to your struct by embedding the log.Logger object.