How Do I Go
Welcome to How do I Go.
You can find this book on-line at: TODO
It is licensed under the Creative Commons Attribution-Non Commercial-Share Alike 3.0 license.
What is this?
This is a FAQ which deals specifically with questions of the form "How do I…?" or "Why does…?"
The inspiration for this documentation is perlfaq. While many sections of perlfaq are answered adequately in other places such as Effective Go and the Go FAQ, the example code in sections 4, 5 and 6 of perlfaq are not as well represented, or at least not in a central place. This document is an attempt to rectify that underrepresentation.
Wherever possible the Go Programming Language Specification is referenced. The acronym GPLS is used to refer to it.
Data Manipulation
Data: Numbers
Why does division give wrong answers?
First, if you have never read anything about how floating point numbers are represented in a machine, stop now, go to a search engine and search for "what every programmer should know about floating point". For bonus fun reading, read about an Excel 2007 bug: http://www.joelonsoftware.com/items/2007/09/26b.html or Eric Lippert’s explaination which is a great read: http://blogs.msdn.com/b/ericlippert/archive/2005/01/20/fun-with-floating-point-arithmetic-part-five.aspx
When doing any math, always be aware of what types are being used.
Integer math results in integers. e.g.
fmt.Println(3/9) 0
Constants in Go may be typed or untyped. Untyped constants are converted to appropriate type when needed. GPLS Constants
When printing floating point numbers, always be aware of what precision is used.
If you are using 32bit floats, you will hit rounding errors much sooner than 64bit. Literals default to 64bit. (This is not really true. The spec says floating point constants are represented with a mantissa of at least 256 bits. At runtime values are converted and 64bit math operations are used. This is also not really true. The floating point units on many CPUs use 80bit IEEE operations.)
e.g.
fmt.Printf("%.20f\n",float32(3)/9)
0.33333334326744079590
fmt.Printf("%.20f\n",float64(3)/9)
0.33333333333333331483
fmt.Printf("%.20f\n",3.0/9)
0.33333333333333331483
How do I make int() work?
It does work. The numbers may not be what you think. See the previous question.
1.0 minus nine tenths minus one tenth is represented as a number very close to zero, but not zero.
e.g.
fmt.Println(int(1.0-0.9-0.1))
constant -8.45304e-132 truncated to integer
A floating point constant which is zero is convertible.
e.g.
fmt.Println(int(5.0-1.0-4.0)) 0
Why isn’t my octal data interpreted correctly?
strconv.Atoi doesn’t know about literal language prefixes.
e.g.
mine := int64(0600)
claim, _ := strconv.Atoi("0600")
fmt.Println(mine == claim)
false
strconv.Atoi is a convenience method for ParseInt(s, 10, 0). Where 10 and 0 are base and bit size. Use base 8 for converting octal.
e.g.
mine := int64(0600)
claim, _ := strconv.ParseInt("0600", 8, 0)
fmt.Println(mine == claim)
true
A side note: The bit size parameter does not change the return type of ParseInt (read the spec and you’ll know that it wouldn’t). It causes ParseInt to return a value out of range error.
Does go have a round() function? What about ceil() and floor()? Trig functions?
Look at the math package.
e.g.
import "math" ... fmt.Println(math.Ciel(1.6)) fmt.Println(math.Floor(1.6)) fmt.Println(math.Round(1.6))
See also, the above question about int().
Combining these two questions we have some interesting results:
e.g.
fmt.Println(int(math.Floor(1.0-0.9-0.1))) -1
In the question about int() we said that int() cannot truncate and that the floating representation of 1.0-0.9-0.1 is a tiny negative number. Calling Floor on that tiny negative number floors it to -1.
Calling Ciel on that tiny negative number ciellings it to 0.
fmt.Println(int(math.Ciel(1.0-0.9-0.1))) 0
Trig functions exist and suffer from the same limitations of floating point numbers as everything else. (Why isn’t math.Sin(math.Pi) == 0? Why isn’t math.Cos(math.Pi/2) == 0?) See the above question about int().
fmt.Println(math.Cos(math.Pi/2)) 6.123233995736757e-17 fmt.Println(math.Sin(math.Pi)) 1.2246467991473515e-16 fmt.Println(math.Cos(0)) 1 fmt.Println(math.Sin(math.Pi/2)) 1
For the same reason, trig identities will not always be exact. No really, go read the "What Every Programmer Should Know About Floating-Point Arithmetic" publication right now.
a30 := math.Pi/6
if (math.Sin(a30) == math.Sqrt(3)/2) { // is equal
fmt.Println("sin(30)==√3/2")
}
i := 0.01
if (math.Tan(i) == math.Sin(i)/math.Cos(i)) { // is not equal for all values i
fmt.Println("Identity not held.")
}
You may be tempted to compare using some epsilon value and this works for some cases, but really, go read WEPSKAFPA.
How do I convert between numeric representations/bases/radixes?
You can use literals with the 0 prefix for octal, the 0x prefix for hexadecimal. http://golang.org/ref/spec#Integer_literals
You can use strconv.ParseInt to parse hexadecimal and octal. https://www.golang.org/pkg/strconv/#ParseInt
deadbeef := "deadbeef" fmt.Println(strconv.ParseUint(deadbeef, 16, 32)) 3735928559 <nil>
Convert from decimal to hexadecimal:
hex := fmt.Sprintf("%x", uint(3735928559)) // hex is "deadbeef"
From octal to decimal
deadbeef := "33653337357" fmt.Println(strconv.ParseUint(deadbeef, 8, 32)) 3735928559 <nil>
Note the use of unsigned integer parsing. If you want to treat your parsed number as signed you will need to cast it.
deadbeef := "33653337357" val,_:=strconv.ParseUint(deadbeef, 8, 32) fmt.Println(int(val)) -559038737
How do I convert from binary to decimal?
Use strconv.ParseInt with a base of 2.
n, err := strconv.ParseInt("01010110", 2, 64)
How do I multiply matrices?
How do I perform an operation on a series of integers?
Go isn’t a functional programming language. There is no generic map function. Use a for loop.
results := make([]int, len(array))
for i:=0; i< len(array); i++ {
results[i] = somefunc(array[i])
}
Why aren’t my random numbers random
The "math/rand" package generates pseudo-random numbers. For cryptographic work use the crypto/rand package.
You can seed the "math/rand" package by calling rand.Seed(). A common seed which will not repeat without clock manipulation is time.Now().UnixNano().
r := rand.New(rand.NewSource(time.Now().UnixNano())) fmt.Println(r.Int())
How do I get a random number between X and Y?
Use rand.Intn(Y-X) and add X.
e.g. between 1 and 10:
num := rand.Intn(10) + 1
This is 1 through 10 inclusive.
The rand.Intn() returns [0,N) so Intn(10) returns a number from the set of integers from 0 through N-1.
Data: Dates
Note that the playground at http://play.golang.org may not have the time that you think it is. time.Now() returns 2009-11-10 23:00:00 +0000 UTC as I write this. It is conceivable that this return value may change in the future.
How do I find the day or week of the year?
Time.YearDay() returns the day of the year for a time.
e.g.
fmt.Println(time.Now()) fmt.Println(time.Now().YearDay()) 2009-11-10 23:00:00 +0000 UTC 314
The second return value of ISOWeek() is the week of the year.
e.g.
fmt.Println(time.Now()) fmt.Println(time.Now().ISOWeek()) 2009-11-10 23:00:00 +0000 UTC 2009 46
How do I find the current century or millennium?
Do some math on the result of Year().
e.g.
fmt.Println("This is century", time.Now().Year() / 100 + 1)
fmt.Println("This is millenium", time.Now().Year() / 1000 + 1)
How do I compare two dates and find the difference?
The time package supports math functions on times and returns durations. There are functions such as Before, After, and Equal to test which time instances are before, after, or equal to the compared time.
e.g.
var now = time.Now()
var date1, _ = time.Parse(someParsableDateString)
if date1.Before(now) {
fmt.Println("The date was in the past")
}
fmt.Println(date1.Sub(now))
The date was in the past
62425h0m0s
How can I take a string and turn it into epoch seconds?
The time package has a Parse function which returns a new time value. That time value has a Unix function.
e.g.
var date1, _ = time.Parse(someParsableDateString) var seconds = date1.Unix()
How can I find the Julian Day?
A duckduckgo search yields this playground post: http://play.golang.org/p/ocYFWY7kpo
How do I find yesterday’s date?
Use a calendar library if you want to catch all the edge cases. Dates and times are always more complex than you may think. I’m not aware of anything like Noda Time for golang, but if you really want to get it right, then you will need to write or use something like it.
Or…
Use time.AddDate(0,0,-1)
Does Go have a Year 2000 or 2038 problem?
Go came out well after 2000, so I sure hope not.
Go uses int64 to store time values, even on 32bit machines. There should be no 2038 problem.
Data: Strings
Before considering some of these items, it will be useful to understand what kind of strings Go uses. This blog post is a good explaination: https://blog.golang.org/strings
How do I validate input?
There is no one way. Your needs may be as simple as comparing input to a list of possible values, or as complex as matching a grammar or consulting a third party service.
How do I unescape a string?
It depends on what you mean by unescape. What is escaped? utf8? c-strings, go strings, other?
How do I remove consecutive pairs of characters?
In Go, you use a loop.
Many other languages suggest a regular expression.
It is up to the reader to decide which is better and why.
How do I expand function calls in a string?
You don’t.
That said, the %v format specifier does the job nicely in an fmt.Sprintf call.
msg := fmt.Sprintf("now its: %v", time.Now())
How do I find matching/nesting anything?
In Go, you use a loop and a simple state machine.
Many other languages suggest a regular expression.
How do I reverse a string?
In Go, you use a loop.
Many other languages have a generic reverse function.
How do I expand tabs in a string?
In Go, you use a loop.
Many other languages suggest a regular expression.
How do I reformat a paragraph?
Create a buffered string output and count width of characters yourself.
How do I access or change N characters of a string?
Strings are slices of runes. Runes are utf8 characters. You can slice a string.
s := "Just another gopher" j := s[0:4] g := s[13:19] fmt.Println(g, "in the hole!")
In go, strings are immutable. This is very familiar if you come from python, C#, or Java, but may be unfamiliar if you come from C or ruby.
s := "Just another gopher" s[0:3] = "yes" // cannot assign to s[0:3]
To change characters, create a new string.
s := "Just another gopher" s2 := "yet" + s[4:] fmt.Println(s2)
How do I change the Nth occurrence of something?
Find the Nth yourself.
package main
import (
"bytes"
"fmt"
"strings"
)
func main() {
const src = `Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec a diam lectus. Sed sit amet ipsum mauris. Maecenas congue ligula ac quam viverra nec consectetur ante hendrerit. Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean ut gravida lorem. Ut turpis felis, pulvinar a semper sed, adipiscing id dolor. Pellentesque auctor nisi id magna consequat sagittis. Curabitur dapibus enim sit amet elit pharetra tincidunt feugiat nisl imperdiet. Ut convallis libero in urna ultrices accumsan. Donec sed odio eros. Donec viverra mi quis quam pulvinar at malesuada arcu rhoncus. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. In rutrum accumsan ultricies. Mauris vitae nisi at sem facilisis semper ac in est.`
count_sit := 0
count_Donec := 0
nth_sit := 3
nth_Donec := 2
var b bytes.Buffer
for _,word:=range strings.Split(src, " ") {
wrote:=false
if word=="sit" {
count_sit++
if count_sit == nth_sit {
b.WriteString("SIT!")
wrote=true
}
}
if word == "Donec" {
count_Donec++
if count_Donec == nth_Donec { b.WriteString("ZOMGDONEC"); wrote = true }
}
if !wrote {
b.WriteString(word)
}
b.WriteString(" ")
}
fmt.Println(b.String())
}
Optimizing the garbage created by caling strings.Split is an exercise left to the reader.
Note that other languages use a regular expression to do this operation. Go can too.
nth := 3
count := 0
src := "One fish two fish red fish blue fish"
r := regexp.MustCompile(`(\w+)\s+fish\b`)
for _, m := range r.FindAllStringSubmatch(src, -1) {
count++
if (count == nth) {
fmt.Printf("The third fish is a %s one.\n", m[0])
}
}
How can I count the number of occurrences of a substring within a string?
If you want to count the number of occurences of a single character in a string, you can simply loop and keep a counter:
s := "ThisXlineXhasXsomeXx'sXinXit"
count := 0
for _, e := range s {
if e == 'X' {
count++
}
}
fmt.Println(count, "Xs")
Or you could use strings.Count
s := "ThisXlineXhasXsomeXx'sXinXit"
fmt.Println(strings.Count(s, "X"), "Xs")
Note that strings.Count is the preferred method and it works with more than a single character.
How do I capitalize all the words on one line?
Call strings.Title.
s := "gone with the wind's director"
fmt.Println(strings.Title(s))
How can I split a character delimited string except when inside some character?
Use the encoding/csv package. It is not just for comma separated files and it handles commas inside of quoted fields correctly.
in := `first_name,last_name,username
"Rob","Pike",rob
Ken,Thompson,ken
"Robert","Griesemer","gri"
"jay,r",wren,"fuk,ka"
`
r := csv.NewReader(strings.NewReader(in))
for {
record, err := r.Read()
if err == io.EOF {
break
}
if err != nil {
log.Fatal(err)
}
fmt.Println(record)
}
How do I strip blank space from the beginning or end of a string?
Use strings.TrimSpace.
s := " some user may have entered this "
fmt.Println(strings.TrimSpace(s))
See also, all of the Trim* functions in the strings package.
How do I pad a string with blanks or pad a number with zeroes?
The fmt package is short for format.
fmt.Printf("%-10s%-10s%10s %06.2f %06d\n","jay","raymond","wren", 20.0/3, 123)
fmt.Printf("%-[5]*[1]s%-[5]*[2]s%[5]*[3]s %0[6]*.[7]*[4]f %0[6]*[8]d\n","jay","raymond","wren", 20.0/3, 10, 6, 2, 123)
How do I extract selected columns from a string?
If you know the columns which contain the data, then you can slice the string.
// column := line[startcolumn:end]
line := "root 28550 2 0 2015 ? 00:00:04 [btrfs-worker]"
fmt.Println(line[9:14],"#")
You can use strings.Split if the columns are separated by a delimiter as long as the delimiter is not part of the data.
line := strings.Split("root 28550 2 0 2015 ? 00:00:04 [btrfs-worker]", "\t")
fmt.Printf("user: %s\tpid:%s\n",line[0],line[1])
If you want to work with comma separated values, then use encoding/csv and See above.
How do I find the soundex value for a string?
Write soundex yourself, or use a library. A quick searches shows this implementation.
How can I expand variables in text strings?
Use text/template and wrap the variables in the templating annotations.
If you can not use a template language you can use strings.Replacer
replacer := strings.NewReplacer("foo", "Fred", "bar", "Barney")
fmt.Println(replacer.Replace("Say Hello to foo and bar"))
What’s wrong with always quoting "$vars"?
Nothing. Go does not expand variables in strings like Perl or PHP. Go’s strong static typing prevents the errors that are prevented in Perl by not always quoting vars.
Why don’t my <<HERE documents work?
Go doesn’t have HERE documents, but it does have multiline strings by using backticks.
s := `I do not need to heredoc when I can backtick`
How do I split a line into fields?
The strings package has a Fields() function which splits on one or more whitespace characters.
Data: Arrays and Slices
What is the difference between a slice and an array?
Rob Pike has an excellent blog post oh The Go Blog titled "Arrays, slices (and strings): The mechanics of 'append'. That should be background reading for any serious go developer. I’ll briefly answer the question here.
Slices wrap arrays underneath and expose all sorts of nice things on top of them. You almost always want to use a slice. If you are unsure, then you almost certainly want to use a slice. An array is a contiguous piece of memory. The length of an array is part of its type. In Go, [16]byte and [24]byte are two different types. A slice is a view into an array.
That is the incomplete answer. For the complete answer read the specification and the above blog post.
How can I remove duplicate elements from a list or array?
I apologize for this answer being too verbatim like perlfaq. Perlfaq is just too well written to not copy it.
Use a map. When you think the words "unique" or "duplicated" think "map keys".
If you don’t care about order of the elements, you could just create the map then extract the keys. It’s not important how you create that has.
a := []int{4, 1, 2, 3, 3, 4, 4, 5 }
m := make(map[int]bool)
for _, i := range a {
m[i] = true
}
newa := make([]int, 0, len(m))
for k, _ := range m {
newa := append(newa, k)
}
a = newa
How can I tell whether a certain element is contained in a slice or array?
Hearing the word "in" is an /in/dictation that you probably should have used a map, not a slice or array, to store your data. Maps are designed to answer this question quickly and efficiently. Arrays and slices aren’t.
If you want to know an element is in a slice or array, iterate the array and look for yourself.
a := []int{4, 1, 2, 3, 3, 4, 4, 5 }
var has_six, has_five bool
for _, i := range a {
if i == 6 {
has_six = true
}
if i == 5 {
has_five = true
}
}
fmt.Println("has six:", has_six)
fmt.Println("has five:", has_five)
How do I compute the difference of two arrays? How do I compute the intersection of two arrays?
Like this: (assuming that each element is unique in a given array)
a := []int{1, 2, 3, 4, 5 }
b := []int{4, 5, 6, 7, 2 }
union := make([]int, 0, len(a)+len(b))
intersection:= make([]int, 0, len(a))
difference := make([]int, 0, len(a))
count := make(map[int]int, len(a))
for _, i := range a {
count[i]++
}
for _, i := range b {
count[i]++
}
for k, _ := range count {
union = append(union, k)
if count[k] > 1 {
intersection = append(intersection, k)
} else {
difference = append(difference, k)
}
}
fmt.Println("union:", union)
fmt.Println("intersection:", intersection)
fmt.Println("difference:", difference)
How do I test whether two arrays, slices, or maps are equal?
What does equal mean? (sorry) Same elements? Same elements and same order? Same keys? Same keys and same values?
reflect has a DeepEquals function.
a := []int{1, 2, 3, 4, 5 }
b := []int{4, 5, 6, 7, 2 }
c := []int{1, 2, 3, 4, 5 }
fmt.Println("a", a)
fmt.Println("b", b)
fmt.Println("c", c)
fmt.Println("a equals b?", reflect.DeepEqual(a, b))
fmt.Println("b equals c?", reflect.DeepEqual(c, b))
fmt.Println("a equals c?", reflect.DeepEqual(c, a))
Works for maps too. For a different definition of equals, write the comparison yourself.
How do I find the first array or slice element for which a condition is true?
Iterate the slice yourself.
b := []int{5, 9, 17, 2 }
var found, position int
for i, e := range b {
if e > 5 {
found = e
position = i
break
}
}
fmt.Println("found", found, "at position", position)
How do I handle linked lists?
Go slices do not have a fixed size. You do not need linked lists if you want to add or remove items. Do your own homework.
How do I handle circular lists?
If you want to cycle through an slice endlessly, you can increment the index modulo the number of elements in the slice.
b := []int{5, 9, 17, 2 }
var i int
for {
fmt.Println(b[i % 4])
if i >= 20 {
break
}
i++
}
How do I shuffle and array randomly?
Choose a shuffling algorithm, for example Fisher-Yates:
func fisher_yates_shuffle(deck []int) {
r := rand.New(rand.NewSource(time.Now().UnixNano()))
i := len(deck)
for (i--) {
j := r.Intn(i+1)
deck[i], deck[j] = deck[j], deck[i]
}
}
...
deck := make([]int, 52)
for i in range(deck) {
deck[i] = i
}
fisher_yates_shuffle(deck)
fmt.Println(deck)
How do I process/modify each element of an array?
Use a for loop and range.
for i := range lines { lines[i] = strings.Replace(lines[i], "coffee", "decaf", -1) }
There is no map function. Go’s lack of generics make writing one require using reflection. Idiomatic Go uses for loops. If you would like to see a generic implementation, Rob Pike wrote one to see how hard it would be. It is available at https://github.com/robpike/filter
If you want to do the same thing to modify the values of a map, range returns a key and value on each iteration.
for k, v := range people_ages { people_ages[k] = people_ages[k] + 1 // Another year older. }
How do I select a random element from an array?
Use the rand.Intn function.
b := []int{5, 9, 17, 2 }
item := b[rand.Intn(len(b))]
How do I permute N elements of a list?
Search for an implementation or write one yourself.
How do I sort an array by (anything)?
The sort package https://golang.org/pkg/sort/ defines an interface named Interface which must be implemented to use sort.Sort. It also includes some common implementations of that interface for []int, []float64, and []string.
func main() {
a := []int{64, 29, 32, 69, 42, 115, 40, 41, 11}
//sort.Ints is shorthand for sort.Sort(sort.IntSlice(a))
sort.Ints(a)
fmt.Println(a)
}
// [11 29 32 40 41 42 64 69 115]
The Reverse function for an interface effectively inverses the response of Less.
func main() {
a := []int{64, 29, 32, 69, 42, 115, 40, 41, 11}
sort.Sort(sort.Reverse(sort.IntSlice(a)))
fmt.Println(a)
}
Implementing the three method interface is easy.
func main() {
a := []string{"my", "mommy", "told", "me", "~not", "!to", "{swear}"}
sort.Sort(TildeFirst(a))
fmt.Println(a)
}
type TildeFirst []string
func(t TildeFirst) Len() int {
return len(t)
}
func (t TildeFirst) Less(i, j int) bool {
if strings.HasPrefix(t[i],"~") || strings.HasPrefix(t[i],"{") {
return true
}
if strings.HasPrefix(t[j],"~") || strings.HasPrefix(t[j],"{") {
return false
}
return t[i] < t[j]
}
func (t TildeFirst) Swap(i, j int) {
t[i], t[j] = t[j], t[i]
}
How do I manipulate arrays of bits?
Use a default int type such at byte, int or int64 and bit mask operations or a third party package such as "github.com/golang-collections/go-datastructures/bitarray"
Why can I call functions on a nil slice?
The Go Blog has a great post Go Slices: usage and internals which goes into depth on arrays and slices.
A Tour of Go has a page on Nil slices.
A nil slice acts like an empty slice so you can take its length, append to it, and things work. Note that this is not the case for a map.
Data: Maps
How do I process an entire map?
The range clause of a for statement returns each key and value. Use a for loop to process an entire map.
for k, v := range mymap {
...
}
Filtering is done the same way, using an if statement inside of the for loop.
For example, to only do something with values when keys start with the string "now:"
for k, v := range some_string_keyed_map {
if strings.HasPrefix(s, "now:") {
doSomething(v)
}
}
How do I merge two maps?
Use a for loop. Also, know what you want by "merge". Do you want to merge map a into map b, merge map b into a, or merge a and b into a new map c?
Also, what do you do when a key exists in each map? does one have preference over another? One does not simply merge two maps. More specificity is required.
// Merging a into b, overwriting b's keys if a has matching keys.
for k, v := range a {
b[k] = v
}
A new map can be created with with the make() built-in and we can specify its capacity to prevent any extra allocations. Although if a and b have colliding keys, there will be overallocation by that number of keys.
c := make(map[string]string, len(a)+len(b))
for k, v := range a {
c[k] = v
}
for k, v := range b {
c[k] = v
}
The Go Blog has a great post Go Maps in Action with things every Go programmer should know about maps.
What happens if I add or remove keys from a map while iterating over it?
Mostly what you would expect. Maps iteration order with range is not defined and cannot be relied upon (it is in fact random). You can delete items and if they have not been iterated yet, they will not be. You can add items and they will either be iterated or not.
See the language specification section on For statements with range clause
How do I look up map element by value?
You could iterate the map looking for the value you want but if you are going to do it more than once you should…
Create a reverse or inverted map.
byval := make(map[string]string, len(my_map))
for k, v := range my_map {
byval[v] = k
}
If you know you have multiple keys with the same value you could invert into a map of slice.
byval := make(map[string][]string, len(my_map))
for k, v := range my_map {
byval[v] = append(byval[v], k)
}
How can I know how many entries are in a map?
Use the len() built-in function.
How can I sort a map (optionally by value instead of key)?
Built a slice of the keys and sort that slice.
keys := make([]string, 0, len(my_map))
for k, _ := range my_map {
keys = append(keys, k)
}
sort.Strings(keys)
Now iterate that sorted keys slice instead of the map when using range.
To sort by value, invert the map (see previous question) and then do the same thing, filling a slice of values (now keys) and sorting it.
You may also want to use a slice of key value pairs an implement the sort.Interface. There are solutions to this same question on stackoverflow. https://stackoverflow.com/questions/18695346/how-to-sort-a-mapstringint-by-its-values
How can I always keep my map sorted?
Go does not have a built in sorted map and does not have the ability to create new generic data structures outside of the built-ins. This is not the Go way, yet?
That said, if you feel that you must do this, there are third party collection libraries, none of them are widely used as gophers tend to prefer to stay in the gopher tunnel of using built in slices and map. You may also consider writing your own with the types you need.
How do I reset a range iteration part way through?
You cannot reset the for-range, but you can restart the entire iteration.
The simplest way to probably to use goto, even though it is considered harmful.
m := map[string]int{"a": 1, "b": 2, "reset":-1, "c": 3, "d": 4}
reset := false
reset:
for k, v := range m {
if k == "a" {
m["e"] = 5
}
if k == "b" {
delete(m, "b")
}
if k == "reset" && !reset {
reset = true
fmt.Println("resetting")
goto reset
}
fmt.Println(v)
}
How can I get the unique keys from two maps?
You can combine the keys into a single slice and then use the above answer for removing duplicates, but that answer says to use a map, so you may as well combine the two maps into a third map.
c := make(map[Whatever]bool)
for k := range a {
c[k] = true
}
for k := range b {
c[k] = true
}
How can I make my map remember the order I put the elements into it?
Keep a corresponding slice of elements.
See the above question about keeping a sorted map.
How can I make the Go equivalent of a C structure/C++ class/map or slice of maps or or slices?
Compose struct instances, maps and slices, or use literals.
data := struct {
Name string
Description string
tags []string
topicWeights map[string]int
}{
"Alice's manifesto",
"A statement for an awesome life",
[]string{"life", "awesome", "manifesto"},
map[string]int{"life": 10, "awesome": 10, "lame": 0},
}
How can I prevent the addition of unwanted keys into a map?
Check the key before you add it. Check the map size before you add it. Write a wrapper type which encapsulated the map behavior you want and use that instead of directly using a map.
Data: Record types.
Is it possible to extend tags of a global struct from another package?
for example you have a
type User struct { Name string json:"name" }
and I want to make it into
type User struct { Name string json:"name" yaml:"bar" }
How can I change one type to another type without copying?
In the same way that a []int can use conversion to become a IntSlice because
type IntSlice []int
If your struct types are identical, other than the tags, then they can use conversion to become another type.
rephrase: Let’s say I have two struct types that are the same except they have different JSON tags (used by different APIs for example). If I have a slice of one type and I want to convert it to a slice of the other type without doing a linear amount of copying, is that a thing that is possible / advisable with Go?
type type1 struct {
Foo string `json:"foo1"`
Bar int `json:"bar1"`
}
type type2 struct {
Foo string `json:"foo2"`
Bar int `json:"bar2"`
}
func main() {
t1 := type1{Foo: "abc", Bar: 123}
t2 := type2(t1)
json.NewEncoder(os.Stdout).Encode(t1)
json.NewEncoder(os.Stdout).Encode(t2)
}
This only works if they are exactly the same. Order must match.
Ignoring the tags was added in Go 1.8: golang/go#16085
What is this casting?
It isn’t casting, its a type assertion.
What is up with values on a package?
where you’re setting a value on a field in a library, not an instance var,
http.DefaultTransport.(*http.Transport).TLSClientConfig = &tls.Config{InsecureSkipVerify: true}`
Data: Misc
How do I keep persistent data across program calls?
Use a marshal format to a file or use a database. For many simple things the standard library gob encoder can be used to write an object graph to a file. JSON may also be a nice format if a human needs to read or write the file.
How do I print a data structure?
The %v and %#v fmt formatters are very useful.
Sometimes, serializing to json can be useful, even moreso with json.MarshalIndent
json.MarshalIndent(my_thing, "", " ")
How do I copy a structure?
The assignment operator shallow copies any values on assignment.
url := returnsAURLptr() newurl := *url
To deep copy, one easy way to is gob Encode and Decode.
If that is not applicable, you will have to copy yourself.
Files and formatters
Files
How do I buffer output?
The stdlib has a bufio package.
How do I read a file line at a time instead of the entire file?
The bufio.Reader type has a ReadLine() method, but don’t use it. As its documentation says, it is low level. Use ReadString('\n') instead.
Use a scanner. (from golang docs)
func main() {
scanner := bufio.NewScanner(os.Stdin)
for scanner.Scan() {
fmt.Println(scanner.Text()) // Println will add back the final '\n'
}
if err := scanner.Err(); err != nil {
fmt.Fprintln(os.Stderr, "reading standard input:", err)
}
}
How do I change, insert, delete a line in a file, or append to the beginning of a file?
The basic operation of doing this is to read or seek to the location of the file to be updated and then writing what is to be inserted or changed and then writing the remainder of the file.
How do I count the number of lines in a file?
See above for reading a file line at a time and use a counter.
How do I delete the last N lines from a file?
The easy concept is to count the number of lines in a file (see above) and then print that number minus N to a new file.
The real question may be how to do it on a large file without doing a lot of copying. One trick is to read the file backwards. Unforntunately, there is no Scanner which can operate on a Reader in reverse. One would need to read some chunk size at a time in reverse, counting the lines numbers and then when the point T is found, use (*File).Truncate(T).
How can I copy a file?
Open the source file for reading, open the destination file for writing. Use io.Copy(dst, src) to copy the bytes.
src, err := os.Open(srcName)
if err != nil {
log.Fatal(err)
}
defer src.Close()
dst, err := os.OpenFile(dstName, os.O_RDWR|os.O_CREATE, 0755))
if err != nil {
log.Fatal(err)
}
defer dst.Close()
io.Copy(dst, src)
How can I copy a file to a certain point?
It depends… If the file is text, use Scanner to find the point and write only what you want written.
src, err := os.Open(srcName)
if err != nil {
log.Fatal(err)
}
defer src.Close()
dst, err := os.OpenFile(dstName, os.O_RDWR|os.O_CREATE, 0755))
if err != nil {
log.Fatal(err)
}
defer dst.Close()
scanner := bufio.NewScanner(src)
for scanner.Scan() {
line := scanner.Text()
if line == `line I'm looking for` {
break
}
io.WriteString(scanner.Text())
io.WriteString("\n")
}
if err := scanner.Err(); err != nil {
fmt.Fprintln(os.Stderr, "reading standard input:", err)
}
If the file is binary, use io.Copy and a LimitedReader.
src, err := os.Open(srcName)
if err != nil {
log.Fatal(err)
}
defer src.Close()
dst, err := os.OpenFile(dstName, os.O_RDWR|os.O_CREATE, 0755))
if err != nil {
log.Fatal(err)
}
defer dst.Close()
l := io.LimitedReader{R:src, N:2048}
io.Copy(dst, l)
What if my file isn’t really text or Scanner can’t handle it?
Scanner buffers each line it reads. If you are reading a mixed format file, for example PDF which looks kind of like plain text, but can have large binary parts throughout it, then you may go beyond Scanner’s default 64kB token size.
You could set Scanner’s buffer using the Buffer(buf []byte , max int) method.
scanner := bufio.NewScanner(src)
buf := &bytes.Buffer{}
scanner.Buffer(buf, 256 * 1024) // max 256kB token size.
for scanner.Scan() {
...
Or, Read chunks. include the last few bytes of the previous buffer when searching for "it". I wrote something called UntilReader which is a bit like LimitedReader only instead of stopping when it has read N bytes, it stops when it has found matching bytes.
You can use it like this:
...
r := &UntilReader{R: src, S:[]byte("needle")}
io.Copy(dst, r)
Everything in the src Reader will be copied to the dst Writer up to and including the bytes of the string "needle".
Here is the implementation of UntilReader:
type UntilReader struct {
R io.Reader
S []byte
b []byte
}
func (u *UntilReader) Read(p []byte) (n int, err error) {
if len(u.S) == 0 {
return 0, io.EOF
}
// The underlying reader could do anything so use ReadFull to fill p.
n, err = io.ReadFull(u.R, p)
if err == io.ErrUnexpectedEOF {
// ReadFull returns ErrUnexpected, but lets make that EOF.
err = io.EOF
}
prev := append(u.b, p...)
ublen := len(u.S)
if ublen > len(prev) {
ublen = len(prev)
}
if i := bytes.Index(prev, u.S); i >= 0 {
u.S = nil
// If this is the first read and a match was found, prev isn't
// offset by ublen, yet add it back to i.
if u.b == nil {
i += ublen
}
return i, io.EOF
}
u.b = prev[len(prev)-ublen:]
return n, err
}
How can I modify a large file, which cannot fit into memory, while reading it?
Similarly to the above UntilReader, wrap the reader and buffer it.
package main
import (
"bytes"
"io"
"os"
"strings"
)
func main() {
var r io.Reader
src2 := strings.NewReader("abcdefghijklmnop")
r = &ReplaceReader{R: src2, S: []byte("de"), D: []byte("Z")}
io.Copy(os.Stdout, r)
io.WriteString(os.Stdout, "\n")
}
type ReplaceReader struct {
R io.Reader
S []byte
D []byte
buf bytes.Buffer
}
func (u *ReplaceReader) Read(p []byte) (n int, err error) {
io.CopyN(&u.buf, u.R, int64(len(p)+len(u.S)))
buf := bytes.NewBuffer(bytes.Replace(u.buf.Bytes(), u.S, u.D, -1))
u.buf = *buf
return u.buf.Read(p)
}
How do I make a temporary file name?
The io/ioutil package has a
TempFile function.
How can I manipulate fixed record length files?
Create a [N]byte array of a certain size and pass to Read() to read only the
fix record length you wish. Use the functions in the encoding/binary
package to turn the bytes into types you intend to read.
TODO
How can I write to a string?
bytes.Buffer is probably what you want. It is a writer. You can use
io.WriteString and call the buffers String() method to get the built string.
How can I use a string as an io.Reader?
strings.Reader is an io.Reader to an underlying string.
r := strings.NewReader("this is my string")
How can I output my numbers with commas added?
See this stackoverflow question How to fmt.Printf an interger with thousand commas
Write it yourself or use a library like message[https://godoc.org/golang.org/x/text/message] or humanize[https://github.com/dustin/go-humanize]
How can I translate tildes(~) in a filename?
First you should consider if this is the right thing to do. Normally the shell is the only thing which expands ~ to the $HOME environment variable. Other programs typically do not do this. It is probably best to accept input filenames as args so that the shell has already expanded them.
If you must continue, use strings.ReplaceAll to replace "~" with os.Getenv("HOME").
For other globbing operations, use filepath.Glob[https://golang.org/pkg/path/filepath/#Glob].
How can I reliably rename a file?
Use os.Rename
How can I lock a file?
Beware: not all platforms support locking.
If you choose to continue you can call syscall.Flock passing the fd returned from Fd() on an os.File
All I want to do is append a small amount of text to the end of a file. Do I still have to use locking?
It depends. The most important part is to understand the open and write semantics of the platform on which you are running. Windows and POSIX behave quite differently in this regard. Ultimately, if two processes (or threads) are going to be opening, writing, and closing a single file, it should probably be locked.
How do I get a files typestamp?
Use os.Stat or if the file is already open, call the Stat method on the *File. The ModTime[https://golang.org/pkg/os/#FileInfo] on the returning FileInfo interface is the modification time of the file.
How do I print to more than one file at once?
MultiWriter returns an io.Writer which writes to all of the writers passed to it when written to. Open the files and pass them to MultiWriter.
How can I read in an entire file all at once?
If possible, you should avoid this. Unless you know with certainty that a file is small enough that it will not use significant amounts of memory, a buffered reader of some kind is always more efficient. See above for reading line by line.
If you must do this, then ioutil.ReadFile reads an entire file into a []byte which you can cast to a string.
How can I read a file by paragraphs?
Use a custom split function with Scanner and do it just like above reading by lines.
func main() {
paras:=`Hello world
this is a test.
What is a paragraph? What does it mean to be a para?
What do you think?
What can be done?`
scanner := bufio.NewScanner(bytes.NewBufferString(paras))
scanner.Split(func(data []byte, atEOF bool) (int, []byte, error) {
if atEOF && len(data) == 0 {
return 0, nil, nil
}
if i := bytes.Index(data, []byte("\n\n")); i >= 0 {
// We have a full newline-terminated line.
return i + 2, data[0:i], nil
}
if atEOF {
return len(data), data, nil
}
return 0, nil, nil
})
i:=1
for scanner.Scan() {
fmt.Println(i, scanner.Text()) // Println will add back the final '\n'
i++
}
if err := scanner.Err(); err != nil {
fmt.Fprintln(os.Stderr, "reading standard input:", err)
}
}
This works, but has the drawback that non-empty lines with spaces are not detected as paragraph separators. Detecting such lines is left as an exercise to the reader.
How can I read a single character from a file? From the keyboard?
Use a byte slice of length 1 and pass it to the Read() function.
b := make([]byte, 1) some_file.Read(b) fmt.Println(b)
This will work on the keyboard too, but not most terminal programs. You’ll
need to turn off terminal echo processing. The easiest way to do this is
probably to use os/exec to call the stty program.
b := make([]byte, 1)
exec
os.Stdin.Read(b)
fmt.Println(b)
How can I tell whether there is a character waiting on a file?
Given Go’s goroutines, you shouldn’t need to do this. Read in a goroutine and communicate over a channel to another goroutine which does what you want when there is data available and when there is not.
Alternately, use a buffered Reader from the bufio package. The Buffered() method on bufio.Reader returns the number of bytes that can be read from the buffer.
How do I do a tail -f? aka follow a file as something else writes to it?
TODO
Why can’t I use "C:\temp\foo" in Windows paths?
Ooops, you just put a tab and a form feed into that string for a filename.
Escape your string.
An easy way of escaping strings like this in Go is to use backticks instead of
double quotes. C:\temp\foo works because backticks don’t expand escape
sequences.
Why doesn’t Glob(".") get all the files?
Because there isn’t a dot extension on most files outside of Windows. Glob uses unix globbing conventions.
Why can I delete a read-only file?
Know how your operating system and its filesystems work. A read-only file is not locked from deletion. In other words: a read-only file is prevented from being changed, but deleting it does not change that file. It does change the directory in which it exists. Check the permissions of the directory containing the file.
How do I select a random line from a file?
An easy way for a small enough file is to read all of the lines into a slice of strings and select a random element from the slice. See the previous question on selecting a random item from a slice.
How do I traverse a directory tree?
The path/filepath package has a Walk function which accepts a WalkFunc which is called for each file or directory in the tree.
How do I delete a directory tree?
The os package has a RemoveAll function.
How do I copy an entire directory?
You could walk the directory and copy each item found, see the previous question about traversing a directory tree.
Alternately, there is a third party package, github.com/otiai10/copy
Common Formats
How do I write a JSON file?
encoding/json allows one to easily marshal and unmarshal JSON. Use a json encoder.
dst, err := os.OpenFile(dstName, os.O_RDWR|os.O_CREATE, 0755))
if err != nil {
log.Fatal(err)
}
defer dst.Close()
err = json.NewEncoder(dst).Encode(something)
if err != nil {
log.Fatal(err)
}
How do I read a JSON file?
Use a json decoder.
src, err := os.Open(srcName)
if err != nil {
log.Fatal(err)
}
defer src.Close()
err = json.NewDecoder(src).Decode(something)
if err != nil {
log.Fatal(err)
}
How do I write a CSV file?
Use the Writer available in encoding/csv.
dst, err := os.OpenFile(dstName, os.O_RDWR|os.O_CREATE, 0755))
if err != nil {
log.Fatal(err)
}
defer dst.Close()
w := csv.NewWriter(dst)
// Use Write, or use WriteAll and pass a [][]string
err = w.Write([]string{"one", "two", "three"})
if err != nil {
log.Fatal(err)
}
How do I read a CSV file?
Use the Reader available in encoding/csv.
src, err := os.Open(srcName)
if err != nil {
log.Fatal(err)
}
defer src.Close()
r := csv.NewReader(src)
// Use Read to get one record or ReadAll to get all of them as [][]string.
r1, err := r.Read()
if err != nil {
log.Fatal(err)
}
r2, err := r.Read()
if err != nil {
log.Fatal(err)
}
theRest, err := r.ReadAll()
r2, err := r.Read()
if err != nil {
log.Fatal(err)
}
How do I write an XML file?
Use the Encoder in encoding/xml.
dst, err := os.OpenFile(dstName, os.O_RDWR|os.O_CREATE, 0755))
if err != nil {
log.Fatal(err)
}
defer dst.Close()
enc := xml.NewEncoder(dst)
err = enc.Encode([]string{"one", "two", "three"})
if err != nil {
log.Fatal(err)
}
How do I read an XML file?
Use the Decoder in encoding/xml.
src, err := os.Open(srcName)
if err != nil {
log.Fatal(err)
}
defer src.Close()
dec := xml.NewDecoder(src)
err = dec.Decode(&something)
How do I parse something?
Write a parser. There are tools but consider writing it by hand. It is often simpler. Watch Lexical Scanning in Go by Roby Pike.
Web Stuff
How do I fetch a URL?
Use the Client in net/http.
resp, err := http.Get("http://example.com/")
io.Copy(os.Stdout, resp.Body)
How do I ignore TLS?
where you’re setting a value on a field in a library, not an instance var, `http.DefaultTransport.(*http.Transport).TLSClientConfig = &tls.Config{InsecureSkipVerify: true}`
How do I fetch N URLs in parallel?
How do a make a web server?
Use the Server and Muxers in net/http.
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "Hello World, %q", html.EscapeString(r.URL.Path))
})
log.Fatal(http.ListenAndServe(":8080", nil))
What REST framework should I use?
None. Use net/http. Seriously. Nearly all of the 3rd party things are built from net/http and only add complexity. Find out what features you need in a 3rd party library before you use one. net/http is complete for the most common tasks.
How do I make a REST API?
Use the servers and muxers in net/http as you would for a web server and return and accept data the the XML, JSON, or other formats you need.