Skip to content

Things (in Go) I’ve been learning this week

Earlier this week, I was replicating an issue that involved creating 1000 Terraform workspaces in S3, and I ended up having a week of learning more Go things in the process.

Since I would like to write here more (ah, the most common refrain on any blog), and also to capture some of the things I’m learning for future-me, I’m going to write some of the things I dug into, and where I was reading about them.

Yeah future-me, you’ve come a long way! I believe in you! Sincerely, past-me.

How do I execute command line stuff in Go (i.e. scripting)

I’ve been writing a bunch of Bash in the last month, to the degree that doing it in Go starts to feel annoying and noisy (ugh, types, being in my way, when I can just do arbitrary system things). Which was a sign that I should probably practice writing this stuff in Go.

So when I needed to create a bunch of stuff in S3, I was like, okay, instead of Bash, let’s make a main.go and run it with go run main.go and script it that way.

Which is how I read a few posts around command line execution, but I really liked this one for breaking things down simply and step-by-step. I also referenced the main Go docs, but the blog post was way handy.

How do I parallelize things in Go?

This led to the very handy, very well-named post: Concurrency and Parallelism in Golang.

Why yes, that’s exactly my question, my internet friend!

This was a hole I fell into after I’d let my computer chug for … a long time … to make 1000 Terraform workspaces in S3. Goroutines and channels remain one of those bits I don’t get to do that often, and so I haven’t absorbed how they work completely.

But I realized — hey, I’m scripting 1000 things to happen, shouldn’t there be a way to make this more efficient than: “make thing one … make thing two …”. Of course, what I learned (…again) is that goroutines do not help with this, because they help with concurrency, not parallelism. That is, they help with organizing the many things, not doing the many things.

I spend more time reviewing the GoByExample on channels again.

Since I completed what I was actually doing, and already went on side-learning quests enough on this, I didn’t dig into this but I suspect that it would involve multiple processes, since once Thing Two is started, it’s messing with stuff happening in Thing One, as far as systemy stuff I was doing. The funny thing I guess is I feel like I got close to using and toward understanding something, and I think that was very helpful, honestly.

How do I tell if something is slower/faster or not?

I was reviewing a PR that changed the behavior of the S3 backend (to support over 1000 workspaces) and I wanted to know if, since this behavior change happened whether or not you have >1000 workspaces, if it measurably impacted command performance.

So I went to learn a little about benchmarking, which is a fascinating topic basically always.

I learned that Go’s testing package includes stuff for benchmarking out of the box (neat!). So I wrote a little test that does:

func BenchmarkMyCommand(b *testing.B) { 
// run the command b.N times
for n := 0; n < b.N; n++ {
exec.Command("terraform", "workspace", "list").Output()
}
}

I suspect something is totally wrong with this idea, but I don’t know what (probably that I could do this with running time, rather than Go). What I do know is I ran it a bunch of times, messed with the commandline flag options (-benchtime, for example, which increases the minimum time the benchmark will run) and performance appeared to be an acceptable difference, so 🚀. And I got my hands on running benchmarks, which was neat.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.