Once changes to the data structures slowed down a bit I started to define them in a more respectable way by putting them in a common library to be used across services. This was great but because naming is extremely difficult, I went back and forth a few times with the module name and its packages. Well, this broke all my services using the common library and gave me dependency naming errors that I knew damn well were wrong. I tried all the fixes I found on the internets:

• go mod tidy
• go clean -modcache
• go clean -cache
• Manually deleting the dependencies in GOPATH

I even tried a nonsensical Hail Mary of deleting my go.mod and generating a new one. Nothing. Eventually I scrolled way down on a Stack Overflow post and found something that didn’t even cross my mind:

Since Go 1.13, the go command by default downloads and authenticates modules using the Go module mirror and Go checksum database.

Ah! That’s why it worked at first but started failing when I made changes to the common library quickly, the changes hadn’t been picked up by the proxy cache yet. This works perfectly if you just need a specific version of a dependency but not so much for rapid development. To directly interact with the dependency’s repo without any caching, you just need to configure GOPROXY like so:

GOPROXY=direct go build main.go

Bonus

Even though this solves my initial problem of quickly updating my common library, I still need to commit changes, push changes, and pull those changes back to my workstation which is pretty cumbersome. Fortunately, Go has an answer for this with the replace directive which allows you to point your dependency to a different location:

go mod edit -replace gitlab.com/common/lib=../common/lib

This will update your go.mod to seamlessly use the local module in ../common/lib instead of the remote module at gitlab.com/common/lib. Then, once you’re done making frequent changes to the dependency, you can put it back to use the original remote location for the dependency:

go mod edit -drporeplace gitlab.com/common/lib

This is awesome for making changes quickly. Now I just have to figure out a way to use this when I want to do rapid development using containers…

Be careful how file path values are written to the git index because they can be set to any value (not just filenames) if you’re manipulating the index manually instead of with git. I created a repo with all the code snippets to make things easier to follow instead having of digging through my project with of unrelated application logic.

Project

I was working on a Go project recently that needed to interact with a git repo that took me down a very unexpected, but interesting, rabbit hole. Without going into a bunch of unrelated detail, my project needed to accomplish the following things in a git repo:

• Read a file in the repo
• Update the contents of the file
• Write the changes to disk
• Add the file changes to git
• Commit the file changes

Implementation

I’m writing this project in Go, so I’ll need a way to interact with git. I decided to use go-git because it was one of the first libraries that popped up. But first, let’s read the file:

func ReadFile(filename string) (string, error) {
	b, err := os.ReadFile(filename)

	return string(b), err
}

Update the file contents that is returned from the function above:

updatedContent := strings.ReplaceAll(content, "great", "good")

Write the updated contents to file:

func WriteFile(filename string, content string) error {
	tmpFile := fmt.Sprintf("%s.tmp", filename)

	fout, err := os.Create(tmpFile)
	defer fout.Close()

	if err != nil {
		return err
	}

	bufOut := bufio.NewWriter(fout)
	defer bufOut.Flush()

	_, err = bufOut.WriteString(content)

	if err != nil {
		return err
	}

	err = os.Rename(tmpFile, filename)

	if err != nil {
		return err
	}

	return nil
}

Finally, add the updated file to git:

func AddToRepo(repo string, filepath string) error {
	gitRepo, err := git.PlainOpen(repo)

	if err != nil {
		return err
	}

	gitWorktree, err := gitRepo.Worktree()

	if err != nil {
		return err
	}

	_, err = gitWorktree.Add(filepath)

	if err != nil {
		return err
	}

	_, err = gitWorktree.Commit("Updated file contents", &git.CommitOptions{})

	if err != nil {
		return err
	}

	return nil
}

Bring it all together to run:

package main

import (
	"bufio"
	"fmt"
	"github.com/go-git/go-git/v5"
	"os"
	"strings"
)

func main() {
	repo := "."
	filename := "my-file"

	filepath := strings.Join([]string{repo, filename}, string(os.PathSeparator))

	content, err := ReadFile(filepath)

	if err != nil {
		os.Exit(1)
	}

	updatedContent := strings.ReplaceAll(content, "great", "good")

	err = WriteFile(filepath, updatedContent)

	if err != nil {
		os.Exit(1)
	}

	err = AddToRepo(repo, filepath)

	if err != nil {
		os.Exit(1)
	}
}

func ReadFile(filename string) (string, error) {
	b, err := os.ReadFile(filename)

	return string(b), err
}

func WriteFile(filename string, content string) error {
	tmpFile := fmt.Sprintf("%s.tmp", filename)

	fout, err := os.Create(tmpFile)
	defer fout.Close()

	if err != nil {
		return err
	}

	bufOut := bufio.NewWriter(fout)
	defer bufOut.Flush()

	_, err = bufOut.WriteString(content)

	if err != nil {
		return err
	}

	err = os.Rename(tmpFile, filename)

	if err != nil {
		return err
	}

	return nil
}

func AddToRepo(repo string, filepath string) error {
	gitRepo, err := git.PlainOpen(repo)

	if err != nil {
		return err
	}

	gitWorktree, err := gitRepo.Worktree()

	if err != nil {
		return err
	}

	_, err = gitWorktree.Add(filepath)

	if err != nil {
		return err
	}

	_, err = gitWorktree.Commit("Updated file contents", &git.CommitOptions{})

	if err != nil {
		return err
	}

	return nil
}

Now let’s make sure git looks like it’s supposed to:

$ git status
On branch master
Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
        new file:   ./my-file
        new file:   my-file

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   my-file

What the hell? I made a change to a single file, but now I have two staged changes and one unstaged change all for the same file??

Troubleshooting

Sanity check

Just to make sure I’m not forgetting something from the way I normally interact with git, let’s try doing it in bash:

file='my-file'

cat "${file}"

content="$(< ${file})"

printf "%s\n" "${content/great/good}" > my-file.tmp

mv my-file.tmp my-file

cat "${file}"

git add "${file}"
git commit -m "Updated file contents"

Nothing exciting here, mostly just the usual flow of git commands. Now let’s check how things look in git here:

$ git status
On branch master
Your branch is ahead of 'origin/master' by 1 commit.
(use "git push" to publish your local commits)

Well, this is what I was expecting all along. So what’s being done differently between git and go-git?

Git index

I started digging in deeper to the git side of things by comparing the state of the repo with each approach. This is the initial state of the git index before any changes are made to the repo:

$ git ls-files --stage my-file
100644 09721149f4f734c171891b7da25c4f7db30ea616 0       my-file

Now let’s run the sanity check implementation and check the index again:

$ git ls-files --stage my-file
100644 47c9021d2cbbd8f691d9ff2e1df9a053ad7b28ee 0       my-file

Okay, things look good. The updated file is there and the hash has been updated as expected (because the file was modified). Now let’s put the repo back to its original state:

$ git reset --hard origin/master

Then run the Go implementation and check the index again:

$ git ls-files --stage my-file 
100644 09721149f4f734c171891b7da25c4f7db30ea616 0       my-file

Weird, the hash is the same as the original file even though it’s been changed. Let’s look at the whole index:

$ git ls-files --stage
100644 47c9021d2cbbd8f691d9ff2e1df9a053ad7b28ee 0       ./my-file
100644 09721149f4f734c171891b7da25c4f7db30ea616 0       my-file

Well this doesn’t make any goddamned sense. The hash for my-file is the same as the hash for the original file and the hash for ./my-file is the same as the hash for the updated file. So does that mean the go-git library is updating the git index wrong?

Spoiler alert: yes it does.

Debugging

I’ll spare you the actual process of setting breakpoints in the go-git library to chase down exactly where this problem arises, but it ends up boiling down to this section of worktree_status.go:

if err != nil || !fi.IsDir() {
    added, h, err = w.doAddFile(idx, s, path, ignorePattern)
} else {
    added, err = w.doAddDirectory(idx, s, path, ignorePattern)
}

In this case, I’m concerned about doAddFile() which passes the filepath as ./my-file and takes me further down into the library where the index is actually being modified in index.go:

func (i *Index) Add(path string) *Entry {
    e := &Entry{
        Name: filepath.ToSlash(path),
    }

    i.Entries = append(i.Entries, e)
    return e
}

And there it is!! The path that is being passed is ./my-file instead of my-file which means the index is being updated with the wrong filename or at least a different filename than the git CLI uses.

I’ll create an issue with the go-git people to see about getting this fixed if it’s not the intended behavior (it would be strange if this was the intended behavior). Either way, now I can just update my application to clean up the filepath before sending it over to go-git. Plus, this little speed bump forced me to dive into the git internals to track down my issue, so I guess it wasn’t a complete waste of time.

Just create a global variable where your main function lives:

package main

import "fmt"

var (
	Version string
)

func main() {
	fmt.Printf("Version: %s\n", Version)
}

And set the flag when you build your application:

go build -ldflags="-X main.Version=1.0.0"

Where main is the package that your variable Version lives. Now when we put it all together we can see the version that was passed in:

$ go build -ldflags="-X main.Version=1.0.0"
$ ./main
Version: 1.0.0

You can take this a step further by setting more details in a Makefile dynamically using info from git. Same idea in the source file:

package main

import "fmt"

var (
	Version string
	Commit  string
)

func main() {
	fmt.Printf("Version: %s Commit: %s\n", Version, Commit)
}

Assuming your project is in a git repo, create the Makefile:

LD_FLAGS := -X main.Commit=$(shell git rev-parse HEAD) -X main.Version=$(shell git describe --always)

build:
    go build -ldflags="$(LD_FLAGS)"

Build using the Makefile and run:

$ make build
$ ./main
Version: some-git-version Commit: some-git-commit

And now you can have version information in all your Go applications.

I’ve often seen guides on how to write an application for Kubernetes and how it will fix all your problems by allowing “organizations [to] achieve increased efficiency and agility”. Overall, that guide makes some good points about Kubernetes offering better scalability and flexibility, but it doesn’t talk about how I’m supposed to use Kubernetes every day as a developer without wanting to kill myself significant friction.

Problem

Many of the benefits that cloud native applications offer can be a huge disadvantage for traditional application development such as isolation and ephemeral runtime environments. For example, when I make changes to an application using a traditional workflow I need to:

vim app.go
make
./app

This simple set of tasks will happen numerous times throughout the day and I can merrily develop software.

But when I make changes to a cloud native application, I need to:

vim app.go
make
docker build -t localhost/some-app:$(git describe --always) .
docker push localhost/some-app:$(git describe --always)
helm install some-deploy some-chart --set image.tag=$(git describe --always)

Not a huge number of tasks, but you have to run multiple applications and this gets very tiresome having to flip through your Bash history all day for every single change. If you think this could be simplified by just using the latest tag you would be wrong because you should never ever use the latest tag. Also, you could write a script to run these steps which wouldn’t be terrible but, now you have this custom script for each project that’s going to definitely be slightly different across projects, and you have to worry about that maintenance especially if you want to use that script for prod deploys.

Plus, if you want to debug your application you’ll have to set up a build stage in the Dockerfile and manually forward ports out of the cluster.

Sane development

Years ago when I was still fairly new to cloud native development I came across a tool called skaffold which helps tie all of these tasks together. All you have to do is configure a skaffold.yaml in your project, then when you make changes it will look like this:

vim app.go
skaffold dev

That’s it! This will:

• compile your application
• build the Docker image
• push the Docker image
• deploy the Helm chart

It will also watch source file and execute these tasks again when it detects changes.

And if you want to debug changes made to your application:

vim app.go
skaffold debug

That will run all the tasks executed by skaffold dev and forward the application’s debug port out of the cluster to your workstation.

Try it

You’ll probably have some frustrating moments with Skaffold at some point. But for me, it is well worth it to have a unified method for developing, debugging, and deploying cloud native applications and avoiding the hell of having to manually operate a Rube Goldberg machine every time you make code changes.

The other day at work, I had a coworker look over some changes I made to one of our Java applications. He gave me some pretty standard feedback: this looks good, fix this, etc. But he also told me that I really shouldn’t be using var for my variable assignments because it “makes the code less readable”. I put a lot of effort in trying to make my code as readable as possible so his comment took me back for a moment. Have I been littering the code I write with this garbage that makes difficult to understand just to use some newfangled language syntax? So I took a few moments to try and understand why he thinks this.

Java type inference

Back in 2018 Oracle released Java 10 with a bunch of fancy new features. One of these features was the introduction of type inference though the use of the var identifier which can really simplify the way your code looks by stripping out a lot of unnecessary information. Take this code for example:

public class MyClass {
    public MyClass() {
        Map<String, String> myMap = new HashMap<>();
        myMap.put("some-key", "some-value");

        for (Map.Entry<String, String> i : myMap.entrySet()) {
            System.out.println(i);
        }
    }
}

The syntax for the loop is super verbose. You already know what the type of myMap is, so why clutter things with typing info? Here’s that same code using type inference:

public class MyClass {
    public MyClass() {
        Map<String, String> myMap = new HashMap<>();
        myMap.put("some-key", "some-value");

        for (var i : myMap.entrySet()) {
            System.out.println(i);
        }
    }
}

That looks much cleaner, and it’s easier to read. Granted this is just a meaningless few lines of code but that reduction of redundant typing info will add up quick in large codebase.

Of course though, you don’t want to use var if it’s going to make your code more difficult to read or put you in a situation to (probably) make dangerous typing assumptions.

Differing opinions

I’m glad I took the time to try and better understand why my coworker has the blanket opinion that using Java’s var makes code less readable, but I still completely disagree. Maybe he doesn’t understand Java’s typing system very well, maybe he just doesn’t like change, or maybe I’m still missing something? Java is a statically typed language with strong typing so the use of type inference is usually a good thing. That’s what I still think at least. As with basically everything else in programming, type inference can make your code easier to read when used properly.

My Hugo website 🎉