.gitignore 💔 Unicode
Have you ever run into a situation where Git just refused to obey your commands? No, I'm not talking about that time you "typo-ed"
git commit and ended up
git reset --hard-ing your repository back to the dawn of the universe, I'm talking about it really, truly, ignoring you.
I have, so let me tell you a story about what happened and how I fixed it so that you can avoid future hair-loss and avoid questioning the nature of your reality.
(For those who want to skip right to the punchline)
Make sure your
.gitignore file is saved in UTF-8 format, or you'll have a bad time.
I like to think of myself as a pretty adept Git user. Sure, I use the odd user interface from time to time, I play around with Git's porcelain commands and I know that there are plumbing commands which I can use if I really need to automate things through the CLI. I've also played around with libgit2 (which you totally should at some point).
None of this prepared me for staring at my editor, wondering why on earth my
.gitignore file didn't seem to be working. I've been around the block, I've written
.gitignore files. Hell, I've even written an app for that.
- ✓ It is named
- ✓ It has the correct file name in it
- ✓ The file name isn't quoted out
- ✓ There isn't another
.gitignoreoverriding my ignore policy
- ✓ The
.gitignoreis in a parent directory of the file I want to ignore
- ✓ The ignored files aren't currently tracked by Git (nor have they ever been)
- ✗ My sanity is dwindling
I'd just created a new project and added a quick
.gitignore file. This was essentially dumped out using
curl https://gitignore.io/api/go > .gitignore to bootstrap the project and I had no reason to doubt that it would work perfectly.
I then added a new entry to the
.gitignore file manually, leaving it looking like this:
# Created by https://www.gitignore.io/api/go # Edit at https://www.gitignore.io/?templates=go ### Go ### # Binaries for programs and plugins *.exe *.exe~ *.dll *.so *.dylib # Test binary, built with `go test -c` *.test # Output of the go coverage tool, specifically when used with LiteIDE *.out ### Go Patch ### /vendor/ /Godeps/ # End of https://www.gitignore.io/api/go # New addition junit.xml
And yet, Git would dutifully offer to add all of my
junit.xml files to my repository whenever I ran
Figuring out the Problem
This part is easy, turns out if you Google ".gitignore is ignored by Git" you'll find this StackOverflow question and the accepted answer will point you at the file encoding being a potential problem.
That's pretty quick to check, so I opened up my
.gitignore in VS Code and turned my gaze to the status bar.
UTF-16 it bellowed back at me, not quite what I was hoping to see.
A quick change to
UTF-8 and suddenly my
.gitignore was working again. What a relief!
Let's quickly dive into what is going on here because it turns out there's more to the story.
For starters, I didn't actually use
curl ... > .gitignore to get the file. Instead I used PowerShell on Windows and did a
Invoke-WebRequest ... | Set-Content .gitignore. That is pretty important because as it turns out, PowerShell on Windows will default to
UTF-16 encoding. If I'd known this, I'd have used
Set-Content -Encoding UTF8 .gitignore and saved myself the headache.
The next issue is that
git doesn't actually handle Unicode all that well. In fact, until relatively recently, it didn't know what a BOM was and even today, it only accepts well formatted
UTF-8 text. The motivation here is that the filesystems it deals with are
UTF-8 compatible and so it should work with the same.
I hope that this comes in helpful the next time you're faced with the same, or if you're simply looking for ways to drive your co-workers crazy.
Site Reliability Engineer, Microsoft