How software developers really spend their day
So here’s what i did this afternoon.
I was migrating some git repositories. A repository is something developers use to safeguard code. One of the most popular systems to do that is called git. For reasons that are beyond the scope of this blogpost the git repositories for my team were located on a personal account instead of a company account. Obviously this is a situation that needs to be fixed.
This task seemed simple enough: copy the repositories, change a couple of references and be done with it. The first step (the copying) was actually finished in just a couple of minutes.
Unfortunately things (and my afternoon) went downhill from there. One of the repositories was used as a Javascript dependency in around 250 projects that my colleagues and i have been working on for the past couple of years. A dependency in a software project is another piece of software that is needed to perform some well-defined task.
I needed to make sure that the more recent of those 250+ projects got an up-to-date version of this dependency, specifically one that was above version 3.0.0. This dependency was included as a direct link to the git repository, opposed to the more regular method of using a ‘registered package’ (this distinction is important later on in this story).
I already had this working (or so i thought), but one colleague mentioned to me earlier that for some reason the dependency didn’t install on his laptop. The thing is, i already had a hunch what the problem could be. One of the most popular options to install dependencies in the Javascript world is called npm, the so called node package manager (note that ‘package’ and ‘dependency’ are more or less the same thing). Because Javascript is in constant flux and things get outdated very fast there is actually another popular package manager that’s basically the same thing as ‘npm’ but does things slightly different. That package manger is called yarn.
I use yarn, my colleague uses npm. And that was, i thought, the cause of the problem.
So, i experimented with the dependency and the quite esoteric syntax to get ‘i need version 3.0.0 or later of this package’ working with the two different package managers (npm and yarn).
It didn’t work. In any of those package managers.
First i thought i was doing something wrong with the syntax. Then after many failed attempts i gave up and decided that maybe this method just wasn’t working. Instead of using the direct link to the git repository i decided to publish the whole thing as a ‘registered package’. This means that the package is published under a short name in a public register (known as the ‘npm registry’) instead of the direct url to the git repository.
However, publishing the package didn’t work. I got a pretty cryptic error message. After some Googling it turns out that my package name was ‘too similiar’ (but not identical!) to another package.
No worries, apparently i could use something called ‘scoped packages’ that would fix this problem and make my package name unique (at least unique enough to stop npm from throwing errors).
Turns out that making my package ‘scoped’ ment i also needed to change lots of code in all of those 250 projects that depended on it.
That seemed like a lot of extra work, so i decided to try one last time getting the git repository link to work as intended.
And then, after a closer look, i realised i didn’t transfer any of the version tags from the old to the new repository, i only transferred the files. Npm and yarn need those tags to install dependencies of a certain version.
So, after transferring the version tags i tried installing the dependency again with ‘npm’ and voila: it worked!
Just as a precaution i checked if my dependency installed with ‘yarn’ as well (remember the story about the two package managers that do the same thing but aren’t the same thing?).
It didn’t.
From a situation where installing dependencies in yarn worked but not in npm i now had the complete opposite.
I got some pretty esoteric error from yarn this time, which is weird, given that i’ve used the official documentation to figure out the exact way to indicate a ‘version range’, and the thing installed with npm perfectly.
So i sighed, and turned to Google again. I looked at all the known bugs and issues with yarn and found a couple that seemed similar to my problem, but none that were really helpful. So i reported a bug with the yarn project, figuring i would probably need to postpone the whole repository migration.
Fortunately i got a response pretty quick and it turns out this was actually a known problem, and already fixed in a new version of the tool that was released in a beta version just a day before. I tried the new version and voila: no problems installing the dependency. This beta version of yarn is probably going to be released in a couple of days, so if i had waited with transferring the repositories just a few days and updated my installation of yarn then i wouldn’t have encountered this problem at all.
After i was finally done migrating the git repositories and making sure all dependencies were installed i looked at the clock. Three pretty useless hours had gone by.
So, is there any moral to this story? Things that could have prevented me from spending my afternoon chasing ghosts and cryptic error messages?
I guess i could have taken a longer look at some of the error messages, so i might have realised earlier my files were transferred but my version tags were not. I guess the yarn project could spend some time rephrasing their error messages to be less cryptic. I guess git might transfer files and tags by default. I guess npm could warn about the problems you might encounter if you use scoped packages.
But in all honesty i think it’s just another story that shows that software development is a difficult, tricky and error-prone practice. Situations like this happen to me virtually every day and i guess, to every other developer in the world as well.
And we, humble programmers, should just accept that as a sad and sobering fact of life.
Anonymous
Great post! Only thing is, how do you account this time to your (often clueless) manager?
hay
Fortunately my manager is not that clueless :) I guess every good manager understands that a lot of time is spent on seemingly useless tasks.
Anonymous
I’d like to give you and every other dev who has gone/and is going through this a hug.
hay
Ah, that’s nice. Thanks!