I found and fixed the build time regression bug. Last month I implemented some foundational changes to my static site generator to generalise how calendar based structures of markdown files are handled. It was mostly a great success, until that is, I noticed that the build times of my site had gone from around 1 min to over 6 mins. That's quite a big regression. I had never seen such a big regression in my build times. It wasn't noticeable until it was fully deployed into production, working with the complete dataset. That's how it goes sometimes, especially when you are resource contrained as I am at the minute.
It gets worse. I had in parallel to that feature been working on setting up an npm modules cache for the build and deploy workflow. The previous few months I had runout of build minutes so I wanted to speed up deploys. The obvious thing was to find a way to skip installation of all the modules. That's possible because most of the time these aren't changing. Anyhow the cache setup ran into issues. The docs are confusing and the stack overflow relevant posts even more confusing. Others have clearly been having similar issues. The long and short of it was that not only did I not manage to get the cache working, but the build times had 6X'd. So I now needed the cache just to get back to the build times I had previously, that were already causing me to run out of minutes.
Against an enormous sychronicity tsunami I managed to get the cache working a few days ago. That was no easy thing, as pretty much everything around me was blocking me. It's hard to describe, it's like the world or some part of the world local to me really didn't want me to get the cache working. There were also problems with Github support which I've written about elsewhere. All sorts of fractaly type patterns in news stories, that had the same exact shape as the things happening to me with my code. I know this will sound strange to some. It's even stranger when it happens to you, I can tell you that.
Anyhow with the cache working, build times were back to the previous level, which were still problematic but probably manageable. I spent this morning, still experiencing a lot of push back from the world around me by the way, trying to find the build time regression bug. Turns out that the foundational work had introduced an easy way to change the order of posts on any given day. I implemented it by reversing the order of the array containing the day's posts. That had a much bigger than expected affect because I have so many links on the linkblog. Each day was invoking reverse. The fix was quite simple. Just iterate over the data in reverse rather that change the data. It's a bit more code, because you have to do the same iteration in both directions so there is some repetition, but it's 6X faster. The over-optimisation caused the problem in the first place.
The build times are now at around 4 mins for the whole site. That's a lot better than the 14 mins builds I was seing a few days ago. I'm happy about the much needed improvements, but I'm still dealing with the backlash from a very angry world that once again appears to be blaming me for everything in the entire universe, with enormous righteousness, starvation, thirst and mutilations. I know this might sound overblown to some, but seriously you should try being in my shoes for a while. One day it will all become clear what has been happening.
[Small precision about build times to clear up any confusion amoung the pedants out there: the site static site generator build 6X'd, but the overall time for the entire build + deploy workflow went from ~7 mins to ~14 mins] #