markjgsmith

Slowing down to speed up

When I got the recent website redesign working and was able to finally get it deployed live, I had been working on the serverside of my new publishing tool for a while, and the static site renderer had been ticking along very nicely without any real issues. However when I switched back to the rendering and finally tried to build my whole website, with over 10 years worth of content, I had a bit of an unpleasant surprise.

The old system was consistently taking about 4-5 minutes to build and deploy, but the new system when I first ran it, took 45 minutes just to do the render. And in fact things would get even worse before they got better. I later discovered that the Github Actions runner had started failing with out of space issues. When I looked into it and ran the build locally it was taking up about 15GB of space. I undid the change I ahad just made and got it back down to 5GB, and the deploy started completing again, but it was clear I had both render time issues and space utilisation issues.

As you can imagine my heart sank considerably. Had I missed something important in my architecture? Did I overlook something fundamental that invalidated my whole approach? Not a great situation. I worked on optimising the rendering for several days. Got the times down to 30 minutes, then 15 minutes, and then 6-7 minutes. At which point I knew that things were back in the right general region, but I could tell there were some big improvements that could be made.

Well I've spent another few days on it, and I“ve made some really huge speed improvements and with it stability and much better HTML and CSS structure. Feeling pretty good about it.