markjgsmith

The fast rendering blog is now live. In the end I was able to get the rendering down to around around 15 seconds for the entire 15 years worth of content. It‘s about 26000 files in total. These are the main features:

1. Renders fast (~15 secs) and the build takes up way less space (~500M).
2. Sleek new design, removed the content boxes, swapping them out for a design focussed on blog posts with supporting links and notes.
3. Added the old newsletter posts back to the mix.
4. Site header background images. The cool thing is the blog remembers the current active header image when a particular post is published, so when you look at old posts, they will look like how they were when they were first published. It‘s all controlled by your post frontmatter, and of course it‘s automatic if you use the render tool to create the content.
5. There‘s also a top right image feature, which makes it really easy to add an image that appears in the top right of your post. Again just a frontmatter field you add to your post.
6. Tags pages - this was on the old blog system, but I‘ve managed to make it much more useful. All posts have clickable tags so you can easily see other similar posts with 1 click.
7. A light weight bucket framework for creating organised HTML, where different content types just slot into place whether it‘s a single post, a flat list, or grouped by day.
8. Themes - The CSS is way way more organised. Themes are installable from an npm module, and you can easily customise things, by adding a user styles.

Since I will likely be changing the header image relatively often, I have added a headers page to keep a record of each new one as I add them. The first one is called The industrial distillation workshop, a humorous look at the new trend in AI development that people seem to be quite freaked out about.

The whole thing is built using my new rendering tool, to which I had to add quite a few things, and seems to be working very well.

Udpate: Here‘s how it looks:

This is a new header background image for the blog, the first one using the new stateful header images feature. It‘s a humorous take on the current moment in the tech industry where it seems everyone is freaking out about AI companies building their models by asking other models zillions and zillions of questions, otherwise known as distillation.

I came up with the theme and idea but it was Gemini that really brought it to life. I basically just said make a humorous image about everyone freaking out about model distillation. I think it‘s kind of spot on.

Obviously I don‘t personally endorse model distillation, to be honest I don‘t really know how to think about it, but I do know that I need to be freaked out about it!

Here's the full resolution image in case it‘s difficult to see the whole thing in the blog header:

I plan to keep a list of the various blog headers I add to the blog here.

This was the original first blog header background image I added on 2025/10/04. I mentioned it here at the time. Previous to this I was running a homepage that used this as the background image. It wasn‘t a blog. It just listed my contact details and linked to the blog which was on a subdomain.

I called it floating above the mountains because when the site title text got superimposed on the mountains it was like I was floating above them. I love mountains. I love snowboarding. Lots of fond memories.

I can‘t quite remember where it‘s from. I think it was part of a blog starter kit I got online somewhere. It was a really long time ago.

Here it is in full:

You can see all the other header images so far on the headers page.

Update: I remembered the source. It was from HTML5 UP.

I‘ve been seeing AT protocol mentioned in several places in the past few weeks. It‘s one of the bunch of social media protocols that emerged a few years ago. I wrote about it at the time, as I read through most of the big hitters. I remember quite liking AT proto, it felt very well put together, well thought out.

Dan Abramov was on Shop Talk Show EP#724 talking about it. He does a really good job of painting the AT Proto picture. I like that it‘s all ultimately just files, that you can look underneath and see what is going on. I get the impression that it wouldn‘t be too difficult to debug AT Proto app code, and I thought the notion that you get your own personal Dropbox folder in the cloud that any app can read and write to, was very compelling. Worth a listen.

A day in the life of a software developer:

• A: We have problem!
• B: OK lets fix the problem.
• A: The problem is fixed!
• B: Ok awesome full steam ahead!
• A: The fix has highlighted a deeper problem.
• B: Oh shit we have to pause the steam ship! Save everything! Let’s fix the deeper problem.
• A: The deeper problem is fixed! We are fucking exhausted!
• B: I think we can see light at the end of the tunnel! Resurrect the steam ship!
• A: It’s a fucking mess but we have boosters now.
• B: I am still quite confused.
• A: How many times have we been through this loop?
• B: Life is loop.
• A: We still have boosters.
• B: I fucking love boosters, just wish it wasn’t so fucking messy so we could actually use them.
• A: We are cleaning up the mess!
• A: You are never going to guess what just happened while we were cleaning!

I‘m still trying to get my head around all the things that I‘ve been through in the past few days trying to optimise the rendering of the website. The difficulty I had writing this blog post is an indication of that for sure. But I do know that I don‘t often get an opportunity like this to use the HTML details tag, so...

Remember, when I started optimising it was taking 45 minutes to render the last 10 years of content.

How long does it now take to render the entire 10 years of the blog?

And also remember it was taking up 5GB of space.

How much space does the render now take up?

It‘s still kind of a mess. There are some small things that need fixing. I have to figure out how to resurrect the steam ship.

Porting the new rendering pipeline to the live blog isn‘t completely straight forward, but luckily the new themes feature will make that a lot easier. The old theme was a hardcoded internal theme, but as part of this whole endeavour I turned the theme system into a full blown real feature. You can now install themes as npm packages, and the new optimised website rendering configuration is the first real external theme.

Definite progress, even if it‘s slow going at times.

When I got the recent website redesign working and was able to finally get it deployed live, I had been working on the serverside of my new publishing tool for a while, and the static site renderer had been ticking along very nicely without any real issues. However when I switched back to the rendering and finally tried to build my whole website, with over 10 years worth of content, I had a bit of an unpleasant surprise.

The old system was consistently taking about 4-5 minutes to build and deploy, but the new system when I first ran it, took 45 minutes just to do the render. And in fact things would get even worse before they got better. I later discovered that the Github Actions runner had started failing with out of space issues. When I looked into it and ran the build locally it was taking up about 15GB of space. I undid the change I had just made and got it back down to 5GB, and the deploy started completing again, but it was clear I had both render time issues and space utilisation issues.

As you can imagine my heart sank considerably. Had I missed something important in my architecture? Did I overlook something fundamental that invalidated my whole approach? Not a great situation. I worked on optimising the rendering for several days. Got the times down to 30 minutes, then 15 minutes, and then 6-7 minutes. At which point I knew that things were back in the right general region, but I could tell there were some big improvements that could be made.

Well I‘ve spent another few days on it, and I‘ve made some really huge speed improvements and with it stability and much better HTML and CSS structure. Feeling pretty good about it.

I've been refactoring the RSS feeds on the blog the past few days. I now generate the feeds without any feed libraries. It's just plain old templates. Total freedom. No more fighting against other people's vision of reality. I needed to add some custom namespaces, and just updated the template and my static site generator handles all the data preparation. Easy.

Over the years I've gotten into some pickles using various RSS libraries. But you don't need them!

I've added the <source:markdown> and <source:linkFull> namespaces. Here's some docs on the source namespace.

I've also added the podcasts back to the blog, and with a little help from an LLM it was super eady to whip up an RSS 2.0 feed with all the things you need to have it work in itunes. With LLMs and templates you can do most things :)

Sometimes when you get into a tangle a tried and tested way to proceed is to step back, list the tangles as best you can, step back again, look at your tangle description as if you were looking at an expensive piece of art in an art gallery.

Take a break.

Take another peek, don't get scared or overwhelmed.

Think about how you could isolate the different tangles, which no doubt all affect each other. They often do. Then create a minimal example. After which you will be several steps closer to your goal.

You might need to create several of these.

If you have been visiting the blog over the past few days you might have noticed that it has changed somewhat. I deployed the latest version a few days ago, and have been doing various tweeks and fixes. I think things are looking pretty good. The design is a lot more sleek, still minimalist and optimised for reading text. On small screens, the header is smaller to give you maximum screen real estate. It‘s mostly black text on white background, and the titles are bold, there are some black borders around content to keep he focus on the posts. It‘s crisp and clear, and since it‘s a static site, all pre-rendered, it should load really fast.

The other big thing is navigation. In the right sidebar, you will see a list of all the years of content, and you can navigate directly in there to any date and see posts. And in the header you will see a "Filter" dropdown that gives you the possibility to select to just view a specific content type, i.e. notes, posts or links, and it should switch to that content type for the specific date period you are currently looking at. I was able to do away with lots of clutter that was previously in the right sidebar, and now the most important links are in the header: About, Portfolio, Contacts and of course Feeds.

Another small tweek, the year in the footer copyright text at the bottom of the screen is automatically rendered based on the current date, so no more forgetting to manually update it and suddenly it‘s out of date. One less thing to worry about.

The thing that‘s not apparent just from looking at it is that the whole thing is rendered using my new static site generator. That‘s a massive boost in terms of robustness. The code is all in one repo and way way better written, way easier to reason about. It should be much easier to add features, without worrying about breaking things. As much as I loved the old system, and it got me very far, it had some rather rough edges. The new system is inspired by what I learnt the first time around, but it‘s a complete rewrite, with loads of new features, and a much more solid rendering pipeline. And is highly programmable.

There are still some things I need to implement, like for instance tags. You can see the tags on blog posts, but they don't link to pages that contain all posts with those tags just yet. More to write on this in the near future, but for now just happy the migration went quite smoothly, even if it was a lot of work.

This is how it looks:

I‘m putting the finishing touches to the re-architecting of the blog. It was all looking great but the realities of managing a stock implementation alongside the customized version for the site were clearly going to be an issue. So I implemented a light weight theming feature.

It‘s just internal for the minute, only one theme which is he blog, but can theoretically be expanded into a full feature later. Sometimes you have to do just enough to get going without boiling the ocean.

Got it working late last night, going to try and see if I can get a live version deployed today.

It‘s been really great to watch on YouTube people from all over the world discover the USA in their travels for the World Cup over the past few weeks. Pretty much everyone is totally in love with what they are finding.

I‘ve travelled quite a bit in the states, having done an east coast to west coast road trip several years ago, so I experienced first hand much of what folks are seeing. The USA really is an amazing place, and the people are super cool too. It‘s very different to what much of the main stream media would have you believe.

And so what a fantastically beautiful thing that it‘s the 250th birthday of the formation of the USA today.

Happy birthday and best wishes to all for the next 250 :)

It‘s really quite mind boggling how many small things there are to fix when you are deploying something for the first time. It feels like it will never end. So many niggly little fixes and things you didn‘t anticipate. It‘s very exhausting.

The good news is that as of a few minutes ago I successfully deployed the new site, fully built using my new rendering tool. I still have a few things to migrate over and I have to rebuild the data to make sure I have the latest, so it‘s not quite ready to show, but the crucial thing is it worked.

Like a lot of people I have only really started paying attention to how money really works the past few years. There is so much content online these days. But it‘s worth remembering this is a very new state of affairs. Two quotes from recent podcast illustrate the point.

First we have Richard Werner, who apparently invented Quantitive Easing. He was on the Liz Truss podcast [9:22]:

And of course the money creation is driven by bank credit, bank credit creation, that’s the more detail, the more precise way of looking at it. Of course that’s not by accident, they are not entirely uninformed at the Bank of England. For example when I published...well I was about to publish my first proof that banks create money out of nothing when they give a loan, each individual bank will create new money. Until then it was controversial. It was debated for a whole century its been debated. Is this true, is this not true.

Well let’s do an empirical test, that’s the scientific thing to do. So I did an empirical test, and it was found out that I was about to publish this, so the bank of England rushed out, to beat me to it, of course they vastly control their own publications, and in March 2014 they put out their own report saying, “oh we just discovered banks create money!”, and they did a video on it, which they had never done, they got all the publicity and that was their goal, and my paper came out later that year, called “Can banks individually create money out of nothing?”. It’s open access so one can google it [...] and so of course they knew this before.

It would be nice to get more info on what exactly his "empirical test" consisted of, but let‘s assume for a second it‘s genuine. Isn‘t it mind boggling that it was only in 2014 that we discovered that banks create money? I still can‘t quite get my head around this fact.

The second one is from Matt Dines on the What Bitcoin Did Podcast [1:04:03]:

I was just an analyst starting out, we were still talking about replacing LIBOR and moving to something else. We knew LIBOR had to go away, and SOFR was already on the table [...] as early as those days, and most of us at the not executive level, we didn‘t understand what the heck was going on.

Like coming through university, I did a masters in finance, it was quantitative focussed, we didn‘t go through any of this, like how the dollar structure worked, this was just not there. You are learning quantitative formulas, number crunching, pricing exotic derivatives. How the system actually works isn't taught to you. It‘s not the focus, maybe that's changing. Actually it‘s the podcast circuit, now that it‘s talked about people can find this and that‘s made all the difference.

It‘s really the explosion of content online by independent media and content creators that has created the situation where we are all learning how these previously very opaque systems are actually organised. Including for the people who work in the finance industry!

This recent surge in interest in the finance industry in general is a quiet revolution. I think it‘s also been happening because so many folks are into crypto and Bitcoin and so are having to figure things out from first principles, and it‘s all been made possible by independent podcasting.

With the rise of version control systems (VCS) like Github, Pull Requests, often called PRs for short, have become very popular with programmers. They are basically a place to gather together all the bits and pieces for a particular thing you are working on before you merge in the new changes from your feature branch into the main branch of your repo. It's useful as a way to mentally separate out changes, and gives you a high level view of what been going on in the project historically without having to see every tiny little change in each file.

As useful as it is for individual programmers, it's even more useful when you are collaborating with other programmers to complete the feature, because it gives you a place to discuss the changes, refactor the changes, review and approve the changes, and even launch automated processes such as linters which ensure the code follows certain conventions, or various flavour of automated tests to ensure that broken core doesn't make it into production.

One of the goals of my new publishing system is to leverage some of these features for content creation. Once you are in a world where everything is a file then all sorts of interesting things become possible. That isn't to say that database powered systems are bad, on the contrary, there can be great synergies, but the point is to use each system where it shines.

In the modern era where AI agents are starting to have a huge effect on things, the collaboration won't just be with other humans, it will be with the agents too. That's why I'm super pumped about my new setup, because all the content I now write passes through PRs. Each new item, whether it's a link, a note or a blog post get's it's own individual PR. It's already really useful, but I think in the future a lot more cool things are going to be possible by hooking up AI agents into the mix, and the tools I've built make it really easy to spin up new ways to publish content.

When I started using the new blog publishing system for all writing on the blog, I only setup blog posts and links. It occurred to me that perhaps one of the reasons I‘m finding it a bit difficult to get back into a good blogging rhythm was because I never setup notes.

Notes are another type of writing unit, not quite as big as a blog post, but more than a link. They are great for small thoughts, things you are pondering, but haven‘t really thought through. Maybe they will eventually turn into a blog post, but also maybe they won‘t. No pressure. Also no title, which is crucial. Sometimes when you have a thought that you want to write about, the hurdle of having to come up with a great title the perfectly describes your thought, is enough to for your initial enthusiasm to disappear into the ether.

That was the idea behind the blogging virtuous circle. Let‘s see if I can get that setup again.

I've been trying to get back into a better blogging rhythm. I can't remember a time where there's just so much doom and gloom everywhere. Whether I open up my RSS reader, or check out what podcasts have dropped, or whether I decide to brave it in the dystopian hall of mirrors that youtube seems to have become, it's 90% people freaking out about something, and often everything.

It's really not making the blogging any easier.

On the other hand, at least I managed to write this post.

Like many I‘ve been watching this latest trend to ban social media sites for under 16s. The first I saw of it was the ban in Australia, but it seems the same will be happening in the UK. Keir Starmer wants to give kids their childhood back, which is a catchy slogan I suppose, but feels a little bit hollow to me.

Yeah perhaps for some that live out in the countryside, you can imagine they will be able to get back to horse riding and hiking or whatever, but for most inner city kids, it‘s not like there are loads of things to do. I read an interview with a teenager that lives in London and when asked what he‘d do when the ban came into force, he paused and said 'I‘ll probably just stare at the wall'. Kind of bleak.

I‘m not totally against the ban though. I can see it could have a positive impact, but you have to pair it with some other programs for the youth if you really are serious about changing things for the better.

If it‘s just the big social media sites, perhaps this will be a boost for the indieweb? The web was in many ways a lot more fun before the social media sites terraformed our digital lives. There was a lot more variety back then. Maybe this is an opportunity for the next generation to build a better digital future for themselves. All you need is an text editor and a bit of knowledge about HTML.

I was just looking at some stock charts way zoomed out to see as much historical context as possible, and wondered what it looked like for some prominent companies that were around for the dotcom era of the web, both for hardware and software.

You can really see the similarity in the massive uptick around the year 2000, and then the current moment. Microsoft was famously not that hot on the internet back then, and so there isn‘t much of a bump, but the hardware companies definitely were, and that‘s very clear for Cisco, who made much of the networking equipment back then.

It sort of makes you go yikes, but it doesn‘t mean necessarily it‘s going to crash anytime soon, in fact I think it‘s unlikely to go that way until there are some more IPOs, but it‘s worth keeping these graphs in mind.

How best to set yourself up for the long road ahead is the big question that nobody knows the answer to.

Marc Andreesen was on the Latent Space Podcast recently. He‘s best known for being co-author of Mosaic, the first web browser to display inline images, and for being the founder of Netscape. There‘s a great bit midway through where he talks about his roots in unix programming and the Unix Mindset.

I started out working on unix systems back in university in the early 2000s, and in my first tech job managing a linux render farm in a visual effects company. Up until that point I‘d only used Windows systems, but having hands on experience it was clear to me very quickly how great these systems were for building and hacking on software.

What‘s cool about this clip is Marc takes all the unix concepts, everything being a file, the shell and being able to easily connect programs together, and he brings all these notions into the modern agentic era, explaining how they underpin all the developments we are now seeing AI.

It‘s all files, and everything is swappable.

Quick update on the Github Actions storage issue I ran into last month. To summarize Github blocked me because it claimed I went over my Actions storage quota. I contacted their support and they were not able to answer my very basic and polite questions, instead sending me pre-written articles that didn‘t fit my situation very well imho. They then stopped replying to me. Eventually the quota got reset with the new billing cycle.

We are 1/3 of the way through the current month‘s billing cycle. I have written four days worth of posts so far this month. Last month in total I had written 13 days of posts when I went over my quota, leading to the suspension. So you would expect that since my publishing workflow uses up the exact same amount of storage every day, that I would be 1/3 of the way through my quota, right?

Well I just checked on the billing page, and I have used up 0 GB of the 0.5 GB included free Actions storage this month. I haven‘t made any changes to how the site builds. It‘s the exact same actions that build the site. Haven‘t changed them for many many months.

Github I tried really hard to figure out your complicated Actions storage quota system. It makes no sense to me.

The deployment of the refactor I mentioned yesterday went very smoothly. The test post and bunch of links I had gathered yesterday, all got picked up by the automated publishing tool, got turned into individual PRs, got put through a batch review process and got released in the early hours this morning.Seems like everything is running as expected.

Today I decided to overhaul the way the system is processing incoming webhooks. The way it was doing things was working but the big refactor exposed some cruft in that part of the code, so I figured out a better way to do things that fits in well with the new architecture, and got it all working, and tested. Just pushed that out earlier, all tests are passing and smoke testing in staging went fine, so no reason to think there will be any issues.

I want to make sure everything is still working as I‘ve just deployed the latest refactor of my publishing software. I previously wrote about the rendering tool, but this is the publishing side of things, it‘s the software that manages the content through a neat publishing workflow once you‘ve pushed it to Github.

As far as refactors go, this was kind of a big one. There had been a apparent misunderstanding between me and the LLM for quite a while. I thought we were on the same page about the architecture, so did the LLM, but we kept running into strange dislocations in the code. It‘s not always obvious immediately that this is happening, because sometimes the LLMs do come up with some really novel ways of architecting things, they have access to basically all of programming through all of time after all. Anyway the same sort of thing kept happening, but in different ways.

I eventually got to the bottom of it, and when I pointed out the ridiculousness of two entities that were clearly supposed to be different things, in the mind of the AI they were the same thing. Anyhow perhaps a story for another blog post.

For now just hoping this makes it onto the blog later tonight. Should know by morning if it worked.

Barely 24 hours after I wrote about Apple and Gemini and that I had been using Gemini a lot recently and now Gemini has had a makeover and looks great but is requireming me to click 10 times for each query :(

Strange.

Interesting to hear the latest from this years Apple developer conference. Apple‘s AI future for now will be with Gemini models. I‘ve been working a lot with Gemini the past few months, and it‘s very good a lot of the time, including for coding, which is what I mostly use it for.

I am struck, especially today actually, by how much Gemini likes to refactor code. In this world where all code is apparently one shotted, it‘s really quite odd how much refactoring one has to do to get anything worth having. I suspect one shotting is the new refactoring.

So Apple and Google...perhaps it is fitting. Always one more one-shottings away from the perfect software that will definitely be the most perfect software ever. This time is different.

The Github Action I added to my blog to run at the start of every month worked. I added it because I got blocked again by Github last month, this time apparently for using hundreds of GB-hours (whatever that is) of storage even though I am using literally the smallest amount of storage possible.

One of the cool things about running a static site website, is I can continue to write offline and sync it back up when I get online again. I realized though that because I was only building the site when I push new content to Github, that once the new billing cycle came around that all the content I had written while offline wouldn‘t automatically get rendered into the site, because all the build triggers had been missed, and the new content would only make it to the site if I added some more content, which would trigger the site to rebuild.

With this extra little monthly workflow, I can be sure that any content I add while offline will definitely make it onto the site even without me writing something new at the beginning of the month. It worked last night, and all the content I added since I got blocked was rendered into the site.

The Github support team, who initially answered my support query, still haven‘t answered my question about what settings I need to use to not go over the storage quota limit. In the past Github support could take several days to reply to a support case but they wouldn't ever just go dark like that. That‘s quite disappointing, especially given how much time I have been spending developing on their platform the past few months.

The Vergecast Podcast is going daily. David Pierce interviewed Casey Neistat to get some advice. Casey is know for having run a daily vlog for years. I used to watch it.

I checked out Ze Frank, another one of the daily vloggers from the mid 2000s that got very popular. He‘s now doing nature related videos narrating in his style. His latest is about bees and wasps and ants.

I wonder what happened to Andrew Baron from Rocketboom. Another one of my favorite dailies from the mid 2000s, when video was just starting on the web. They used to do all sorts of things both indoors and outdoors.

Oh yeah there was also Mike Rugnetta and The Idea Channel. Actually not sure if this was a daily, but somehow in my mind it falls into a similar bucket.

As far as I know the Vergecast isn't video, but I‘m interested to see where they take it with an audio only daily pod. Potentially lot‘s of things to explore.

I found myself needing some good music to code to the other day and after a bit of searching found an LTJ Bukem album I hadn't heard before:

• LTJ Bukem - Cosmic Twilight Vol.2 @ Tipper & Friends, Spirit Of Suwannee Music Park // 03.04.22

Seems like it was released in 2022. I haven't really been following the drum and bass scene for many years now, so it makes sense that I wouldn't have seen it. It's a great album. Really reminded me of the times I got into drum and bass around 1996. I feel like back then drum and bass was much lighter, things were very chill. Bukem uses the "Amen Break" a lot in his tracks. I thought he came up with it, but turns out folks had been using that sample for years.

Anyhow he's got several of his albums on youtube, Logical Progression being his most famous, and probably the best place to start:

• LTJ Bukem presents Logical Progression (CD-B mixed set, original 1996 version) Intelligent DnB

This live album is pretty great too:

• LTJ Bukem studio set '95 - Love Of Life "Master Mix" soulful dnb Cut 4 YT

I‘m still trying to figure out exactly why the storage in my Github Actions has gone through it‘s monthly quota. Github support did get back in touch with me, but that got delayed because replying to the emails for some reason didn‘t add the message to the support case thread. I didn‘t realise initially so that wasted a few days. When I copied those messages into the thread, they replied with what looked like a pre-prepared reply with some links but not really answering my question.

I am genuinely confused by their storage quota. I have been using it for several years in the same exact way. Every day an action runs and builds the website if there was new content added, and during that build I create an artifact of the built website in order to pass it between two Github Action jobs. It‘s a pretty standard thing to do as far as I am aware. The artifact is around 100MB, and I set a retention policy of 1 day, which is the minimum amount. I don‘t even need the artifact to last a day, as soon as the build is finished it can be deleted as far as I am concerned.

Well Github have changed their billing system relatively recently, within the past few months, so I suspect it‘s got something to do with that, because I am literally using the least amount of actions storage as possible. The way it's always worked is as long as I was under the 500MB limit every day, and I always am because of the 1 day retention setting, then the quote is not exceeded.

But now when I look in the billing section they are talking about GB-hrs, and it lists 100s of GB-hrs. I have no idea what this means, but it just seems utterly ridiculous to me because I am as far as I am aware using the least amount of storage possible. I politely pointed this out to the support person, and they have gone dark on me. It's been several more days now and no reply. Anyway the billing cycle resets in 4 days.

I‘ve added a scheduled start to the build so that as well as building on push to main, it now builds once at the start of each month. That will hopefully ensure that any content I push during these weird periods were Github is blocking me for whatever reason they have decided to block me for this time, always makes it onto the website.

According to the Github account usage page I have used up all my Actions storage quota of 500MB, so it is blocking all my builds that use artifacts.The website's build uses artifacts to pass the built website between jobs, which is the Github recommended way to do things. Thus all builds of the website and thus all publishing to the website is completely blocked. I haven't even been publishing much the past few weeks!

This isn't the first time Github has blocked me completely but it is the first time it has done it on the basis of storage. In the past it has always been because of running out of free build minutes. The thing that is very strange about the storage quota is that as far as storage goes I am very conservative, setting a retention of 1 day on all artifacts I generate, and I only run 1 build per day, so it's very odd that I am over the quota suddenly. I have been publishing without storage problems in this, the recommended way, literally for years.

When I did some investigation on the command line using Github's gh tool I could see hundreds and hundreds of old artifacts on the system which would have totalled several GB in size. They were all "expired", because of the retension policy, but I would have expected these to get cleaned up. According to the LLM helping me I did everything correctly, and it looks like something on Github's side is suddenly trigering them to block.

Anyway I manually deleted all the expired artifacts, even though I shouldn't have to, so there are literally zero artifacts now, but even after a day the system is still blocking everything. The Github website says it will reset in 12 days.

They always seem to find some reason and way to block you, even when you spend years and years making everything perfect. I have emailed support, so hopefully I will hear back soon. This always seems to happen right when I'm putting the final fixes to software I'm building. Quite difficult to test a blogging tool when you aren't allowed to build your blog. The one small win is that since it's a static site powered by git I can still write posts offline, but right now it doesn't feel like much consolation.

Bugs in code is not a lot of fun at the best of times, but finding a bug in a mocking library is the sort of thing that can really ruin your day, or week, or worse.

The past few days I‘ve been deep in a refactor, and I eventually hit a point where I was fighting the tools rather than the code. There were just too many moving parts, and it became impossible to see the woods for the trees. What appeared to be a straightforward bug in my integration tests turned into a two-day troubleshooting marathon. It’s a strange irony of working with LLMs: the ability to generate code so quickly means you can sometimes dig yourself into an even deeper hole than the one you are already in, with terrifying efficiency. You end up iterating on a flawed premise until you’re so far down you’ve lost all perspective on the original problem.

My way out of that cycle is to step away from the main project and build a "minimal". I’ve been doing this for years. I’ve even started a dedicated section in my portfolio to track them. The goal is to isolate the problem domain completely. By stripping everything back to a few lines of code and a single test case, the friction has nowhere to hide. In this case, it turned out that Node’s native fetch behaves just differently enough during a 204 handshake to throw Nock into a tailspin unless you explicitly override the transport.

I don't just build these repos to fix a single bug, they serve as a way to document the "why" so my future self can return to the problem and actually remember the details. It’s easy to forget the nuances of a library conflict six months down the line, but having a working, isolated example provides a path back to stable ground. It’s about being pragmatic, clearing the brambles in a small patch so you can see the through-line in your main project.

Here's the latest minimal: minimal-modern-node-http-mocking

The good news is that when I dove back into the main project I was able to fix the integration test I had been spinning on almost immediately. I feel like this latest minimal is a textbook example of when to use this technique.

Side note, a long list of very strange synchronicities for me this week, including what seemed like a name drop of the mocking library in a podcast I listened to moments after deciding to fix the bug. What's really weird, as far as I can tell they were talking about something completely unrelated to programming. It made no sense whatsoever in their context.

Barely a single day after getting my review and approve publishing software fully working, and the White House annouces some sort of regime where they will review and approve AI models. I will file this away in the, at this stage, unbelievably large "what are the chances" folder.

Long day today, but managed to fix all the integration tests that had broken during the latest refactor. It‘s nice to finish the day with the latest build running smoothly. I had to re-record all the test fixtures, but that‘s been made a lot easier following recent changes I made. Previously I had to create all the test mocks by hand, which was really laborious and error prone, but now it‘s all automatic. Anyway, nice to have some progress :)

This is another one of those test posts. I‘ve been busy with more refactoring of my blogging software. This time I‘ve been getting the integration tests working correctly. Thankfully the developer tools and libraries are pretty great. It‘s a bit laborious to get things setup initially and the LLMs have sent be down several pretty bad dead ends. But things are progressing well in general.

It‘s really quite amazing the sort of refactoring of your code you can get done when working with LLMs. They make it possible to see much much further into the future, and you can get a much better idea of whether something is possible or not, but also if you can't quite see the through line, but you know it‘s there, you can head off in the general direction, and work things out as you go.

This happened several times to me this week, I had a really good sense that it was possible, but it was really lot of very abstract changes, and I couldn't quite get a handle on it. Instead of fighting, I just said, what are the top 3 things we could do to the code base to put us in a position that would make the particular refactor possible. The LLM had a good sense of where we were heading as we'd been discussing it for a while, and so we charted a course and started on the top 3 things.

By the time we finished the top 3 things, it had cleared enough of the brambles and tangles that it was much much more obvious how the refactor could be done.

But don‘t fall into the perfection trap, you have to be pragmatic, there are times, where getting to stable ground is more important that having everything perfect.

Seems like the bug I fixed yesterday is properly fixed. It is at least with my links, this post tests it's also fixed in blog posts.

I hate to go on about it, but since I started blogging again it‘s been strange thing after strange thing after strange thing.

More bizarre and weird synchronicities than you can shake a stick at. I just fixed a bug in my review and approve workflow, the sort that you only find when you have it running in production. It took me all day yesterday to debug and write the fix. Anyhow I was just about to push the code to production and suddenly all my shells across my whole machine suddenly become unresponsive. All of them in Vscode, and all in iTerm2 and even the system Termnal. Had to reboot. That's two reboots in two days.

I was just about to write a test post to test things out, so this is the test post. I think I prefer the dull and boring test posts. Does everything have to be a high wire act?