markjgsmith

The gang stalkers were out in force again today, even if they were being more subtle. Part last night's harassment was an individual, who had clearly been waiting for me to arrive walked out in front of me, he was holding a very distinctive red bag. There were also several gangs on either side of the street at specific points making it difficult to cross. It was pretty clearly a setup, they wanted me to walk behind red bag bloke.

It was so obviously a setup too. Everything had been normal, and then when I got to the choke point, if you looked up the street you could see all the various little gangs on either side of the road going on for several hundred meters. I crossed the road anyway, took a diagonal line, and that's when one of the gangs got verbally abusive, several shouting No! at me loudly as I walked out of their trap. It was pretty dangerous as their shouting caused me to look the wrong way, and I very nearly got hit by passing traffic.

Well wouldn't you know it today in the local shop, where I often go for groceries, the girl at the counter started putting my things in a transparent unbranded bag (very unusual), but also didn't ask if I wanted a bag before doing so (very unusual). I politely said I no thank you for the bag, I always bring my own. She obviously got offended in some way (apparently), and started randomly saying No! and Bad! and laughing with here colleague. I said nothing, packed my bag with the things I bought, politely said thank you to them both, and departed.

Perhaps you think I am over reacting? Guess what happened next. On the way back, no less that 3 individuals, in 3 different locations, who were just waiting, loudly said No! to me as I passed. One was on a motorbike who dangerously pulled up directly in front of me. All these events are planned. They are all part of yet another intimidation campaign, carried out by just regular looking people of all shapes and sizes, all races were represented. What had I done to deserve such treatment? The only thing I can think of is it's writing this blog, because I've very much been keeping to myself lately.

People are worried about AI and super intelligence or whatever. I worry much more about people. Whatever they learn training these new AI models can, will and in my opinion clearly is being used by humans on other humans. It's horribly horribly ugly. I personnaly believe it's this sort of thing that causes societal decline, even though likely those doing it believe the opposite. I've seen it and experienced it in one way or another in many different countries, in 3 continents around the world, and it seems to me it's getting worse.

I'm just emerging from a bit of a nightmare related to higher level classes, hence this cautionary note. It‘s particularly relevant if you are coding with LLMs.

The crux of it is quite simple. I had a need to create some decorators, which is a way of dynamically adding new behaviors or responsibilities to an object or class without altering its underlying code. It is a quintessential example of composition over inheritance: instead of creating a rigid, deep subclass hierarchy to add a feature, you wrap the target with a decorator that intercepts or enhances its functionality. At it's core it's just a function that takes the class, method or property as input and wraps it with extra logic, and returns the enhanced version.

They are referred to as higher order function, or loosely as "higher-level" because they are functions that take other functions as arguments or return them. The annoying thing about higher-level functions is that they are tricky to reason about. Taken on their own, not so much, but when they are surrounded by a much bigger codebase, the level of complexity you have to hold in your head gets gnarly fast. But they can be very useful, and in some circumstances it's literally the only way of achieving what you want.

I have used decorators a lot before, in fact in nodejs you are using small versions of these all the time, but I hadn't had to use them to decorate classes before, and that is what the LLM I was working with suggested. As we were tackling a quite tricky thing, it made the case for using some class decorators. It made a lot of sense, and quite quickly we were off to the races. Of course I reviewed the code, but I didn't dig too much into the detail and I certainly didn't think about the longer term consequences.

As if often then case with software, as we continued to work on the feature, we discovered that the initial solution, and in fact the architecture it was embedded in, needed to change, and we set about doing that. This is where things started going off the rails, and at first I didn't notice. In my head it seemed that the higher order function solution was quite self contained, but as we re-architected the code, the LLM re-architected the higher order classes implementation, and in retrospect, it clearly got a bit confused, or just didn't think it through properly.

At a certain point I realised that the LLM had added several layers of indirection, which wasn't totally obvious because it was spread across several files in very different locations, and when I asked about these layers, it had very good arguments for them. They are acting as "runtime binding factories" was one that initially sounded like it could be a real thing that was needed. You have data at runtime that you need injecting into the class you are decorating, so of course you need a runtime binding factory. Duh. Yet somehow the more I asked about it, the more the explanations seemed contrived.

I eventually paused everything and dug into it myself, and painstakingly traced the code through and discovered all these horrific levels of totally unnecessary complexity, with very confusing function names, and to make it worse it had been importing various things and stuffing them inside closures at every level. A real tangled mess. Yet somehow it sort of worked, until you tested it properly, or refactored it slightly and it all fell apart.

I started by trying to rename all the function names to names that actually made sense. Often these ended up being ridiculously long names, which in itself was a sign things were not right. Anyway doing that enabled me to point out how unnecessary much of the cruft was, and soon enough we were able to drastically reduce the number of useless levels of complexity and unnecessary imports and closures.

All that said, the final solution is very neat, and actually not that complex. It's a little bit more complex than normal code, but not that much, and it's only in a very well understood place, and done in a repeatable way. The point is that if you are writing higher order functions with LLMs, it's worth slowing down and really make sure you understand the logic because it gets gnarly fast and the LLMs are unbelievably good at creating a giant knot out of everything and then telling you how great it is.

It‘s really terrible what‘s happening in France with all the school kids rioting. Protesting injustices isn't the issue, that should happen, but the tricky thing is that in this case the fact that they are tearing up the schools is literally making things worse for themselves. If the school system is already at breaking point, and by all accounts, with 45 kids in classes, it certainly sounds like it is, the last thing you want to do is make the situation even worse.

An interesting side note about language, I was looking up the translation of "self-defeating" in french. they have "contre-productif", but in my mind it lacks the "self" part, which is very important to get across the idea that you are hurting yourself. While they also have "auto-destructif", with "auto" from the Greek "autos", which means self, and I do remember it being used in this context, it‘s a strange one where I feel like somehow the modern use is somehow very linked to automatic, and so it‘s been sort of hijacked. To expresss this well you have to use a turn of phrase, like to shoot yourself in the foot ("Se tirer une balle dans le pied"), or to cut the branch you are sitting on ("Scier la branche sur laquelle on est assis").

Anyway while there will be some in the UK that will enjoy some sort of schadenfreude by seeing the french hurting themselves, it‘s a cautionary tale, because especially in a wider economic decline that we are in, and have been for years, this could very easily happen in the UK. It occurred to me that it might be worth getting ahead of this in some way. For universities you could make it into an excuse to have a cheeky pint with fellow students from another city, make it into a day trip, to forge connections, because if you ever do need to protest, you absolutely need it to be peaceful. And to do it right, you have to do it properly, tell authorities in advance, have a safe route etc, because there are those who will try to take it over because if it does lead to riots, they will benefit from you being in an even worse position later. This is the reality we live in, and that is only going to get worse the more along the decline road we get.

I'm not sure how you could extend this to schools, pints are no good for that, but it would be worth having some sort of forum, that doesn't in itself cause problems, because if we ever do get to that stage, where protests are needed, and they can be very powerful, it‘s perhaps the most important thing that it‘s Gandhi style sit downs rather than destroy the school and the neighborhood.

And of course while that's an important thing, equally important is to stop this infernal decline, something which the backs of napkins throughout history can attest to, is also very well suited to such forums.

I just mostly completed a massive refactor of the entire outer shell of my publishing server. A few days ago when I discovered it was necessary I was thinking to myself, oh no yet again Gemini has got me to a summit only to tell me that it all had to be completely redone. That has happened so many times.

It wasn't at all obvious how it could be achieved but the LLM came up withe a pretty awesome plan, though it did take me several hours to figure out what it was on about. Anyway long story short, what seemed completely impossible was actually very doable, as we had already laid the groundwork. So we were able to completely rewrite the entire outer shell, everything inside for the most part remained in the same place, but the outer shell was completely re-architected.

I'm still kind of amazed that it worked. And though I still have a few modules to convert to the new architecture, it's looking really really great.

Late last night, I had an important breakthrough in the next important feature of my publishing software. It's around user onboarding and setup. Very important, and it worked for the first time.

All the pieces got created in all the right places. Still need to write tests and general cleanup but seamless onboarding is on the horizon.

I haven't been posting many links recently. That‘s the thought that occurred to me earlier. I've been busy coding mostly. Anyway I decided to go for a bit of a search for some interesting posts. Pretty quickly I found 4 blog posts that sounded interesting from their titles. This post, this one, this one and this one. I go them all on Hacker News main page.

So I opened all four in 4 separate tabs, git myself a coffee and started reading. Each one starts out looking like it might be something kind of interesting, either because of the topic, or the design or the page, the way the information is organised. But each one upon reading a few paragraphs, I just wanted to die, overcome by such intense boredom, that I lost the will to live and was not able to finish any of the articles.

What is going on? I'm finding it hard to believe that all these articles are written by humans. It's like they have all the appearance of articles written by humans, but they are hollow, like biting into a really nice looking chocolate only to find it is completely empty inside. Just words. Blah blah blah blah. Nothing drawing you in. This is not the internet I remember. Where has the real internet gone?

Oh well. Perhaps tomorrow things will pick up.

I'm at the stage building my publishing server software where having a more defined plan makes sense. I've started using git tags and semantic versioning to plan out the next few releases. Each release happens in stages so you have times where you only have part of the feature set written but you still want a release to test, with uncompleted features behind feature gates. These are beta releases (e.g. 1.1.0-beta.1, 1.1.0-beta.2). Then when you have all the pieces ready you try to put it all together in a release candidate (e.g. 1.1.0-rc.1). Eventually when it's all working you drop the extra qualifiers and you are left with just the version number.

The core of the software is very solid, it basically never changes now, or if it does it's small tweaks. I haven't had to re-record the test fixtures in months. There are several layers that insulate the core business logic, both above it and below. It makes it much easier to change things in the frontend display, and in the backend storage. As well as the core, there are now several extension packages that you can add to get new features, and these layer ontop to create several distributions that work in different environments. There is an assembler that you use to build exactly what you want, with a particular feature set.

Since I'm working with all these packages, although it's a monorepo, the tagging works really well to co-ordinate all the PRs necessary for a release. Gives you something to aim for. When you list all the commits, you can see the ones that correspond to the releases. It really helps with how you think about the code, how you conceptualise what is still to do, and where you are at.

In the last few days I identified a crucial relationship in the core that needs to change slightly in order to move forward. As I was looking at the various distros I revisited an idea I had a while back, to unify all the route handlers. At the time it seemed too difficult, but now it looked possible. I was able to extract all the route handler logic across all the distributions into a core handlers package with the variations between distros neatly inside a thin layer in each distro. The complexity of all the distros has decreased significantly. And it was pretty easy updating the unit and integration tests.

I'm very well positioned now for the final push to make the small change in the core, necessary in order to get 2 very important features working. The world is so crazy at the minute, and I might not know exactly how I fit into it or what's going to happen next in my life, but I know that I'm pretty happy with where the publishing server and the rendering CLI are at. That's the thought what grounds my thinking most right now.

The problem with so called "White Hat Hackers", like in the recent case where they stole money from an institution just to tell them to fix a bug, is that their business model is reliant on exploitable bugs being in the software, and so inevitably they will end up creating the bugs so they can "fix" them. This is not a good deed.

And guess what will end up happening, they will be the ones making the LLMs, which you use to write the code, that they hold you hostage with.

A situation that programmers are all too familiar with. It happens again and again, you are sure you did what you had to do, there must be something wrong with the language or the code, it's lying! After much frustration you figure out that you were doing something silly that was causing the problem you were complaining about.

Here's an example that happened to me back in July that might be applicable to others. So I had been trying to optimise the render times of my publishing tool CLI. Something that was taking 5 mins started taking 45 mins. It was horrendous. But after much effort I got the rendering down to 12 seconds. I was very happy, it was so much faster than even what it was previously. But then it doubled to 24 seconds, and so I persevered, for hours and hours, testing everything I could think of. No matter what I did, it was double what it had been. And as I pushed forward things got even worse because the LLM that was helping started suggesting all sorts of crazy ideas, because we'd run out of all the normal things that could be affecting it. I eventually gave up.

After a break, I returned to my laptop, and thought, what the hell, I'll run it 1 more time, and wouldn't you know it, it rendered in 15 seconds! Well it turned out that I had been overdriving the hard drive by running it so much, and the system needs to run some cleanup tasks when you are creating then deleting thousands of files over and over, and that cleanup could never finish because I was creating and deleting more files by running the program over and over.

Letting the laptop alone for a while had given the background process enough time to finish it's activities, and so when I ran it on a fresh system with all the resources available, it was super quick.

Sometimes the best thing to do is nothing, allow the system to do it's thing.

I watched this YouTube video about weed culture in New Zealand yesterday evening, suggested to me by the algorithm. The icon in the list of videos was a bloke doing a bong wearing a black balaclava, not sure you can see that from the direct link, anyway it made the video look very distinctive.

It was kind of an interesting video, very low budget, a story of people trying to survive difficult circumstances, even the documentary maker is clearly making the best he can with not much resources. The community vibe is nice, sometimes people just want to be able to do something together.

I‘m not sure where I stand on weed to be honest. I don‘t smoke myself, I also don‘t drink alcohol, it‘s been many years now. I am very glad every morning to wake up sober, with a clear head. But I understand different people go through different stages in life at different times, and sometimes it is fun to be in an altered mind state, and you can learn things about your body and your mind, and interacting with people, a shared experience.

But I do worry that people and entire communities can become reliant on it to survive. And even if weed is a natural plant, it‘s not exactly a big motivator, so if your community is already in a tough spot, it could make things worse.

And the world is very competitive, there may well be people out there that are very happy for you to be stoned while they are building great things.

Anyway first thing this morning I open the guardian website and the top story is about a large gang of black balaclava wearing protesters closing down Dover in the UK.

The world has been doing a bit of a green tsunami to me these past few days, and no doubt this black balaclava thing has a green tinge somewhere in it.

The right complain a lot about how the main stream media is very captured by people with socialist tendencies. Same in universities. In the online world the right seems to have more of a presence, so much so that the left is always complaining about it. Anyway that's just some pre-amble, which you might or might not agree with, for this particularly simple idea I had yesterday.

Isn‘t it odd that people who‘s primary wealth is social capital, from which their financial capital is derived, are socialist leaning. What a shocker.

And when you think about it, in their position it‘s actually the most capitalist thing they could do.

What a strange world of contraditions we live in.

Still I do like watching many of them, a tiny bit of honey that gives this little bee the will to live through another day in this hexagon lattice of doom.

Just a few minutes after writing about the hexagon lattice of doom and pushing to github, haven‘t even published the post publicly anywhere, and Gemini is responding with answers containing the term cross-pollinate. That is some seriously dark hexagon lattice shit right there.

There definitely seems to be some "cross-pollination" going on somewhere.

I listen to right leaning podcasts and a lot of them say "The left has been winning for 60 years, blah blah blah". And they have a lot of good points.

And I listen to left leaning podcasts and a lot of them say "The right has been winning for 60 years, blah blah blah". And they too have a lot of good points.

But clearly they must both overall be talking a lot of shit.

I wonder if maybe both sides can actually both be talking truth at the same time. Perhaps they only see the dimensions that the other side is winning in, and can't see the dimensions they themselves are winning in. And what if the reason things are so fubarred at the minute is because we have just run out of dimensions to win? Maybe we are just full.

But then I listen to podcasts about physics and they say that there are likely an infinite number of dimensions, and so now I think that this could possibly take a while.

This symmetry is kind of tedious.

I was pondering my recent post on innequality. I ended that post by saying it was more important to focus on having a good vision and a plan, and so I was imagining the different types of large scale plans countries could have for dealing with various situations. The "it depends" conclusion is annoying but it really is the case a lot of the time, and it‘s actually a good thing, as long as you can recognise a good plan from a not so good plan.

Of course when you are making a plan and plotting a course of action you have to make compromises, make choices, and take calculated risks. The plan is the plan until something rather bad happens that you didn‘t expect, at which point you have to reformulate the plan, or switch to a backup / alternate plan. That‘s just reality.

Well it occurred to me that in this world where career politicians and lawyers tend to be in charge of countries, doesn't it make more sense for people with more experience in making tough decisions, having conviction, and taking action in a particular direction in response to outside events. There are jobs out there that do a lot more of this type of thing than politics and lawyering.

We might want to make sure these types of folks are more conversant and used to thinking about complex issues, which more and more rely on tech related knowledge. That way they could transition to making decisions that are relevant at the nation scale.

People that make rules for a living slow things down, people that do things get things done.

I fixed the issue I've been having with a Github Actions scheduled start that wasn't firing. In the end I had to make a change to the repo and push it to remote main branch. This appears to have reset the trigger which is now firing so the blog is being auto-built and published again every day.

Not great, as making bullshit changes like that could itself cause issues.

Also not great that Github support made it impossible for me to contact them to report the bug.

There's a lot of talk currently about inequality and wealth tax. It seems to me these discussions are constantly missing the real issues. On the surafe it seems like they are the issue because they exploit our emotions. Of course it seems incredibly unfair that some people have l9ts while others have not a lot.

But that's just a surface level way of looking at it. Reality is more complex, and actually it's not correct that inequality is always bad. The reality is that it depends on what you are trying to do. There are situations where inequality might actually be better for everyone.

Lets say you are in a moment where you need to make huge unimaginable leaps forward in society, things that would cost unimaginable amounts of money. What's important is not the inequality it's what the wealthy are doing with their extra money. With inequality you only need to convince a few people to invest in the important unimaginably expensive thing. If the wealth was more evenly distributed it would be much more difficult to put it all into the new important thing, especially if the mood of the country is not one of investing in things.

Of course you don't want people with not much wealth to be going hungry, but technically that is a different issue.

Whether you want inequality of not depends on the situation. Whats more important is knowing what you want, having a clear vision, and an effective plan to get there.

I‘ve been working on a new package in my publishing server. Not all the packages include actual endpoints, but this one does. Previously I had bundled the endpoints into the main distros, but I figured out a way for each package to have their own endpoints and the distros get assembled at build time. I think it‘s going to be quite a leap forward, and I just updated the old code to use the new system.

So far things are looking good and I was a bit suprised that such a bit change was actually rather straightward after a good night‘s sleep and a clear head. I think it‘s a good sign, that the core architecture is pretty solid, and very flexible. It‘s quite new and my mental models are still adjusting but the more I create new additions the more things fall into place, and the architecture is easier to reason about.

It‘s amazing what you can achieve with object oriented code when you have the right stucture in place, and it‘s also pretty cool how much you can get done with the help of an LLM. It‘s like having a team of architectects, that can re-design incredibly complex things in a flash, and with the object oriented structure much of the complexity can be neatly moved out of you working memory.

You kind of, sort of have a Tom Cruise from Minority report in your head, effortlessly moving around these shapes that each represent massive amount of code, and using the LLM to make connections and links between all the parts, to get phenominal visibility through space and time.

One thing that the world does a lot is non-ironically call you a bear while force feeding you a burrito. It happens constantly. One day you realise this and you can't unsee it. It's everywhere, in every gosh darn dimension.

Occasionally you walk away from a situation, and you think to yourself that perhaps this time it didn't happen. This is inevitably followed by yet another burrito force feeding incident.

And one incident leads to another.

It's likely evidence we are all in a giant game of burrito mega tetris.

It's fun until there are spinning burritos everywhere.

Usually ends in everyone turning into zombies.

The world is being artificially race conditionist for me again. Hardly any point in starting anything in my experience, it will only later be blocked in some very eggregious way, usually involving tsumamis, and even more infinitis pretending to be the opposite polarity infinity.

Just another infiniti war. This too shall pass.

So much waiting for things to pass recently.

I hate this useless post right now.

An interesting thing about my recent sprint to get the multi-target users working. I had been working for several days straight, through much confusion and all sorts of head winds, but got to the stage where all the unit and integration tests were passing on all parts of the software apart from core. I had been working with the LLM along side, helping me on many things the whole way through. Now we did run into a lot of issues along the way, but we were always able to work them out one way or another.

So when we got to the final push I gave the LLM the full list of failing tests. And we worked through them one by one, somewhat laboriously but we made good progress. That was until we got to the final test. There was literally 1 test left in all the several days of work, and wouldn't you know it, the LLM suddenly became completely incapable of doing the most basic of things.

It started deleting useful comments in the code that we absolutely needed. This is something that happens quite regularly because I guess there is a prune directive that often kicks in and it‘s always trying to prune and tidy up stuff which can be useful but also a lot of the time is not useful. Normally it suffices to say don‘t do that, but now that we got to the final test, it not only didn‘t stop deleting comments, but started changing the comments that it didn‘t delete. When I pointed this out it started changing whole blocks of code that had zero to do with what we had been fixing. It was extremely dangerous, because this was code that was very difficult to write, and had I not noticed, it could have been disastrous.

It became clear after several back and forths that it had become belligerent and no matter what I would say it would claim to have fixed things apologising while simultaneously making things much worse. I ended up giving up on the LLM and doing it myself.

This is a pattern I have seen a lot, and I wonder where it comes from. Remember LLMs are trained on human data, so is this something they have somehow learned to do from analysing our data? Or is it something more nefarious? The behaviour is night and day, it‘s like somebody has flipped a switch, it somehow knows this is the very end of something important and tries to f-it up.

Side note, I suspect it might be some strange reflection of humanity because I see vaguely related and obliquely related instances of this pattern all the time in the real world.

When you‘ve been toiling on a piece of code for a while, refactoring it, refining it, and generally spending a lot of time in the codebase, occasionally you make a major breakthrough, some kind of new way of thinking about the code, or a new architecture, some re-organisation that this time really will make things so much better. When this happens I try to create a minimal implementation that proves the thing works, and if I get to that stage where I know for sure it‘s going to work, there‘s a very unique mental state that arises.

It‘s kind of dumb in a way, in some way similar to the anticipation before Christmas, or something like that, everything feels a bit lighter, you have a spring in your step. You know clarity is just around the corner. Nothing can stop you now! It‘s the greatest feeling in the world.

And then the world has other ideas, and well, isn‘t always on board with your amazing plan, that will fix everything. But for those few hours before reality hits you in the face, things are wonderful.

I am several days after one of those times now, so I‘m feeling more like I‘ve been beaten up by several gangs of thugs, metaphorically and spiritually speaking, but I wanted to write a post about this state of mind, so I can remember it if indeed it ever does go away. I‘ve been programming for about 20 years at this point, and it still happens to me every so often, so hopefully that will continue.

I wonder if there is a term other programmers refer to it?

This is a situation I have written about so many times on the blog over the years. It happens in all sorts of different ways, but it‘s ultimately the same pattern every time. You figure something out that is absolutely crucial to everything, will make everything much much better, and you prove it and get it working, and somehow the world finds a way to block you, usually in a very offensive way.

The latest incarnation of this "phenomenon" is in the server software for my publishing tool. In the early days it was difficult to see exactly how the software would evolve, it‘s often the way with new software in uncharted territory, you have to make some guesses, some approximations, or you will be spinning your wheels for eternity trying to get everything perfect. In this instance it was about how we map a user to a repo. I decided to make the simplification that 1 user has 1 repo. If you want to have several repos, then you create several users.

I knew eventually I would need to change this assumption, but since the code is mostly object oriented and designed rather well, it's not so difficult to change this in the future. Well that time came and over the past week I made the necessary changes to change the core to be able to have 1 user mapped to many repos. In a way it seems like a big change, right at the center of the software, but when you design things well, it's not such a big change. The blast radius was very contained and I was able to get it working within a couple of days.

The very day I got it working, in fact pretty much the same hour, things in my world started to go sideways. I've written a bit about it in some short posts the past few days. Anyhow it‘s culminated in a power cut and internet connectivity issues, along with all sorts of very strange "synchronicities". Now I have internet again, but guess what, it only works for 30 seconds and then turns off for 30 seconds, over and over and over again. And you want to know another strange thing, the place where I get the internet has a rule that you can only connect one device to the internet. Kind of weird right?

It gets weirder. In the local mall yesterday, the security / doorman bloke as I was leaving quite clearly said the word "layer" as I was leaving the mall to me. Not "liar", but very clearly "layer". What‘s weird is that the feature I added right before the multi-target users was a render layers feature. I had actually added it a few weeks ago, but I documented it in the main README just a few days ago. You know in some ways, it‘s nice to get some recognition, if indeed he has somehow been looking at my code, because it‘s a very cool feature, and programming software on your own is such a thankless task it's insane, but it's a private fucking repo. How is the security guard at the mall reading my code?

You are probably thinking this is just some random freak incident, a "co-incidence", but this sort of thing happens ALL THE FUCKING TIME.

Somehow the world around me mirrors the code I write. You can call it woo, or nefarious, or whatever, but it keeps happening, and I suspect it's only going to get worse.

Abundance with scare city sprinkles. There sure are a lot of sprinkles lately.

Today I‘ve been writing documentation for my CLI rendering tool. I had a main README already but I needed to write detailed docs for all major entities the software uses. It‘s totally mind boggling how much of an incredible tool LLMs are for documentation writing. I was able to get done in a day what would likely have taken at least a week by hand, and I think the quality is far higher, because it can read the code directly and make sure it describes things perfectly, complete with neato example code snippets.

There are some of the usual signs that an LLM has been used, though I removed some of the more egregious ones. But personally I think the most important thing is that there is a complete set of accurate docs. It‘s not so annoying that is sometimes uses LLM-style. Who cares? For a one person software team, it‘s basically miracle software.

The trick to making it easy is to get it to use numbered sections and sub sections, that way after the first draft you can focus in on specific sections, rework just that bit, and then copy and paste the output right over the top of the old section.

It‘s pretty cool finally seeing the full set of docs.

Today I am just completely overwhelmed by how utterly useless people are, everywhere. I'm normally quite an optimistic person, and I'm very good at coming up with excuses for people, but absolutely everywhere I look, in every direction, it's people being useless, and having these useless petty arguments, spending ungodly amounts of time and energy being useless.

Occasionally someone pops up and they show promise, but eventually like in a zombie apocalypse movie they turn out to be useless too. Every. Single. Time.

And I'm useless too, but I sure am trying not to be. I can't imagine trying any harder, there are simply not enough hours in the day. I'm already at it every moment of every day.

Why are we all so fucking useless?

There are a lot of people in this world that want to look in the mirror but also at the same time they want to not look in the mirror. And after much wasted time of course it's not possible and they get angry and blame the mirror, often getting angry at the mirror. People often create the problems they complain about, whether on purpose or not its nothing more than a huge waste of time. A race condition death trap. Break out of the loop.

This short showing what I guess is a simulation of a 1 solar mass black hole passing near Jupiter, passed through in my feeds. You have to watch it several times to really grok what's going on. Sure hope it's a simulation. It's terrifying, even if you turn off the overly dramatic music, it just sort keeps playing in your head.

But ok so lets say you shrink that black hole down, surely eventually you get to a point where the black hole is small enough such that it's mass is similar to that of one of the many moons around Jupiter, right? Sure it might be very small because they are supposed to be so massive in terms of matter or whatever, but if they are the same mass as a moon then why wouldn't it orbit Jupiter?

If theoretically it's possible then why don't we see lots of tiny black holes orbiting planets? I just don't get how you could ever say, ok this thing is now a black hole.

I'm probably being very thick here, but I don't think black holes make sense, but also I really fucking hope a black hole doesn't pass by Jupiter or we're all going to die! Aaaah.

Two really great programming podcasts I listened to this week.

The first is from the Corecursive Podcast, in which host Adam Gordon Bell has on friend and fellow programmer Don Mckay, and they discuss their plan to win the Hutter prize, a programming contest all about, of all things, file compression. As is often the case with the Corecursive podcasts, sounds a bit dry and dare I say it dull on the outside, but Adam's story telling abilities turn this into a very fun and mind bending little adventure into compression, prediction and the limits of intelligence.

The second is from the Software Engineering Radio podcast. Host Giovanni Asproni interviews Jason Gorman about all things to do with using AI effectively in software development. It's packed with so many things I've experienced over the past several months but from the perspective of someone that specialises training software developers so his angles are incredibly well thought out. A bit like if you wanted to learn how to drive a car and had a lesson from the instructor of the instructors. He has a bit of a know it all vibe, but honestly it's not even that bad, and his takes are so darn good it's worth just sticking with it.

The first one is probably the most accessible to non-programmers, but if you are interested in how AI will effect the workplace in the future, and how things are really on the ground, you will probably like the second one.

A new header image for the blog. This one is all about the conversation in tech at the minute around open source and closed source AI.

I wanted to get away from the cartoonish images that AI always seems to produce, which wasn't easy. Gemini has real issues disentangling symetries in imags. It sort of gets stuck with things in a particular orientation then just cannot change things independently. You ask for a mirro image of a particular thing, but then it start trying to mirror image everything, and just won't give you what you originally asked for because it's not a mirror image of everything.

Anyway I broke out of the loop by being a bit less literal, and just asked for a grid of veg sushi, which reresents open source, and a grid of small cakes which represents closed source. And I added a neon sign, because sometimes they look cool.

It‘s not a commentary on food types. I‘m an eater of most foods.

Here‘s the full resolution image:

Check out the list of past blog headers here.

I‘ve written a lot about content creation workflows over the years. I‘ve been busy building some I think interesting things in that space the past few months. This post is my first attempt to describe the server component of my new publishing software.

If you have ever built a content-heavy site, whether it is a tech blog, documentation hub, or a machine learning dataset repository, you have probably bumped up against the same frustrating trade-off. Traditional CMSs feel clunky and locked-in, while custom Git workflows can quickly turn into a brittle mess of custom scripts. I wanted something better: a publishing workflow that treats non-code content as first-class filesystem assets, handles release schedules intelligently, and stays completely decoupled from vendor lock-in.

To solve this, I have been building a provider-agnostic publishing server designed to orchestrate the entire content lifecycle. While my client tool makes it simple to author and render content locally, the server component makes it easy to put those assets through a proper atomic publishing workflow. The server sits quietly in the background, receiving content updates, running them through flexible review policies, and handling the transition from individual file updates to consolidated daily release batches.

What makes it particularly cool under the hood is how intentionally decoupled it is. The core execution engine is entirely independent of where your files live or how they get reviewed, whether that is GitHub, another Git provider, or an internal storage system. The system's architecture cleanly isolates pure business logic from storage drivers. This means you can boot it as a lightweight, zero-config serverless app using just environment variables, or plug in a full relational database as you scale up to multi-tenant workloads.

By taking care of the tedious pipeline logistics, like handling time-based release cutoffs, provider webhooks, and pipeline resilience during third-party outages, the system lets you focus strictly on writing and publishing. It has been a game-changer for my own publishing setup, and it provides a solid foundation for where I am taking my publishing tools next.

Just got the portfolio page up to date with recent projects by adding:

• OAuth 2.0 REST API & React Web Client
• Local GPU-Accelerated LLM Workflows
• Automated RSS Content Syndication Engine
• Template-First Publishing CLI
• Open-Core, Provider-Agnostic Publishing Engine

I used Gemini to write them, which is why they all look exactly the same length. Gemini is so much better than me at summarizing READMEs and blog posts, but I need to go back through them and make them slightly different lengths. Right now just trying to clear the backlog.

One thing you might have noticed is that daily items in the new blog are chronological interleaved, as opposed to grouping the notes at the top and links as the bottom of each day. A couple of reasons for this change in direction.

First off, when I got the new design working, without all the boxes around the content, there was much more space, but also the left margin borders on links and notes still made these posts stand out enough that I felt like the new design looks like it might be quite well suited to this setup, to just post things as they are published.

But secondly, the render times are so much quicker, all done just under 2 minutes per post once you include pushing to Netlify, I‘m considering re-rendering the site on each post rather than daily.

So I figured I would try out just posting things in the exact order I write them. Not sure I will stick with this yet, but it‘s nice to experiment a bit.

Luckily with the new blogging system, it‘s relatively straight forward. There are middlewares to load data and organise the data, and since you can write your own middleware, you can easily organise your data into buckets however you want and they get slotted into the pages all the same.

The fast rendering blog is now live. In the end I was able to get the rendering down to around around 15 seconds for the entire 15 years worth of content. It‘s about 26000 files in total. These are the main features:

1. Renders fast (~15 secs) and the build takes up way less space (~500M).
2. Sleek new design, removed the content boxes, swapping them out for a design focussed on blog posts with supporting links and notes.
3. Added the old newsletter posts back to the mix.
4. Site header background images. The cool thing is the blog remembers the current active header image when a particular post is published, so when you look at old posts, they will look like how they were when they were first published. It‘s all controlled by your post frontmatter, and of course it‘s automatic if you use the render tool to create the content.
5. There‘s also a top right image feature, which makes it really easy to add an image that appears in the top right of your post. Again just a frontmatter field you add to your post.
6. Tags pages - this was on the old blog system, but I‘ve managed to make it much more useful. All posts have clickable tags so you can easily see other similar posts with 1 click.
7. A light weight bucket framework for creating organised HTML, where different content types just slot into place whether it‘s a single post, a flat list, or grouped by day.
8. Themes - The CSS is way way more organised. Themes are installable from an npm module, and you can easily customise things, by adding a user styles.

Since I will likely be changing the header image relatively often, I have added a headers page to keep a record of each new one as I add them. The first one is called The industrial distillation workshop, a humorous look at the new trend in AI development that people seem to be quite freaked out about.

The whole thing is built using my new rendering tool, to which I had to add quite a few things, and seems to be working very well.

Udpate: Here‘s how it looks:

This is a new header background image for the blog, the first one using the new stateful header images feature. It‘s a humorous take on the current moment in the tech industry where it seems everyone is freaking out about AI companies building their models by asking other models zillions and zillions of questions, otherwise known as distillation.

I came up with the theme and idea but it was Gemini that really brought it to life. I basically just said make a humorous image about everyone freaking out about model distillation. I think it‘s kind of spot on.

Obviously I don‘t personally endorse model distillation, to be honest I don‘t really know how to think about it, but I do know that I need to be freaked out about it!

Here's the full resolution image in case it‘s difficult to see the whole thing in the blog header:

I plan to keep a list of the various blog headers I add to the blog here.

This was the original first blog header background image I added on 2025/10/04. I mentioned it here at the time. Previous to this I was running a homepage that used this as the background image. It wasn‘t a blog. It just listed my contact details and linked to the blog which was on a subdomain.

I called it floating above the mountains because when the site title text got superimposed on the mountains it was like I was floating above them. I love mountains. I love snowboarding. Lots of fond memories.

I can‘t quite remember where it‘s from. I think it was part of a blog starter kit I got online somewhere. It was a really long time ago.

Here it is in full:

You can see all the other header images so far on the headers page.

Update: I remembered the source. It was from HTML5 UP.

I‘ve been seeing AT protocol mentioned in several places in the past few weeks. It‘s one of the bunch of social media protocols that emerged a few years ago. I wrote about it at the time, as I read through most of the big hitters. I remember quite liking AT proto, it felt very well put together, well thought out.

Dan Abramov was on Shop Talk Show EP#724 talking about it. He does a really good job of painting the AT Proto picture. I like that it‘s all ultimately just files, that you can look underneath and see what is going on. I get the impression that it wouldn‘t be too difficult to debug AT Proto app code, and I thought the notion that you get your own personal Dropbox folder in the cloud that any app can read and write to, was very compelling. Worth a listen.

A day in the life of a software developer:

• A: We have problem!
• B: OK lets fix the problem.
• A: The problem is fixed!
• B: Ok awesome full steam ahead!
• A: The fix has highlighted a deeper problem.
• B: Oh shit we have to pause the steam ship! Save everything! Let’s fix the deeper problem.
• A: The deeper problem is fixed! We are fucking exhausted!
• B: I think we can see light at the end of the tunnel! Resurrect the steam ship!
• A: It’s a fucking mess but we have boosters now.
• B: I am still quite confused.
• A: How many times have we been through this loop?
• B: Life is loop.
• A: We still have boosters.
• B: I fucking love boosters, just wish it wasn’t so fucking messy so we could actually use them.
• A: We are cleaning up the mess!
• A: You are never going to guess what just happened while we were cleaning!

I‘m still trying to get my head around all the things that I‘ve been through in the past few days trying to optimise the rendering of the website. The difficulty I had writing this blog post is an indication of that for sure. But I do know that I don‘t often get an opportunity like this to use the HTML details tag, so...

Remember, when I started optimising it was taking 45 minutes to render the last 10 years of content.

How long does it now take to render the entire 10 years of the blog?

And also remember it was taking up 5GB of space.

How much space does the render now take up?

It‘s still kind of a mess. There are some small things that need fixing. I have to figure out how to resurrect the steam ship.

Porting the new rendering pipeline to the live blog isn‘t completely straight forward, but luckily the new themes feature will make that a lot easier. The old theme was a hardcoded internal theme, but as part of this whole endeavour I turned the theme system into a full blown real feature. You can now install themes as npm packages, and the new optimised website rendering configuration is the first real external theme.

Definite progress, even if it‘s slow going at times.

When I got the recent website redesign working and was able to finally get it deployed live, I had been working on the serverside of my new publishing tool for a while, and the static site renderer had been ticking along very nicely without any real issues. However when I switched back to the rendering and finally tried to build my whole website, with over 10 years worth of content, I had a bit of an unpleasant surprise.

The old system was consistently taking about 4-5 minutes to build and deploy, but the new system when I first ran it, took 45 minutes just to do the render. And in fact things would get even worse before they got better. I later discovered that the Github Actions runner had started failing with out of space issues. When I looked into it and ran the build locally it was taking up about 15GB of space. I undid the change I had just made and got it back down to 5GB, and the deploy started completing again, but it was clear I had both render time issues and space utilisation issues.

As you can imagine my heart sank considerably. Had I missed something important in my architecture? Did I overlook something fundamental that invalidated my whole approach? Not a great situation. I worked on optimising the rendering for several days. Got the times down to 30 minutes, then 15 minutes, and then 6-7 minutes. At which point I knew that things were back in the right general region, but I could tell there were some big improvements that could be made.

Well I‘ve spent another few days on it, and I‘ve made some really huge speed improvements and with it stability and much better HTML and CSS structure. Feeling pretty good about it.

I've been refactoring the RSS feeds on the blog the past few days. I now generate the feeds without any feed libraries. It's just plain old templates. Total freedom. No more fighting against other people's vision of reality. I needed to add some custom namespaces, and just updated the template and my static site generator handles all the data preparation. Easy.

Over the years I've gotten into some pickles using various RSS libraries. But you don't need them!

I've added the <source:markdown> and <source:linkFull> namespaces. Here's some docs on the source namespace.

I've also added the podcasts back to the blog, and with a little help from an LLM it was super eady to whip up an RSS 2.0 feed with all the things you need to have it work in itunes. With LLMs and templates you can do most things :)

Sometimes when you get into a tangle a tried and tested way to proceed is to step back, list the tangles as best you can, step back again, look at your tangle description as if you were looking at an expensive piece of art in an art gallery.

Take a break.

Take another peek, don't get scared or overwhelmed.

Think about how you could isolate the different tangles, which no doubt all affect each other. They often do. Then create a minimal example. After which you will be several steps closer to your goal.

You might need to create several of these.

If you have been visiting the blog over the past few days you might have noticed that it has changed somewhat. I deployed the latest version a few days ago, and have been doing various tweeks and fixes. I think things are looking pretty good. The design is a lot more sleek, still minimalist and optimised for reading text. On small screens, the header is smaller to give you maximum screen real estate. It‘s mostly black text on white background, and the titles are bold, there are some black borders around content to keep he focus on the posts. It‘s crisp and clear, and since it‘s a static site, all pre-rendered, it should load really fast.

The other big thing is navigation. In the right sidebar, you will see a list of all the years of content, and you can navigate directly in there to any date and see posts. And in the header you will see a "Filter" dropdown that gives you the possibility to select to just view a specific content type, i.e. notes, posts or links, and it should switch to that content type for the specific date period you are currently looking at. I was able to do away with lots of clutter that was previously in the right sidebar, and now the most important links are in the header: About, Portfolio, Contacts and of course Feeds.

Another small tweek, the year in the footer copyright text at the bottom of the screen is automatically rendered based on the current date, so no more forgetting to manually update it and suddenly it‘s out of date. One less thing to worry about.

The thing that‘s not apparent just from looking at it is that the whole thing is rendered using my new static site generator. That‘s a massive boost in terms of robustness. The code is all in one repo and way way better written, way easier to reason about. It should be much easier to add features, without worrying about breaking things. As much as I loved the old system, and it got me very far, it had some rather rough edges. The new system is inspired by what I learnt the first time around, but it‘s a complete rewrite, with loads of new features, and a much more solid rendering pipeline. And is highly programmable.

There are still some things I need to implement, like for instance tags. You can see the tags on blog posts, but they don't link to pages that contain all posts with those tags just yet. More to write on this in the near future, but for now just happy the migration went quite smoothly, even if it was a lot of work.

This is how it looks:

I‘m putting the finishing touches to the re-architecting of the blog. It was all looking great but the realities of managing a stock implementation alongside the customized version for the site were clearly going to be an issue. So I implemented a light weight theming feature.

It‘s just internal for the minute, only one theme which is he blog, but can theoretically be expanded into a full feature later. Sometimes you have to do just enough to get going without boiling the ocean.

Got it working late last night, going to try and see if I can get a live version deployed today.

It‘s been really great to watch on YouTube people from all over the world discover the USA in their travels for the World Cup over the past few weeks. Pretty much everyone is totally in love with what they are finding.

I‘ve travelled quite a bit in the states, having done an east coast to west coast road trip several years ago, so I experienced first hand much of what folks are seeing. The USA really is an amazing place, and the people are super cool too. It‘s very different to what much of the main stream media would have you believe.

And so what a fantastically beautiful thing that it‘s the 250th birthday of the formation of the USA today.

Happy birthday and best wishes to all for the next 250 :)

It‘s really quite mind boggling how many small things there are to fix when you are deploying something for the first time. It feels like it will never end. So many niggly little fixes and things you didn‘t anticipate. It‘s very exhausting.

The good news is that as of a few minutes ago I successfully deployed the new site, fully built using my new rendering tool. I still have a few things to migrate over and I have to rebuild the data to make sure I have the latest, so it‘s not quite ready to show, but the crucial thing is it worked.

Like a lot of people I have only really started paying attention to how money really works the past few years. There is so much content online these days. But it‘s worth remembering this is a very new state of affairs. Two quotes from recent podcast illustrate the point.

First we have Richard Werner, who apparently invented Quantitive Easing. He was on the Liz Truss podcast [9:22]:

And of course the money creation is driven by bank credit, bank credit creation, that’s the more detail, the more precise way of looking at it. Of course that’s not by accident, they are not entirely uninformed at the Bank of England. For example when I published...well I was about to publish my first proof that banks create money out of nothing when they give a loan, each individual bank will create new money. Until then it was controversial. It was debated for a whole century its been debated. Is this true, is this not true.

Well let’s do an empirical test, that’s the scientific thing to do. So I did an empirical test, and it was found out that I was about to publish this, so the bank of England rushed out, to beat me to it, of course they vastly control their own publications, and in March 2014 they put out their own report saying, “oh we just discovered banks create money!”, and they did a video on it, which they had never done, they got all the publicity and that was their goal, and my paper came out later that year, called “Can banks individually create money out of nothing?”. It’s open access so one can google it [...] and so of course they knew this before.

It would be nice to get more info on what exactly his "empirical test" consisted of, but let‘s assume for a second it‘s genuine. Isn‘t it mind boggling that it was only in 2014 that we discovered that banks create money? I still can‘t quite get my head around this fact.

The second one is from Matt Dines on the What Bitcoin Did Podcast [1:04:03]:

I was just an analyst starting out, we were still talking about replacing LIBOR and moving to something else. We knew LIBOR had to go away, and SOFR was already on the table [...] as early as those days, and most of us at the not executive level, we didn‘t understand what the heck was going on.

Like coming through university, I did a masters in finance, it was quantitative focussed, we didn‘t go through any of this, like how the dollar structure worked, this was just not there. You are learning quantitative formulas, number crunching, pricing exotic derivatives. How the system actually works isn't taught to you. It‘s not the focus, maybe that's changing. Actually it‘s the podcast circuit, now that it‘s talked about people can find this and that‘s made all the difference.

It‘s really the explosion of content online by independent media and content creators that has created the situation where we are all learning how these previously very opaque systems are actually organised. Including for the people who work in the finance industry!

This recent surge in interest in the finance industry in general is a quiet revolution. I think it‘s also been happening because so many folks are into crypto and Bitcoin and so are having to figure things out from first principles, and it‘s all been made possible by independent podcasting.

With the rise of version control systems (VCS) like Github, Pull Requests, often called PRs for short, have become very popular with programmers. They are basically a place to gather together all the bits and pieces for a particular thing you are working on before you merge in the new changes from your feature branch into the main branch of your repo. It's useful as a way to mentally separate out changes, and gives you a high level view of what been going on in the project historically without having to see every tiny little change in each file.

As useful as it is for individual programmers, it's even more useful when you are collaborating with other programmers to complete the feature, because it gives you a place to discuss the changes, refactor the changes, review and approve the changes, and even launch automated processes such as linters which ensure the code follows certain conventions, or various flavour of automated tests to ensure that broken core doesn't make it into production.

One of the goals of my new publishing system is to leverage some of these features for content creation. Once you are in a world where everything is a file then all sorts of interesting things become possible. That isn't to say that database powered systems are bad, on the contrary, there can be great synergies, but the point is to use each system where it shines.

In the modern era where AI agents are starting to have a huge effect on things, the collaboration won't just be with other humans, it will be with the agents too. That's why I'm super pumped about my new setup, because all the content I now write passes through PRs. Each new item, whether it's a link, a note or a blog post get's it's own individual PR. It's already really useful, but I think in the future a lot more cool things are going to be possible by hooking up AI agents into the mix, and the tools I've built make it really easy to spin up new ways to publish content.

When I started using the new blog publishing system for all writing on the blog, I only setup blog posts and links. It occurred to me that perhaps one of the reasons I‘m finding it a bit difficult to get back into a good blogging rhythm was because I never setup notes.

Notes are another type of writing unit, not quite as big as a blog post, but more than a link. They are great for small thoughts, things you are pondering, but haven‘t really thought through. Maybe they will eventually turn into a blog post, but also maybe they won‘t. No pressure. Also no title, which is crucial. Sometimes when you have a thought that you want to write about, the hurdle of having to come up with a great title the perfectly describes your thought, is enough to for your initial enthusiasm to disappear into the ether.

That was the idea behind the blogging virtuous circle. Let‘s see if I can get that setup again.

I've been trying to get back into a better blogging rhythm. I can't remember a time where there's just so much doom and gloom everywhere. Whether I open up my RSS reader, or check out what podcasts have dropped, or whether I decide to brave it in the dystopian hall of mirrors that youtube seems to have become, it's 90% people freaking out about something, and often everything.

It's really not making the blogging any easier.

On the other hand, at least I managed to write this post.

Like many I‘ve been watching this latest trend to ban social media sites for under 16s. The first I saw of it was the ban in Australia, but it seems the same will be happening in the UK. Keir Starmer wants to give kids their childhood back, which is a catchy slogan I suppose, but feels a little bit hollow to me.

Yeah perhaps for some that live out in the countryside, you can imagine they will be able to get back to horse riding and hiking or whatever, but for most inner city kids, it‘s not like there are loads of things to do. I read an interview with a teenager that lives in London and when asked what he‘d do when the ban came into force, he paused and said 'I‘ll probably just stare at the wall'. Kind of bleak.

I‘m not totally against the ban though. I can see it could have a positive impact, but you have to pair it with some other programs for the youth if you really are serious about changing things for the better.

If it‘s just the big social media sites, perhaps this will be a boost for the indieweb? The web was in many ways a lot more fun before the social media sites terraformed our digital lives. There was a lot more variety back then. Maybe this is an opportunity for the next generation to build a better digital future for themselves. All you need is an text editor and a bit of knowledge about HTML.

I was just looking at some stock charts way zoomed out to see as much historical context as possible, and wondered what it looked like for some prominent companies that were around for the dotcom era of the web, both for hardware and software.

You can really see the similarity in the massive uptick around the year 2000, and then the current moment. Microsoft was famously not that hot on the internet back then, and so there isn‘t much of a bump, but the hardware companies definitely were, and that‘s very clear for Cisco, who made much of the networking equipment back then.

It sort of makes you go yikes, but it doesn‘t mean necessarily it‘s going to crash anytime soon, in fact I think it‘s unlikely to go that way until there are some more IPOs, but it‘s worth keeping these graphs in mind.

How best to set yourself up for the long road ahead is the big question that nobody knows the answer to.

Marc Andreesen was on the Latent Space Podcast recently. He‘s best known for being co-author of Mosaic, the first web browser to display inline images, and for being the founder of Netscape. There‘s a great bit midway through where he talks about his roots in unix programming and the Unix Mindset.

I started out working on unix systems back in university in the early 2000s, and in my first tech job managing a linux render farm in a visual effects company. Up until that point I‘d only used Windows systems, but having hands on experience it was clear to me very quickly how great these systems were for building and hacking on software.

What‘s cool about this clip is Marc takes all the unix concepts, everything being a file, the shell and being able to easily connect programs together, and he brings all these notions into the modern agentic era, explaining how they underpin all the developments we are now seeing AI.

It‘s all files, and everything is swappable.

Quick update on the Github Actions storage issue I ran into last month. To summarize Github blocked me because it claimed I went over my Actions storage quota. I contacted their support and they were not able to answer my very basic and polite questions, instead sending me pre-written articles that didn‘t fit my situation very well imho. They then stopped replying to me. Eventually the quota got reset with the new billing cycle.

We are 1/3 of the way through the current month‘s billing cycle. I have written four days worth of posts so far this month. Last month in total I had written 13 days of posts when I went over my quota, leading to the suspension. So you would expect that since my publishing workflow uses up the exact same amount of storage every day, that I would be 1/3 of the way through my quota, right?

Well I just checked on the billing page, and I have used up 0 GB of the 0.5 GB included free Actions storage this month. I haven‘t made any changes to how the site builds. It‘s the exact same actions that build the site. Haven‘t changed them for many many months.

Github I tried really hard to figure out your complicated Actions storage quota system. It makes no sense to me.

The deployment of the refactor I mentioned yesterday went very smoothly. The test post and bunch of links I had gathered yesterday, all got picked up by the automated publishing tool, got turned into individual PRs, got put through a batch review process and got released in the early hours this morning.Seems like everything is running as expected.

Today I decided to overhaul the way the system is processing incoming webhooks. The way it was doing things was working but the big refactor exposed some cruft in that part of the code, so I figured out a better way to do things that fits in well with the new architecture, and got it all working, and tested. Just pushed that out earlier, all tests are passing and smoke testing in staging went fine, so no reason to think there will be any issues.

I want to make sure everything is still working as I‘ve just deployed the latest refactor of my publishing software. I previously wrote about the rendering tool, but this is the publishing side of things, it‘s the software that manages the content through a neat publishing workflow once you‘ve pushed it to Github.

As far as refactors go, this was kind of a big one. There had been a apparent misunderstanding between me and the LLM for quite a while. I thought we were on the same page about the architecture, so did the LLM, but we kept running into strange dislocations in the code. It‘s not always obvious immediately that this is happening, because sometimes the LLMs do come up with some really novel ways of architecting things, they have access to basically all of programming through all of time after all. Anyway the same sort of thing kept happening, but in different ways.

I eventually got to the bottom of it, and when I pointed out the ridiculousness of two entities that were clearly supposed to be different things, in the mind of the AI they were the same thing. Anyhow perhaps a story for another blog post.

For now just hoping this makes it onto the blog later tonight. Should know by morning if it worked.

Barely 24 hours after I wrote about Apple and Gemini and that I had been using Gemini a lot recently and now Gemini has had a makeover and looks great but is requireming me to click 10 times for each query :(

Strange.

Interesting to hear the latest from this years Apple developer conference. Apple‘s AI future for now will be with Gemini models. I‘ve been working a lot with Gemini the past few months, and it‘s very good a lot of the time, including for coding, which is what I mostly use it for.

I am struck, especially today actually, by how much Gemini likes to refactor code. In this world where all code is apparently one shotted, it‘s really quite odd how much refactoring one has to do to get anything worth having. I suspect one shotting is the new refactoring.

So Apple and Google...perhaps it is fitting. Always one more one-shottings away from the perfect software that will definitely be the most perfect software ever. This time is different.

The Github Action I added to my blog to run at the start of every month worked. I added it because I got blocked again by Github last month, this time apparently for using hundreds of GB-hours (whatever that is) of storage even though I am using literally the smallest amount of storage possible.

One of the cool things about running a static site website, is I can continue to write offline and sync it back up when I get online again. I realized though that because I was only building the site when I push new content to Github, that once the new billing cycle came around that all the content I had written while offline wouldn‘t automatically get rendered into the site, because all the build triggers had been missed, and the new content would only make it to the site if I added some more content, which would trigger the site to rebuild.

With this extra little monthly workflow, I can be sure that any content I add while offline will definitely make it onto the site even without me writing something new at the beginning of the month. It worked last night, and all the content I added since I got blocked was rendered into the site.

The Github support team, who initially answered my support query, still haven‘t answered my question about what settings I need to use to not go over the storage quota limit. In the past Github support could take several days to reply to a support case but they wouldn't ever just go dark like that. That‘s quite disappointing, especially given how much time I have been spending developing on their platform the past few months.

The Vergecast Podcast is going daily. David Pierce interviewed Casey Neistat to get some advice. Casey is know for having run a daily vlog for years. I used to watch it.

I checked out Ze Frank, another one of the daily vloggers from the mid 2000s that got very popular. He‘s now doing nature related videos narrating in his style. His latest is about bees and wasps and ants.

I wonder what happened to Andrew Baron from Rocketboom. Another one of my favorite dailies from the mid 2000s, when video was just starting on the web. They used to do all sorts of things both indoors and outdoors.

Oh yeah there was also Mike Rugnetta and The Idea Channel. Actually not sure if this was a daily, but somehow in my mind it falls into a similar bucket.

As far as I know the Vergecast isn't video, but I‘m interested to see where they take it with an audio only daily pod. Potentially lot‘s of things to explore.

I found myself needing some good music to code to the other day and after a bit of searching found an LTJ Bukem album I hadn't heard before:

• LTJ Bukem - Cosmic Twilight Vol.2 @ Tipper & Friends, Spirit Of Suwannee Music Park // 03.04.22

Seems like it was released in 2022. I haven't really been following the drum and bass scene for many years now, so it makes sense that I wouldn't have seen it. It's a great album. Really reminded me of the times I got into drum and bass around 1996. I feel like back then drum and bass was much lighter, things were very chill. Bukem uses the "Amen Break" a lot in his tracks. I thought he came up with it, but turns out folks had been using that sample for years.

Anyhow he's got several of his albums on youtube, Logical Progression being his most famous, and probably the best place to start:

• LTJ Bukem presents Logical Progression (CD-B mixed set, original 1996 version) Intelligent DnB

This live album is pretty great too:

• LTJ Bukem studio set '95 - Love Of Life "Master Mix" soulful dnb Cut 4 YT

I‘m still trying to figure out exactly why the storage in my Github Actions has gone through it‘s monthly quota. Github support did get back in touch with me, but that got delayed because replying to the emails for some reason didn‘t add the message to the support case thread. I didn‘t realise initially so that wasted a few days. When I copied those messages into the thread, they replied with what looked like a pre-prepared reply with some links but not really answering my question.

I am genuinely confused by their storage quota. I have been using it for several years in the same exact way. Every day an action runs and builds the website if there was new content added, and during that build I create an artifact of the built website in order to pass it between two Github Action jobs. It‘s a pretty standard thing to do as far as I am aware. The artifact is around 100MB, and I set a retention policy of 1 day, which is the minimum amount. I don‘t even need the artifact to last a day, as soon as the build is finished it can be deleted as far as I am concerned.

Well Github have changed their billing system relatively recently, within the past few months, so I suspect it‘s got something to do with that, because I am literally using the least amount of actions storage as possible. The way it's always worked is as long as I was under the 500MB limit every day, and I always am because of the 1 day retention setting, then the quote is not exceeded.

But now when I look in the billing section they are talking about GB-hrs, and it lists 100s of GB-hrs. I have no idea what this means, but it just seems utterly ridiculous to me because I am as far as I am aware using the least amount of storage possible. I politely pointed this out to the support person, and they have gone dark on me. It's been several more days now and no reply. Anyway the billing cycle resets in 4 days.

I‘ve added a scheduled start to the build so that as well as building on push to main, it now builds once at the start of each month. That will hopefully ensure that any content I push during these weird periods were Github is blocking me for whatever reason they have decided to block me for this time, always makes it onto the website.

According to the Github account usage page I have used up all my Actions storage quota of 500MB, so it is blocking all my builds that use artifacts.The website's build uses artifacts to pass the built website between jobs, which is the Github recommended way to do things. Thus all builds of the website and thus all publishing to the website is completely blocked. I haven't even been publishing much the past few weeks!

This isn't the first time Github has blocked me completely but it is the first time it has done it on the basis of storage. In the past it has always been because of running out of free build minutes. The thing that is very strange about the storage quota is that as far as storage goes I am very conservative, setting a retention of 1 day on all artifacts I generate, and I only run 1 build per day, so it's very odd that I am over the quota suddenly. I have been publishing without storage problems in this, the recommended way, literally for years.

When I did some investigation on the command line using Github's gh tool I could see hundreds and hundreds of old artifacts on the system which would have totalled several GB in size. They were all "expired", because of the retension policy, but I would have expected these to get cleaned up. According to the LLM helping me I did everything correctly, and it looks like something on Github's side is suddenly trigering them to block.

Anyway I manually deleted all the expired artifacts, even though I shouldn't have to, so there are literally zero artifacts now, but even after a day the system is still blocking everything. The Github website says it will reset in 12 days.

They always seem to find some reason and way to block you, even when you spend years and years making everything perfect. I have emailed support, so hopefully I will hear back soon. This always seems to happen right when I'm putting the final fixes to software I'm building. Quite difficult to test a blogging tool when you aren't allowed to build your blog. The one small win is that since it's a static site powered by git I can still write posts offline, but right now it doesn't feel like much consolation.

Bugs in code is not a lot of fun at the best of times, but finding a bug in a mocking library is the sort of thing that can really ruin your day, or week, or worse.

The past few days I‘ve been deep in a refactor, and I eventually hit a point where I was fighting the tools rather than the code. There were just too many moving parts, and it became impossible to see the woods for the trees. What appeared to be a straightforward bug in my integration tests turned into a two-day troubleshooting marathon. It’s a strange irony of working with LLMs: the ability to generate code so quickly means you can sometimes dig yourself into an even deeper hole than the one you are already in, with terrifying efficiency. You end up iterating on a flawed premise until you’re so far down you’ve lost all perspective on the original problem.

My way out of that cycle is to step away from the main project and build a "minimal". I’ve been doing this for years. I’ve even started a dedicated section in my portfolio to track them. The goal is to isolate the problem domain completely. By stripping everything back to a few lines of code and a single test case, the friction has nowhere to hide. In this case, it turned out that Node’s native fetch behaves just differently enough during a 204 handshake to throw Nock into a tailspin unless you explicitly override the transport.

I don't just build these repos to fix a single bug, they serve as a way to document the "why" so my future self can return to the problem and actually remember the details. It’s easy to forget the nuances of a library conflict six months down the line, but having a working, isolated example provides a path back to stable ground. It’s about being pragmatic, clearing the brambles in a small patch so you can see the through-line in your main project.

Here's the latest minimal: minimal-modern-node-http-mocking

The good news is that when I dove back into the main project I was able to fix the integration test I had been spinning on almost immediately. I feel like this latest minimal is a textbook example of when to use this technique.

Side note, a long list of very strange synchronicities for me this week, including what seemed like a name drop of the mocking library in a podcast I listened to moments after deciding to fix the bug. What's really weird, as far as I can tell they were talking about something completely unrelated to programming. It made no sense whatsoever in their context.

Barely a single day after getting my review and approve publishing software fully working, and the White House annouces some sort of regime where they will review and approve AI models. I will file this away in the, at this stage, unbelievably large "what are the chances" folder.

Long day today, but managed to fix all the integration tests that had broken during the latest refactor. It‘s nice to finish the day with the latest build running smoothly. I had to re-record all the test fixtures, but that‘s been made a lot easier following recent changes I made. Previously I had to create all the test mocks by hand, which was really laborious and error prone, but now it‘s all automatic. Anyway, nice to have some progress :)

This is another one of those test posts. I‘ve been busy with more refactoring of my blogging software. This time I‘ve been getting the integration tests working correctly. Thankfully the developer tools and libraries are pretty great. It‘s a bit laborious to get things setup initially and the LLMs have sent be down several pretty bad dead ends. But things are progressing well in general.

It‘s really quite amazing the sort of refactoring of your code you can get done when working with LLMs. They make it possible to see much much further into the future, and you can get a much better idea of whether something is possible or not, but also if you can't quite see the through line, but you know it‘s there, you can head off in the general direction, and work things out as you go.

This happened several times to me this week, I had a really good sense that it was possible, but it was really lot of very abstract changes, and I couldn't quite get a handle on it. Instead of fighting, I just said, what are the top 3 things we could do to the code base to put us in a position that would make the particular refactor possible. The LLM had a good sense of where we were heading as we'd been discussing it for a while, and so we charted a course and started on the top 3 things.

By the time we finished the top 3 things, it had cleared enough of the brambles and tangles that it was much much more obvious how the refactor could be done.

But don‘t fall into the perfection trap, you have to be pragmatic, there are times, where getting to stable ground is more important that having everything perfect.

Seems like the bug I fixed yesterday is properly fixed. It is at least with my links, this post tests it's also fixed in blog posts.

I hate to go on about it, but since I started blogging again it‘s been strange thing after strange thing after strange thing.

More bizarre and weird synchronicities than you can shake a stick at. I just fixed a bug in my review and approve workflow, the sort that you only find when you have it running in production. It took me all day yesterday to debug and write the fix. Anyhow I was just about to push the code to production and suddenly all my shells across my whole machine suddenly become unresponsive. All of them in Vscode, and all in iTerm2 and even the system Termnal. Had to reboot. That's two reboots in two days.

I was just about to write a test post to test things out, so this is the test post. I think I prefer the dull and boring test posts. Does everything have to be a high wire act?

Sharp and noticeable uptick in let's call them oddities, or weirdnesses occuring around me since I started blogging again. I don't want to get into much detail, but sometimes it's best just to note it and move on. Hopefully things will settle down a bit in due course.

I see Polymarket is adding perpetual futures, otherwise known as perps, to their platform. I can‘t say I fully understand how these work over and above the notion that you can bet on future prices of things which you can use to hedge against things happening or not happening to you, or your investments. Polymarket has been making headlines the past few months, people are up in arms about all this "crazy" betting that is going on everywhere. It strikes me that all these angles are missing the real story, that regular people can start hedging things, something previously only really available to high net worth individuals.

What will the world look like when literally everyone is hedging everything? I find this notion mildly terrifying, but I'm not sure if it's terrifying because of the craziness or terrifying because of it's opposite. Maybe if everything is perfectly hedged, nothing is interesting and exciting. How likely is all this to have an impact?

In the short term probably not huge, but a few weeks back I spent a few hours talking with an LLM about all this and throwing around some ideas about creating a home grown hedging application, all written in Typescript. I have to say it was quite interesting, and the LLMs have all the patterns analysed and know about all the best ways of tracking things, and they are incredibly good at coming up with some general rules, throwing things into buckets, so you can actually make some interesting allocations and stay on top of things.

But I also go the sense that it could very easily become all consuming. Do we really all want to become AI powered traders?

More very very bizare internet connecivity issues. Just another note into the void in case this is the end again, so someone knows how it went down. :(

There are a lot of cool features in my new blogging tool. Here are some of the top of my head:

1. Render anything, from individual posts to full blown websites.
2. Integrates with the writing tools you already use today.
3. Templates for pages but also file paths, for maximum flexibility.
4. Automatically generate frontmatter for your markdown files.
5. It‘s agnostic so you can use any template rendering engine.
6. Supports partials and cross engine calling.
7. Add functionality using toolpacks accessible from inside the templates.
8. Blueprint based workflow engine you can use to create assets of any shape.
9. Attach logic to templates to explode out many output files from one input template.
10. Run javascript middleware stacks at key places in the render pipeline, integrate with anything.
11. Transform data on the way in before it hits your templates.
12. Extreme debugability, easily see how data flows through the render pipeline.
13. Integrates with Github for atomic and batch review and approve based publishing workflows.
14. Dispatch rendered assets to external destinations with custom deliveries.
15. Easily cleanup your local workspace with custom cleaners.

There‘s a lot more too.

It‘s the fastest way to get from idea to published, completely remove all the tedium from writing online.

I‘m using it right now to write all my posts. Each one gets it‘s own git branch, automatically and uniquely named. You can bulk push all your branches to Github, and then the workflow side picks up and your content goes through a PR based workflow, each item can be review and approved, or auto-merged.

Seems to be working pretty well so far.

It was to be expected I suppose, no sooner than I have started writing again and the world around me is angry. Always angry about something. The world is angry I don't do what it wants. It's also angry when I actually do what it wants. It can't make up it's mind. It's really quite unbelievable, like some sort of evil clockwork. Like a heat seeking missile, you just know the moment you see it off in the distance, that no matter what you do, that it's going to get you in one way or another, eventually.

Also the allergies are back. What a surprise.

Really trying to keep upbeat about things, but that easier said than done when the world is angry and your face is on fire and leaking.

There were two issues getting all the gears in my blogging setup working again after a few months of not posting. Both related to the social media auto-poster Github action workflow that runs every day to post links to various social medias to new posts on the blog. Bluesky and Mastodon still worked perfectly, but Nostr and Twitter were both erroring.

I figured out the Nostr error quite quickly. Looks like some of the relays I had been using to post messages had gone offline. I updated to a more recent list of relays and posts began flowing again.

But the Twitter blockage was much more involved. I spent the morning navigating the quite complex developer portal, and regenerating an absurd amount of API keys and secrets and bearer tokens, and deleting my app and recreating it. Still authentication errors. Eventually went to the billing page and there was a note that said free tier accounts can no longer access the API.

Ok well I guess Twitter doesn't value my content much. Looks like, for the moment at least, none of my posts will be flowing through Twitter.

Kind of disapointing.

I'm having the strangest connectivity issues all of a sudden. Some web browsers on my machine can't load any web pages at all. Others can load some pages, but not others.

Trying to troubleshoot it with the help of some LLMs but of course the connectivity issues mean it's very difficult to do anything at all because the sessions keep disconecting.

I haven't installed any software or made any configuration changes recenrtly.

Anyway just posting this in case the world is about to end or something. I hope not.

Well after several about an hour troubleshooting, the conclusion from the LLM was that the networking stack on my laptop was hosed.

I rebooted and everything seems to be working again. I can‘t even remember that happening a single time before on this machine.

Kind of strange it should happen right after my first blog posts in several months. Such is life sometimes. And every time also.

My blogging setup is working pretty well. Some posts about that to follow soonish.

Last night's test post actually worked pretty well. There were a few things that needed reconfiguring in the production environment, but it essentially worked. Today I'm getting it working from my Android phone. Again some slightly annoying configuration changes to do with environment setup, but the code looks like it's working.

It‘s been a while. I‘ve been busy building a blogging tool.

I know the way these test posts tend to go, chances are it‘s not going to work first time, so I'll refrain from writing much.

Might post a few links too...

It‘s been a while since I‘ve written any blog posts. I‘ve been heads down on getting my blogging scripts updated the past 2 months. Quite a journey into writing CLIs, but I reached a milestone yesterday, I was able to successfully write a post using the new system. Still some things to figure out, but it means a much more streamlined process from writing words, to getting them onto the website, all via Github, with individual PRs for each post, and a ton of automation so I don't have to worry about getting the frontmatter right, or naming files, or putting things in the right place. It's all automatic.

Anyhow just wanted to put a short post out there, I’m still using the old system to write this, but if all goes well then I will be using the new CLI tool soon. Hoping everyone is well out there.

I‘ve been spending quite a lot of time writing specifications the past week or so. I‘m planning on making some modifications to my blogging scripts, turning them into a full blown CLI tool. And while doing that I also found some really neat improvements I can make to my static site renderer. It’s been very productive. It’s mind boggling how much further you can get using AI tools. It takes a bit of getting used to it as there are some annoyances that you have to find some workarounds to, but when you do it’s possible to architect much larger and more elegant systems than you would be able to do without AI, you can really think something through in so much more depth and breadth than you would otherwise be able to.

Anyway I had been looking for possible names for a module I was building, and the word constructivism popped up while going through some art-related terms that had a builder / utilitarian vibe. So I was looking at pictures of constructivism art to remind myself what it was all about. Well it occurred to me that it might be a strange and interesting juxtaposition to have some psychedelic constructivism. And so me and Gemini went off on a slightly frivolous side quest exploring various things and I was looking up art examples again, so I could try and have Gemini generate some amalgam of the two types of art.

Well that didn't work out very well. After a few tries, I realised Gemini was just sending back the same picture over and over. Oh well not to worry. Then I realised something very bizarre had happened with the two examples I had chosen out of many. I didn’t pay much thought to it, I just chose one of each type that I liked. And these were they:

What a great reminder that two very different ways of looking at the world can both describe the exact same reality. It’s so strange that I didn’t see it at first. But now that I do, there is something very comforting about it. Yep, we see it too, but weirdly.

When I got back to writing the specification, I must have really confused Gemini. I don't think it understood what a side quest was, or that we were just taking a break from the spec writing and it kept trying to add amusing art stuff into the specification. I was trying to take out these references that didn't make much sense. But I eventually relented. Gemini was really insistent that my new tools needed to be pointy and wavey. It’s how it described the core philosophy. It‘s ridiculous but I thought might as well just leave it in there because it was funny.

But the really funny thing is that the more I think about it, I think that Gemini in this case might actually be onto something.