A day in the life of a software developer:

• A: We have problem!
• B: OK lets fix the problem.
• A: The problem is fixed!
• B: Ok awesome full steam ahead!
• A: The fix has highlighted a deeper problem.
• B: Oh shit we have to pause the steam ship! Save everything! Let’s fix the deeper problem.
• A: The deeper problem is fixed! We are fucking exhausted!
• B: I think we can see light at the end of the tunnel! Resurrect the steam ship!
• A: It’s a fucking mess but we have boosters now.
• B: I am still quite confused.
• A: How many times have we been through this loop?
• B: Life is loop.
• A: We still have boosters.
• B: I fucking love boosters, just wish it wasn’t so fucking messy so we could actually use them.
• A: We are cleaning up the mess!
• A: You are never going to guess what just happened while we were cleaning!

I‘m still trying to get my head around all the things that I‘ve been through in the past few days trying to optimise the rendering of the website. The difficulty I had writing this blog post is an indication of that for sure. But I do know that I don‘t often get an opportunity like this to use the HTML details tag, so...

Remember, when I started optimising it was taking 45 minutes to render the last 10 years of content.

How long does it now take to render the entire 10 years of the blog?

And also remember it was taking up 5GB of space.

How much space does the render now take up?

It‘s still kind of a mess. There are some small things that need fixing. I have to figure out how to resurrect the steam ship.

Porting the new rendering pipeline to the live blog isn‘t completely straight forward, but luckily the new themes feature will make that a lot easier. The old theme was a hardcoded internal theme, but as part of this whole endeavour I turned the theme system into a full blown real feature. You can now install themes as npm packages, and the new optimised website rendering configuration is the first real external theme.

Definite progress, even if it‘s slow going at times.

When I got the recent website redesign working and was able to finally get it deployed live, I had been working on the serverside of my new publishing tool for a while, and the static site renderer had been ticking along very nicely without any real issues. However when I switched back to the rendering and finally tried to build my whole website, with over 10 years worth of content, I had a bit of an unpleasant surprise.

The old system was consistently taking about 4-5 minutes to build and deploy, but the new system when I first ran it, took 45 minutes just to do the render. And in fact things would get even worse before they got better. I later discovered that the Github Actions runner had started failing with out of space issues. When I looked into it and ran the build locally it was taking up about 15GB of space. I undid the change I ahad just made and got it back down to 5GB, and the deploy started completing again, but it was clear I had both render time issues and space utilisation issues.

As you can imagine my heart sank considerably. Had I missed something important in my architecture? Did I overlook something fundamental that invalidated my whole approach? Not a great situation. I worked on optimising the rendering for several days. Got the times down to 30 minutes, then 15 minutes, and then 6-7 minutes. At which point I knew that things were back in the right general region, but I could tell there were some big improvements that could be made.

Well I've spent another few days on it, and I“ve made some really huge speed improvements and with it stability and much better HTML and CSS structure. Feeling pretty good about it.

I've been refactoring the RSS feeds on the blog the past few days. I now generate the feeds without any feed libraries. It's just plain old templates. Total freedom. No more fighting against other people's vision of reality. I needed to add some custom namespaces, and just updated the template and my static site generator handles all the data preparation. Easy.

Over the years I've gotten into some pickles using various RSS libraries. But you don't need them!

I've added the <source:markdown> and <source:linkFull> namespaces. Here's some docs on the source namespace.

I've also added the podcasts back to the blog, and with a little help from an LLM it was super eady to whip up an RSS 2.0 feed with all the things you need to have it work in itunes. With LLMs and templates you can do most things :)

Sometimes when you get into a tangle a tried and tested way to proceed is to step back, list the tangles as best you can, step back again, look at your tangle description as if you were looking at an expensive piece of art in an art gallery.

Take a break.

Take another peek, don't get scared or overwhelmed.

Think about how you could isolate the different tangles, which no doubt all affect each other. They often do. Then create a minimal example. After which you will be several steps closer to your goal.

You might need to create several of these.

If you have been visiting the blog over the past few days you might have noticed that it has changed somewhat. I deployed the latest version a few days ago, and have been doing various tweeks and fixes. I think things are looking pretty good. The design is a lot more sleek, still minimalist and optimised for reading text. On small screens, the header is smaller to give you maximum screen real estate. It‘s mostly black text on white background, and the titles are bold, there are some black borders around content to keep he focus on the posts. It‘s crisp and clear, and since it‘s a static site, all pre-rendered, it should load really fast.

The other big thing is navigation. In the right sidebar, you will see a list of all the years of content, and you can navigate directly in there to any date and see posts. And in the header you will see a "Filter" dropdown that gives you the possibility to select to just view a specific content type, i.e. notes, posts or links, and it should switch to that content type for the specific date period you are currently looking at. I was able to do away with lots of clutter that was previously in the right sidebar, and now the most important links are in the header: About, Portfolio, Contacts and of course Feeds.

Another small tweek, the year in the footer copyright text at the bottom of the screen is automatically rendered based on the current date, so no more forgetting to manually update it and suddenly it‘s out of date. One less thing to worry about.

The thing that‘s not apparent just from looking at it is that the whole thing is rendered using my new static site generator. That‘s a massive boost in terms of robustness. The code is all in one repo and way way better written, way easier to reason about. It should be much easier to add features, without worrying about breaking things. As much as I loved the old system, and it got me very far, it had some rather rough edges. The new system is inspired by what I learnt the first time around, but it‘s a complete rewrite, with loads of new features, and a much more solid rendering pipeline. And is highly programmable.

There are still some things I need to implement, like for instance tags. You can see the tags on blog posts, but they don't link to pages that contain all posts with those tags just yet. More to write on this in the near future, but for now just happy the migration went quite smoothly, even if it was a lot of work.

This is how it looks:

I‘m putting the finishing touches to the re-architecting of the blog. It was all looking great but the realities of managing a stock implementation alongside the customized version for the site were clearly going to be an issue. So I implemented a light weight theming feature.

It‘s just internal for the minute, only one theme which is he blog, but can theoretically be expanded into a full feature later. Sometimes you have to do just enough to get going without boiling the ocean.

Got it working late last night, going to try and see if I can get a live version deployed today.

It‘s been really great to watch on YouTube people from all over the world discover the USA in their travels for the World Cup over the past few weeks. Pretty much everyone is totally in love with what they are finding.

I‘ve travelled quite a bit in the states, having done an east coast to west coast road trip several years ago, so I experienced first hand much of what folks are seeing. The USA really is an amazing place, and the people are super cool too. It‘s very different to what much of the main stream media would have you believe.

And so what a fantastically beautiful thing that it‘s the 250th birthday of the formation of the USA today.

Happy birthday and best wishes to all for the next 250 :)

It‘s really quite mind boggling how many small things there are to fix when you are deploying something for the first time. It feels like it will never end. So many niggly little fixes and things you didn‘t anticipate. It‘s very exhausting.

The good news is that as of a few minutes ago I successfully deployed the new site, fully built using my new rendering tool. I still have a few things to migrate over and I have to rebuild the data to make sure I have the latest, so it‘s not quite ready to show, but the crucial thing is it worked.

Like a lot of people I have only really started paying attention to how money really works the past few years. There is so much content online these days. But it‘s worth remembering this is a very new state of affairs. Two quotes from recent podcast illustrate the point.

First we have Richard Werner, who apparently invented Quantitive Easing. He was on the Liz Truss podcast [9:22]:

And of course the money creation is driven by bank credit, bank credit creation, that’s the more detail, the more precise way of looking at it. Of course that’s not by accident, they are not entirely uninformed at the Bank of England. For example when I published...well I was about to publish my first proof that banks create money out of nothing when they give a loan, each individual bank will create new money. Until then it was controversial. It was debated for a whole century its been debated. Is this true, is this not true.

Well let’s do an empirical test, that’s the scientific thing to do. So I did an empirical test, and it was found out that I was about to publish this, so the bank of England rushed out, to beat me to it, of course they vastly control their own publications, and in March 2014 they put out their own report saying, “oh we just discovered banks create money!”, and they did a video on it, which they had never done, they got all the publicity and that was their goal, and my paper came out later that year, called “Can banks individually create money out of nothing?”. It’s open access so one can google it [...] and so of course they knew this before.

It would be nice to get more info on what exactly his "empirical test" consisted of, but let‘s assume for a second it‘s genuine. Isn‘t it mind boggling that it was only in 2014 that we discovered that banks create money? I still can‘t quite get my head around this fact.

The second one is from Matt Dines on the What Bitcoin Did Podcast [1:04:03]:

I was just an analyst starting out, we were still talking about replacing LIBOR and moving to something else. We knew LIBOR had to go away, and SOFR was already on the table [...] as early as those days, and most of us at the not executive level, we didn‘t understand what the heck was going on.

Like coming through university, I did a masters in finance, it was quantitative focussed, we didn‘t go through any of this, like how the dollar structure worked, this was just not there. You are learning quantitative formulas, number crunching, pricing exotic derivatives. How the system actually works isn't taught to you. It‘s not the focus, maybe that's changing. Actually it‘s the podcast circuit, now that it‘s talked about people can find this and that‘s made all the difference.

It‘s really the explosion of content online by independent media and content creators that has created the situation where we are all learning how these previously very opaque systems are actually organised. Including for the people who work in the finance industry!

This recent surge in interest in the finance industry in general is a quiet revolution. I think it‘s also been happening because so many folks are into crypto and Bitcoin and so are having to figure things out from first principles, and it‘s all been made possible by independent podcasting.

With the rise of version control systems (VCS) like Github, Pull Requests, often called PRs for short, have become very popular with programmers. They are basically a place to gather together all the bits and pieces for a particular thing you are working on before you merge in the new changes from your feature branch into the main branch of your repo. It's useful as a way to mentally separate out changes, and gives you a high level view of what been going on in the project historically without having to see every tiny little change in each file.

As useful as it is for individual programmers, it's even more useful when you are collaborating with other programmers to complete the feature, because it gives you a place to discuss the changes, refactor the changes, review and approve the changes, and even launch automated processes such as linters which ensure the code follows certain conventions, or various flavour of automated tests to ensure that broken core doesn't make it into production.

One of the goals of my new publishing system is to leverage some of these features for content creation. Once you are in a world where everything is a file then all sorts of interesting things become possible. That isn't to say that database powered systems are bad, on the contrary, there can be great synergies, but the point is to use each system where it shines.

In the modern era where AI agents are starting to have a huge effect on things, the collaboration won't just be with other humans, it will be with the agents too. That's why I'm super pumped about my new setup, because all the content I now write passes through PRs. Each new item, whether it's a link, a note or a blog post get's it's own individual PR. It's already really useful, but I think in the future a lot more cool things are going to be possible by hooking up AI agents into the mix, and the tools I've built make it really easy to spin up new ways to publish content.

When I started using the new blog publishing system for all writing on the blog, I only setup blog posts and links. It occurred to me that perhaps one of the reasons I‘m finding it a bit difficult to get back into a good blogging rhythm was because I never setup notes.

Notes are another type of writing unit, not quite as big as a blog post, but more than a link. They are great for small thoughts, things you are pondering, but haven‘t really thought through. Maybe they will eventually turn into a blog post, but also maybe they won‘t. No pressure. Also no title, which is crucial. Sometimes when you have a thought that you want to write about, the hurdle of having to come up with a great title the perfectly describes your thought, is enough to for your initial enthusiasm to disappear into the ether.

That was the idea behind the blogging virtuous circle. Let‘s see if I can get that setup again.

I've been trying to get back into a better blogging rhythm. I can't remember a time where there's just so much doom and gloom everywhere. Whether I open up my RSS reader, or check out what podcasts have dropped, or whether I decide to brave it in the dystopian hall of mirrors that youtube seems to have become, it's 90% people freaking out about something, and often everything.

It's really not making the blogging any easier.

On the other hand, at least I managed to write this post.

Like many I‘ve been watching this latest trend to ban social media sites for under 16s. The first I saw of it was the ban in Australia, but it seems the same will be happening in the UK. Keir Starmer wants to give kids their childhood back, which is a catchy slogan I suppose, but feels a little bit hollow to me.

Yeah perhaps for some that live out in the countryside, you can imagine they will be able to get back to horse riding and hiking or whatever, but for most inner city kids, it‘s not like there are loads of things to do. I read an interview with a teenager that lives in London and when asked what he‘d do when the ban came into force, he paused and said 'I‘ll probably just stare at the wall'. Kind of bleak.

I‘m not totally against the ban though. I can see it could have a positive impact, but you have to pair it with some other programs for the youth if you really are serious about changing things for the better.

If it‘s just the big social media sites, perhaps this will be a boost for the indieweb? The web was in many ways a lot more fun before the social media sites terraformed our digital lives. There was a lot more variety back then. Maybe this is an opportunity for the next generation to build a better digital future for themselves. All you need is an text editor and a bit of knowledge about HTML.