markjgsmith

My blocked man switch worked

The Github Action I added to my blog to run at the start of every month worked. I added it because I got blocked again by Github last month, this time apparently for using hundreds of GB-hours (whatever that is) of storage even though I am using literally the smallest amount of storage possible.

One of the cool things about running a static site website, is I can continue to write offline and sync it back up when I get online again. I realized though that because I was only building the site when I push new content to Github, that once the new billing cycle came around that all the content I had written while offline wouldn‘t automatically get rendered into the site, because all the build triggers had been missed, and the new content would only make it to the site if I added some more content, which would trigger the site to rebuild.

With this extra little monthly workflow, I can be sure that any content I add while offline will definitely make it onto the site even without me writing something new at the beginning of the month. It worked last night, and all the content I added since I got blocked was rendered into the site.

The Github support team, who initially answered my support query, still haven‘t answered my question about what settings I need to use to not go over the storage quota limit. In the past Github support could take several days to reply to a support case but they wouldn't ever just go dark like that. That‘s quite disappointing, especially given how much time I have been spending developing on their platform the past few months. #

Vergecast going daily

The Vergecast Podcast is going daily. David Pierce interviewed Casey Neistat to get some advice. Casey is know for having run a daily vlog for years. I used to watch it.

I checked out Ze Frank, another one of the daily vloggers from the mid 2000s that got very popular. He‘s now doing nature related videos narrating in his style. His latest is about bees and wasps and ants.

I wonder what happened to Andrew Baron from Rocketboom. Another one of my favorite dailies from the mid 2000s, when video was just starting on the web. They used to do all sorts of things both indoors and outdoors.

Oh yeah there was also Mike Rugnetta and The Idea Channel. Actually not sure if this was a daily, but somehow in my mind it falls into a similar bucket.

As far as I know the Vergecast isn't video, but I‘m interested to see where they take it with an audio only daily pod. Potentially lot‘s of things to explore. #

Some great LTJ Bukem albums

I found myself needing some good music to code to the other day and after a bit of searching found an LTJ Bukem album I hadn't heard before:

• LTJ Bukem - Cosmic Twilight Vol.2 @ Tipper & Friends, Spirit Of Suwannee Music Park // 03.04.22

Seems like it was released in 2022. I haven't really been following the drum and bass scene for many years now, so it makes sense that I wouldn't have seen it. It's a great album. Really reminded me of the times I got into drum and bass around 1996. I feel like back then drum and bass was much lighter, things were very chill. Bukem uses the "Amen Break" a lot in his tracks. I thought he came up with it, but turns out folks had been using that sample for years.

Anyhow he's got several of his albums on youtube, Logical Progression being his most famous, and probably the best place to start:

• LTJ Bukem presents Logical Progression (CD-B mixed set, original 1996 version) Intelligent DnB

This live album is pretty great too:

• LTJ Bukem studio set '95 - Love Of Life "Master Mix" soulful dnb Cut 4 YT

Monthly catchup action

I‘m still trying to figure out exactly why the storage in my Github Actions has gone through it‘s monthly quota. Github support did get back in touch with me, but that got delayed because replying to the emails for some reason didn‘t add the message to the support case thread. I didn‘t realise initially so that wasted a few days. When I copied those messages into the thread, they replied with what looked like a pre-prepared reply with some links but not really answering my question.

I am genuinely confused by their storage quota. I have been using it for several years in the same exact way. Every day an action runs and builds the website if there was new content added, and during that build I create an artifact of the built website in order to pass it between two Github Action jobs. It‘s a pretty standard thing to do as far as I am aware. The artifact is around 100MB, and I set a retention policy of 1 day, which is the minimum amount. I don‘t even need the artifact to last a day, as soon as the build is finished it can be deleted as far as I am concerned.

Well Github have changed their billing system relatively recently, within the past few months, so I suspect it‘s got something to do with that, because I am literally using the least amount of actions storage as possible. The way it's always worked is as long as I was under the 500MB limit every day, and I always am because of the 1 day retention setting, then the quote is not exceeded.

But now when I look in the billing section they are talking about GB-hrs, and it lists 100s of GB-hrs. I have no idea what this means, but it just seems utterly ridiculous to me because I am as far as I am aware using the least amount of storage possible. I politely pointed this out to the support person, and they have gone dark on me. It's been several more days now and no reply. Anyway the billing cycle resets in 4 days.

I‘ve added a scheduled start to the build so that as well as building on push to main, it now builds once at the start of each month. That will hopefully ensure that any content I push during these weird periods were Github is blocking me for whatever reason they have decided to block me for this time, always makes it onto the website. #

Blocked by Github yet again

According to the Github account usage page I have used up all my Actions storage quota of 500MB, so it is blocking all my builds that use artifacts.The website's build uses artifacts to pass the built website between jobs, which is the Github recommended way to do things. Thus all builds of the website and thus all publishing to the website is completely blocked. I haven't even been publishing much the past few weeks!

This isn't the first time Github has blocked me completely but it is the first time it has done it on the basis of storage. In the past it has always been because of running out of free build minutes. The thing that is very strange about the storage quota is that as far as storage goes I am very conservative, setting a retention of 1 day on all artifacts I generate, and I only run 1 build per day, so it's very odd that I am over the quota suddenly. I have been publishing without storage problems in this, the recommended way, literally for years.

When I did some investigation on the command line using Github's gh tool I could see hundreds and hundreds of old artifacts on the system which would have totalled several GB in size. They were all "expired", because of the retension policy, but I would have expected these to get cleaned up. According to the LLM helping me I did everything correctly, and it looks like something on Github's side is suddenly trigering them to block.

Anyway I manually deleted all the expired artifacts, even though I shouldn't have to, so there are literally zero artifacts now, but even after a day the system is still blocking everything. The Github website says it will reset in 12 days.

They always seem to find some reason and way to block you, even when you spend years and years making everything perfect. I have emailed support, so hopefully I will hear back soon. This always seems to happen right when I'm putting the final fixes to software I'm building. Quite difficult to test a blogging tool when you aren't allowed to build your blog. The one small win is that since it's a static site powered by git I can still write posts offline, but right now it doesn't feel like much consolation. #

Bugs in mocking libraries

Bugs in code is not a lot of fun at the best of times, but finding a bug in a mocking library is the sort of thing that can really ruin your day, or week, or worse.

The past few days I‘ve been deep in a refactor, and I eventually hit a point where I was fighting the tools rather than the code. There were just too many moving parts, and it became impossible to see the woods for the trees. What appeared to be a straightforward bug in my integration tests turned into a two-day troubleshooting marathon. It’s a strange irony of working with LLMs: the ability to generate code so quickly means you can sometimes dig yourself into an even deeper hole than the one you are already in, with terrifying efficiency. You end up iterating on a flawed premise until you’re so far down you’ve lost all perspective on the original problem.

My way out of that cycle is to step away from the main project and build a "minimal". I’ve been doing this for years. I’ve even started a dedicated section in my portfolio to track them. The goal is to isolate the problem domain completely. By stripping everything back to a few lines of code and a single test case, the friction has nowhere to hide. In this case, it turned out that Node’s native fetch behaves just differently enough during a 204 handshake to throw Nock into a tailspin unless you explicitly override the transport.

I don't just build these repos to fix a single bug, they serve as a way to document the "why" so my future self can return to the problem and actually remember the details. It’s easy to forget the nuances of a library conflict six months down the line, but having a working, isolated example provides a path back to stable ground. It’s about being pragmatic, clearing the brambles in a small patch so you can see the through-line in your main project.

Here's the latest minimal: minimal-modern-node-http-mocking

The good news is that when I dove back into the main project I was able to fix the integration test I had been spinning on almost immediately. I feel like this latest minimal is a textbook example of when to use this technique.

Side note, a long list of very strange synchronicities for me this week, including what seemed like a name drop of the mocking library in a podcast I listened to moments after deciding to fix the bug. What's really weird, as far as I can tell they were talking about something completely unrelated to programming. It made no sense whatsoever in their context. #

Review and approve trending

Barely a single day after getting my review and approve publishing software fully working, and the White House annouces some sort of regime where they will review and approve AI models. I will file this away in the, at this stage, unbelievably large "what are the chances" folder. #

Recording integration test fixtures

Long day today, but managed to fix all the integration tests that had broken during the latest refactor. It‘s nice to finish the day with the latest build running smoothly. I had to re-record all the test fixtures, but that‘s been made a lot easier following recent changes I made. Previously I had to create all the test mocks by hand, which was really laborious and error prone, but now it‘s all automatic. Anyway, nice to have some progress :) #

More refactoring, more integration tests

This is another one of those test posts. I‘ve been busy with more refactoring of my blogging software. This time I‘ve been getting the integration tests working correctly. Thankfully the developer tools and libraries are pretty great. It‘s a bit laborious to get things setup initially and the LLMs have sent be down several pretty bad dead ends. But things are progressing well in general. #

Refactoring using LLMs

It‘s really quite amazing the sort of refactoring of your code you can get done when working with LLMs. They make it possible to see much much further into the future, and you can get a much better idea of whether something is possible or not, but also if you can't quite see the through line, but you know it‘s there, you can head off in the general direction, and work things out as you go.

This happened several times to me this week, I had a really good sense that it was possible, but it was really lot of very abstract changes, and I couldn't quite get a handle on it. Instead of fighting, I just said, what are the top 3 things we could do to the code base to put us in a position that would make the particular refactor possible. The LLM had a good sense of where we were heading as we'd been discussing it for a while, and so we charted a course and started on the top 3 things.

By the time we finished the top 3 things, it had cleared enough of the brambles and tangles that it was much much more obvious how the refactor could be done.

But don‘t fall into the perfection trap, you have to be pragmatic, there are times, where getting to stable ground is more important that having everything perfect. #

Another test post

Seems like the bug I fixed yesterday is properly fixed. It is at least with my links, this post tests it's also fixed in blog posts. #

Fixing bugs and weirdnesses

I hate to go on about it, but since I started blogging again it‘s been strange thing after strange thing after strange thing.

More bizarre and weird synchronicities than you can shake a stick at. I just fixed a bug in my review and approve workflow, the sort that you only find when you have it running in production. It took me all day yesterday to debug and write the fix. Anyhow I was just about to push the code to production and suddenly all my shells across my whole machine suddenly become unresponsive. All of them in Vscode, and all in iTerm2 and even the system Termnal. Had to reboot. That's two reboots in two days.

I was just about to write a test post to test things out, so this is the test post. I think I prefer the dull and boring test posts. Does everything have to be a high wire act? #

Oddities trending

Sharp and noticeable uptick in let's call them oddities, or weirdnesses occuring around me since I started blogging again. I don't want to get into much detail, but sometimes it's best just to note it and move on. Hopefully things will settle down a bit in due course. #

Polymarket and perps

I see Polymarket is adding perpetual futures, otherwise known as perps, to their platform. I can‘t say I fully understand how these work over and above the notion that you can bet on future prices of things which you can use to hedge against things happening or not happening to you, or your investments. Polymarket has been making headlines the past few months, people are up in arms about all this "crazy" betting that is going on everywhere. It strikes me that all these angles are missing the real story, that regular people can start hedging things, something previously only really available to high net worth individuals.

What will the world look like when literally everyone is hedging everything? I find this notion mildly terrifying, but I'm not sure if it's terrifying because of the craziness or terrifying because of it's opposite. Maybe if everything is perfectly hedged, nothing is interesting and exciting. How likely is all this to have an impact?

In the short term probably not huge, but a few weeks back I spent a few hours talking with an LLM about all this and throwing around some ideas about creating a home grown hedging application, all written in Typescript. I have to say it was quite interesting, and the LLMs have all the patterns analysed and know about all the best ways of tracking things, and they are incredibly good at coming up with some general rules, throwing things into buckets, so you can actually make some interesting allocations and stay on top of things.

But I also go the sense that it could very easily become all consuming. Do we really all want to become AI powered traders? #

More weird internet connectivity issues

More very very bizare internet connecivity issues. Just another note into the void in case this is the end again, so someone knows how it went down. :( #

Cool things about my new blogging tool

There are a lot of cool features in my new blogging tool. Here are some of the top of my head:

1. Render anything, from individual posts to full blown websites.
2. Integrates with the writing tools you already use today.
3. Templates for pages but also file paths, for maximum flexibility.
4. Automatically generate frontmatter for your markdown files.
5. It‘s agnostic so you can use any template rendering engine.
6. Supports partials and cross engine calling.
7. Add functionality using toolpacks accessible from inside the templates.
8. Blueprint based workflow engine you can use to create assets of any shape.
9. Attach logic to templates to explode out many output files from one input template.
10. Run javascript middleware stacks at key places in the render pipeline, integrate with anything.
11. Transform data on the way in before it hits your templates.
12. Extreme debugability, easily see how data flows through the render pipeline.
13. Integrates with Github for atomic and batch review and approve based publishing workflows.
14. Dispatch rendered assets to external destinations with custom deliveries.
15. Easily cleanup your local workspace with custom cleaners.

There‘s a lot more too.

It‘s the fastest way to get from idea to published, completely remove all the tedium from writing online.

I‘m using it right now to write all my posts. Each one gets it‘s own git branch, automatically and uniquely named. You can bulk push all your branches to Github, and then the workflow side picks up and your content goes through a PR based workflow, each item can be review and approved, or auto-merged.

Seems to be working pretty well so far. #

The return of allergies

It was to be expected I suppose, no sooner than I have started writing again and the world around me is angry. Always angry about something. The world is angry I don't do what it wants. It's also angry when I actually do what it wants. It can't make up it's mind. It's really quite unbelievable, like some sort of evil clockwork. Like a heat seeking missile, you just know the moment you see it off in the distance, that no matter what you do, that it's going to get you in one way or another, eventually.

Also the allergies are back. What a surprise.

Really trying to keep upbeat about things, but that easier said than done when the world is angry and your face is on fire and leaking. #