2025/08/09 #

Changing tires

I‘ve been reading about running LLM‘s locally for the past few days. It‘s something I looked into briefly before but it always seemed too complicated. I‘ve noticed that since OpenAI released their open source models, people on various podcasts have been talking about this more, so I‘ve been checking projects out and what not. It‘s seeming more achievable, I think partly because the tech has progressed but also my understanding of the space has evolved too.

It‘s tough not to get sucked into rabbit holes on some of this stuff. I am trying to spend a bit of time reading about it in the mornings, but then you have to put your half baked research aside and get on with your current project. Web development is strange in that you have to constantly be taking small bites at things, and eventually what was not possible, becomes possible. You have to do both, and then find time to write about it too. #

Today’s links:

  • Maple AI - Private AI chat that is truely secure - "End-to-end encryption means your conversations are confidential and protected at every step. Only you can access your data - not even we can read your chats". It‘s still cloud based but it‘s much more private and you can run more powerful models. I feel like I want a hybrid solution where I can run local models and private models in the cloud. trymaple.ai #

  • Is it, like, OK to say ‘like’ now? - I actually remember when people started saying 'like' all the time. It seemed to happen over night, at least in the UK and in Europe. And yet now, it‘s strange to think there was a time when people didn‘t say it, like ever. Sort of like there was a time when there was no internet, no mobile phones. I hadn‘t really thought about it much, but it did have a profound effect on how we experience the world. www.theguardian.com #

  • CodeRunner: Run AI Generated Code Locally - "CodeRunner is an MCP (Model Context Protocol) server that executes AI-generated code in a sandboxed environment on your Mac using Apple's native containers". I could definitely see this could be part of a local AI rig. github.com #

  • Getting started with Podman AI Lab - I remembered this morning that Podman has this AI Lab extension. Re-read the blurb and I wonder if whether it might turn out to be the open source dark horse. I'll have to try this out sometime soon. It‘s been a long time since I used a Redhat product. Perhaps not the coolest kid on the block, but maybe for running AI agents dependability might be more important. developers.redhat.com #

For enquiries about my consulting, development, training and writing services, aswell as sponsorship opportunities contact me directly via email. More details about me here.