markjgsmith

home > archives > everything > 2023 > 01

2023/01/25 #

  • cat << EOF > The importance of good RSS feed GUIDs

    The past few days I've been trying to migrate my existing RSS feeds to the new site. The blog, linkblog and podcast all have distinct feeds, and they are currently hosted on seperate subdomains, which each resolve to a seperate server. The linkblog and podcast feeds are both created by the initial version of my static site generator, whereas the blog's feed is built using a static site generator called Jekyll.

    When I wrote the initial implementation of the ssg tool, I just went with what seemed like the default settings of the feed generation library I was using. That library expects each feed item to have a post url, which makes sense because clearly consumers of the feed are highly likely to want to know where the post is located on the web. The library docs also state that you can provide a GUID for each feed item, but that it's optional, and if you don't provide one, then it will just use the post url as the GUID. That seems to make sense because urls are essentially unique, which is good because GUIDs need to be unique. So what's the problem?

    Well the problem only really happens if you decide to move your website. Of course each post on your new website will at the very least have a new domain name, but if you change website software, then the structure of the urls could change too, depending on how the new software organises the site pages. With new feed item GUIDs, there's a chance readers of your feed will suddenly see duplicate items appear in their feed reader.

    The reason for this is because many feed clients use the GUIDs as a way to keep track of items in the feed. If you change the GUIDs, then existing posts could very well end up looking like totally new feed items, and be displayed as such, right next to the ones with the original GUIDs. Readers will unnexpectedly be faced with doppleganger feed items! It's freaky enough in real life, but in an RSS reader? It's almost too scary to contemplate.

    I wish I would have been aware of this when I initially set the feeds up. All the feed items have a uuid which I could easily have used. Now I'm faced trying to figure out a way to keep old GUIDs the same. There's a relatively easy fix for the podcast. Since there aren't that many posts I could easily go through and set the uiid to the post url. That way they will remain unchanged in the new location. The podcasts data will look a bit strange, but at least things will look good in Apple Podcasts. An other way to approach the problem is to add some special cases to the feed generation code, which I could do, but I really don't want this sort of special case brittleness in the codebase.

    Changing the data for the linkblog isn't really an option because there is in the order of 170000 posts. I'm not sure what I'll do there. I haven't checked what Jekyll uses for GUIDs. I'm guessing it's post urls.

    Does it even matter? I doubt there are many folks actually subscribed. And it might very well be that the duplicates issue only really arises for the latest items. After all the RSS feeds only actually contains the last 20 posts for the blog abd 50 posts for the linkblog.

    That's pretty much what you need to know about RSS feed GUIDs. I'm still not sure what I'll do. I think I'll likely try to make it look good in Apple Podcasts, and just go ahead and change to using uuids for the blog and linkblog. If you know of any reason I should do it differently please email me.

    EOF

Today’s links:

2023/01/24 #

  • cat << EOF > Safely Moving Your Podcast Feed

    A quick note on how to safely move your podcast RSS feed in case I need to refer to it later for some reason.

    I recently migrated my main website off of Heroku and I’m consolidating most of my sites into the one site to make things easier to manage. I’ve managed to migrate most things successfully, but one remaining item is the RSS feeds. This article covers the podcast feed migration.

    Apple has published a couple of useful articles:

    The main idea is to setup a redirect on the existing feed location which returns an HTTP 301 and the location of the new feed. You also need to add a special custom <itunes:new-feed-url> tag to the new feed. With these items in place clients that use the feed will get redirected to the new feed location, and they should also be able to see the new feed url from the custom tag, and update their settings with the new location.

    Finally you have to ensure that the existing post GUIDs in your RSS feed don’t change. Without doing this, existing podcast clients could end up with duplicate episodes being listed because they will appear as new episodes.

    Once setup you need to keep the redirect and custom tag in place for 4 weeks. During that period Apple Podcasts will update the feed in your account. The reason for doing it this way is to ensure that existing subscribers to your feed continue to receive posts without needing to update their podcast client.

    Here are the steps from the docs:

    1. Set the web server where your feed is hosted to return an HTTP 301 redirect response when it receives a request for the old feed.

    2. Use the itunes:new-feed-url tag in your new feed to point to the new URL. This response updates Apple Podcasts with the new feed URL and your subscribers continue to receive new episodes automatically.

    I have my existing podcasts site hosted at Netlify. While setting up the redirect I ran into some issues so I opened a ticket in the community forum:

    If all else fails then you can also directly change the feed in Apple Podcasts account. However doing it this way will cause existing subscribers to stop receiving new posts.

    As I write this I’m still waiting to get an answer about the redirect. Hopefully the migration will happen without any major issues.

    EOF

Today’s links:

2023/01/22 #

  • cat << EOF > Perpetual website DAOs

    One of the things website owners don't spend much time thinking about is what will happen to their website after they die. Once they aren't there to maintain it, and perhaps more importantly, to pay the hosting and donain name costs, then the site will most likely ultinately go offline.

    Some folks make arrangements with long running organisations / foundations to host their site, but that night not akways be possible if you don't have content that these institutions are interested in preserving, or just don't have the contacts to make that happen. There's also the internet archive which could keep a backup, but that won't help the live site.

    In web3 there exists somethibg called a decentralised autonomous organization, or DAO for short. They are essentially code hosted on a blockchain with an associated digital wallet address. People are using them for all sorts of things, often to help with project governance and organisational tasks.

    What if you created a DAO to pay the costs to keep your site online? Theoretically that way you could keep your site online after you pass. I'm not an expert in this area, just thinking out load.

    Your DAO would need a way to make money, it could be related to the content, but maybe it could be a simple AI that could access the stock markets. Make little bets everyday, make a few pennies here, a few pennies there. It could be as simple as donations from site visitors.

    It's an interesting thought experiment, because one could imagine a whole ecosystem that would be built to accomodate such activity. Assuming your site is fully static files, your website DAO might want to be able to perform tasks such as migrations between hosting providers, offsite backups etc. Who knows maybe even a bit of web design to keep it looking fresh?

    I wonder if anyone else is thinking about self sustaining DAOs?

    EOF

Today’s links:

2023/01/07 #

  • cat << EOF > The list of titles that didn't make it

    Several weeks ago now I started a new section in my newsletter which I've ended up calling the "list of titles that didn't make it". It has it's origin in a list I had been keeping every week of possible titles for the newsletter. At the end of the week I would read through the list and pick one, or sometimes disguard the entire list and go with something totally different. At least with a list I can remember a lot of the interesting topics that I spent time reading, listening and thinking about.

    One week I was struggling to find a title, and I think I was rushed for time, and the week had been going so terribly that in a sort of burst of exasperation I just thought, well fuck it post the whole darn list, which I think, in it's own way, was quite illustrative of the moment.

    So I did, and I immediately felt a lot better. A weight off my mind.

    The next week, I did the same thing again, and kept doing it. The list rapidly turned into a place where I was venting frustrations and worries, but also a place where I was exploring thoughts and ideas. I don't think it has completeky settled into something concrete yet. It varies week to week. I often look back at them and wonder what the f I was thinking or doing at the time, but I just post it anyway, warts and all.

    Sometimes the entries are real title candidates, but other times it's more like a sentence or a short paragraph that I don't have any intension of turning into a title, but they usually express something I think is in some way important. I don't edit them, and only occasionally remove some. They are often cringey, controversial and risqué. They don't necessarily convey my point of view, often just taking the form that they popped into my head. They could be from any point of view, and often are meant to get the reader, which includes future me, to think about certain aspects of an idea that are new or unnexplored. Sometimes they are humorous, other times very serious, or a mixture.

    They have so far been overwhelmingly dark. There's a lot of turmoil both in the world and in my life at the minute and it's definitely showing in the lists. I'm going to keep doing them, because it's helping me deal with the crazy world, but it's also a bit of a barometer of the present moment that perhaps some find interesting. If you don't like them, feel free to skip over them.

    Now you know what those wierd lists are.

    EOF

Today’s links:

For enquiries about my consulting, development, training and writing services, aswell as sponsorship opportunities contact me directly via email. More details about me here.