markjgsmith

home > archives > everything > 2023 > 04

2023/04/27 #

  • Oauth shows open protocols often get messy

    Yesterday I wrote a commentary and writeup piece about ActivityPub, one of the contender protocols that has emerged in the federated social media space. A hot topic at the minute as many creators are looking for Twitter alternatives. What I discovered is that the visions of these protocols often sound wonderful, but the realities when you actually try to build something with them aren't always smooth sailing.

    That's not to say that it isn't worth the effort, open solutions do offer fantastic long term benefits, but the road is invariably long and bumpy. With that in mind, it's useful to look at other open protocols just to get a sense of the types of difficulties involved in creating such a thing. One such open protocol is OAuth which has been used for authentication these past 10 years on pretty much all major APIs.

    Robin Guideber writes:

    The real-world OAuth experience is comparable to JavaScript browser APIs in 2008. There’s a general consensus on how things should be done, but in reality every API has its own interpretation of the standard, implementation quirks, and nonstandard behaviors and extensions. The result: footguns behind every corner.

    Guideber would know, he's apparently been involved in building the authentication for many large APIs including amoung others Google (Gmail, Calendar, Sheets etc.), HubSpot, Shopify, Salesforce, Stripe, Jira, Slack, Microsoft (Azure, Outlook, OneDrive), LinkedIn, Facebook.

    His article is ultimately a sales pitch for a product he's working on, however the piece is interesting nonetheless because it highlights very detailed real world examples of how these protocols often get stretched and used in different ways.

    ActivityPub, which everyone is talking about at the minute, feels somewhat complicated to me. I'm interested to see whether Nostr gets much traction mainly because it's much simpler, but it too has issues. I wrote about some of these last week in a writeup piece about my experience setting up the Damus iOS app, which is a Nostr client.

    Hopefully this gives a bit of perspective on open protocols. They come in all shapes and sizes, and adoption is hard to predict. It's clearly worth it in some cases. The world would be very different without, to name just a few, pop3 and iMap for email and TCP/IP, HTTP and FTP for the web.

Today’s links:

2023/04/26 #

  • What’s ActivityPub like really?

    David Pierce from the Verge has an article out about ActivityPub and the future of social media protocols. It’s a great piece, well researched and really sells the decentralised social networks vision.

    The crux of his pitch:

    If our current social system was decentralized, you’d be able to post a picture on Instagram and I could see it and comment on it in the Twitter app. Your friends could read your tweets in their TikTok app. I could exclusively use Tumblr, and you could read all my posts in Telegram. Different apps would have different strengths and weaknesses, different moderation policies and creator tools, but you’d have the same set of followers and follow the same accounts no matter which platform you use

    Sounds pretty cool.

    So what is ActivityPub?

    It’s a technology through which social networks can be made interoperable, connecting everything to a single social graph and content-sharing system. It’s an old standard based on even older ideas about a fundamentally different structure for social networking, one that’s much more like email or old-school web chat than any of the platforms we use now. It’s governed by open protocols, not closed platforms. It aims to give control back to users and to make sure that the social web is bigger than any single company

    This new world would be very different, better in many ways:

    A couple of dominant platforms would be replaced by millions of interoperable smaller ones

    and

    No one will be tied to a platform and any platform that tries to tie down users will lose them instead

    Great I love the vision. Sounds amazing. It’s worth reading the whole article to get the full picture.

    After reading that I wondered how difficult would it be to post a link from my linkblog into the ActivityPub. At which point I realised I’d brushed up against the first hurdle, because one does not simply “post to the ActivityPub”. It’s not a website, it’s a protocol. First you have to find a website that supports ActivityPub and then you post to there using the ActivityPub protocol. Fine it’s not as simple as just using the Twitter API, but how complicated can it be?

    The first thing I did was check npm for ActivityPub nodejs modules. Lots of search results, but they are almost all for creating your own ActivityPub server. The activitypub-express module indeed looks very comprehensive. It appears to be quite straight forward to add ActivityPub routes and functionality to an express server. That’s great but way more than what I want to do, probably would need to hook up a database. I just want to post a link mate.

    Next step was to do a Google search to find the specification, and some tutorials. The spec shows up immediately, but one glance at that will have most developers running for the exits. No one reads the W3C specifications. Fuck that shit.

    But look there’s an ActivityPub Rocks! website. Hey that reminds me of when HTML5 got released. They had cool HTML5 Rocks websites for that too. This must be what I’m looking for. Click…page loads…uh…there must be some kind of mistake. This website most definitely does not rock. There’s nothing on there about posting a link. It’s all about implementing servers.

    I then decide to look at existing implementations. I find that Mastondon has a tutorial about posting a message, which looks somewhat doable but it’s got Ruby code in it. Yeah I know, I should be able to replace it with nodejs but I already don’t understand a lot of the terminology, and my brain is already going yuk, especially after skimming through the horror that was the W3C spec earlier. And why do the JSON examples have property names that have differing font sizes. That’s not normal. What is this JSONish text blob? I’ll come back to this.

    A bit more googling and I find this article which is a bit rambley and very hodge podge, but is reassuringly clean and nicely formatted black text on white background. It basically points out which are the important parts of the spec, and which parts can be safely skipped.

    That was actually quite useful with a few good examples.

    Ok so ActivityPub uses a special type of JSON called JSON-LD that includes URLs where you fetch sub parts of an object from different locations on the web. Sounds interesting but potentially complicated.

    Anyway after reading the full article I have a better idea of the vocabulary being used, the various entities etc. Still feels very complicated. Plus it appears to take the form very similar to email where users have inboxes and outboxes. Which is weird because haven’t we spent the last 10 years trying to specifically get away from email?

    I go back and re-read the mastodon tutorial, which makes a lot more sense now. Basically just some HTTP post requests with JSON payloads, and some sort of complicated crypto signatures. Feels quite doable but also I’m feeling like the reality of ActivityPub is very scattered and messy. I’m certainly not super eager to dive in. It has all the signs of an ecosystem littered with cracks and crevices, though there are a few large implementations so it likely does work.

    The vision sounds amazing, but existing developer resources are a bit underwhelming and disorganised. I want to like ActivityPub and suspect that I probably would end up liking it, but I don’t have the time or resources to get involved, I’ll have to put a pin in this for now.

    Update: this just dropped in my podcatcher as I was publishibg this blog post. I havent had a chance to download and listen to it yet but their shows are usually very good quality and sometimes include suprisingly good opto-electronic based AI generated rap:

Today’s links:

2023/04/24 #

  • Materials Science is pretty cool actually

    Tyler Cowan published a great interview with Jessica Wade, who is a researcher at Imperial College, London. Jessica is doing some really fascinating research using incredible tools that enable you to peer into the nano world at a molecular level. Debugging the very fabric of reality.

    I did my Materials Science & Engineering Masters at Imperial. It’s a long time ago now, in the mid 90s. I wonder how things have changed. Back then the Materials department was part of the Royal School of Mines, along with other disciplines like Geology. It was a bit of an odd mixture of the really old school with the very futuristic. I did some Raman spectroscopy myself as well as electron microscopy and IR spectroscopy. Very large and expensive equipment. Many fond memories, though writing dissertations was a bit of a slog.

    I chose Materials because it had an interesting balance of maths, physics and chemistry. It had a bit of a reputation for being somewhat less hardcore than say mechanical or electrical engineering, but there was something kind of cool about it. That’s what I thought anyway. Turns out that much of the interesting parts of things in the real world actually centre around materials science. It was becoming true back then, but it’s even more true as we transition to a digital world.

    The difficulty for me back then was that it was very hard to see how anything I was learning would translate into the real world. After I graduated, I interviewed for some materials engineering jobs that sounded great on paper, but in practical terms, it would have been 8 hours a day in sterile labs looking through microscopes. I had absolutely nothing in common with the people I met at these companies, nice though they were.

    Everything these companies did felt so far removed from how their products were used in the real world. After a tour of the labs, some of the lab coated employees there asked me if I had any questions, and I simply had nothing to ask them. Total blank. I was completely disinterested in what they were doing. It just wasn’t cool in any way. Even though it was futuristic everything somehow felt really archaic. There was no energy, enthusiasm or purpose. It felt empty.

    The jobs were also always in small towns, far from the capital, and I just couldn’t envisage not being in London. These days the idea of living in a small town seems kind of nice to me, but in my 20s, it felt like some form of suicide. What would I do at the weekends for heavens sake?

    After a few years working in gastropubs, an interesting cultural detour in itself, I ended up going back to university to do a Masters in Computer Science at University College London. That was awesome. I read Levi’s Hackers book, and started to get into the whole history of Silicon Valley, entrepreneurs, open source & the free software movement. Now this felt like something was going on. These were the companies building on top of the materials science tech. It felt like this was were the future was at.

    Looking back it’s not so surprising to me that I went in this direction. I’d spent years following bands, going to live gigs, being immersed in mythologies, stories and fashions. In cultural scenes. That’s where all the action was. I was looking for something similar but in technology. As it happened, technology had its own version of this, but much of that had been happening in software rather than hardware. And at the turn of the millennium things were starting to get very interesting because the world wide web was starting to really catch on, and companies like MySpace and later Facebook and Twitter were taking off. Tech was beginning to enter the mainstream cultural space.

    These days I wonder what the path is like for Material Science graduates. Educational wise it feels to me that there are so many more resources that connect the science to the real world. Independent media being produced by scientists and shared in places like YouTube, Wikipedia and podcasts. I love consuming all this content. I’m fascinated by it. Plus software has become pervasive basically everywhere both in industry and academia. Open source development is the norm now. Everyone codes.

    Would I have stayed in science if that would have been available to me as I left college? Hard to say. Quite possibly. I ended up working in film visual effects and then tech startups.

    The state of the world is very different today than it was back in the 90s. The tech landscape is massively different. We’ve been through a couple of boom bust cycles, we’ve had web2.0, crypto has started making serious progress, most people use computers, user generated content is normal, there’s an exciting Cambrian explosion in generative AI. We have moved forward technologically but also culturally, and we are starting to wrestle with the impact of all this tech. Two areas that are making the science and engineering disciples cool again are Space Travel and Quantum Computing.

    Elon Musk and SpaceX is building rocket ships that can carry 50x the load of previous rockets into orbit. These are using new materials designed from first principles to push the boundaries of what is possible. They plan to turn humanity into a multi-planetary species by colonising Mars. There are also plans to send missions to the moons of Jupiter and mine asteroids. That’s pretty darn cool.

    The latest All-in podcast has a great interview with one of the SpaceX engineers involved in the latest launch, which ended in disassembly but was nonetheless a big success. Really worth listening to:

    Quantum Computing is very much at the early stages of development however real world production hardware is beginning to be commercialised. The UK government has bought some of these recently. All this tech is brand new, which means new materials need to be developed. New companies will be build. New software will be written. New applications will be found. New tools will be forged. There’s so much fertile ground in this space both technologically and culturally. Are we going to see films written about quantum computer hackers? Surely. Technology and culture is at a point where materials science and engineering disciplines are very very cool.

Today’s links:

2023/04/20 #

  • I miss watching films

    I read an interview yesterday with film director Nastia Korkia, where she answers questions about her new film Almost Spring (Pochti Vesna). It’s about the normalisation of corruption in everyday russian life. After reading the article I remembered how much I used to enjoy watching films, especially weird and unusual foreign films. Some of you reading might be aware I used to work in the film visual effects industry.

    The way the director describes the film making process and how she tried to re-create the ambience of the time and place where the story unfolds really speaks to how magical film is as an art form, a window into different cultures and places.

    It’s been a few years since I have watched a movie. Life circumstances have for the moment at least removed that from my day to day activities. At first I was just too busy, but since the pandemic, without going into any details, it’s basically become no longer possible for me. Even as I write these words the world is strongly pushing back against me.

    Anyway I just wanted to say I miss watching films a lot.

  • Nostr and Damus first impressions

    I’ve been listening and reading to much bitcoin, crypto and blockchain content these past few months. I decided it would be interesting to try out Nostr, which is a blockchain simple client & relay based social messaging protocol and network. Loads of the bitcoin people are talking about it, especially since Elon bought Twitter. It’s sort of like Twitter except it’s decentralised, so there’s no central company that runs it. Rather than use a username and password, you use public/private keys to create your identity.

    Nostr is a simple, open protocol that enables global, decentralized, and censorship-resistant social media.

    Though it’s similar to Twitter in many ways, you post messages to a timeline and can see other people’s messages, which you can reply to, and you can post pictures and videos, but in other ways it’s different and a bit confusing. You have to connect to relays which forward your messages throughout the network for instance.

    I installed Damus, which is the iOS app. It installed without any issues and opening it up for the first time was mostly quite straight forward. The app is minimal, with all of the functionality you would expect for a messaging app. It’s a nice looking app.

    They automatically set you up with some relay servers and one or two users so that you can see some messages in the public timeline. That makes big difference, even if it’s a bit odd to only have messages from one user viewable. In my case it was the account of one of the project devs.

    I suppose it’s the same for everyone. He was going on about quite technical dev stuff which didn’t make much sense to me. Not a huge deal, but I can imagine that would scare off most normal non dev folks. I clicked around and was able to view a few threads where others had commented on some of his messages. It all felt a bit disjointed, but made a bit of sense.

    After looking through the settings and filling out some basic details I was ready. Then the big unanswerable question: where is everyone? How the heck do you find people?

    I had seen that some folks have been sharing their Nostr npub on their Twitter profile. Npub’s as far as I can tell are the user’s public shareable key. I found one of these and stuck it into the search box, which tried to autocomplete it into some sort of hash tag. The search results that came back were weird long text strings. Didn’t look like it had found anyone.

    Then I searched for a few usernames of bitcoin podcasters. That did return what looked to be the right accounts, but this is where I started wondering how safe this all was. Was it safe to click on long strings of text? There were loads of messages with folks sharing text strings, but what happens if I click on one that then somehow hacks my phone? Nothing against bitcoiners and crypto people but they are all using nyms, voice modifiers and balaclavas and you read about hacks like every few days in the news. I decided to hold off clicking on anything and following anyone.

    I found this Github repo from cryptoquick that has some general Nostr info but also suggests usernames of other devs to follow. They come with long text strings that don’t start with npub. Are these npubs or something different? It’s not clear.

    There is also a way to attach a domain name to your username by going through a mostly automated verification process. Looks like it’s relatively easy to do, you have to put a special text file on the domain you use to prove it’s your domain. I guess I could do that once I’ve figured out how to add some users.

    By this stage I’d used up all my available energy and good will, and the thought of having to determine who to follow was too much. It just felt like there was a quite high non zero chance I could screw up my phone if I made s mistake. So that’s where I left it. I can see all the messages from the initial dev bloke and whoever replies to him, but that’s it. I’ll have to revisit this when I have more time. How do you safely find people on this thing?

Today’s links:

2023/04/08 #

  • My favourite javascript proposals (2023)

    The 95th meeting of the TC39 just finished, meeting notes have been published, and blog posts are being written. Figured I might as well pick out my favorites, based on what I can see being most useful.

    I'm writting this without an internet connection however I have the article listing the proposals linked above saved. Based on descriptions alone these to me look like the most interesting / least boring:

    • Import Attribute - mentions importing JSON, which I was trying to do recently and was suprised to discover that it doesn't work with impprt, I do this with require all the time, so this might be cool, if it's what I'm thinking it is
    • Async Context - Looks interesting, but also looks like it could turn out to be super boring, it will depend on what you can actually do with the feature
    • Await Dictionary - still not totally sure what this is, though any additional async/await control structures would likely be awesome given how popular async/await has become, I use it all the time now
    • Time Zone Canonicalization - again not totally sure what this is but I read through temporal's github project a while ago and thought it looked kind of cool, dates and especially timezone stuff in javascript has always been very annoying, so improvrments there would no doubt be very welcome

    Right those are my initial picks. Time to find an internet connection.

    Incidently, wouldn't it be incredible if Safari could save a page but also automatucally save all the main article's linked pages? IMO, that would be the best feature literally ever.

    After checking online, it's mostly as I thought, and the asyc context proposal does imo look like it could be useful for a few advanced use cases that I can think of.

Today’s links:

For enquiries about my consulting, development, training and writing services, aswell as sponsorship opportunities contact me directly via email. More details about me here.