[DJ]: I never expected to be on a podcast where we would be giving updates on a live mission to the moon, Justin, because when I was growing up, all of that stuff was in the distant past.
[DJ]: But here we are.
[JM]: It's fascinating to feel like we are reliving history, but this time in high-definition 4K.
[JM]: It's really, really surreal.
[JM]: I remember as a kid...
[DJ]: Watching the moon landing, right?
[JM]: I was not around for that.
[JM]: But I do remember being at my grandparents' house and everyone was making a big deal out of some space launch that happened in the mid-70s.
[JM]: But all the moon-related missions had all been done before then, so...
[JM]: like you, I haven't seen any of those until this past week.
[JM]: And it's been really interesting to see people at NASA get so excited about something that was done 50 years ago.
[JM]: And presumably they're excited about it for the same reason we are, which is we weren't there when this happened before.
[JM]: And they're getting super excited to
[JM]: see the details of craters and see all these things that to some degree feels a bit old hat. We've done it before, like okay, we're just covering ground we've already covered, and not even that because we're not even on the moon yet -- we're just flying by it, and yet it really is very cool and exciting for I don't know what reason. I guess just because it's happening in real time and we're watching it, I guess, that's how I felt is just watching it all unfold in real time. Yeah I don't know -- it's cool.
[DJ]: I think there is something to having a connection to the past. I was talking to a friend of mine recently about another topic that we usually don't cover on this show, but maybe we will on this episode a little, and that is large language models. It says here, "Pause for laughter."
[JM]:
[DJ]: We were talking about AI, so-called AI generated art, for example, and whether it could ever be compelling.
[DJ]: And something that came up in the discussion was interesting.
[DJ]: We sort of differentiated between art can be compelling if the experience of it moves you.
[DJ]: And although most or all LLM-generated art thus far...
[DJ]: I at least have not generally found to be moving.
[DJ]: It's at least hypothetically possible that one could generate a text or an image with an LLM that would create an emotional response in the person viewing it.
[DJ]: Okay.
[DJ]: But another aspect of art is the historical context.
[DJ]: Like when you go to an art gallery and you look at a painting by one of the masters of hundreds of years ago,
[DJ]: I think there's something there that isn't just about the effect that the sight of the colors on the canvas has on you, but there's also a connection to what the artist and what the people around the artist were going through at that point in time and what influenced that art and how that art influenced the people that came after.
[DJ]: Right.
[DJ]: And like that's how curation at museums works generally is that like curators aren't just like, look at how look at how this one is red and that one is blue.
[DJ]: Right.
[DJ]: There's always a lot of context of like this is what was going on in Greenwich Village in the 1960s that led to thus and such.
[DJ]: So I wonder if there's something similar here where most of us have at least some knowledge of the historical context, this notion that like the moon landings brought a large number of people together and gave them a shared sense of mission and accomplishment.
[DJ]: And that's inherently valuable no matter what period you're living through.
[DJ]: And so one of the things that feels nice to me about this Artemis two mission is having a little bit of that sense of, you know what it was like in, yeah, like the 1960s.
[DJ]: Well, we're still doing things like that for some definition of we, of course.
[DJ]: And I'd imagine that if you actually work at NASA and you have like so much more sort of institutional history, like I work at the same place as the people who made this happen.
[DJ]: Now we're making it happen again.
[DJ]: That's got to be really exciting.
[JM]: For sure.
[JM]: And the fact that we have technology that is so much more advanced than we did when we did this last time around is also interesting.
[JM]: I saw that one of the things that the first visitors to the moon definitely didn't have was internet access.
[JM]: And the current astronauts heading there have 250 megabit per second connections to the internet via lasers, which is really...
[JM]: Just like something that feels like we've read it out of science fiction, and yet this is what's actually happening right now.
[DJ]: Yeah, that's amazing.
[DJ]: And also, I think that despite being advertised as a gigabit internet connection, that is effectively faster than my home internet.
[DJ]: So that's quite impressive.
[JM]: Latency might be a little higher, but...
[DJ]: Do you think so?
[JM]: Probably, given the distance.
[JM]: Yeah, I would imagine, you know, light can only go so fast.
[DJ]: It's going that fast through a vacuum, though.
[DJ]: So I nod my head like I know what difference that makes, but...
[JM]: In the interest of full disclosure to our listeners, neither of us are physicists.
[DJ]: I am not a professional laserologist, it's true.
[JM]: Speaking of the opposite of physicists, I came across an entertaining site called the Idiocracy Proximity Index.
[JM]: And if you go to idiocracy.wtf,
[DJ]: And a better use of that top-level domain, I cannot imagine.
[JM]: Probably the best usage of that TLD I have seen.
[JM]: And the mission of this site is to track how close reality is to Mike Judge's 2006 prophecy.
[JM]: And if you don't get that reference, that's because (A), it was 20 years ago, and (B), it was a movie called Idiocracy.
[JM]: And as of this moment, the Idiocracy Proximity Index,
[JM]: on the scale ranging from functional society on the left to full idiocracy on the right, we are currently at 78%.
[JM]: Or to put it another way, Idiocracy is basically a documentary at this point, as the site accurately describes.
[DJ]: I feel like Mike Judge has already had many reasons to be smug for many years as the creator of lots of award-winning satires, but I just hate to think how smug he must feel now.
[JM]: Yeah, you really got to give it to him.
[JM]: He really nailed this.
[JM]: One of the fun things about this site is it's not just like this person made up a number and said, all right, we're at 78%.
[JM]: There's actual criteria listed as in this thing happens in the movie versus this is the reality that we're living in.
[JM]: And then there's a percentage match for each of those things.
[JM]: ranging from 55% on the low end to 92% on the high end.
[JM]: And then the blend of all of those things is how we arrive at 78%.
[JM]: But I feel confident that we will eventually get there.
[JM]: We will eventually arrive, maybe not at 100, but close enough that we can say like, all right, we've achieved effective full idiocracy.
[DJ]: That attitude makes it sound like that's the goal.
[DJ]: That you're just like, I'm confident that we will achieve this goal.
[DJ]: Like you're the CEO of the worst company that's ever existed.
[DJ]: You're just like, listen, people, if we really roll up our sleeves...
[JM]: Like our intrepid astronauts, I feel confident that we will arrive at our full mission completion.
[DJ]: But yeah, this site follows, to me, it is of a piece with sites like Web 3 Is Going Great.
[DJ]: Have you seen that?
[JM]: Yes.
[DJ]: And there's another similar site.
[DJ]: And I think that one was created by, I believe her name's Molly White.
[DJ]: I'm doing that off the dome.
[DJ]: So I hope that's correct.
[DJ]: That's another one of these sites that tracks things going on in the real world that are obnoxious and depressing but does so in a way that is enlivening and hilarious.
[DJ]: I appreciate that these things are out there cataloging, I guess, how ridiculous everything is.
[JM]: There's a ticker at the top that says things like "Secretary of State sponsored by Mountain Dew" and other entertaining breaking news.
[DJ]: And now I'm honestly not sure whether those are references to the movie or reality.
[JM]: That's the scary part, isn't it?
[JM]: I can't tell either.
[JM]: All right, moving on.
[JM]: In other news, Claude Code is a tool that one can use in a terminal to generate code using the Claude model by Anthropic.
[JM]: And last week it was reported that the source code to Claude Code has leaked.
[JM]: Apparently there was some oversight by someone at Anthropic that resulted in a map file being
[JM]: published in their NPM registry, and as a result, the source code became available.
[JM]: Someone posted a link to it to Twitter, which was subsequently downloaded and reposted to every possible place on the internet, including hundreds or perhaps even thousands of times as GitHub repositories.
[JM]: Most of which, if not all of which, at this point have been taken down due to presumably requests from Anthropic.
[JM]: The fact that it's still available on Twitter, that the original link, as far as I can tell, is still there and downloadable, should surprise nobody who's paying attention, because I can just see the current owner of Twitter being like, "Oh yeah, I'm leaving that one up."
[JM]: That just seems so perfectly onbrand.
[DJ]: It does, yes.
[JM]: And there have been a lot of interesting insights into the source code to Claude Code since it became available, including an April Fool's joke that was just about to be released when this code leaked, where you could type slash buddy and you would get a little Tamagotchi-style companion creature with ASCII art sprites.
[DJ]: but Claude Code already prominently includes a little Tamagotchi-like creature.
[JM]: Yes, I think this one was more interactive or animated.
[JM]: It wasn't just some static image that appeared on launch.
[DJ]: Well, this one in the bold path blazed by the Tamagotchi, this one would die if you did not keep feeding it and giving it attention, which might also be a good metaphor for our current obsession with large language models.
[JM]: Another insight that was unearthed is that there is an undercover mode in which there are instructions specifically that the model should strip all references to Claude from commits and open source pull requests.
[JM]: And you can take guesses, I suppose, at why they would do this.
[JM]: Some people assumed it was so that Anthropic employees, when contributing to open source projects, wouldn't have the fact that they're using Claude to generate code be clear or transparent.
[JM]: But regardless of the motivation, it's interesting that this instruction is in there.
[JM]: I think one of the more damaging things for Anthropic isn't so much the leak of the code itself as much as it is the features that are referenced in it, as in the features that are coming, because that way their competitors can tell to an extent what their future roadmap looks like.
[JM]: For example, I think one of the most significant features that became clear by looking at the source code is an autonomous agent mode.
[JM]: And part of that is that it runs continuously in the background and has a skill called Dream, which performs nightly memory distillation, which is another way of saying when it's idle, it's essentially analyzing recent interactions and distilling those interactions down to some kind of actionable memory for subsequent interactions.
[DJ]: That sounds useful, actually.
[JM]: It's something that I know some people do manually.
[JM]: And it is kind of interesting the way that they've even used the word "dream", which is a reference to how we believe our own brains work, where while we're sleeping, we seem to believe that our brains use that time to write certain information that we've learned to long-term memory or something to that effect.
[JM]: I'm not a physicist nor a specialist in neurology.
[DJ]: Yeah, I'm not a laserologist or a brainologist, just for the record.
[JM]: One of the other insights was the quality of the code, which someone described as eye-wateringly spaghetti code.
[DJ]: I mean, in fairness to Anthropic, I would say something like 99% of all the code I've ever encountered in my professional career could be described as eye-wateringly spaghetti code.
[JM]: Fair, but have you ever encountered a function that is over 3,000 lines long?
[DJ]: Yes.
[JM]: Wow.
[DJ]: I mean, really, I hate to take the wind out of your sails.
[DJ]: I once encountered a function that I think was 10,000 lines long.
[JM]: Oh, wow.
[JM]: Okay.
[DJ]: There were no plans to refactor this.
[DJ]: It's okay.
[DJ]: I think the client that that software was written for was dissolved by the Canadian government a short time later, so it didn't matter.
[JM]: Well, I was blown away by the fact that there is a print function in this code base that is over 3,000 lines long, the file in which this function is contained is 5,600 lines long.
[JM]: So more than half is the single function.
[JM]: It has 12 levels of nesting at its deepest and defines 21 inner functions and closures.
[JM]: And is very clearly, according to the people that have analyzed the source code, generated by Claude Code itself.
[JM]: It is very clear that Anthropic uses Claude Code to work on their products.
[JM]: It is something that their employees talk about.
[JM]: So it's not a secret.
[JM]: And I suppose if you have seen functions that are 10,000 lines long, then clearly you can write spaghetti code even without these tools.
[JM]: So I guess it doesn't really prove very much.
[DJ]: I'm sensing a little bit of goalpost-moving in some of the commentary we're discussing on this.
[DJ]: People want so badly to think badly of the company and the tools involved, and some of that is deserved perhaps,
[DJ]: that you get these takes that are like, "But their code's really messy."
[DJ]: And I'm like, bro, are you a professional software engineer?
[DJ]: Because like all production systems that have slowly accumulated over 10 years are really messy because the idea of clean code or perfect code or whatever, unfortunately only exists in our heads.
[DJ]: You don't have a mandate to do it when you're actually shipping.
[DJ]: And then the other part is just like, did you know they use Claude Code to build Claude Code?
[DJ]: Yeah.
[DJ]: Have you ever heard the term dogfooding?
[DJ]: You're supposed to use the tool that you're building while you're building it so that you can actually, you're supposed to be the first best user of the tool because then it gives you the most insight into how it should work.
[DJ]: So some of this stuff is a little like, it just, it feels like we're clawing for reasons to criticize.
[DJ]: There's lots of good reasons to criticize Anthropic.
[DJ]: I thought the most cogent take I saw on this leak was the reason this leak is alarming is not because they have a messy function in their code.
[DJ]: The reason this leak is alarming is if they can screw this up, like what else can they screw up?
[DJ]: Like with all the data of ours that they have collected, right?
[DJ]: That was kind of the concern that resonated the most with with me.
[JM]: Yeah.
[JM]: And you're right.
[JM]: Like they use Axios, which was involved in a very high profile security breach.
[JM]: So when you see what feels like incompetence by leaking this code combined with using Axios, you're right.
[JM]: Like it does tend to undermine confidence in what they're doing.
[DJ]: It's the, oh man, I haven't read Spider-Man in a while...
[DJ]: It's the "With great power comes great responsibility" argument, which is if companies like Anthropic and OpenAI feel like they're trying to take over the entire world from basically the rest of tech.
[DJ]: Like they're basically saying everyone should use all our tools for everything.
[DJ]: No other software needs to exist, etc.
[DJ]: Like the way that they're totally disrupting the tech industry, we should hold them to the highest standards possible.
[DJ]: Because if a company says like, hey, pipe all of your proprietary information through our systems so we can help you build stuff.
[DJ]: And then it's like, oh, by the way, we leaked our own source code.
[DJ]: Um, that should be alarming.
[DJ]: And it is alarming.
[DJ]: I will say, though, that the thing where their code includes a regex of just hard-coded swear words to try to detect whether the user is getting annoyed with them, that is kind of silly.
[JM]: That was definitely one of the more amusing parts of this was this use of regular expressions for detecting negative sentiment in users' prompts, which is then logged so that they can know when people are getting frustrated.
[JM]: And I saw at least one person remark that an LLM company using regexes for sentiment analysis is peak irony.
[JM]: And I think that's probably accurate, right?
[JM]: Like we all kind of feel that that's inherently funny.
[JM]: Yes, obviously using a regular expression here is faster and cheaper.
[JM]: But the fact that they're using that tool as opposed to, I don't know, using a large language model to detect, you know, negative sentiment is still hilarious.
[DJ]: So it is like it's conceptually hilarious.
[DJ]: But again, I like I hate to find myself coming to the defense.
[DJ]: But I think being pragmatic is good, actually.
[DJ]: So the thing where it's like, well, this spaghetti code function has 12 nested functions.
[DJ]: Well, that's bad if human beings need to review the code.
[DJ]: But arguably, if that function has a robust enough test suite, and you're just going to use agents to work on it, it kind of doesn't matter what the code works looks like.
[DJ]: And for the regex thing, like, honestly, in my like day job, I find myself sometimes giving feedback to people where I'm like, instead of running that through an LLM, which costs money and more important, and even more importantly, like makes the user wait several seconds.
[DJ]: Have you considered like pattern matching?
[DJ]: Have you considered like one of the many non-large-language-model-related
[DJ]: computer science solutions to this? So again I can't actually blame them in practice for saying, you know what, just put a map of strings that contain common curse words in there and call it a day, because this thing has to run in like 0.1 milliseconds. Yeah, that probably is actually the right solution even though it is still funny.
[JM]: To nitpick, this is a logging function that doesn't need to be done in real time.
[JM]: This can just detect the sentiment and log it.
[JM]: It can run in some cron job that runs 10 hours later.
[JM]: It has nothing to do with the current user's interaction, but it does cost more money.
[JM]: They have to consume more of their precious
[JM]: large language model resources that they're already struggling to keep up with demand for.
[JM]: So I can understand doing the quick-and-dirty solution that they went with.
[JM]: Perhaps a little less understandable is the anti-distillation defense that they baked into Claude Code.
[JM]: And when enabled, this injects anti-distillation
[JM]: fake tool responses to every API request, which causes the server to silently put decoy tool definitions into the model system prompt.
[JM]: Why would they do this?
[JM]: Well, the goal is to prevent someone who's trying to scrape Claude Code's API traffic
[JM]: to train a competing model.
[JM]: The goal here is to poison the training data to make this distillation attempt less useful.
[JM]: And someone noted that Quen 27B, which is a model made by a competitor, was distilled on Opus 4.6 and has some known issues with tool use specifically, indicating perhaps that this anti-distillation defense has been effective in thwarting
[JM]: other competing model makers from using Claude Code to do their distillation.
[JM]: I do think it's kind of amusing that Anthropic is trying to prevent people from using their tool to create derivative works when that's precisely how they created the model in the first place.
[JM]: So it's a little, one could say hypocritical to try to prevent the very thing that allowed Anthropic to create this model in the first place.
[DJ]: I know.
[DJ]: It's hypocritical, but 100% predictable given the incentives, right?
[DJ]: Like this is arguably the problem with these companies existing as for-profit corporations in the first place.
[JM]: No, it's totally predictable.
[JM]: No question.
[JM]: I just find it interesting.
[JM]: And like I said, kind of amusing.
[JM]: All right, moving on to GitHub, who has been having a bit of trouble recently with uptime.
[JM]: And people have noticed this quite a bit recently.
[JM]: It's been affecting...
[JM]: their ability to get things done.
[JM]: And GitHub used to have a way of looking at historical uptime, but I think decided that was perhaps not very flattering and removed it in favor of just a current status.
[JM]: Like these systems are functioning, these are not.
[JM]: So people have stepped in to create
[JM]: The missing GitHub status page and GitHub's historic uptime, which are two different sites that provide a bit of insight into how GitHub's been performing.
[JM]: And spoiler alert, it's not good.
[JM]: The last 90 days, according to the missing GitHub status page, shows uptime of 89.43%.
[JM]: And the reason why that's funny is because
[JM]: People usually talk about uptime in terms of nines, right?
[JM]: Like, you know, five nines, however many, you know, 99 points.
[JM]: How many nines can you get really after the decimal point is usually the implicit thing that's being discovered here.
[JM]: And now GitHub is delivering zero nines.
[DJ]: They have, there is a single nine in there.
[DJ]: It's just in the wrong position.
[JM]: Yes, which in terms of how that is usually scored is zero nines.
[JM]: It's funny because when I originally looked at this, they only had two nines.
[JM]: It was like 99 point something and then it was down to one and now it's down to zero, which is really quite an achievement.
[DJ]: I mean, it's horrifying, really, if you consider a piece of infrastructure for the internet, for lots of businesses, for open source projects.
[DJ]: Having less than several nines of uptime is not generally considered acceptable.
[DJ]: Yeah.
[DJ]: It's maybe some combination of the ubiquity of GitHub, like the too-big-to-fail nature of it, and maybe its ownership by Microsoft, which, same deal.
[DJ]: That prevents any sort of...
[DJ]: I haven't really seen any accountability about this, and I apologize if I've missed it, but I haven't seen a statement from GitHub recognizing that this is essentially a crisis and that they're going to do better.
[DJ]: Instead, it just seems like...
[DJ]: At least from my perch, like working for a company that uses GitHub, you know, sitting there going like, oh, there's another outage today.
[DJ]: You know, it's like, oh, this part of our CI pipeline just isn't working right now.
[DJ]: It's like, this does not seem to bode well.
[DJ]: But what I haven't heard is GitHub running around reassuring us all.
[JM]: Yeah, just to check my own sanity, I'm looking at the notes where I've been following the story.
[JM]: And the original link that I put down here was from just under two months ago.
[JM]: And the headline is GitHub appears to be struggling with measly three nines availability, which now seems almost quaint two months later.
[DJ]: Yeah, exactly.
[DJ]: And that was only two months ago.
[JM]: So people were already saying GitHub might be in trouble back when that was their availability.
[JM]: And I think you're right to call out the Microsoft acquisition.
[JM]: Because if you look at GitHub's historic uptime, which is a page that will be in the show notes, and there's a graph of average uptime by month from April of 2016, presumably when GitHub came on the scene, through the present day.
[JM]: And there is this vertical line drawn.
[JM]: about November of 2018 when this acquisition happened.
[JM]: And this graph is a damning indictment of Microsoft's stewardship of GitHub's uptime because it goes from being this horizontal line of green dots on the 100% uptime line to just a sea of yellow and red indicating very shoddy uptime post-acquisition.
[DJ]: Now, Justin, we have to remember that correlation does not imply causation because if you really look at that graph, Microsoft acquired them in like 2019-ish.
[DJ]: But when the uptime really starts to fall off a cliff is right around March of 2020.
[DJ]: And I would like to remind you what other significant event happened in the world around that time.
[DJ]: So I think possibly the real solution is that GitHub is suffering from long COVID.
Yeah.
[JM]: Well, if this keeps up, it's going to be more like forever COVID.
[DJ]: If this keeps up, we're all going to have to move to self-hosted Forgejo installations.
[JM]: Well, it's funny that you mentioned that because if you look at the status page for Codeberg, which is powered by... Say it.
[JM]: I don't even know how.
[DJ]: Powered by a fork of getee.
[DJ]: Getea?
[DJ]: What is it with these names, Justin?
[DJ]: Why can't anybody come up with a good name?
[DJ]: I mean, Codeberg at least is a pretty good name.
[JM]: Forgejo.
[JM]: I know that's wrong.
[JM]: Forgejo.
[JM]: I know Forgejo is not how it's pronounced, but that's what I am willfully mispronouncing it as.
[JM]: So if you look at the uptime history for Codeberg listed by service, it's 100, 100, 100, 99.91, 100, 100, 99.86, and more hundreds after that.
[JM]: I understand that vastly fewer people are using Codeberg than using GitHub.
[JM]: And yet their overall uptime is just night and day, way, way above GitHub's at this point.
[DJ]: Uh-huh.
[DJ]: I saw a thing the other day from one of the heads of operations at GitHub, I think.
[DJ]: It was saying that, well, I can't remember the exact numbers, but it was something like in, I think, the first quarter of 2026 on, I think it was like on GitHub Actions.
[DJ]: They had something like, I don't know, 12 times as much activity as they'd had in all of 2025 put together.
[DJ]: So like there are real problems with GitHub.
[DJ]: I do also think that those problems might come from a genuine massive increase in scale and not simply the idea that Microsoft has assigned only the F tier employees to work on GitHub or whatever.
[DJ]: And, and also like,
[DJ]: Yeah, I have a fondness for Codeberg.
[DJ]: But quite frankly, if Codeberg, if something like 1% of 1% of the activity on GitHub was suddenly shunted to Codeberg, Codeberg would go down tomorrow and not come back.
[DJ]: Like, sorry, guys, that's what would happen.
[DJ]: So some of this is not really like this organization is good.
[DJ]: And this organization is bad.
[DJ]: I think I think it really is more about the sort of unprecedented wave of software that has that has come about over the past year or so in particular.
[JM]: Yeah, but I don't think you can look at this graph of GitHub's historic uptime and conclude that there was this massive wave around the time that Microsoft acquired GitHub.
[JM]: Like, I don't think this timing is unrelated to the acquisition.
[JM]: And I feel like particularly with the kinds of resources that a company like Microsoft has, they should be able to deliver better uptime
[JM]: But I feel like they just don't put as big a priority on that as, say, deploying their copilot LLM in all the places where people sometimes don't necessarily want it.
[JM]: Well, speaking of uptime or downtime, as it were, I think it's been interesting also to see how Anthropic has struggled with keeping their Claude service available and some of the outages that they've had.
[JM]: I had one friend of mine say, "Claude was out for two hours today."
[JM]: "I had no idea how to do anything."
[JM]: And this wasn't even about code generation.
[JM]: This was just about using Claude for non-code related things.
[JM]: I do wonder what that's going to be like.
[JM]: It's like that old XKCD comic where the manager sees their engineers goofing off and asks them why they aren't working.
[JM]: And they say, "Code's compiling."
[DJ]: That feels quaint.
[JM]: Right?
[JM]: And now it's going to be, oh, well, the LLM is down.
[DJ]: That's really for real, right?
[DJ]: It is kind of insidious how you get used to using this tool all the time.
[DJ]: And then if your internet connection drops, you can't get any work done.
[DJ]: But then also, if the company providing the LLM service has an outage, you can't get any work done.
[DJ]: And like,
[DJ]: I think we are going to want and need, as I am fond of pursuing, alternatives like things that run models locally on device or on a company's own infrastructure or something like that.
[DJ]: Because this really is such a stark dependency.
[DJ]: I ran into this just yesterday on a personal project.
[DJ]: I was working with Claude Code and...
[DJ]: Claude Code just decided you're not authenticated anymore. You know how you were logged in on that last request? Well on this one you're not, so please log back in. And you know it's relatively straightforward to do that: you type in a command, it opens a web browser, in this case my web browser like I have a cookie that's already authenticated with Claude, so you just have to basically click okay, but the first time I tried to do that it didn't work, so I tried again -- it didn't work.
[DJ]: And I tried again and it didn't work.
[DJ]: And so I was like, I guess I'm done using Claude Code for the morning.
[DJ]: And sure enough, like I came back that afternoon and tried exactly the same thing and it worked immediately.
[DJ]: So, but, but it did just like, it pointed out to me like, oh, okay.
[DJ]: So now if I'm like working on something, I'm in the flow state, I'm focused on this project.
[DJ]: My ability to keep working on it or not is somewhat dependent on this service being available.
[DJ]: And I don't like that.
[DJ]: And it's the case, whether your friend was joking or not, that generally for the things I use these tools for, I do know how to do them myself if I have to.
[DJ]: But nonetheless, you know, we like we should acknowledge that the more you use a tool, the more that you build habits around its presence, the more difficult is its absence.
[DJ]: And we have that with lots of things, our smartphones, Internet connection, etc.,
[DJ]: that we don't really notice until they go away.
[DJ]: And we suddenly realize like, oh, I actually have to think really hard about how to do X or Y or Z.
[DJ]: And this is just another dependency like that.
[DJ]: So we should keep that in mind.
[JM]: There does indeed seem to be a bit of angst regarding the degree to which we will become dependent on these tools and how well we will function when they go down or become unavailable for some reason.
[JM]: Because you're right that you and I can function when that happens.
[JM]: But there's going to be a whole new wave of people that won't be able to, that have just become so accustomed to using these tools, not in lieu of their own abilities, but because those abilities were never needed.
[JM]: And it'll be interesting to see how that all plays out.
[DJ]: I was on the train the other day, and I happened to glance over the shoulder of a fellow commuter, and I saw that they were using -- I think it was ChatGPT -- on their phone, there was a chatbot of one of them, anyway... They seemed be using it to describe a difficult social situation they were going through.
[DJ]: Like I just, I saw something about whether someone's behavior was acceptable, something like that.
[DJ]: Again, and then I looked away, like I did not spend a bunch of time eavesdropping on this person's private thing.
[DJ]: But that just stuck with me because, you know, we've heard lots of lurid stories about people becoming dependent on these tools, not just for things like writing code or writing emails, but for portions of their like mental well-being, right?
[DJ]: You know, this notion of using this tool is like a therapist, which is a real bad idea for a whole bunch of reasons that, you know, I'm not going to catalog all of them right now.
[DJ]: But like, I've occasionally used large language models to help me process my own thoughts.
[DJ]: I’ve tried to do it very carefully, and keeping very much at the front of my mind, things like the fact that the way these things are trained, they will tend to produce output that reads as confirming. So the danger of going, to use common parlance, like is it valid for me to feel upset in this situation, like just understand that you’re gonna get some version of, "Yes, absolutely." Right?
[DJ]: And so if you're going to use a tool like that, again, I think it really is much more useful not to think that you're like asking a human being what to think and instead use it to basically say, I have a bunch of ideas in my head.
[DJ]: Like I just I want to get them on the screen so that I can consider them.
[DJ]: But the point being that you consider them, what you're doing is, in my opinion, like what you're doing is the skill of introspection.
[DJ]: The thing is, you can do that with like a piece of paper and a pen just as much as you can do it with chat GPT.
[DJ]: So to your point about skills, like what I am really desperate for is for people to continue to actively develop skills.
[DJ]: Skills of managing their own mental well-being, which I mean frankly lots and lots of us are bad at anyway, right? Like, they don't teach you that in school, at least I don't think they do -- I haven't been in school in a very long time, but I'm guessing they still don't teach you that in school but...
[DJ]: People are already bad at this, and outsourcing stuff like your ability to think through how to have a challenging conversation with another person to like a large language model service that might be offline at that moment is such a bad idea.
[DJ]: So yeah, I'm pessimistic about this new dependency that we're building into our lives in the same way that we have done with
[JM]: the internet in general, and it's interesting to see the variance in our tolerance for uptime and availability, and maybe not even our tolerance but just what we are currently observing, right? Like if our electricity went out as frequently as Github's service went down, or even as you said, the internet itself -- if our internet connections were as fragile and flaky as GitHub's uptime,
[JM]: it would probably be a bigger deal.
[JM]: We would probably be a lot more vocal about it.
[JM]: And that might have something to do with the degree to which these things are different, right?
[JM]: But I do hope we arrive at a point someday where people care enough about service uptime and availability that they try to deliver it
[JM]: with as much uptime as, say, our electricity delivery is usually provided.
[JM]: Because otherwise, as you said, our dependence on it, coupled with frequent outages, is going to start to become a real problem.
[JM]: Which is why it's nice when we can run models locally.
[JM]: And one of the use cases that I have found recently relates to dictation.
[JM]: I am using a tool called MacWhisper for dictation, and it uses an excellent downloadable model called Parakeet.
[JM]: But I find that sometimes the punctuation isn't quite right.
[JM]: It'll put periods in between words that don't belong.
[JM]: Sometimes things are miscapitalized.
[JM]: And MacWhisper has a feature where you can feed the output of your dictation into the large language model of your choice, and you can run the prompts of your choosing.
[JM]: One of the default prompts is called Cleanup.
[JM]: It's designed to fix a lot of the punctuation and capitalization issues that I just mentioned.
[JM]: The problem that I have with it is latency.
[JM]: I find that push to speak dictation doesn't work really well unless it's fast.
[JM]: If you hold it down and you talk for 10 seconds and you let go and you have to wait for another 10 seconds, it makes it really hard to use dictation effectively.
[JM]: And the round tripping to process this transcription through the large language model and back is what accounts for that extra latency.
[JM]: I originally was doing this with Gemini because Google gave me a bunch of free credits to try their model with, but it was just too slow to effectively use.
[JM]: So I turned it off.
[JM]: So yesterday I was thinking about it and I realized part of the issue is the network latency.
[JM]: And part of it is even when I was running local
[JM]: models, I was using larger models than I probably needed.
[JM]: They were just too slow for the task at hand.
[JM]: So I decided to experiment with the just released Gemma for open weights model from Google.
[JM]: And they have a small 2 billion parameter model that I've been experimenting with.
[JM]: And I have to say, I'm really impressed with the results.
[JM]: I barely notice any difference between a
[JM]: disabling the LLM post-processing and using it with it enabled.
[JM]: And for me, this is really just an excellent use case for a local large language model, something that operates really fast because there's no network latency.
[JM]: And obviously I love the fact that it's all happening on device.
[JM]: So from a privacy perspective, I don't have to really think too much about what I'm dictating.
[JM]: There's even an iPhone version of
[JM]: on an app called Google AI Edge Gallery, which is a bit of a strange name for an app.
[JM]: But apparently, this app can run Gemma 4 on device on your iPhone, which I guess could be an interesting way to run something on device until Apple ships their allegedly Gemini powered on device models later this year.
[JM]: One other thing that I'll mention quickly is I recently experimented with a tool called LlamaBarn, which is a way to run local models via llama.cpp, which is a tool I've mentioned in the past.
[JM]: But instead of running in a terminal, it's a Mac app that you can download and run in your menu bar.
[JM]: which is a lot more convenient than running the llama-server commands in the terminal.
[JM]: But there were a few issues that I ran into.
[JM]: One of them is that I don't see ways of adding the plethora of command line flags available to llama-server in LlamaBarn.
[JM]: And so using this dictation post processing as a use case, I couldn't replicate the same speed using LlamaBarn, presumably because some of the command line flags that I'm adding, I can't add in the Mac app.
[JM]: And there are a couple of other minor things related to LlamaBarn, but it does seem like a cool tool that I will use once they get some of these rough edges sorted out.
[JM]: What about you, Dan?
[JM]: Haven't you been working with local models recently?
[DJ]: Yeah, I have.
[DJ]: And my principal machine for running local models is a Linux box.
[DJ]: So I don't think LlamaBarn is usable for me.
[DJ]: But in terms of having an easier user interface to models...
[DJ]: than I'd get with raw llama.cpp.
[DJ]: I've been using LM studio and it's, I've been happy with it for the small experiments I've been doing so far.
[DJ]: Cause it gives you both like a, the common chat bot interface that you can use directly.
[DJ]: And it's also able to stand up a server endpoint for,
[DJ]: that I can hit from another machine on my local network.
[DJ]: And I recently got that working reliably.
[DJ]: Very long story short, there was some Mac OS network permission stuff that I had to dig out of the Arcana and then flip a switch and suddenly everything worked.
[DJ]: So that was fun.
[DJ]: It wasn't fun actually.
[DJ]: But it is very cool to now be able to write software that wants to interact with an LLM and do that on a model that's running locally on my hardware.
[DJ]: So, for example, I wrote a little app that I use to process my financial transactions, like my bank statements.
[DJ]: I really don't want to load my bank statements into cloud or chat GBT.
[DJ]: Thanks.
[DJ]: But, uh, but now instead I can use a large language model to cat, to categorize bank transactions.
[DJ]: This is something large language models are quite good at.
[DJ]: Like this is actually a use case that large language models are good for is take a bunch of semi-structured information like the random names of businesses that show up on your bank statements and,
[DJ]: and try to figure out which one of those is a coffee shop and which one is a piece of software you bought.
[DJ]: And so I'm really enjoying being able to run that on hardware and software that's under my control.
[DJ]: So I'm definitely going to check out Gemma 4 and other maybe smaller models because the last experiment I was doing, I was using like Quen 3 80 billion, which is a gigantic model, which I'm able to run because I have a lot of GPU memory in this machine.
[DJ]: But it definitely felt like using a for the thing I was doing with it.
[DJ]: It definitely felt like using, you know, a jackhammer to drive a nail into a wall to hang a picture, like just too much tool for the job.
[DJ]: So it will be fun to play with some of these smaller things and see if I can achieve similar results faster and maybe heating up my GPU heatsink a little less.
[JM]: All right, that's all for this episode.
[JM]: Thanks, everyone, for listening.
[JM]: You can find me on the web at justinmayer.com, and you can find Dan at danj.ca.
[JM]: Reach out with your thoughts about this episode via the Fediverse at justin.ramble.space.