Chris: Hello, friends, and welcome back to your weekly Linux talk show. My name is Chris.
Wes: My name is Wes.
Brent: And my name is Brent.
Chris: Hello, gentlemen. Coming up on the show this week, Greg KH has been busy.
Chris: Seven Linux kernels landed on Saturday.
Chris: Almost all of them were in LTS. We'll break down what changed and what releases
Chris: just might fit your home lab best.
Chris: And then why generating truly random numbers is so dang hard and how it's gone so bad in the past.
Chris: We'll talk about that and we'll round it out with some boosts,
Chris: some picks, and a lot more. So before we get there, let's say time appropriate
Chris: greetings to our virtual lug. Hello, Mumble Room.
Chris: Hello, guys.
Mumble: Hello, Brent.
Chris: And hello up there in the quiet listening and everybody watching live over at
Chris: jblav.tv, making it a Tuesday on a Sunday.
Chris: And good morning to our friends at Defined Networking.
Chris: Check out Defined.net slash unplugged and sign up for Managed Nebula 100 hosts
Chris: for free, no credit card required.
Chris: Nebula is one of my favorite technologies, and Manage Nebula makes it so straightforward,
Chris: and now they give you a lighthouse to get going real quick.
Chris: Picture a Mesh VPN completely and totally under your control with a design that
Chris: is intelligent and low resources from the beginning.
Chris: Something designed to connect some of the most important and sensitive data
Chris: around the world is now available to you and, through the magic of cryptography,
Chris: is very straightforward to set up.
Chris: You get strong encryption, direct device-to-vice connections,
Chris: and you can self-host Lighthouse Nodes, giving you complete control over the
Chris: network that you depend on. That's huge.
Chris: So go get started. 100 hosts for free. No credit card required at Defined.net slash unplugged.
Chris: Go check it out. We love it. Defined.net slash unplugged.
Chris: You know, we just have a really quick housekeeping this week.
Chris: We wanted to say thank you. We meant to say thank you last week when we were
Chris: remote at Brent's place.
Chris: But it dawned on us afterwards, once again, a big thank you to everybody who
Chris: helped us get those headsets.
Chris: It's almost been, what, two years now?
Brent: Those things have been just solid. I don't think they've never ever caused us
Brent: a technical issue we've had to solve. Everything else has, but never those headsets.
Chris: It's crazy. It's crazy what a reduction in complexity and gear that was for us, too.
Wes: Really? Yeah. I mean, at that point, it's them and a mixer of some kind that
Wes: can handle enough inputs.
Chris: Yeah.
Wes: And then a computer.
Chris: And without them, it's big microphones and stands and separate headphones and
Chris: wires for all of that to work. And it's nuts. It's nuts. So thank you so much.
Chris: We just wanted to take a moment.
Chris: And it's like every time we use them, we're like, oh, so, so freaking grateful,
Chris: the members and the listeners, too. People contributed from all around.
Brent: I want to thank you two boys And the Mumble Room for doing a nice little send
Brent: off Of my beloved studio That was nice That thing has done quite a few episodes
Brent: So I want to say thanks for your One last hurrah in there It was really great.
Chris: Seven kernels on a Saturday. Greg KH shipped a bunch of stable kernels in one
Chris: window. I'm hoping he's taking a summer vacation after this.
Chris: Six of them were LTS point releases and some big numbers in here, boys. Check this out.
Chris: 6.12.100.
Chris: 6.1.180. Get ready for this. It's still kicking.
Chris: Linux 5.10.262.
Wes: Yeah, those are basically point release numbers, right? so those get bumped
Wes: up as Greg releases new ones so you can kind of tell how long he's been doing those.
Chris: Yeah.
Brent: I'm glad he's keeping track because we're not.
Chris: Linus also tagged Linux 7.2 RC5 over the weekend, so he called it quite the massive RC5.
Chris: This really kind of brings it up to a total of seven. So we got these six stable
Chris: point releases, and then the new 7.2, which kills AppleTalk, is reaching RC5.
Chris: And it has quite a bit of changes in it because there's been a lot of AI-assisted
Chris: patching that's been hitting the kernel recently.
Chris: It's a lot. What do you say, Wes? You want to break some of it down for us?
Wes: Yeah, I guess just take it to handle on what kernels we're talking about here.
Wes: Well, sort of before the big push, Greg released 7.1.5. 7.2 is not out,
Wes: right? So we're doing the 7.1 thing.
Chris: Yes, thank you.
Wes: And then, of course, as you said, Linus released 7.2 RC5. We're expecting 7.2
Wes: to come out maybe sometimes around August 16th, somewhere in there.
Wes: Obviously, depending on how many RCs Linus deems necessary.
Wes: And then, so that was July 26, and then on July 30, yeah, six LTS point releases.
Wes: So we got, we'll go from the newest and then kind of get older.
Wes: We got 618.41.
Chris: Wow.
Wes: That's the newest, quote unquote, active LTS, right? 618.
Chris: Okay.
Wes: That has an end of life coming up in December 2028. Then we had,
Wes: let's see here, 612.100, Debian 13 Trixie and RHEL 10 base. That EOL is also December 2028.
Chris: That's, I mean, 2028's not that far away, but pretty good. So these are going
Chris: to, this will be the kernel that ends up in people's RHEL 10.
Chris: So this will be pretty widely used kernel 612-100.
Wes: And it's the first time we start seeing the involvement of the civil infrastructure
Wes: platforms SuperLTS that's going to try and keep this thing alive until December 2035.
Chris: I'm sorry.
Chris: What did you say?
Wes: Yeah, 2035.
Chris: Wow, yeah, yeah. That's incredible.
Wes: So basically, it's a Linux Foundation project, the civil infrastructure platform,
Wes: with a bunch of industry folks, right? Like Siemens, Toshiba,
Wes: Hitachi, Mux, so people that.
Chris: Build like- I hope Google's in there with Android.
Wes: Right? Long-term embedded industrial systems. And so it kind of exists to extend
Wes: kernel.org, which often does it for like roughly two years.
Wes: But they're trying to offer stuff more like 10 plus, 20, you know?
Chris: So if you want to run 612-100, you can say that you're running a super LTS kernel.
Wes: Yeah, that's right. Okay, we also got 66147. That has an end of life of 2027.
Wes: That's getting pretty close, December as well.
Wes: 61180, that's a Debian 12 bookworm base. That AOL is December 2027 as well.
Chris: Also coming up.
Wes: And would you believe it? We are still pushing updates to the 5 series kernel.
Wes: Yes, we're on 7 now. that's like two giant numbers ago uh,
Wes: so this is 5 15 213 that's the ubuntu 2204 lts base that's coming up soon though
Wes: end of life december 2026 as far as obviously ubuntu does their own work but
Wes: yeah in terms of kernel.org.
Chris: Right right then they're on their own after that.
Wes: And then the last but the least i don't know that's up to you uh 5 10 262 that's
Wes: the debian 11 era kernel.org end of life December 2026 coming right up but this
Wes: one is also under that super long term support
Wes: to January 2031.
Chris: When you see that, when you see they get the super long-term support,
Chris: I think the obvious giveaway here is like these have ended up in some sort of,
Chris: widely deployed corporate system, some image that's on a bunch of consumer devices
Chris: or control devices, something like that.
Wes: Yeah, and things shipped that like aren't easy to update or maybe are certified.
Wes: And so you need, if you change the kernel, you got to do a whole new certification.
Wes: You might not be able to do updates in place.
Chris: But a lot of these drop off in 2027. A couple of drop off in December of this year.
Chris: Two of them drop off. So like Greg's list, I don't know if he's responsible
Chris: for the super long term, but like Greg's list does shorten a little bit in a year or two.
Wes: And that's part of sort of the two-year thing, right? It's like the kernel.org
Wes: team has to maintain stable kernels that like, you know, actual people pull from.
Wes: And they work on whatever the latest kernel is all the time.
Wes: It's just, you know, you have to handle who's going to actually support this,
Wes: which is where it is nice to see the companies involved in wanting to run ancient
Wes: kernels stepping up to sort of take on some of that ownership.
Chris: So, Brent, I think last time we were looking at some of the patch sets that
Chris: were hitting 7.2, there was a lot of stuff in video, a lot of video drivers.
Chris: What are we seeing that got your attention in 7.2 RC5? The new freshness.
Brent: There is new freshness. There's AMD GPU compute stuff in there.
Brent: that's a 42x regressions though on some of the 7.0 point releases so stable
Brent: diffusion xl via comfy ui went from a nine second to uh 388 seconds.
Chris: That's a pretty significant regression.
Brent: Okay so.
Chris: All right so maybe 7.2 don't don't use it on amg compute systems just yet.
Brent: I mean some of those are fixed upstream in 7.0.13 but not every single distro
Brent: has picked them up yet so So take your chances or do some research, I would say.
Chris: I also saw a fair amount of pent-up networking work. Wes had a stat here that
Chris: over a third of the patches that landed were networking alone.
Chris: And again, I think, Brent, that's more of like the AI-assisted code stuff only
Chris: sitting in the networking branch. It just, to me, seems like...
Chris: We're going to need, like, a spring cleaning next year or something.
Brent: How much are you hoping to clear out? As much as we cleared out?
Chris: Well, if we're willing to get rid of AppleTalk, then where is the line, boys?
Wes: I think you're right. We're probably going to see more of that as basically
Wes: every subsystem in the kernel gets more and more pressure.
Wes: The stuff that is no longer load-bearing is just the desire to cut it is going to increase.
Chris: But to your point about, like, that massive 42x regression compared to Linux 7.0,
Chris: So that stuff stings, but most people that are using that stuff in a production
Chris: workload where they're using the actual AMD compute infrastructure,
Chris: I doubt they're going to install 7.2.
Chris: And AMD does seem to be actively contributing upstream, so I imagine they're
Chris: going to be working on that.
Wes: I think part of the story here is kind of like what and why do you choose a particular kernel?
Wes: How do you go about doing that, right? So it's like some of these things are,
Wes: yeah, you hit it if you are upgrading, but do you need to upgrade for hardware
Wes: enablement? Do you need some sort of new feature? Are you really looking for
Wes: a potential performance improvement?
Chris: There's another wrinkle to it as well, as sometimes the drivers or system support
Chris: do get backported to these older releases.
Chris: So you might see that they started contributing drivers at the 7.0 series,
Chris: but sometimes, like the NVIDIA stuff...
Chris: IOMMU stuff, that stuff gets backported. And so the older kernels can sometimes
Chris: end up with that. So I think you could think about it in tiers here, right?
Chris: If you're looking through an LTS lens and you're just looking at long-term stable
Chris: kernels, you want to set up a box and forget about it for as long as possible.
Chris: I mean, you still need to update, but you won't have to redo your kernel.
Chris: Then that's, I think, right off the bat, that's Linux 6.18.
Chris: It is the current, quote-unquote, bleeding-edge LTS. It has some of the newer
Chris: AMD, NVIDIA stuff backported.
Chris: And it will last until 2028. That gets you a pretty long runway.
Wes: Yeah, I suppose that comes down to like what are you supporting and how many
Wes: updates are you willing to do? How long can you go without changing kernels?
Chris: Yeah. Or like you said, is it just doing stupid file serving and maybe DNS?
Chris: And it's like something you run on a Raspberry Pi or it's a low-end networking box.
Chris: Then you don't need 6.18. Something like 6.12 will be around until possibly 2035.
Chris: So talk about a box for networking infrastructure you could just set and forget
Chris: and just do quarterly updates on.
Chris: And you're not going to have to worry about a new kernel. So 6.12 could potentially
Chris: be around until 2035. That's the one for those machines.
Wes: It's funny it feels old, right? 6.12 kind of sounds a little old.
Wes: But, I mean, it was a kernel we really talked about a lot on the show.
Wes: It was a big one. Preempt RT stuff.
Wes: I mean, there's just a bunch of stuff. And I think it was pretty widely deployed,
Wes: which is why it's getting that long-term support.
Chris: Right. Preempt RT. Right. Also, the note in here that it's considered one of
Chris: the most battle-tested kernels of all of them.
Chris: And that's their opinion. So, okay. Let's go with that. So, 6.12 seems like
Chris: a real winner if you don't need super modern hardware and you're cool with the
Chris: setup and forget it kernel.
Chris: Then after that, like, I think it sort of just drops off and it's,
Chris: okay, well, you're just going to go with whatever distro you want.
Chris: This is really if you're even making an active decision, right?
Chris: Right, because Debian's going to manage their kernel release.
Chris: Ubuntu will manage theirs. There's going to be stuff that they're just going
Chris: to take care of for you, and you may end up on a different kernel at some point
Chris: when you just upgrade the distro.
Chris: But if you are actively making a choice, and I'd like to know how many of you
Chris: really are actively making a choice, which kernel do you actually pick?
Chris: Like, I'd like to know how many of you are actually doing that. I mean, we do.
Wes: Yeah, we definitely do. I mean, we're running this end kernel.
Wes: And then back in the day when this was a KDE Neon box here in the studio,
Wes: you were doing the mainline tool to just kind of get the freshest kernels.
Chris: And on FakeNez, we're doing the LTS kernel. We're doing one of the LTS kernels
Chris: just because we hardly ever update that box and we don't want it to break when we do ZFS updates.
Chris: And so we did go to the LTS gear for the rolling arch-based server.
Chris: And so I wonder how many people out there are, A, actively choosing their kernel.
Chris: I bet it's a minority. And I wonder, B, which kernels they're choosing.
Chris: So boostin, boost.jupiterbroadcasting.com and let us know.
Wes: Yeah, I like that. It turns out you can have a rolling distro with a stable
Wes: kernel or a stable distro with a rolling kernel.
Chris: I like it. I mean, on our desktop boxes, we just go full rolling with everything.
Chris: But on the server, it's not that crazy, I think, to have an LTS kernel and a
Chris: rolling user land. I mean, we've been proving it for how many years on that
Chris: fake NAS box, abusing it, doing rando updates.
Brent: Isn't it just lots of luck, though? I don't know.
Wes: I think the cottonwood is structural. It helps at this point.
Chris: It's really about, well, for us, right, it was down to we wanted rock solid ZFS support.
Chris: And sometimes ZFS doesn't keep up with the current kernel. And if you load the
Chris: current kernel, you can sometimes not have a loadable ZFS. And so for us,
Chris: it just made a lot of sense. I think if we were doing Bcache FS or ButterFS on that machine.
Wes: Probably wouldn't be a big deal.
Chris: It probably wouldn't matter. But then if we were trying to use an NVIDIA driver or an AMD driver.
Wes: Then it could be.
Chris: It could be. So again, I'd like to know how you pick and what you pick.
Chris: Let us know. But let's talk about 7.2 because I feel like we've been building
Chris: to this for weeks, especially since we learned about the death of AppleTalk
Chris: coming at 7.2. So what stands out to you, Wes?
Wes: Yeah, there's a lot of exciting stuff.
Wes: A few highlights for today, though. The cache-aware task scheduler.
Chris: Yeah, buddy.
Wes: The scheduler now knows which cores share a cache.
Wes: It groups tasks that share data onto the same cache domain, so you stop paying
Wes: cache miss penalties across cores. Now, we should be clear, this really only
Wes: counts if you are on a system like Epic or Xeon that has multiple cache domains.
Wes: You're not going to see it on a single cache domain device.
Wes: But if you have that, this is basically like a free performance when you don't
Wes: do anything except upgrade your kernel and things get faster.
Chris: See all those Xeon CPUs, boys?
Brent: Sweet.
Chris: I'm going to go dust off that old Mac Pro upstairs that's like 15 years old that's got Xeons in it.
Chris: And I'm going to... Actually, I do have an active Xeon system in production that I think about it.
Wes: Oh. So I love it.
Chris: I love it.
Wes: Well, time to update.
Chris: Mm-hmm.
Wes: Or it will be in August.
Chris: Yeah.
Wes: Okay. Not the only work on the scheduler, though.
Chris: Okay.
Wes: And scheduler stuff. I mean, not only is it one of the trickiest areas,
Wes: I think, of the whole kernel, but it can tend to really, you know,
Wes: it can have regressions pretty easily. It can be very noticeable,
Wes: or it can have a lot of performance impact.
Wes: So another change is a fairer GPU scheduler.
Wes: The kernel's DRM scheduler used to default to high priority for GPU jobs.
Wes: Now it defaults to fair. So GPU time gets distributed more evenly when you've
Wes: got multiple things competing for it.
Wes: And if you're running GPU pass-through to a VM or something like that,
Wes: while also running all your fancy desktop compositing as you do, this should help.
Wes: if you've just got like one gpu you know churning away at like a ml task or
Wes: just rendering one thing specifically you probably won't notice but again these
Wes: are hardware specific but that even makes them you know a little more challenging to do yeah,
Wes: So it's impressive we're seeing the changes.
Chris: Okay, you got anything for a guy that's just got a regular old computer that
Chris: doesn't have like Xeons or, you know, multiple Goopoos?
Wes: Well, we do have memory reclaiming. How about that?
Chris: All right. I like that.
Wes: The kernel's page reclaim got a lot smarter in 7.2.
Wes: MGLRU improvements are showing 30 to 100% higher throughput on things like MongoDB,
Wes: which is kind of just a test workload.
Chris: All right.
Wes: And your page allocator isn't stuck and always handing out 2MB anymore.
Wes: It can hand out 16K, 32K, 64K pages, smaller bits.
Wes: It means less overhead on every memory lookup. And that means instead of having
Wes: to allocate more stuff than you actually need, programs can request and get
Wes: more reasonable amounts.
Chris: I can't believe that wasn't a thing already.
Wes: Yeah. There's definitely some of that in here.
Chris: You're telling me the huge page allocator in Linux has been stuck at 2 megabytes forever?
Wes: Yeah.
Chris: Wow. Okay. That's a big one. That's a big one. All right. You got me.
Wes: And it should mean container startup and teardown in particular should be noticeably faster.
Chris: All right. All right. You got me on board, Wes.
Wes: And maybe you don't want to be using it, but if you do...
Chris: What?
Wes: Your pal swap also got faster.
Chris: Oh, you know, you never know. Sometimes you got to swap.
Wes: Yeah. Back in the old days, the bad old days, swap used to route different memory
Wes: types through different code paths. That's gone, cleaned up, unified.
Wes: Less bookkeeping, less bloat.
Wes: and at the end of the day that just makes it faster.
Chris: I like to put my swap on a ram disc so i'm glad to see that performance has been improved.
Wes: Now it's not all good news necessarily there are some regressions here um that,
Wes: you know that's being worked on though but rfs in particular is in a bit of
Wes: active surgery um large folio supports on by default now okay and they've improved
Wes: things there was a previous direct io regression that got a fix
Wes: a 59 throughput fix no less.
Chris: Jeez, okay.
Wes: But, as a result of that, you know, so you had to do that to get the new fix.
Wes: You had to get the performance improvement.
Wes: We got large folio support that speeds things up. Right. But,
Wes: it did also introduce regression itself.
Wes: In particular, if you're running KVM guess with RAM backed by a ButterFS file,
Wes: so it's like a particular setup, it can cause a deadlock.
Chris: Hmm.
Wes: But they're working on the patch. I don't know yet if that's going to land in
Wes: 7.2 ultimately. I hope so. But, we haven't gotten there yet.
Chris: And then they will backport it to Linux 6.12.
Wes: And perhaps in a way where if this is only in the RC, then perhaps then it's
Wes: a way that you just never have a possibility of getting it at all.
Brent: I'm like working through all my VMs and like, is this going to hit me?
Brent: And I think actually one of them does.
Chris: Yeah, I could see that getting me. What you need is a super fast external storage
Chris: in a different file system.
Chris: Maybe some new stack for the universal serial bus could enable this,
Chris: Wes? Maybe a new iteration?
Wes: Oh, maybe. Yeah, we are seeing USB 4 streaming support coming.
Wes: There's not hardware really, I think, out there a ton of it yet that you could
Wes: actually use. So this is the case where the kernel sort of leads the way with,
Wes: people doing upstream development, right? Where before the hardware exists,
Wes: it's in the kernel. So then when the hardware does exist, you just plug it in
Wes: and the kernel supports it.
Brent: That's so great.
Wes: Also stuff that isn't super used yet and isn't a little more experimental,
Wes: but fun to watch because we've been paying attention is sub scheduler support
Wes: in the sketch X system that's, that enables you to write your own scheduler.
Chris: What's the difference between a scheduled task and a sub scheduled task, Wes?
Wes: Well, like you can, you have a finer grain on, you know, because,
Wes: and you can change, you can basically have tasks that have their own schedulers, right?
Wes: So you can have, instead of one scheduler that has to know about how to schedule
Wes: everything for every type of task, it can then have its own subscheduler that
Wes: can schedule just that type of task.
Chris: That's awesome.
Wes: And then you can optimize it more, like maybe specifically you're trying to
Wes: run a database server, right? And the database does all kinds of stuff that
Wes: other kinds of applications don't do. And so you might need a scheduler that's
Wes: aware, like, oh, give the database priority on this kind of thing.
Chris: Dang it. You're going to make me run 7.2.
Wes: Well, that was the idea.
Chris: Dang it.
Wes: I don't know why. I just like the number 7.2. It's got a good round shape in my brain.
Chris: Yeah, it does.
Wes: It's got a nice color, so we should all be running it.
Chris: There has been. I mean, I was teasing earlier, But there has been a bit of cleanup
Chris: in the 7.2 kernel release as well.
Brent: Quite a bit of cleanup. You mentioned the networking stack.
Chris: Mm-hmm.
Brent: And you mentioned a third of the patch of RC5 is networking alone.
Brent: So mostly driver-side stuff. And Linus himself said, nothing in there strikes
Brent: me as particularly strange or scary.
Brent: So don't worry, guys.
Chris: Oh, man.
Brent: Also, there's a, hey, you remember the Sega Dreamcast? Yeah.
Chris: Did you guys play that? Yeah. That was like a peak late nineties, really.
Brent: Oh, I played the heck out of that. Yeah. That was 1998.
Chris: Wow.
Brent: Well, the Dreamcast is still getting driver fixes. So there's a mouse driver
Brent: null pointer dereference.
Wes: It had drivers at all?
Chris: And a mouse? Okay.
Brent: I mean, we should investigate maybe. So there was a null pointer dereference
Brent: since 2017 and that's been fixed.
Chris: That's got to be an LLM fix.
Wes: For sure.
Chris: That's got to be.
Wes: I mean, it's kind of interesting almost that this happened. And are people on
Wes: the list being like, why do we still support the Dreamcast?
Brent: You can't have AppleTalk.
Wes: But the Dreamcast is totally fine.
Chris: I like to think there's a couple of load-bearing Dreamcasts out there running
Chris: the LAMP stack. That's what I like.
Wes: Look, it's been hosting my personal website since 1998.
Chris: Or it's like some kernel developer's email server from like,
Chris: yeah, 99 or something like that.
Chris: Like in 2000, he gave up on it and just turned into a Linux mail server.
Chris: And they're still maintaining it.
Brent: Well, they might be running off disks, so GDROMs, those disks,
Brent: mount properly now in the Gamecast, just in case you're having that problem.
Brent: But don't forget, i486 is also gone from that.
Chris: Wow, right.
Brent: Yeah.
Wes: Hey, 14,000 lines removed, though.
Brent: It's true. That's quite a few. That's 80 whole files.
Brent: And Linus says here, zero real reason for anybody to waste one second of development effort on this.
Brent: It launched in 1989, supported for 37 years in total, which is 18 years after
Brent: Intel stopped making them.
Brent: 6.12 LTS still supports it, though, till 2036. So don't worry, Chris.
Chris: That 6.12 is the MVP kernel, guys.
Wes: Yeah, you want your Apple talk, you want your 486.
Chris: Honestly, if we're still going in 2035, 2036,
Chris: hand to sky, I hope, we should load up 612 and get a 486 box that we can still get booting going.
Brent: Yeah.
Chris: That would be pretty great.
Chris: And the Dreamcast, right? All right. I mean, this is pretty solid,
Chris: right? We've got a lot of kernel releases here. I mean, a lot of times you're
Chris: not even going to have to think about this. We like to think about it.
Chris: We like to pick. But I think a lot of us don't actually have to think about it that much.
Chris: We hope that maybe you're the kind of person that does give it some thought.
Chris: Seven kernels. Two of them are finally aging out at the end of this year.
Chris: And one that does kind of break your GPU performance, so be careful.
Chris: But the clear MVPs for home labs, if you need modern graphics hardware,
Chris: is 618. and if you just want to set it and forget it forever, is 6.12.
Chris: I mean, 6.12 is getting the fixes that we talked about for the performance regressions.
Chris: It's getting support until 2035, and it's going to support 486.
Chris: Probably also has AppleTalk in there. I mean, probably has AppleTalk, right?
Wes: It does.
Chris: Oh, my goodness. What an MVP of a kernel.
Brent: Are we calling it the legacy kernel?
Chris: I think we should call it the MVP super kernel. I mean, that's what they call it. It's the MVP super.
Brent: Let's do it.
Chris: We got all kinds of links in the show notes. if you would like to nerd out and
Chris: read more. And there is some good reading in there. And you know where to find
Chris: those show notes over at linuxunplugged.com.
Brent: Well, we got a really nice email into the show this week, or more accurately,
Brent: into my personal email. And here's the title.
Brent: Serious Security Advisory for Your Cold Card.
Brent: And I thought this was a scam at first because I've been getting a lot of really,
Brent: quite sophisticated spam into my email. And it's been very entertaining to watch.
Brent: But nope, this one's real.
Brent: And it got me a little nervous about what the heck is happening out there.
Brent: Do you guys have any idea why I got this email?
Chris: This is really a tragic story.
Chris: As of this weekend, almost 2,000 addresses have been drained of nearly $90 million worth of Bitcoin,
Chris: affecting cold card wallets, initially the cold card Mark III devices,
Chris: and then later expanded to include certain Mark IV, Mark V, and cold card Q
Chris: firmware versions, basically everything but the most current emergency firmware
Chris: that was released over the weekend.
Chris: And it comes down to an issue in how the cold card generated random numbers.
Chris: You know, we're going to get into this. Randomness has been a real challenge
Chris: and a challenge for Linux, too, that has gone wrong in catastrophe kind of ways.
Chris: So in this case, the cold card had a firmware bug that sent the seed generation down the wrong path.
Chris: Instead of using the hardware that was built in to derive a seed phrase or a
Chris: key that had low entropy, it was falling back to like a Python script.
Chris: In fact, a micro Python script that roughly generated an entropy of 40 bits
Chris: on the Mark III and 72 bits on the later devices.
Wes: Yeah, it's essentially the micro Python runtime provides sort of like a random,
Wes: a basic random generated. It'd be fine if you were using it for,
Wes: like, you know, an offset in a cron job or something like that.
Wes: But it is not meant to be cryptographically secure.
Chris: Yeah, it doesn't generate complex enough cryptography that it can't be figured out.
Chris: So the hardware was designed to generate strong randomness. The hardware was
Chris: present. It was working, but it just wasn't being used, which is really a tragedy.
Chris: And so because people can now mathematically derive the...
Chris: the seeds, they are able to essentially restore wallets on remote systems and
Chris: drain the cold cards without ever having your cold card connected to the internet
Chris: or your computer active.
Wes: Yeah. So essentially without the hardware part that was supposed to be like
Wes: the secure random entropy, the fallback random number generator was seeded from
Wes: the chip's UID, which is like a unique ID on the chip that has a limited possible
Wes: amount of numbers, and a boot time timer state.
Wes: And neither of those are truly secret. And a lot of them, it turns out to be
Wes: kind of closer to a predictable range than something random.
Wes: And then so if you can kind of constrain the space of things that the initial
Wes: secrets generated from, then you can explore that space and the rest of it's
Wes: all just deterministic.
Wes: And then you can generate a whole bunch of private keys at once and then just go check them.
Chris: Yeah. And before the computing machine became easily accessible to us,
Chris: analysis confirmed in the past that the hardware worked. That's what people
Chris: really focused on in their audits is that the hardware did what the hardware said it did.
Chris: But they didn't test if the actual path to activate it was working until LLM-assisted
Chris: security reviews came along and discovered it.
Wes: Yeah, because the code to use the hardware is in the binary.
Wes: Like it's built in, it's available. It's just never called.
Chris: Yeah, yeah. So the code exists, it works, but they failed to verify it was actually being used.
Chris: And so as a result, attackers have been able to access people's funds.
Chris: And that's a tragedy because people thought they were doing the best thing they
Chris: could for themselves. They were keeping them offline and all of that.
Chris: And it got us thinking about some of the previous times in Linux where random
Chris: number generation has gone horribly wrong.
Chris: It's not necessarily a hardware story itself. It's really about the difficulty,
Chris: of generating true randomness on deterministic systems, right?
Chris: Because if you give a computer the same input, you're going to get the same
Chris: output every single time.
Chris: So the limitation then is how do you generate something that is truly uniquely
Chris: random on a deterministic system?
Chris: And it has to be something that's sound and secure and hopefully as future-proof as possible.
Chris: So when something like this happens, well, you don't get rugged.
Chris: And that is more difficult than it seems. And it's something that Linux has
Chris: been working on since the very early days, like since 1994.
Wes: Yeah. It's essentially cryptographic security is essentially a bet,
Wes: right? It's like not a law of physics. It's not a guarantee.
Wes: You are just trying to do everything you can to have a giant pool of randomness to pull from.
Wes: and then keep that hidden and secret and intact across the entire chain.
Wes: And there really is a chain. Like it's not just, you know, give me number and
Wes: done. It takes a lot of setup.
Wes: So kind of how it works today, the very first step is some of the most important
Wes: and that's getting quote unquote unpredictable inputs. And this is stuff that
Wes: comes from a large space that's really hard to guess.
Wes: So Linux, the kernel is continuously collecting small little timing variations
Wes: and environmental noise and pieces of the system that are sensitive to things
Wes: like that. So when does a hardware interrupt fire? Which one fired?
Chris: It can even be like noise from the disk, user input, just things that the system
Chris: generates that are unique.
Wes: Totally. CPU timing, bootloader provided seeds you can use if you want to try,
Wes: to bootstrap randomness in a VM.
Wes: You can also use hardware things either like included by your CPU manufacturer
Wes: or an add-on device or something like that.
Wes: And part of the idea here is that no, you're not trying to make sure to have
Wes: every single source needing to be perfect on its own, you can try to mix them
Wes: together so that even if some don't have as much randomness in there as you
Wes: think, you're still getting a good final randomness on the other side.
Wes: So after you're able to find a bunch of different sources, you mix that into your entropy pool.
Wes: And you need to be careful because the most dangerous time is when you boot something up fresh.
Wes: Because there hasn't been a lot of time for environmental noise to get into
Wes: the system. There hasn't been that you haven't really done anything yet.
Wes: It's all kind of starting code cold from the very non-random initial firmware
Wes: and like, you know, first code that runs on the system.
Wes: And so a lot of work over the years has been around like, do we need to block
Wes: if there's not enough randomness or not? What does it mean to have enough randomness or not?
Chris: And Intel has tried to solve it with their RAND kind of built-in stuff that's
Chris: trying to generate some noise. But, of course, that thing's a black box.
Chris: You can't use it as the one source of truth.
Wes: Yes. And that's where, like, the kernel folks have said, like,
Wes: it's OK to use, we think, but we're not going to rely on it in terms of being
Wes: the only option. We're going to make sure we mix it into a bunch of other stuff.
Wes: So even if it is compromised, that limits the damage.
Chris: Yeah, I would say, in a way, the random generator in Linux is a story of learning
Chris: that we need multiple sources and we need to make sure that we don't trust any one particular source.
Chris: We have a clip from Jason Donfeld, and he took over the Linux random generation
Chris: system from Theodore So, who originally created it in 94.
Chris: And this was some of Jason's observations when he started working on it.
Chris: And, you know, he's a developer and he tells it like it is. But I think if you
Chris: look through that, it's a fair assessment.
Clips: It's in Random.c, which is really old code. It dates back, as far as I could
Clips: find, to version 1.3.30. I
Clips: mean, it's really ancient code. There's a copyright date on it from 1994.
Clips: So I think that even the code was living someplace else before there.
Clips: But actually, considering when it was written, it's not so bad.
Clips: it's held up remarkably well despite the knowledge that was around in that time,
Clips: and I think for all intents and purposes it's okay, it's held up it's been done,
Clips: with some reasonable intelligence but yeah,
Clips: Still, over time, there's been a lot of complexity added to it.
Chris: But that complexity is kind of necessary in order to generate randomness that
Chris: doesn't have one single point of failure as well.
Wes: Yeah, I think his point, too, is sort of identifying, like, what's the essential
Wes: complexity and what's sort of the complexity that's just crept in over years
Wes: of maintenance? And how do we keep what we need, but also keep it simple enough to understand?
Chris: And so on Linux, as an end user, you perceive this as there's dev random and there's dev u random.
Chris: And when I was reading about this, one of the things that I thought was really
Chris: fascinating is early on, DevRandom, or maybe it was DevURandom,
Chris: I can't remember, but one of them would essentially let an application ask for a random number.
Chris: but wouldn't force the application to wait for the correct answer.
Chris: And so applications would query, but this thing would take a minute to generate,
Chris: and then they would be like, okay, got it, when they didn't actually have like
Chris: a truly random number generated. And so later on, they had to create an API,
Chris: that would force the application.
Chris: It's just called get random. It's the get random API on the Linux kernel.
Chris: They had to create this API that would essentially manage all of that for the
Chris: application so they could, in a sense, force the application to wait for true
Chris: randomness to be generated.
Chris: but then for compatibility reasons they had to like allow the api to also keep
Chris: that working in the background and just allow the application to move on and
Chris: think it had everything it needed like it's it's this funny kind of line they
Chris: have to walk every time they try to make a change to the system so i imagine complexity does seep in.
Wes: Yeah and i think part of it was just being confused around like the intentions
Wes: like you random versus random was like never super clear but what the new interface
Wes: is able to do and get random is it has um underneath there's crng ready
Wes: and so the kernel now tracks whether its random number generator has been seeded with enough
Wes: quote-unquote real entropy to be trusted and that's the gate you want
Wes: to wait for it doesn't really matter if you've generated a bunch after that
Wes: which is sort of where the you random versus random didn't ultimately make sense
Wes: but what does matter is especially at early boot
Wes: has the pool itself had enough mixed in entropy yet and that you probably do
Wes: want to wait for and that's where it becomes important to like,
Wes: you know not roll your own stuff to rely on standard apis to rely on trusted
Wes: audited libraries that are using the correct code paths because that can and
Wes: does change over the years.
Chris: There's been improvements. Is there anything else you want to touch on before
Chris: we talk about when it went horribly wrong?
Wes: No, I guess just the rough idea is,
Wes: you grab random bits, you mix them into a pool, you wait until that pool has
Wes: sufficient entropy, then you use deterministic pseudo-random number generator
Wes: stuff like cha-cha 20 in the current kernel and that sort of stretches that value.
Wes: it can't add more stuff, but it can turn like initial secret seeds into big
Wes: long streams of random numbers.
Wes: And that's what you actually pull from. And they sort of generate ephemeral
Wes: keys and then generate stuff and then throw them away. And there's a whole process
Wes: around that, but it's not adding new entropy. It's just using the entropy you
Wes: have to get as many random bits as you actually need for user space.
Chris: Okay, so when you don't get enough random bits, you have situations like the
Chris: cold card or a situation that lasted multiple years, CVE 2008-0166,
Chris: also known as Debian's OpenSSL bug.
Chris: I remember this one. And it started very innocently. A Debian package maintainer
Chris: was trying to silence a memory analysis tool, a Valgrant warning about OpenSSL
Chris: utilizing uninitialized memory.
Chris: And it seemed like, okay, well, this is an easy two-line fix. Clean that up.
Wes: And normally that's good, right? Because why are you reading from memory that
Wes: hasn't been initialized? It can't possibly have a value you want.
Chris: Right. So they remove those lines of code. They ship it. Everything seems to be fine.
Chris: The issue was is that they had basically really cut down the entropy of the
Chris: keys that were getting generated for things like OpenSSL, which you can imagine what that impacts.
Chris: In fact, it didn't just impact OpenSSL. It was like everything that needed a
Chris: cert. So SSH, all your X509 keys, like everything got affected by this.
Chris: And the worst part was it was out in the wild for two years before researchers discovered it.
Clips: So this bug, which affected OpenSSL, the random number generator,
Clips: was introduced two years ago due to a Debian-specific patch.
Clips: So it's a Tebian-specific bug,
Clips: and remained undiscovered for two years, almost exactly two years until Luciano
Clips: discovered this, and it was published on May of 2008.
Chris: Think about the scope of this for a moment, right? You've got to reissue and
Chris: revoke every key that was created from 2006 to 2008.
Chris: There were, I mean, just...
Chris: Maybe millions, could have been in the millions that were affected by that.
Chris: It was a massive, massive issue.
Wes: Yeah, it also affected a chunk of live Tor relay nodes at the time, which isn't great.
Wes: Under the hood, what was happening is that OpenSSL was actually intentionally
Wes: using uninitialized memory as an unpredictability source. The idea is on different
Wes: systems, other processes, whatever has been in that system, how it came up in
Wes: the hardware, all of that will be different.
Wes: And so it's one of the very few times you might actually legitimately read from uninitialized memory.
Wes: Valgrind doesn't know that. And there hadn't been any infrastructure or communication
Wes: or clearly comments around it or whatever to, like, make it clear.
Wes: And so when you remove that, it wasn't like there was no other sources,
Wes: but it was essentially only the process ID that was left in the pool of entropy.
Wes: And so that's like a number between 1 and, like, 32K at the time.
Wes: And that's not enough, right? Right. You'll start having. And so what how they
Wes: found it is that they found real world collisions out there.
Brent: Right.
Wes: And that's the only way you can, because if you just look at output from this
Wes: random number generator, you can do statistical tests on it.
Wes: You can look at just sampling that one value won't tell you.
Wes: It doesn't tell you how big of a space it came from you. The only way to do
Wes: it is by looking at huge samples of the outputs in the real world, effectively.
Wes: And so it's it's really hard to do, especially for a developer making a change.
Wes: And it's kind of the thing that tends to favor the attacker.
Chris: It strikes me, too, that if an automation tool led to a mistake like this today,
Chris: we would probably spend 90 percent of the discussion having an existential panic
Chris: about automation tools and 10 percent of the discussion about the actual bug.
Chris: But back then, it wasn't that wasn't the conversation at all.
Chris: And it was so it was so massive. And there was a lot of downstream issues as well.
Chris: And it's reminiscent of the cold card issue.
Chris: When you dig through it, you know, it looks like this person that made this
Chris: commitment, I don't know for sure, but it looks like maybe they were a Python
Chris: person and they didn't understand the C code and they didn't really know what they were looking at.
Chris: They thought they applied a fix that solved an error that was happening when
Chris: they were compiling the firmware.
Chris: They would get this compiler error over and over again, and the guy went in,
Chris: or the gal, I don't know who it was, went in and made the change that ultimately
Chris: bypassed the hardware and used the MicroPython.
Chris: And they resolved the build error and considered it fixed.
Wes: And all the tests were making sure that the build flag was there to say enable
Wes: the hardware RNG, and it was there.
Wes: It was just supposed to be a one, and it was a zero. And there was no check
Wes: on the value. There was only ever a check that the var existed.
Chris: So I just sat there for 20 months just waiting for somebody to come along with a tool to find it.
Wes: And it is a case, right, if you did add your own passphrase thing on top,
Wes: if you had brought in your own sources of randomness.
Chris: Yeah, which you can with a cold card. You can provide your own entropy.
Chris: But if you didn't. You have to do that, right? And if you're buying a device
Chris: like that, you assume it's...
Wes: Part of the value was like, well, this has a hardware source.
Wes: That's part of why I'm buying it.
Brent: It's so sad that the exact thing they were promising is the thing they didn't do.
Chris: Yes.
Brent: Unintentionally.
Chris: And it was for people that had all the best intentions. But it makes me realize,
Chris: you said at the beginning, or towards the middle there, Wes,
Chris: you said, you know, don't reinvent the wheel. Try to use things that exist.
Chris: In a crazy twist of fate, people that rolled on a Linux box and used the Linux
Chris: random number generator are better off.
Chris: And I think there's a lesson in that. And it's why Linux is so important to
Chris: us as a society is that it is a general technology platform where there are
Chris: people with mutual incentives that are improving the system and making it and testing it.
Chris: And every time we go outside of that and we recreate something, you take on risk.
Chris: And this time it's a type of risk that was at the firmware provider level when
Chris: a lot of people were thinking about physical security and air gapping and 9-volt
Chris: batteries and all this kind of stuff.
Chris: And we're not really focused on the actual threat.
Wes: And it also speaks to open source is not enough. Maybe that will help in our
Wes: new era of automated reviews. But the code was out there, right?
Wes: It wasn't that it was, obviously the hardware is somewhat hidden, but the code was there.
Chris: You know, it wasn't like a truly open source license. And I think if it had
Chris: been like a GPL license, maybe more people would have built off of it and discovered
Chris: this. But because they didn't want people doing that, they did like some sort
Chris: of like read only you can view license kind of crap.
Chris: And this is also one of the side effects of that.
Chris: And it just goes back to like, that's why Linux is so important,
Chris: is we have solved some of these problems and they continue to be monitored and
Chris: maintained and it's done right.
Chris: And it really comes back to like we invented it once and we've continued to iterate on it.
Chris: And what started back in 1994 as kind of a simple system has developed into
Chris: this truly robust multi-source of noise and entropy, truly great as we can get
Chris: on these determined systems, random number generator.
Chris: And that's just one of the many gifts of Linux is that these things like networking
Chris: and disk IO and random number generation and cryptographic signals,
Chris: all this stuff is handled by Linux for us. Such a gift.
Wes: We get a world-class product for free. You don't have to go buy a proprietary,
Wes: cryptographic enhanced private OS from Microsoft or from Qualcomm or for whatever.
Chris: And just trust that they got that aspect right.
Wes: Right. No, you just use the standard kernel that is open source and anyone can download.
Brent: Oh my goodness, gents, there are some boosts here. We're going to start with
Brent: our baller, Greg the Lawyer.
Brent: Greg came in with 133,333,
Brent: sets.
Chris: Whoa!
Chris: Thank you, Greg. Appreciate that.
Brent: Greg is also adhering by the rules of the show. One of the boosts is a row of sticks.
Chris: Was that a row?
Brent: And it said, well, the other one is actually.
Wes: It is now.
Brent: We've said that, I think, twice. He wants to reward early movers on Buzz.
Brent: Not for me, but as a Mac user since 1984, I appreciate those early adopters.
Chris: Aw, thank you.
Wes: Aw, sweet.
Chris: Yeah, Greg came coming in with a live boost here because we got low support
Chris: for last week's episode and we realized we probably should have waited a year
Chris: to cover Buzz, and so thank you.
Brent: Greg.
Wes: And thanks for hanging with us, audience.
Chris: Also a few Satoshis in there to feed those chickens. I do have,
Chris: I wonder how I could get those public in a way that wouldn't drain my internet.
Chris: But I do have a few cameras set up on those chickens. But do I want to dedicate,
Chris: one to two megabits of my precious Starlink upload link? I don't know.
Chris: But if I can think of a way.
Brent: Isn't that why you got the mini?
Chris: Well, no, I do the backup. Oh. The connecting backup.
Brent: We've been planning this for months, really.
Chris: And then if I have to switch to it for production, maybe I pause the streams or something.
Wes: What about a system where people, it's like you boost and then the stream is
Wes: opened for a certain amount of time?
Chris: Opens the Ngrok tunnel to your knees.
Wes: Yeah, so you get like 10, you know, send in a couple sets and you get like 10
Wes: minutes of chicken camp.
Chris: That's great. They are the best chickens. You would not believe.
Chris: Like Labradors. Like they get on our lap and they just snuggle.
Chris: It's ridiculous. And Boss, the rooster, he's got a routine now in the evening
Chris: around sunset. He comes in and watches TV on the couch with us.
Brent: What's his favorite show?
Chris: I mean, mostly it's Magnum, but last night he was watching YouTube.
Chris: He doesn't mind. He just likes the colors. Nice. Turd Ferguson comes.
Chris: And thank you very much, Greg.
Chris: You basically brought up the
Chris: whole trend for the episode. Turd Ferguson comes in with a row of McDucks.
Chris: That's 22,222 Satoshis. Oh, it's a live boost.
Chris: The cold card mess has me tired. We trusted the hardware because it was offline,
Chris: but the bug was already baked in and waiting fun times ahead.
Wes: Yeah, pretty brutal. And then it can just sort of disappear out from under you.
Wes: You don't have to do anything. It's just moved in a transaction,
Wes: and the next time you check, it's gone.
Chris: You're like Brent and you take a few days, you go check, and it's gone.
Brent: I did check this morning because I thought that was a responsible thing to do.
Brent: Not gone yet. Don't try anything on me. But I've got some homework to do after the show.
Wes: Can you describe your gold card more to me?
Chris: You know, it makes me think like, yeah, what else? That is a good point.
Brent: Oh, that's a good question.
Chris: We're just getting this thing rolling, right? Because what do you think the
Chris: coincidence is that it happened a few days after Kimmy K3 with Open Weights came out publicly?
Chris: And it's one of the only really current top tier models that doesn't restrict
Chris: you from doing cybersecurity stuff. And then three days later, this exploit's found.
Wes: I think it depends on how long it took to do some of the pre-work.
Wes: But it's a good question.
Chris: We may find they were cooking for months. Or we may find that they slammed it
Chris: in 10 minutes with K3. Because the reason why I bring that up is now people
Chris: have tried it, and it finds it in 5, 10 minutes. It's just immediate.
Wes: Yeah, I think the main question is from finding it, how long does it take to
Wes: generate all of the private keys?
Chris: It's a massive sweep and all of that. Yeah, yeah, for sure.
Wes: And I assume there was some staging before you actually submit all the transactions.
Wes: but who knows that's a great question.
Chris: I'm sure we'll be i mean that's the thing about the blockchain people are already putting it all.
Wes: Together well e scott boosts in with 2371 sats,
Wes: what what what what happened with nextcloud i have two separate instances running
Wes: i even moved slash data off my windows 7 ntfs share to a proper zfs mirror with
Wes: backups also this is a star trek.
Chris: So a quick version is I had like a cold stop will not be using xCloud ever again
Chris: moment just because the data it wiped out was some of the most valuable data possible.
Chris: And so what happened is I went to view a text file just using the viewer like
Chris: it's in the web interface and you click on a .txt file,
Chris: and it opened and the instant it opened, it wiped the document,
Chris: it put some escape characters in the top left corner and then it immediately
Chris: wrote that to the file system.
Chris: and it should have just been a document viewer. It shouldn't have been writing,
Chris: and it did it instantly. And Wes, you looked it up, and it turned out to be
Chris: some sort of in-browser issue that is very rare but happened on the client-side rendering.
Chris: And I realized for what I need, there was just too much going on there.
Chris: And the fact that by just viewing my most important document,
Chris: which is what I used NextCloud for, was for its privacy, totally offline,
Chris: off the Internet, by just viewing that document, I wiped it out.
Chris: And that was when I was like, I don't need any of this. And I decided at that
Chris: moment I was going to stop using it, which has been a real pain in my butt.
Chris: But when something fails that catastrophically on you, sometimes it just leaves
Chris: a bad taste in your mouth.
Chris: And that's after it worked for me for a long, long time.
Wes: And it doesn't mean it still couldn't be a great CalDev suite.
Wes: There's lots of roles it still makes sense for, especially if you have robust
Wes: backups and it's just one place to look at them.
Chris: It was just sort of the last straw, honestly, because it's been a lot to upgrade
Chris: and manage over the years. and I sort of have rolled back what I use it for to some degree.
Chris: And so that was sort of the last thing I was really, really relying on it for and it bit me.
Wes: And made you reconsider. Do I even need it?
Chris: Yeah.
Wes: I did look it up. 2371 is a well-known year in Star Trek. So we got a few things.
Wes: Well, third season DS9 begins then.
Wes: It's also the year USS Voyager launched on its ill-fated mission.
Wes: And it's Star Trek Generations is set in that year when Captain Kirk meets Captain Picard.
Chris: That was such a great time for Star Trek. That was such a great time.
Chris: Thank you, sir. Appreciate the boost.
Wes: Thank you, E-Scott.
Brent: Unicorn,
Brent: 1999 came in with 6,000 sats.
Brent: At some points in open source and with some projects who had to sign off that
Brent: you were the author of the code and had not used any source that was also not clean.
Brent: But now with AI, that doesn't really matter. It seems the who owns AI code question
Brent: is not completely solved.
Chris: Wait, why is that? Why is that?
Wes: Well, there's still active legal debates around it as one. So,
Wes: like, for instance, some courts have said that pure, like, if all you did was
Wes: one-shot code, it can't be copyrighted at all.
Wes: So I think the question of, you know, exactly the law has not caught up with
Wes: the current tool set is sort of one aspect of it.
Wes: Like we have some cases that mostly dealt with code in the era of browser tabs
Wes: and nothing that's really dealt with the case of you're doing project management
Wes: on an agentic harness and then you edited some parts of it or you reviewed it.
Wes: So I think that's where some camps concerns are coming from,
Wes: just in that it is still an open legal question.
Chris: But it seems like if my self-driving car runs into Brent and smashes his van
Chris: up, I'm legally responsible for that.
Chris: It doesn't matter if the AI degenerated that.
Wes: Yeah, but that's a legal sort of – it's a separate set of law essentially,
Wes: right? Like this was a judge making a case about if you could copyright pure AI-generated content.
Chris: Oh, copyright, copyright.
Wes: Yes. So like when you make something – not everything is copyrightable,
Wes: but if it is a copyrightable expression, not just the idea but the expression,
Wes: you get to sign copyright.
Wes: to it that's how you add a license to your code files but they're saying if
Wes: all you did was prompt a single ai to generate code that cannot currently be.
Chris: Copyrighted i just think in the context of most most not all but most free software
Chris: project this is really ant humping because who's responsible is the person who
Chris: submits it and this is sort of full stop,
Chris: i mean this anything else is just sort of ant humping now when you're working
Chris: on a giant international project maybe that's really where things matter like
Chris: that i think but that's my opinion.
Brent: There were some explorations of laws in europe though that recently suggested
Brent: that open source software might be responsible if people use it in certain contexts so,
Brent: that's an interesting angle unicorn 99 here wants to clarify though i'm not
Brent: anti-ai i'm anti-corporate exploitation.
Chris: Boy sure the corporate ai aspect is so obnoxious so tiring i agree with you
Chris: there it's the worst about it it's the worst thing about it and yet We need
Chris: their precious GPUs, at least for now.
Chris: Hey, Mr. Mayhem's back with 16,000 Satoshis.
Chris: Look at that. Boost the roost, boost the roost, boost the roost.
Chris: Something broke. It's a fill-in boost. Well, boost the roost indeed. Thank you.
Chris: Appreciate that. And that is all of the paid boosts, but we do have some member
Chris: boosts. Would you like to kick that off, Wes?
Wes: Oh, you know how I love a good member boost.
Chris: Hey, Unicorn's back.
Wes: Oh, Unicorn1999 comes in with a member boost.
Chris: Hey!
Wes: I think tech people bringing up the copyright issue is not only crap,
Wes: but partially how copyright is always used by corporations against us users.
Wes: Like, it was okay back in the day for ISPs to host Usenet servers full of TV
Wes: shows, but not okay for people to download and share.
Chris: We don't talk about Usenet. We don't talk about Usenet. Yeah,
Chris: no, I think it's like we're talking about two different things here, Unicorn.
Chris: Because everything you just said I completely agree with. I just think there's
Chris: hypocrisy in folks that, you know, have their RAR stack set up or whatever,
Chris: or rip DVDs, then now concerned that LLMs are ripping content,
Chris: right? There's just hypocrisy in that.
Chris: But yeah, using copyright as a kludge is never good.
Wes: There's also sort of like, we're seeing projects now, which is perhaps reasonable
Wes: in that they are much more established,
Wes: projects, but we see projects now that are having concerns over copyright questions
Wes: that maybe were different than how Linux was treated and being actively developed
Wes: in the ESCO era, where there were similar sort of copyright questions.
Chris: Mm-hmm, mm-hmm, mm-hmm, mm-hmm, yeah.
Brent: Mm-hmm.
Chris: All right, well, Greg, the lawyer's here with a memboost. Welcome back to the
Chris: States. I hope the trip was flock-free. You know, I'm not sure.
Chris: I doubt it, though. I doubt it.
Chris: He says, I got fed up with Dario's nonsense and switched back to Codex.
Chris: I figure their model is better because they broke out first.
Chris: The weird feeling when switching VC-funded frontier models is how I fight in part for open source.
Chris: Yeah, although if it makes you feel better, the Codex is open source.
Chris: The backend isn't, but the Codex client is open source.
Wes: Amadeus member boosts in.
Wes: My workflow of cleaning up my own drives is using a great Rust tool called Dust.
Wes: Find out what takes up space, old hugging face models, old Proton versions from
Wes: Steam, Podman containers, et cetera.
Wes: Works great, and I'm using it on every machine I control or manage.
Chris: The Podman containers makes me chocolate.
Wes: Thank you for concluding the very realistic selection of actual disk usage.
Chris: Yeah, yeah, really, yeah.
Wes: And Dust is a Apache 2 license.
Chris: Good to see. Nice little extra pick there.
Brent: We've got Joshua here with a free members boost.
Brent: I'm a software engineer and always have ideas for side projects,
Brent: but no time to actually make the most of them.
Brent: Cloud Code has really changed that, though, but I worry about making my new
Brent: projects open source because of public backlash, since they're all written with
Brent: AI. Do you have any advice?
Chris: I would bet Joshua that you would actually find the reverse problem is going
Chris: to be your issue is that if you open source it and it's popular,
Chris: you're going to have a lot of people that are contributing back with AI assistant code development.
Chris: You're going to have a lot more of that than your people that drive by your
Chris: repo and get upset at you.
Wes: Yeah. Like I think it would be different. Like if you if you vibe something
Wes: up and go share it on three subreddits at once. And like there's there are versions
Wes: of that where you can get yourself in trouble. But if all you're doing is sort
Wes: of putting it out there, I don't think you'll receive too much backlash.
Chris: Anonymous comes in with a member boost.
Chris: It never occurred to me that a nutcase might not only have a thousand tabs open,
Chris: but that nutcase might also have them scattered across dozen of windows.
Chris: Absolute madness!
Chris: I know, I know.
Wes: Are we, it says anonymous, but I don't know. I kind of smell the PJ booze.
Brent: I don't think I've been called a nutcase before and certainly not for my tab habits.
Wes: Well, I came from the whole squirrel thing.
Chris: Yeah, as a squirrel father, I think you would put yourself in the nutcase category.
Brent: Yeah, that's fair. That's actually kind of a compliment. Thank you, anonymous.
Wes: Well, SensorSmile comes in with our last member boost.
Wes: I'm finally ready to start playing with AI. However, I'm not sure where to start.
Wes: I have a local Quen model up with
Wes: Lama CPP. The built-in chat interface is boring. How do I code with this?
Wes: Hermes? How do you do long-running agent? A beginner's guide for the tech experienced.
Chris: This is SeniorSmile. You're like, you're starting with Bitcoin by installing
Chris: a node before you've acquired sats.
Chris: I mean, not necessarily a bad way to go to truly learn from the ground up, but,
Chris: you might You might just spend a month with OpenCode and OpenCode Go subscription,
Chris: and learn that because what they're going to have for you is agents that are,
Chris: A, optimized for coding, B,
Chris: work right out of the box with tool calls.
Chris: And when you're going to do your own Quinn and your Lama CPP,
Chris: I love it, but you're going to need to use a front-end interface.
Chris: You're going to have to pick one that understands all that stuff,
Chris: that knows how to work with that stuff. And that, when you're just getting into
Chris: all of this, feels like a little extra complexity before you've even really
Chris: figured out what they can even do for you.
Wes: And they could also help like if you do get open code going or whatever.
Chris: Then you can have it set it up for.
Wes: You exactly you can have open code talk to Lama CPP no problem but that's just
Wes: more stuff you kind of get in that isn't actually playing with AI but you can
Wes: have the AI sort of bridge the gap and then try out both local and cloud hosted.
Chris: Not too bad there Westpain alright thank you everybody who supported this episode
Chris: with a boost we had 18 of you stream sats as you listened along and collectively
Chris: you all stacked 17,625 Satoshis for the show.
Chris: And you bring it all together with everybody who boosted. We stacked a humble
Chris: but very grateful 198,570 Satoshis.
Chris: Not the best hourly rate when you split it four ways, as we do,
Chris: or five ways, really. I think it was like five in there, but we still appreciate the support.
Chris: It is a lean season. When you hear the dynamic ads, you know we're hurting.
Chris: But we really appreciate the
Chris: members and the boost. It's also one of our favorite segments on the show.
Chris: It's stuff we never really plan to talk about, but often is great conversation.
Chris: And you can send a boost by going to boost.jupyterbroadcasting.com or jupyterbroadcasting.com
Chris: slash membership or linuxunplugged.com slash membership to be a member.
Chris: That's it from us for the boost. Thank you, everybody who supported us.
Chris: And we got a couple of pics before we get out of here. This is one I've been
Chris: sitting on for a while because I thought maybe we would use it for our Texas trip.
Chris: But I think I'd like to put it out there and get a sense if the audience likes
Chris: it and if anybody deploys it before we use it.
Chris: But I could see this being very useful for us in the future.
Chris: It's called Trek. I already like the name.
Chris: It's a self-hostable travel trip planner with real-time collab features,
Chris: interactive maps, progressive web support, single sign-on, planning budgets,
Chris: packing lists, and more.
Chris: I mean, this thing, and it looks good. It looks like a top-grade commercial
Chris: application that you can self-host.
Chris: And then you can work with your friends, your family, whatever it might be, together to plan a trip.
Chris: And, like, you know, make your big plans, put your budgets in there,
Chris: get it all on a calendar. You can subscribe to the calendars.
Chris: Like, you know what I'm saying?
Chris: Like, this is a real tool with interactive maps and stuff like that.
Chris: So you get a journal for every trip.
Chris: You get real-time collaborative planning. You see it all on a map.
Chris: It has lists and categories.
Chris: It has – you can put in, like, we're going to go on a vacation.
Chris: You can have it pre-visualize that. You can track costs per person.
Chris: You can drag things around and drop them on the calendar. And Wes, it's got an API.
Wes: Ooh, okay. Okay.
Wes: This looks really nice.
Chris: Yeah, I'm pretty excited about it. Like I said, I haven't had a chance to try
Chris: it myself, but I think somebody out there in the audience might be willing to
Chris: give it a go and let us know.
Chris: It is AGPL3.0, and it is primarily TypeScript.
Wes: It's got a Docker Compose setup, Kubernetes if that's your style. Yeah, okay.
Chris: Yeah, yeah.
Wes: The real-time collab part sounds quite interesting. We could just be on a call,
Wes: workshopping a trip, trying things out.
Chris: Teasing Brent about how long he has to drive, because we'll just put it right
Chris: there on the map in front of him.
Wes: Right.
Brent: It'll probably help me with that part. One of the big questions I have is, will it nix?
Chris: Well, of course. It does also have a Docker if you just want to go that way.
Chris: But from that, you could get to a declarative Podman configuration and about
Chris: two licks and two shakes of a lamp's tail.
Wes: Oh, it includes built-in MCP as well as an API. Great.
Chris: I mean, you could see how maybe our agents use this to have a plan and they
Chris: just tell us when they're done.
Chris: All right, Brentley, you got a little sneaky pick. I believe that you wanted
Chris: to pull forward, as they said.
Brent: I'm representing this sneaky pick as producer Jeff actually brought this up to me and he said, Brent.
Brent: This is perfect for me and therefore probably perfect for you.
Brent: It's a to-do app that is called C'est Fait, which is French,
Brent: but I'm going to declare that everybody else can probably just say C-Fit.
Chris: Oh, yeah. I would have sold it.
Brent: You tell me what you should call it.
Wes: Can we get the French one more time?
Chris: I would have said C'est Fait.
Brent: Or C'est Fait.
Chris: C'est Fait. I know, because it's got a cat. So I would have gone with cat,
Chris: you know, because it has a cat logo.
Brent: Yeah, yeah. Yeah, that works too. A silent F, right? In French,
Brent: that really means it's done, which I think it's kind of clever.
Chris: Ah, I gotcha. Okay.
Brent: But it's built for people who, quote, want keyboard-centric efficiency,
Brent: which I have been looking for forever. Jeff, you know me so well.
Brent: It looks like you can use it comfortably from the command line via TUI.
Brent: You can use a desktop GUI if you want or on the go with a native Android app.
Brent: Ain't no Electron in here.
Chris: It's got a TUI and a graphical app and an Android app.
Wes: This is impressive.
Brent: I don't know how none of us found this It's also.
Wes: It's apparently available in FlatHub already, FreeBSD There's obviously,
Wes: it's in FDroid and Google Play There's Windows releases maybe There's Mac OS
Wes: stuff, the only thing I'm not seeing is Nix really, but we can.
Chris: Get that done As a long time, and slightly ashamed Todoist user This,
Chris: this cooks I think this is blowing Todoist away Even at the UI layer.
Wes: To find shortcuts on the fly Wow Yeah.
Brent: It's nice, there's a little detail here It's 69.5% rust.
Chris: All right. You know what?
Brent: Does that count?
Chris: We'll give it to it. It's over 50%. It's over 50%. Everyday usage.
Chris: Press the question mark and navigate to the help tab.
Chris: Configurations in the CLI-2E under the hood.
Chris: All right. So, PJ, you've been trying this? Like, what's the dealio? What do you think?
Mumble: It really did catch my eye. I found it in F-Droid just going to do some updates,
Mumble: and it was right there on the front.
Mumble: Yeah. I really do like it. I like that the Linux desktop app and the phone app,
Mumble: they look very similar. They're not these giant, there's no giant changes between the two.
Chris: Right.
Mumble: So you don't need to necessarily learn all the key bindings because I never
Mumble: will remember them anyways.
Mumble: And I really like the subtasks. You can tree out things. I dream of an idea
Mumble: of having dependency tasks.
Mumble: This ain't quite that, but the subtasks kind of get you there. So I've been using it.
Chris: So it's like a, it's like subtasks isn't quite a project, but you can have a
Chris: task with a bunch of things you got to do to complete it.
Mumble: Yeah, exactly. So I have, I'm kind of using it for goals.
Mumble: Like I have, you know, truck repairs, right. And then a bunch of subtasks under
Mumble: that of the different things I want to do for my truck, you know,
Mumble: house repairs or home things I want to do. And I can have a bunch of subtasks
Mumble: for that. And it's been working out pretty well for that. I like that a lot.
Mumble: Just a dedicated place. I can put the things I want to get done or,
Mumble: oh, I've got 15 minutes to kill. What kind of thing can I do?
Mumble: Well, here's a whole list of things I want to try. I want to do.
Chris: Right.
Mumble: And I'll forget about them otherwise.
Chris: How is the self-hosting side set up the server side?
Mumble: I'm using NextCloud to sync everything up.
Chris: Oh, so you don't have to run a backend in particular? You could just have it sync?
Mumble: Whatever CalDev server you want to use, yeah.
Chris: Oh, okay. I see. I see. So they include Radical, a self-hosted lightweight feature
Chris: complete thing you can use, or, yeah, NextCloud. Huh.
Chris: This is great, Jeff. Nice find. I'm glad Brent, I'm glad he pulled it forward.
Brent: Mm-hmm.
Chris: I may give this a try afterwards. Especially if I could store it,
Chris: of course, I don't know. If I could store it in something, we'll see.
Wes: Yeah, especially if we can compose it with a few other things.
Wes: The fact that it's got the Android layer solved, there's a lot of promise.
Chris: There's a lot I like about it. And the client's in Flathub, which is nice.
Chris: And the Android version's in F-Droid.
Wes: Which is nice. Okay, so PJ's now game correspondent, right? And I guess he's also like a...
Chris: Productivity.
Wes: Yeah, productivity expert.
Chris: Productivity John. PJ still works for that. Productivity John.
Wes: Lifestyle expert.
Chris: Yeah. And a few other things. Solar, long hair, floor showers.
Wes: Oh, floor showers. He's got that down.
Chris: He does have the floor showers down.
Brent: Also our on-site Californian expert.
Chris: It's true. If you want to know anything about California, you just ask PJ. All right.
Chris: Well, we'll have links to everything we talked about at linuxunplugged.com slash 678.
Chris: We're slowly working our way to 700, so we hope you stick with us and join us
Chris: live. You can make it a Tuesday on a Sunday, 10 a.m. Pacific,
Chris: 1 p.m. Eastern, over at jblive.tv.
Chris: I say before we go, we tell people some pro tips. We should always,
Chris: maybe we should start the show with these.
Wes: Instead of burying it all the way at the absolute end of the show?
Chris: Right.
Brent: We should, like, do a reverse show sometime. We should start from the bottom.
Wes: Flip the duck. Surely a bot could flip the duck.
Chris: We should flip the show.
Brent: All right, next week.
Chris: We should think about that. That could be a lot of fun. Don't you think?
Wes: Probably.
Brent: Oh, yeah.
Wes: At least tell us what we like and don't like about doing it.
Chris: How do you handle the intro and the outro? Do you do the outro at the beginning?
Chris: Because that would really screw with new folks. Like, you lose them immediately.
Brent: Oh, go for it. Come on. What's one episode?
Chris: I don't know. I think if we could, the intro is the only thing.
Chris: We need some input on that, but I would totally be down for a reverso episode. That could be fun.
Brent: No, it works perfectly. You start and you say, about the same bat time,
Brent: same bat channel, you know, make it a Linux on a Tuesday, on a Sunday, and join us Sunday.
Chris: But then we play the outro music in what?
Wes: No, it starts with the outro music.
Brent: We'll figure it out.
Wes: I don't think we will. And then you intro the show at the end.
Chris: I think we need help.
Brent: Drew will save us.
Chris: Oh, yeah, sure. Yeah. All right. Okay, very good.
Wes: But in the meantime, JSON. It's in the cloud and in the feed.
Wes: We put JSON in our XML, but then we also put like text in there in the form of a VTT.
Chris: That's true. So you can have it either way.
Wes: And we put in a link. It's not the whole MP4 but there's a link to an MP4 embedded within.
Chris: Yeah. So you could watch it in video version and you may be surprised there's
Chris: visuals but because we're such effing pros you can just listen to the audio
Chris: version and it never detracts from that to such a degree you wouldn't even know
Chris: there's a video version. How about that? Ha!
Chris: So yeah, you can find that MP4 in the feed as well and our Mumble Room is always
Chris: going every single Sunday. So do join us live. It's fun. It's unique.
Chris: And not a lot of podcasts do it.
Chris: And, of course, links to everything we talked about at our website,
Chris: linuxunplugged.com. And the back catalog and all the other great shows over
Chris: at jupiterbroadcasting.com. As long as you want to check it out,
Chris: you might as well go to the calendar. It's over there in a contact page as well.
Chris: Would you believe it? And last but not least, you can get a message and support
Chris: the show at boost.jupiterbroadcasting.com. And you members, don't you forget
Chris: you get a free boost every single episode because we appreciate you so much.
Chris: In fact, we appreciate you just for listening.
Chris: Thank you so much for listening to this week's episode of your Unplugged program.
Chris: And you know what we're going to do?
Chris: We're going to work on a show right after this, get cooking on another one,
Chris: and we're going to see you right back here next Tuesday, as in Sunday.