Chris: Hello, friends, and welcome back to your weekly Linux talk show. My name is Chris.
Wes: My name is Wes.
Brent: And my name is Brent.
Chris: Hello, gentlemen. Well, coming up on the episode this week, the kernel release
Chris: that deprecates our childhood has landed.
Chris: There's a lot to like. There's one thing that really stings,
Chris: but 7.3 is looking like such a banger. I'm not sure if I'm even going to care.
Chris: We'll tell you more about that.
Chris: Then, how sync things saved the day this week.
Chris: Plus, we have some great boosts and picks that you're going to love.
Chris: and a lot more. So before we get into all of that, let's say time-appropriate
Chris: greetings to that virtual lug. Hello, my room.
Mumble: Hey, Chris. Hello, Brent.
Chris: Hello, and hello up there in the quiet listening. Nice to see you up there.
Chris: You guys looking good up there. Handsome as ever. Go say good morning to our
Chris: friends over at Define Networking as well. Define.net slash unplugged. Go meet Manage Nebula.
Chris: Decentralized VPN built on the open-source Nebula platform that we love and use.
Chris: Nebula is so awesome. Oh, I'm so, so grateful for Nebula, especially as I have
Chris: been having issues with other mesh networks. It really is so superior and simple
Chris: in its way and in its design too, yet extremely secure.
Chris: And I love that I have my own lighthouse, so I completely control the infrastructure. I own everything.
Chris: But like friends and family, I can hook them up with Define Networking's managed
Chris: Nebula. They automatically provision a lighthouse now for people,
Chris: but you can also run your own.
Chris: It's really great. It's a great company. It's a great product.
Chris: It's open source, independently audited, and it's built with modern encryption.
Chris: So go get started. Up to 100 hosts, absolutely free. No credit card required.
Chris: Go to defined.net slash unplugged. Redefine your VPN experience.
Chris: See why I love it. Own your infrastructure. Defined.net slash unplugged.
Chris: Thank you to Defined for sponsoring the Unplugged program.
Chris: Well, with the release of Linux 7.2, it is the end of AppleTalk and it is the
Chris: end of our sticker. We'll have to remove it from the store this week.
Chris: So if you want to get your commemorate sticker, now's the time.
Chris: It's like we're making a coin for AppleTalk, but it's a sticker.
Chris: I'll not dwell on this. The audience knows how I feel about this.
Wes: But get them while you can.
Chris: Also, we wanted to make a mention of the passing of a longtime Samba contributor,
Chris: Steve French. He passed away recently after dealing with an illness.
Chris: He's been around forever. He's been contributing since 2002.
Chris: He worked at Microsoft. He worked at IBM. He's worked in Azure.
Chris: He worked on my favorite project, KSMBD.
Chris: And so he had to step down for health reasons this week, and then he did pass away.
Wes: Our condolences.
Chris: Yeah, really. A longtime contributor. So our thoughts are with his family.
Chris: And they do have a new maintainer stepping up for CIFS.
Chris: So there will be. And I guess this individual has a history with NTFS, the rewrite.
Wes: Yeah, we have Paulo Alcantara and someone we've mentioned multiple times on the show, Namjai Jian.
Wes: Those folks are now co-maintaining CIFS, CIFS, which is Namjai's third file
Wes: system, the NTFS rewrite, ex-fat, and now this.
Chris: So it sounds like it's in pretty good hands.
Wes: It does indeed.
Chris: But we did want to note, because, you know, it's part of the community.
Chris: And I think it's worth observing, and our thoughts are with the family.
Chris: out there, and it is good to see the project continue to get support as well.
Wes: This is how open source gets made.
Chris: Mm-hmm. Mm-hmm. All right. Well, then, let's get into Linux,
Chris: speaking of the kernel, Linux 7.2.
Chris: So just as we finished up the live stream, I think Linus probably got done listening
Chris: to the show, clearly. And then decided to release.
Wes: Oh, better get some work done.
Chris: Yeah. Pushed out Linux 7.2. I don't think he wants to push it out before the
Chris: show's over because he doesn't want to be distracted. I understand that.
Wes: It might distract us, too.
Chris: And this was classified as a very busy kernel cycle release due to increased
Chris: patch and reporting churn due to AI-assisted development.
Chris: We do have a quote from Linus. You want to take it?
Wes: Well, this last week of the release was, once again, bigger than I would have wished for.
Wes: But hey, with the whole quote-unquote new normal thing, if I delayed releases
Wes: for that reason, we'd probably never have a release at all.
Wes: There are a number of fairly late reverts. The DRM scheduling reverts stand
Wes: out, but there's a few other ones in here too.
Wes: It may not be pretty, but it's the correct way to deal with,
Wes: oh, that code wasn't ready and caused problems. People will be trying that again later.
Wes: But while the DRM reverts stand out and are some of the bigger patches in here,
Wes: there's a lot of small fixes all over, mostly in drivers, but networking remained
Wes: fairly active this last week, too.
Wes: And then there are a few random patches to architecture files and the perf core.
Wes: With this, the merge window for 7.3 obviously opens tomorrow,
Wes: and I already have 40 pending pull requests.
Wes: I appreciate all the timely people. You know who you are.
Chris: Yeah, we'll get into 7.3 here in just a moment. You know, when Linus calls out
Chris: a subsystem, it's probably worth paying attention. When he calls out networking
Chris: specifically, it's probably worth paying attention.
Chris: So I think we'll get to that. But the thing that you and I have been watching, And,
Chris: Something that I think we talked about before on the show, so we'll just give
Chris: a brief mention, is there's been some real improvements to, quote-unquote, schedulers.
Chris: CPU schedulers, GPU schedulers.
Wes: The scheduling system, all kinds of stuff.
Chris: Scheduling got a big work, a lot of work in 7.2. So we have a now more cache-aware CPU scheduler.
Wes: Yeah. This is a pretty big deal.
Wes: And it's almost surprising in a way that we haven't had it already,
Wes: right? Like, caches are a huge part of CPU and computer performance, of course.
Wes: And these days, especially on systems with, like, multiple dies,
Wes: right, it's not just one giant CPU, right? It's, like, multiple CPUs,
Wes: each with their own last-level cache right next to them.
Wes: And so understanding, like, where is a process or a thread running,
Wes: and then what caches is it near, and then what data is in those caches can be a big deal.
Wes: So if you think about something like a database, right, maybe you're doing a
Wes: query where you have a lot of working memory that is, the whole point is you
Wes: want a query optimizer that can parallelize things, right? So you're not waiting
Wes: on trying to seriously do stuff.
Wes: So maybe you have a list of things you're trying to look up.
Wes: Well, both subqueries are going to need to be accessing that data, right?
Wes: So if one of the subquery threads is running on a different CPU core,
Wes: suddenly it's got to do the inter-core communication to go get the data from the other cache.
Wes: So you don't really want to do that. So now the goal is upgrade the scheduling
Wes: system so that it can figure out, like it starts tracking stuff in a multiple
Wes: way. So one is just obviously sort of threads of the same thing, right?
Wes: But it can figure out when things that are running need to share the same cache
Wes: and then try to schedule them on the same cores appropriately.
Chris: Woo!
Chris: That's a nice one.
Wes: There's multiple steps to it. They're learning a lot, or they're taking advantage
Wes: of a lot of neat stuff that happens.
Wes: Like the hardware actually has cache monitoring stuff built in.
Wes: So then the first level is basic structural stuff. It understands what the system
Wes: has and the basic schedule threads together.
Wes: But then it can actually go check to see what the layout the hardware tells
Wes: it is doing in real time. And it can even go sample the misses to figure out
Wes: like, oh, we tried to hit something in the cache, but that's actually on the other system.
Chris: Really?
Wes: We might need to start considering if this continues actually moving it to the other core.
Brent: Wow.
Wes: Now you don't want to do that too much, right? Because then you could just be
Wes: bouncing around all the time. That could be a problem. So it's a delicate balance,
Wes: as is everything in scheduling.
Wes: But this is a huge upgrade for just it being really smart.
Wes: Now, it will matter more on, you know, big systems with lots of CPU cores and
Wes: that kind of thing. So you might not see as much of an impact on your single-core laptop or whatever.
Chris: Yeah.
Wes: But it's a big upgrade for our friend, the Colonel.
Chris: Linus had made a comment that it may not be pretty, but it's the correct way
Chris: to deal with, oh, that code wasn't ready and caused problems.
Chris: People will probably try that again later.
Chris: Is he referring to the GPU scheduler stuff that got reverted?
Wes: Yes.
Chris: Yeah, okay.
Wes: 7.2 was meant to switch the DRM scheduler from first in, first out to a new CFS-style FAIR policy.
Wes: So we already have this on the CPU side. This was bringing it over to the GPU side.
Wes: But then an RX 9070 XT owner hit a sustained 100% GPU load.
Chris: Uh-oh.
Wes: Yeah, games dropped to 10 FPS. The Wayland session froze. Had to reboot to recover.
Wes: And then here's the part that maybe wasn't as good of an idea.
Wes: When this happened, it wasn't just that the default was swapped.
Wes: It was the only schedule. The other stuff was ripped out, and all you were left
Wes: with was fair. So the first in, first out, and the RR policy,
Wes: those weren't there. There was no runtime escape hatch.
Wes: So five days before the release, someone from Egalia shipped a 20-patch revert,
Wes: smoke-tested on a Steam Deck, and it ships first in, first out.
Wes: As the defaults again, sort of unrips things out, but now it still adds fare
Wes: as an opt-in experiment. So instead of doing a big change to making this default,
Wes: now we're still in sort of test mode.
Chris: All right. That makes sense. Thank you. Thank you.
Wes: What is nice, though, is the reporter, AB, tested some of the fixes himself.
Wes: This is a very active collaboration process, including, quote,
Wes: the patch Claude gave me.
Chris: Also, I think MongoDB users will be happy to hear this.
Chris: There was some work in Linux's memory management. The multi-gen LRU memory reclaim
Chris: system has been cleaned up.
Chris: And as a result, kernel developers have reported performance improvements approaching,
Chris: so, you know, results will vary, but approaching 30% in some MongoDB workloads,
Chris: along with fewer unexpected out-of-memory situations.
Wes: Ooh, I like that one. You know, some of us have a lot of tabs, and it gets dicey.
Chris: I, my, my bet is not everybody's going to see that 30%, but heck,
Chris: even a 10% is pretty great. Just by a better memory reclaim.
Chris: And Brent, I know you're going to love to see this one. Faster,
Chris: leaner swap in Linux now.
Brent: Delicious.
Chris: Are you guys still using the swap file?
Wes: Well, I have one. I don't know that it gets used a ton.
Chris: That's what I mean, really, is are you actually, uh, like when you install your
Chris: Linux box, are you deploying a swap file? Not are you actively using it necessarily, but...
Chris: I mean, any system that runs for a really long time generally swaps a few things out.
Wes: Yeah. Yeah, I have 3.7 gigs used, 12 gigs free in MySwap right now.
Brent: MySwap is zero currently, which is, I think, a little unusual,
Brent: but I use it heavily for hibernation.
Brent: Like, sometimes when I'm traveling with a laptop and stuff, I want to maintain,
Brent: as much battery power as possible depending on what I'm doing.
Brent: Maybe I'm going on a train at some point of the trip or something like that,
Brent: And so having hibernate available for me is quite valuable. So that's the main reason I have it.
Chris: Yeah. That's one of the reasons I have one too. Uh, I also find it better for
Chris: crash debugging and also, um,
Chris: Some of my systems only have 16 gigs of rewrite, so I do actually get the swap out.
Chris: So it's good to see that there is ongoing rewrite of the swap table infrastructure in Linux 7.2.
Chris: The improvements reduce the amount of static metadata Linux needs to keep around
Chris: for swap and improves how that metadata itself is managed.
Wes: Nice. Yeah, one example here. A system with a 1TB swap device can save roughly
Wes: 512 MB of RAM just from the reduced metadata overhead.
Wes: Which is, you know, nice. It'll add up over time.
Chris: That's a massive savings with how expensive RAM is right now.
Chris: That's no joke. So again, a system with one terabyte swap device,
Chris: which is a pretty big swap device.
Chris: will save roughly 512 megabytes of RAM just from the reduced metadata overhead.
Chris: Does anybody out there in the audience have a terabyte swap device?
Chris: Boost.Jupyter Broadcasting.
Brent: Yeah, right.
Chris: You've got to tell them. And why?
Brent: How'd you get there?
Chris: Mm-hmm, mm-hmm, mm-hmm. All right, we've got a couple other items.
Chris: There's one I really want to talk about. Is there a couple before we get to
Chris: the file system stuff that you want to talk about? I know we have faster pipes.
Wes: Yeah, we do have faster pipes. We covered a bit about this in 6.7.2.
Wes: There was basically going to be improvements to make piping things between processes
Wes: faster, kind of all around, especially on the right path.
Wes: And now we have a few numbers under memory pressure in particular.
Wes: Throughput is up 21 to 48% and right latency is down 17 to 33%.
Chris: Whoa, this is a, yeah.
Wes: Yeah, and I mean like pipes and, you know, FIFOs are used all over the place.
Chris: Yeah, yeah.
Brent: I find it so crazy how, you know, Linux isn't just a baby anymore,
Brent: and yet we're getting insane numbers of performance upgrades in seemingly every
Brent: single version. It's just bonkers.
Chris: It really is like, I wonder, like, was this all just there for the taking all
Chris: along? And you just, you work through it, I suppose.
Chris: Brent, I'd like to see you test this next one somehow. USB 4.0 stream.
Chris: So in 7.2, you now, two machines can send raw streams of data directly over
Chris: USB 4.0 or a Thunderbolt cable.
Brent: Well, who needs Ethernet anymore, I say?
Chris: Well, it is actually pretty fast. It is quite fast, actually.
Wes: It's neat, too, because before, you know, you could obviously do this and then,
Wes: like, stand up your own little local network or whatever and set up IPs and,
Wes: you know, get it going. You could do it.
Wes: But this, you just get, like, a slash dev slash TB stream zero device.
Wes: And you can just cat and read to write or DD or whatever, and it just shoots it across the USB cable.
Chris: It's going to be so fast. That's going to be, you know, because you don't have
Chris: the overhead of the protocol of Ethernet.
Brent: Nice. Oh, my goodness.
Wes: And it can also still coexist with normal Thunderbolt networking.
Wes: So it's not like it's the only thing you can do on the cable.
Chris: Okay. All right. Well, while we're talking about performance,
Chris: I think the thing that I'm probably the most excited about because it'll have
Chris: the biggest practical impact for all of us is ButterFS is getting faster in
Chris: this release. ButterFS is getting faster.
Chris: We talked about large folios in the last episode.
Wes: They're enabled by default now. Yeah. There's also experimental support for huge folios up to 2 megs.
Chris: But I think the big wins come from better direct I.O. performance,
Chris: faster sequential writes, and a new interface that exposes the raw checksums
Chris: to user space, so that way you can build tooling around that and all of that.
Wes: Yeah, including things like perhaps deduplication, Brad. Oh, hello.
Chris: So, you know, this is the beginning of an API, right? This is the beginning of an API.
Chris: But the direct I.O. performance and the sequential writes are resulting in real
Chris: performance improvements for ButterFS. Amazing.
Chris: I think we'll probably see Laravel benchmark that pretty soon,
Chris: and I'm going to keep an eye out for that. So I think that's probably the thing
Chris: in 7.2 I'm the most excited about.
Chris: So people upgrading soon on rolling releases, you know, do they?
Chris: I don't know. You probably have to do something to upgrade your ButterFS format. I don't know.
Chris: But it's something to keep an eye on. Something I'll probably play with, and I'll update you.
Brent: There's one here I'm excited about, inline storage encryption.
Brent: There's a new device mapper target called DMInlineCrypt.
Brent: And it's designed to take advantage of hardware that can perform inline block
Brent: device encryption rather than doing all of that encryption through the traditional DM crypt path.
Brent: Based partly on work that happened in Android, of all things,
Brent: and could eventually become a useful alternative to DMCrypt on hardware with
Brent: dedicated encryption support. You know how I like that encryption. You make sense on it.
Wes: On mobile, you probably do have the opportunity to have custom stuff and not
Wes: as much CPU as you might want.
Chris: And they have been tested in the fire. Android did not start super strong in
Chris: this area, and they've made a lot of progress.
Wes: And, hey, look, it turns out that having Linux on your phone actually pays dividends
Wes: that the whole kernel could use, proving out the entire model that we've been saying all along.
Chris: It's also worth pointing out, not that you guys don't know this,
Chris: but it's worth just reminding us all that these tools like this,
Chris: if they're being actively supported and contributed to by a large,
Chris: well-financed group of people,
Chris: it's probably something to consider just because you want tools in this category
Chris: that are being actively supported and maintained.
Chris: Um, there's a few other things in here, but the one that really jumped out to
Chris: me is faster proc. Like when we're talking about, this is really the performance kernel.
Wes: Yeah, it really is, right?
Chris: This is silly, but like, what? Faster slash proc? What?
Brent: I didn't know that was an issue.
Chris: Yeah.
Wes: Proc exposes a whole bunch of metadata and info about like the running kernel
Wes: and all that. And there's, um, there's a bunch of programs, especially command
Wes: line programs that want to go read that stuff to actually go.
Wes: Right. Cause it's a way that you don't have to have use this calls to try to
Wes: get the same information. you can get it right out of the file system.
Wes: It's super handy. It's standard Unix. Yeah, right?
Wes: So there's a bunch of stuff that uses it. And so if you make that faster,
Wes: you make all those things faster.
Chris: Yeah, absolutely. It's like Brent said. It's like we're just seeing these optimizations
Chris: in areas after all these years.
Brent: But who thinks Steven go down there and be like, huh, what can I do here?
Chris: A clanker? I don't know. Maybe a clanker? I don't know. Some autist,
Chris: I don't know. Maybe like somebody who's really passionate?
Wes: I don't know. I've got a few Pats and such rock I can tell you guys about.
Brent: There you go.
Chris: Uh-huh, uh-huh.
Wes: Well, we should probably mention we now have the safer open at command because
Wes: we have a sticker based on it.
Chris: Right.
Wes: Or we only open it if it's a regular file.
Chris: That's right. Only regular files here. Only regular files.
Wes: And this has shown up in a few different vulnerabilities. This prevents an application
Wes: from unexpectedly being redirected to something like a pipe or a device node
Wes: when what you want is like a text file.
Chris: Yeah. I think that's a good thing
Chris: to call out. There's so many things we could get into here with this.
Chris: It's like this kernel isn't a headliner banger feature. This is the kernel feature.
Chris: It's all these things from like CPU schedulers to GPU schedulers to file systems
Chris: to drivers and networking that have seen improvements.
Chris: Or just new little features like the fact that if you had Thunderbolt or USB
Chris: 4, you can now plug machines directly into them. Like that just kind of got a little bit better.
Chris: So it's, I mean, 7.2 is a really good release.
Chris: Is there any word if there's LTS talk around this one?
Wes: I don't know. That's a good question. I've seen discussion around a few possibilities
Wes: from this one, or maybe 7.3 or 7.4, perhaps.
Chris: Well, 7.3 would be a wild one to make an LTS.
Wes: There's a lot going on.
Chris: Let's just talk about this really quick, because it's an early development.
Chris: The merge window just opened last Sunday, obviously.
Chris: But there's a lot more scheduler work in here. A lot more scheduler work.
Wes: Yeah. Right now, there's some focus, particularly on how scheduling interacts
Wes: with C groups, because, of course, the control group mechanism is involved in
Wes: maintaining a lot of access to different resources, which impacts how you want to schedule things.
Wes: So instead of having to traverse a bunch of hierarchical run queues,
Wes: it's moving to a single run queue with new methods for distributing C group weights.
Wes: there's also a new c group mode knob um you can do smp up max concur tasks uh
Wes: concurs the new default and this is basically a mode you can choose so that you pick how,
Wes: how much cpu weight right like how much your share of the cpu is going to be
Wes: gets but divided between the different okay that seems great,
Wes: especially if you care a lot about you know scheduling things and especially
Wes: c groups are used a lot in our buddy containers.
Chris: Yes in our friend.
Wes: Yeah the hierarchical version had a lot of long-standing oddities,
Wes: including how nice, you know, the nice command priorities got handled across
Wes: groups. So that might improve from some of this work, too.
Chris: Oh, okay.
Wes: Yeah. Early testing on an old Sandy Bridge machine paired with an RX 580 showed
Wes: enough improvement that the
Wes: work has been pitched partly as a win for gaming on lower-end hardware.
Brent: Nice.
Wes: So you might not start thinking about gaming on lower-end hardware,
Wes: but that's where it ends up with the kernel sometimes.
Chris: Well, I mean, a lot of those handheld devices, those little built,
Chris: you know, purpose-built, run on Linux. Makes sense. Hmm.
Chris: Okay. Now this next one, I can't even believe. So I'm going to say it out loud,
Chris: but I still don't believe it. Okay.
Chris: In Linux 7.3, in speaking of schedulers, there appears to be up to a 90%,
Chris: I'm going to say that figure again, around 90% reduction in a nasty source of
Chris: x86 scheduling latency.
Chris: Here's what strikes me about this. Number one, 90% is a big number.
Chris: Number two, x86 is a big platform. This isn't ARM. This isn't like some sort
Chris: of maybe risk chip that nobody's using.
Chris: This is x86. What? A 90% reduction in a nasty source of scheduling latency?
Chris: Now, I realize this must be an edge case here or something, right?
Chris: Because this is massive.
Wes: Yeah, and part of it is just sort of how we've been doing things.
Wes: So, like, back to, you know, you have multiple cores and you have multiple caches.
Wes: And one of those things is the page cache, right? Sort of like how do you have
Wes: the files, what you've copied off the disk to memory for working with.
Wes: Well, those different page caches can get out of sync. And so you need to maintain
Wes: coherence across all of these things.
Wes: So you do what's called a TLB shootdown, which is where the one that figures
Wes: out that something needs to be invalidated sends out interprocessor interrupts
Wes: to all the other cores to say like, hey, you got to take this out of your cache
Wes: and go get it fresh because it's no longer valid.
Wes: Well, the calling CPU can wind up sitting with preemption disabled while waiting
Wes: for the remote CPUs to finish that operation.
Wes: And so ByteDance actually observed weights causing up to 16 milliseconds of
Wes: scheduling latency on a 16-core system just from this.
Wes: So Linux 7.3 changes that mechanism so CPUs can now be preempted while waiting in these situations.
Wes: And in ByteDance further testing, the P99 maximum preemption-disabled interval
Wes: fell to roughly 1.5 milliseconds, which the patch series described as producing
Wes: as much as a 90% latency reduction. So there's your 90% in the affected workload.
Wes: With one particular workload, which may not be yours, but it is aimed in particular
Wes: at latency-sensitive and real-time style workloads, which you know Linux is doing more and more of.
Wes: And it removes an architectural source of scheduler stalls that can affect ordinary machines, too.
Chris: So you heard Wes there, too. He mentioned this is patchwork from ByteDance,
Chris: and this is now merged for Linux 7.3 after 10 rounds of patch reviews.
Chris: But I don't think this is the only set of patches that ByteDance has committed
Chris: recently. It seems like their contribution to the kernel is kicking up. And is ByteDance...
Chris: Are they the TikTok company?
Wes: Yeah.
Chris: Right? The TikTok company? What?
Wes: I mean, you know, we've said it in the cliche way of, right,
Wes: like, I don't know, meta runs Linux servers or whatever. But I think there's
Wes: a real component where, like, if you're trying to do media distribution at scale,
Wes: you really push what computers can do.
Chris: Yeah.
Wes: And that means you might need to interface with the kernel. Whether you like
Wes: the content or not is up to you.
Chris: So this is the world we live in now. And, I mean, that's a serious contribution.
Chris: That's a real contribution.
Chris: Something we've talked about on and off for a little while now,
Chris: but really seems like it's coming together in Linux 7.3, is this subscheduler idea.
Wes: So cool.
Chris: So cool. So I think this could be one of the more consequential long-term changes
Chris: because you're going to see vendors take advantage of this.
Chris: Network vendors, hardware vendors. So they have now, I guess,
Chris: like a subscheduler system where you can build your own subscheduler for Linux.
Wes: Yeah. Yeah, so you have a root scheduler, and that can now give a C-group subtree,
Wes: back to the C-groups, to another scheduler, including with revocable access
Wes: to individual CPUs so you can control what it's scheduling over.
Wes: So in practical terms, you could imagine different scheduling policies for a
Wes: few different workloads on the same machine, right?
Wes: So maybe you're trying to do latency-sensitive audio capturing,
Wes: but you've also got a background job doing some transcription in bulk, right?
Chris: Or syncing, or whatever you just have running.
Wes: Or you're trying to do games, but you've got containers that are running that
Wes: are doing image processing at the same time.
Chris: As one does.
Wes: Or I could see this a lot, right? You might have, you're running a web server,
Wes: right? And you might want a particular scheduler for your database that's different
Wes: from how you want to run it for the Nginx reverse proxy.
Chris: Or, you know, to bring it to the desktop, my kid, right? Constantly,
Chris: he's got the Discord going with like 15 people in the call. He's got the game
Chris: going. He's got OBS recording and streaming to the call.
Chris: And, you know, and then you want the game to perform well at the same time.
Chris: And like that, that seriously taxes the system when you're doing all those things.
Chris: So you could see a world where there's a SteamOS scheduler, subscheduler.
Wes: And this sort of scales with how much investment you want to put,
Wes: right? Like with this SketchX stuff, that of course is built on being able to
Wes: write like BPF programs that can do scheduling, which is furthermore means you
Wes: can take a standard Linux kernel.
Wes: You don't have to build your own modular. You don't have to do potentially dangerous
Wes: and, you know, require lots of testing changes on general scheduler algorithm changes.
Wes: You get to implement your own, just, you know, adding it in from user space
Wes: and then get that attached to particular programs and workloads that you need to run.
Chris: Yeah.
Wes: That's powerful.
Chris: What other OS is doing that? All right. So, Brent, you know, how much would you pay?
Chris: We told you you're going to get performance updates to ButterFS and Linux 7.2.
Brent: Delicious.
Chris: But how much would you pay for a whole other round of large performance wins
Chris: in ButterFS and Linux 7.3?
Chris: Because, buddy, I'm going to give it to you for the price of free.
Brent: Yeah.
Chris: Another round of ButterFS optimizations will be coming in Linux 7.3,
Chris: and some of them are kind of a big deal.
Chris: I mean, I'm sure you went through this, Wes, but DirectIO, again, gets some attention.
Chris: ButterFS can now use IOMAP to bounce buffers instead of failing back to a buffered
Chris: I.O. Don't know what that means at all, but it seems good.
Wes: Yeah, well, it's, you know, buffered I.O. has buffers, right?
Chris: I mean, I know that. I know what a buffer is.
Wes: Well, and so, and DirectIO is unbuffered, right? And so now it's basically a
Wes: setup where in some situations, direct IO didn't really work.
Wes: You had to fall back to using the slower buffered stuff.
Wes: Now, in that same situation, Butterfest is internally using an IO map bounce
Wes: buffer of its own that lets it stay on the more direct IO path.
Chris: But gives like direct IO a little buff if it needs it?
Wes: I think so.
Chris: Huh. Okay, thank you. All right. See, now I get it.
Wes: And that's why the case that was sort of the study for this reportedly goes
Wes: from roughly 50% of theoretical throughput up to about 90%.
Brent: What?
Chris: I'm sorry?
Brent: That's massive again.
Wes: So a 45% increase.
Brent: Are they using the same math that we're using?
Chris: This is really something. I mean, just in the next couple releases,
Chris: there are so many performance wins.
Wes: Yeah, really. I mean, there's more removing an unnecessary one Jiffy delay in a non-SSD logging path.
Wes: That's like a 5x throughput improvement in one test. And then another is interesting.
Wes: it replaces the use of an x-array data structure with a local least recently
Wes: used lru list for inhibited extent buffers that one's a 3x improvement.
Chris: All right i mean it's funny that a three it says three to five x by the way
Chris: but it's funny that a 3x improvement seems kind of tame at this point i know just so great still,
Chris: ah i mean like where you at butter fs haters time to update time to update,
Chris: All right, there's several other things in here we could talk about.
Wes: Yeah, how do you like the name FailFS?
Chris: I love it.
Brent: Seems like one of our challenges is coming up.
Chris: I feel an episode already. I hear the name, and I'm already picturing the episode.
Wes: Yeah, it's great, right? So you know what you love about file systems?
Wes: They let you open and write to files and do stuff with files, right?
Chris: Definitely, very handy.
Wes: Yeah, well, FailFS is a file system where every possible operation fails.
Brent: What?
Wes: Okay.
Chris: All right, all right, cool.
Wes: So what's cool about this is it's a security idea, right? So what you do is
Wes: you open a file and you get a file descriptor, which is basically like a handle to the file, right?
Wes: And after that, like you've done the security checks, the kernel scene,
Wes: if you have permissions, all of that stuff has already happened.
Wes: And then you have a handle you can use to read and write from the file.
Wes: What FailFS lets you do for a security system perspective is you can,
Wes: if you know all of the files that something needs to access,
Wes: you pre-open them and then you launch something and you give it a FailFS file system.
Wes: And that means there's no way to ever escape its sandbox. All it can ever use
Wes: is whatever FDs you pass it when you spawn it.
Wes: So it's an idea to pre-restrict it. So it's not in a true root.
Wes: It's not stuck in some folder. It's not in a weird mount. It's just the only
Wes: file system it knows about is not the root file system at all. It's fail.fs.
Chris: That's so funny.
Chris: It's such a funny way to solve the problem.
Wes: It's interesting because it builds on a few, I think in 6.7.2 maybe,
Wes: we talked about null FS, which was something cleaning up sort of early pivot
Wes: routes, switch routes stuff, where you had like a, just a super minimal file
Wes: system that can be the root of the whole tree of file systems.
Chris: Right.
Wes: So fail FS is kind of a similar, like not really quite a, it's like more of
Wes: an abstract file system, but they play important roles in the kernel.
Chris: That's a good one.
Brent: We've been talking about file systems a lot this year because there's been a
Brent: lot of really great news. And I've got to say, FailFS seems like the easiest,
Brent: file system to implement. You just make a lot of mistakes for every single implementation of a file.
Chris: Easy testing.
Brent: Exactly.
Wes: In this case, if you don't implement a bug, it's itself a bug.
Chris: I see BPF is getting some new fancy BPF-MT-MISC. Is this a mount?
Wes: No. Okay, so it's bin format MISC, which is maybe you're familiar.
Wes: their bin format is what lets you basically register different types of binary
Wes: files. Like you can do this with an app image, right? Or you could do it with,
Wes: a wine or you do it with like a jar file to launch a Java application.
Wes: And so this is where the kernel looks at the magic bytes at the start of a file.
Wes: That helps it determine what that is. And then you can register executions or run times for that.
Chris: For BPF.
Wes: Well, so there's two pieces. There's actually a NixOS angle here.
Wes: Oh. Uh-huh, yeah. So right now it's all fixed, right? And maybe you know with
Wes: NixOS, you have to embed the dynamic library interpreter, the LD,
Wes: in basically it has to hard code that in all of the binaries it produces.
Wes: And this is where the patch elf stuff comes in and all that kind of stuff.
Wes: And it means you can't really relocate that stuff. It needs this fixed, hard-coded path.
Wes: What's coming in 7.3 is a bunch of that's being made more dynamic.
Wes: And in particular, you could implement an algorithm that was like,
Wes: hey, wherever I am, go look up in like dot dot lib from me and just find the
Wes: interpreter right there.
Chris: Oh, that would be really nice.
Wes: So then you could start shipping the interpreter that you need to run sort of
Wes: hermetically sealed in with the rest of your NixOS closure.
Chris: Oh, boy.
Wes: And then, so that's one level of dynamic stuff. And then on top of that,
Wes: you're now also going to be able to attach BPF programs to do that role of figuring
Wes: out what interpreter to run.
Chris: So you could make your own interpreter?
Wes: Yeah, you can have your own interpreter and you can choose very dynamically
Wes: at runtime which interpreter is selected.
Chris: Wow. Dang.
Wes: It's pretty early and like this is going to be this is like the kernel opens
Wes: the features. It doesn't actually add the stuff you do. So this will be downstream
Wes: dependent on distros and users and stuff to actually register the interpreters
Wes: and like write them and all of that.
Wes: But I certainly expect we'll see an XOS at least.
Chris: This is the stuff that makes the Linux kernel the king.
Brent: Oh, yeah.
Chris: It is the king. And the other kernels, they can't even with this stuff. It's really something.
Chris: I suppose in order for the king to maintain its reign, it must shed some older
Chris: projects from time to time.
Chris: And in the upcoming Linux 7.3, not 7.2, those were just released,
Chris: but 7.3, which is coming out in October, 32-bit ARM support has been deprecated.
Brent: Whoa.
Chris: Yeah. So I believe it will survive through 7.3, but it's officially deprecated,
Chris: which doesn't mean it's gone, but it means it's now marked deprecated. So there's that.
Wes: One thing we maybe should mention, too, is, which I don't know if it will be
Wes: fixed in 7.3 or not, but presumably it would be, there's a NTFS vulnerability going around right now.
Wes: Essentially, not the newer NTFS, but the older Paragon NTFS,
Wes: this seems to be live now.
Wes: Essentially, you can set basically a version of Unix security model permissions,
Wes: you know, CHMOD stuff on NTFS files.
Wes: And that includes the set UID bit.
Wes: And the old driver in a lot of configurations for desktops will mount,
Wes: will like auto mount an NTFS thumb drive with that set.
Brent: Oh, no.
Wes: Yeah, so you can have a specially crafted NTFS thumb drive that gets you root just by plugging it in.
Brent: Well, I mean, that could be handy sometimes.
Chris: You know, that's really something. But I think the largest change,
Chris: maybe, it's not a feature that's being added to Linux 7.3.
Chris: A third to a half of the patches look like they are AI-generated low-priority
Chris: fixes that are queued up in Linus' next-next patch set.
Chris: And going through that, they say, quote, we are completely overwhelmed.
Wes: Yeah, these are the networking maintainers.
Chris: Yeah, the networking maintainers say we're completely overwhelmed.
Chris: Their counter move is AI reviewing AI with meta-funding a multi-model review
Chris: for every networking patch.
Chris: But the practical result seems to be really just a lot of trimming from the
Chris: kernel of older network drivers that are not really maintained because they're
Chris: getting poked at by clankers.
Chris: And so they have a whole AI reviewing, AI submissions process now,
Chris: thanks to just a ton of compute contributed by Meta.
Wes: Yeah, they describe it as managing patchwork, automating common process complaints,
Wes: editing commit messages, and maybe applying patches, which already got reviewed
Wes: by tags from people we trust. So they're experimenting actively.
Chris: Yeah, I suppose they have to with the volume they're going to get,
Chris: right? Because the Linux kernel has to be the one everybody thinks of when they
Chris: all of a sudden I can build software.
Chris: I'm going to go to the Linux kernel, right? A lot of people think that, but it's interesting.
Chris: that 7.2 was widely impacted by AI-assisted development and 7.3 is both in contributions
Chris: and again in pull requests and things like that.
Chris: So it's essentially continuing to escalate. And so they're responding by adding
Chris: more compute on their side. I think that's going to be a story worth following.
Wes: We also saw Andrew Morton talking about using Gemini to not write the code,
Wes: but write some of his memory management subsystem patch summaries this cycle.
Chris: Look at that.
Brent: Well, for years, we've had the project of, well, thinking about first,
Brent: and then also just tightening up our production infrastructure behind the shows.
Brent: Big dreams. Some of it's actually happening recently. And Chris,
Brent: Dropbox has been a big part of that, but I know we've had issues with cloud
Brent: syncing in the last several years.
Brent: And nobody's excited about cloud stuff anymore, and you're making some changes.
Brent: What did you replace?
Chris: Well, so, you know, I'm a fan of SyncThing. I want to be clear, I've used it for years.
Chris: When it first launched, I started using it. I was initially with BitTorrent
Chris: Sync, and I was sad when that went away, but I happily Those were the days.
Chris: It was. It was great. But SyncThing has served me well.
Chris: And I've mostly used it for, like, back-end large file transfer media,
Chris: stuff like that. And I've held on to Dropbox because we got this sweet 15-terabyte deal.
Brent: Yeah, yeah, yeah.
Chris: It's crazy cheap. And how do you say no to 15 terabytes these days?
Chris: But I hate Dropbox client. I don't really like Dropbox the company.
Chris: So I kind of compensated and got by for a while by using the Mistral open source
Chris: sync client that was really lightweight.
Chris: You could run it headless. You could declaratively configure it.
Chris: And it gave me Dropbox like back in the OG days of Dropbox again.
Chris: And it just ran headless in the background and synced my damn files.
Wes: That's all you want.
Chris: And used their storage, right? And that worked well for a couple of years.
Chris: And then Mistral announced that they're shutting down. And I thought,
Chris: okay, I'm going to have to migrate soon.
Wes: Right. So either you have to go back to the standard clients or...
Chris: And then, as sort of like to really poke me to get going, I don't know,
Chris: like an old machine joined the Dropbox. I have my suspicions. Yeah.
Chris: All of the files got a sync conflict in my Dropbox. No.
Wes: That doesn't sound good.
Chris: A little bug in how the exclusion folders work with Mistral,
Chris: and it started syncing to all my systems. Like, you know, terabytes of data.
Chris: Terabytes of data. So all my hard drives started filling up, and I realized it.
Chris: So at first I shut down the sync client on my workstation upstairs because I'm sitting there.
Chris: And then it dawns on me, every other computer is also doing this same thing right now.
Brent: Oh, my goodness.
Chris: So then it's like a mad dash before it fills up their hard.
Brent: Drive right.
Chris: Or whatever it.
Brent: Might been there been there.
Chris: Yeah yeah that's a nightmare you've been looking at sync stuff this or the last couple of weeks too.
Brent: Yeah i uh i guess a similar problem actually i've been,
Brent: trusting cloud syncing less and thought geez with these new mesh networks certainly
Brent: there's something better we could do right and not have to rely about it with
Brent: a always on uh system in the in the cloud.
Brent: So you pushed me towards sync thing. I've complained for years that I can never wrap my head around,
Brent: it how it works and how to configure it and stuff like that but somehow on my
Brent: third try around i got over that and it's actually quite simple.
Chris: I think that i don't know what the issue was well the biggest head shift right
Chris: is um like with next cloud sync or dropbox sync it's user login and then you
Chris: have these folders and you sync these folders down,
Chris: and with sync thing there's no user right i mean you can pass protect the dashboard
Chris: but it's these keys that you exchange and then they discover themselves and
Chris: they sync and you share these keys and that's how you set it up,
Chris: which is actually great because it's accountless which means anybody that has
Chris: sync thing can just grab that you can share that key with them and they can
Chris: sync they don't have to go create an account anywhere,
Chris: but it's a total flip of the traditional file sync model
Chris: where you have an account you log in and it just syncs down your files but like
Chris: i think both you and i have found recently that next cloud isn't working for
Chris: us that might have been the direction i would have gone on for these files because
Chris: I do really like having a web interface. I think that's really nice.
Brent: Yeah, I share a bunch of stuff with other people too.
Chris: Yeah, sharing is also a big one.
Brent: Yeah, it's a tricky one. The other thing that got me with SyncThing,
Brent: I think, was the configs for each client.
Brent: You have to kind of put your head into different perspectives when you're configuring
Brent: stuff. And also there's this concept of like,
Brent: I forget what they call it now, maybe Wes, you'll remember, but of like the
Brent: quote unquote cloud, but it's really just the amalgamation of all the files
Brent: of all of the devices into this
Brent: one like virtual file system almost that you can think of it that way.
Brent: And you get to pick and choose which folders you bring down into this particular,
Brent: device. So it's interesting, very powerful when you start learning the config.
Chris: It just sort of slips into the background too. It does have a web UI.
Chris: There are some desktop clients, but it just sort of slips into the background.
Chris: So you're not really thinking about it. It's just like you just have these files
Chris: that persist across all your machines. And it just sort of does it.
Wes: It's best when it's at the layer of infrastructure magic.
Chris: So I had this problem happening, right, where all my systems are getting blasted
Chris: with files to the point where it was going to affect production.
Chris: So I shut it all down and figured I got to redo this. And I wanted to replace
Chris: it at that point with SyncThing.
Chris: I just felt like it's been the most reliable.
Chris: and it was probably my safest bet now that I'm also not using Nextcloud. Sort of the way to go.
Wes: That's a big change though, right? Because you kind of get, it's a very different architecture.
Wes: And in particular, if you don't have a cloud relay, you're going to need a way
Wes: for these things to talk to each other.
Chris: That is a thing. That is a thing. I will say, I think, at first I thought it
Chris: was going to be a downgrade because of that.
Chris: Because I don't have a web UI, I thought this is not going to be as functional.
Chris: But what I realized is that with SyncThing, because it has an API,
Chris: There's actually a lot more visibility into it than I have with Dropbox.
Chris: So with sync thing, I can see which peers are connected, what folders are caught
Chris: up, what's still outstanding in the sync.
Chris: And I can see, does this particular host have that file or not?
Chris: And I can expose that in a way that an agent can read or a script can check
Chris: or Prometheus endpoint can monitor.
Chris: And so then it starts to become a lot more like kind of like this black box
Chris: of sync and like this production monitor thing.
Chris: which is even better than i had with dropbox so then it becomes like oh i'm
Chris: not just replacing dropbox i'm like improving the functionality here so i leaned in on that,
Chris: i i really i wanted to lean in on making sure my hermes agents,
Chris: could understand where files were at and where they're needed to be reviewed
Chris: because it's something my hermes agents can do because i put this on that system
Chris: as well they can write a file to a folder for them and then they know when the
Chris: workstation i'm at has receive that file. That's really powerful.
Brent: Yeah, yeah, yeah.
Wes: Especially when you want, you really need to be, you know, the number of times
Wes: where we've tried to sync something at the studio and we've assumed that it
Wes: synced and then we discover later after we've left the studio that in fact it
Wes: did not fully sync or like one file out of the seven we needed wasn't there or whatever.
Chris: Yeah.
Wes: You just don't want that. Yeah.
Chris: It's come up. I mean, it's come up recently too and, you know,
Chris: you need to be able to change these things.
Chris: You need to be able to make it flexible too and all of that and Dropbox didn't
Chris: really meet that requirement.
Chris: It didn't really do it for me. So that part feels like a huge upgrade now.
Chris: So I think the trickier part, really, was all of the systems are different.
Chris: I mean, I have my NIC systems, but then my Aurora host here is running,
Chris: you know, UBlue, SilverBlue.
Chris: The soundboard machine over here is an old Mac, you know, so that's a totally
Chris: different system. So that part...
Wes: So it's not one platform that you have to build across.
Chris: Yeah.
Wes: You got to make this work across a disparate set of platforms with different hardware.
Chris: Yeah.
Brent: Did that mean you also had, like,
Brent: like disparate folder structures across those systems as well like how did you.
Chris: Implement this in a way that was quote-unquote smooth yes it was you know like
Chris: especially like on the mac oh,
Chris: yeah like slash users slash username like the whole path is totally different
Chris: but then on some of my systems i have those scary raids and so i put the sinks
Chris: on the scary raid several of them and then and then like the laptops they're
Chris: just in the home folder because i don't have more than one hard drive and a laptop usually usually
Chris: so it's just in my home folder so like literally And then you have my Hermes agent's host,
Chris: which stores that in a completely different location too. So literally all my hosts.
Wes: That guy's probably the only one who knows where all the paths are.
Chris: Well, that's the thing. So I focused on the main folders that I use,
Chris: not all of them, just the main ones I use today.
Chris: And then I had the agent figure out where they all need to go and which systems
Chris: we could just hot swap and take over the existing Dropbox location.
Wes: Nice, right. Because you were working from like a unified pre-model mostly.
Chris: Why sync everything again if I don't have to?
Wes: Yeah.
Chris: But that could also go very wrong on some systems. So on some systems,
Chris: I had to make the call not to do that. And I had to create a new folder pattern.
Wes: Start from scratch.
Chris: Yeah, yeah, yeah.
Wes: Okay, well, that sounds like maybe you ran into some issues.
Wes: Some of this sounds like you've handled it. You've obviously been using the
Wes: same thing for years. But I'm wondering, did anything go wrong?
Wes: Did you have any scary moments? What was the process like?
Chris: I think the scariest moment was I forgot to include the Mac.
Chris: I don't think about the Mac.
Wes: But you need a lot of soundboard files.
Chris: That's the soundboard files. Yeah, exactly. said i was like oh right but what
Chris: is really fantastic with this setup is everything's documented all the endpoints
Chris: i mean the clanker can even connect in and read the database yeah
Chris: so it is really straightforward for for just you go in there and say deploy it on a new host,
Chris: and it uh i have some guidance in there like it caught okay well this folder
Chris: is your backups folder that's 20 gigabytes do you want to sync that to the soundboard
Chris: machine i do not i do not want to sync that so that was kind of nice so i kind of i have,
Chris: i'm really only dealing with 10 folders that i actively use and i'm considering
Chris: just keeping everything else on dropbox removing all the dropbox clients and
Chris: then using our clone to mount it when i do need something from the archive,
Chris: because i'm paying such a cheap price it's like 50 a year or something for 15
Chris: i got like this grandfather deal it's a cloud backup i mean so nuts for 15 terabytes
Chris: of cloud backup So our clone could be a good way to interface with that.
Brent: Can I get access?
Chris: So the Mac was tricky.
Chris: Yeah, right?
Brent: Yeah.
Chris: The Mac was the scary part because I almost completely forgot it.
Chris: I just totally blanked it out.
Brent: Did it, was it smooth for you going cross-platform?
Chris: It wasn't too bad. Yeah, I mean, you know, the nice thing is you got Homebrew
Chris: on the Mac. You can use that.
Chris: I think Aurora. So Aurora is a, like I said, it's a silver blue,
Chris: uBluehost, and I went Podman.
Chris: I just did a Podman container on that, a rootless Podman container.
Chris: And I don't think Syncthing cares.
Wes: And then I assume it's just riding back on the back of your existing mesh infrastructure.
Chris: Right?
Wes: So a container plus mesh, and there you go.
Chris: It's really good. I'm really liking it. I find it to be such an upgrade,
Chris: and I thought it was going to be a downgrade. Screw that Dropbox.
Brent: Are you using Dropbox and sync thing looking at the same folders?
Brent: Because I would assume that's going to cause you some issues.
Chris: Yeah, yeah, yeah, yeah. No, I had to be careful about that. But it took me a
Chris: second to realize I had to shut it down on one of my hosts.
Brent: That's all it takes.
Chris: But when I scrambled, I didn't just stop this thing. I'm like,
Chris: all right, disable the service, rebuild. This thing's gone.
Chris: I did not want. So I just removed the client.
Chris: So I don't have to worry about that except for I think it was the Mac.
Chris: In fact, I think Dropbox is actually running right now.
Chris: I even disabled it from starting, and it still starts. I hate macOS so much.
Chris: so yeah it's still this it's it's such a pain in the butt and i don't think
Chris: about the mac so the mac is an issue for sure i'm just gonna uninstall it after
Chris: the show off of here and i don't know i don't really need to access the archive
Chris: from from this thing one you know what i realized.
Wes: I think i got our clone on the mac.
Chris: We could put linux on this i've been waiting to replace this with a linux box
Chris: right an actual but we could just put linux on this because it's an intel mac.
Wes: Yeah.
Brent: There's the whole soundboard thing. Isn't that the reason?
Chris: Yeah, we have to, no, we have to build a soundboard.
Wes: But we need to do that anyway.
Chris: We have to do that anyway.
Brent: We've been saying it for like, what, 10 years?
Chris: Well, like, what's the point of starting the project if you don't have any hardware to run it on?
Brent: Fair enough, fair enough. Well, let's do that after the show.
Chris: Right, right.
Brent: Hmm.
Chris: So that's been an issue. You know, but being able to run it in Podman worked
Chris: out great. And then it's just installed natively on the Nix systems.
Brent: So are you managing your configs through Nix as well?
Chris: Yeah, for those hosts. Yeah.
Brent: Yeah, for all those hosts.
Chris: Everything.
Brent: That's how I started doing it. It's so nice. I even...
Chris: It is very nice.
Brent: Yeah.
Wes: And it's positive pressure to use NixOS for more and more things.
Brent: I would... One recommendation, if you're going to throw, like,
Brent: a mobile device in the mix,
Brent: which I don't know if you want to at this point, but I found it really nice
Brent: to have the clanker just, like, make a config file for the Android system and just suck it in.
Chris: You can import it?
Brent: Yeah, you can import them pretty nicely. And that keeps you from like having
Brent: to thumb all the config options and stuff on the phone.
Brent: So that would be one of my recommendations if you get there.
Wes: I wonder, presumably you could
Wes: also maybe get some port forwards going on your Android and have it API.
Chris: You're right. Or just it'd be on the mesh too.
Wes: Yeah, right.
Chris: Yeah. So I think,
Chris: if you do dabble in the clankers, I really want to make a strong recommendation
Chris: that if you need to solve a sync problem, you look at sync thing.
Chris: because my favorite part of this is I built a wrapper called Hermes sync thing.
Chris: And so the agents are not doing like raw credential passes over the network
Chris: and like looking things up in the LLM every single time.
Chris: So the agents aren't using like the raw credentials and connecting to the API
Chris: without instruction every single time.
Chris: And the LLM is just kind of figuring out how to do it every time when I want
Chris: to check on the status of stuff. I've created this little wrapper and this wrapper,
Chris: automatically has all the presets to go see if all the hosts is connected,
Chris: if the folders are caught up, what might be outstanding from any individual hosts.
Chris: You know, like, for example, what does Aurora still need before we can go on
Chris: air? Because we've got to get the clip synced up.
Wes: I like that.
Chris: And I can ask it, did this file reach this host? And all the clinker is doing
Chris: in the background is calling this Hermes sync thing.
Wes: And that's a wrapper that just has the endpoints you want.
Chris: Yes, predefined setup. And so it's not, and the advantage there,
Chris: it isn't creating the workflow every single time. So you're saving tokens,
Chris: you're reducing the impact of hallucination because it's fairly straightforward.
Wes: You can do like a multi-stage, right? So it's like you have a skill describing
Wes: this, how to work with it, and telling it how to use your wrapper.
Wes: And then the wrapper can embed all of the implicit knowledge of like how to
Wes: get those core tasks done.
Chris: And the wrapper can query the destination machine's own sync thing database.
Wes: Nice.
Chris: And see, okay, is it present? Is it valid? Is it synced? Was it ignored?
Chris: Was it not deleted? Whatever. So like you get beyond just proof of delivery,
Chris: you can get proof of removal and all of that.
Chris: And it's just using the wrapper every single time, which I could go down on
Chris: the command line and run myself if I wanted to.
Chris: But the nice thing is, is when you do something like that in that wrapper is
Chris: you can use a free model, a cheaper model.
Chris: It doesn't have to be like a big, you know, frontier class model.
Chris: It can be an open source, even eventually self-hosted model,
Chris: but it can execute these little queries for me and tell me if everything synced.
Wes: There's no debugging, there's no code writing, it's just column tools.
Chris: Yeah, it's pretty great.
Wes: So I was going to ask if you were going to try and come up with a web interface
Wes: replacement, but it sounds like for the most part, this is good enough.
Chris: Yeah, I still toyed with the idea of putting some sort of web file manager in
Chris: front of the sync thing directly.
Wes: Just for like upload-download or something like that.
Chris: Yeah, or this one-off machine.
Wes: Right.
Chris: Or mobile, that could be another way to do it on mobile. I don't know what sort
Chris: of state of the art there is.
Chris: Like, I don't think it's PHP file manager anymore. So if anybody has recommendations
Chris: for that, I would definitely like them. Because I do think that could be useful,
Chris: but I don't think it's as necessary as this. It's been pretty nice.
Wes: So are you done? I mean, is there anything left that you need to build in this
Wes: thing besides the web interface?
Wes: Are you happy with the deployment? Is it on all your machines?
Chris: Yeah, everything's deployed. It's gotten really nice. You know,
Chris: I think the trick was is deploying from one central machine and just having
Chris: it reach out over SSH and just blast it onto everything with their various esoteric setups.
Chris: I think the missing piece still, like I mentioned might be the R clone mount.
Wes: Right.
Chris: And just coming up with a better way to take advantage of that 15 terabytes while I have it.
Chris: Not sure. But I think there's, that's the work still to be done.
Chris: There's no errors. Everything seems to be sinking.
Chris: And then I do have a couple like this machine. I got to go clean up Dropbox
Chris: on. Same thing is just, it feels like this,
Chris: a sleeper of a project that we don't talk about, that we take for granted.
Brent: I think it's just assumed that we've all implemented it.
Brent: Like that's the feeling I get is that it's been around forever.
Brent: It's been really stable for a long time.
Brent: It's just kind of, it's like SSH. It's just there when you need it.
Brent: But for anybody who hasn't tried it, you have any suggestions or tips or pitfalls
Brent: to avoid or anything like that?
Chris: And if you have SyncThing, do yourself a favor. Seriously, listen to Chris's hot tip here.
Chris: Download OpenCode. Use the free aux model that's in there. Use any of the free
Chris: models. Point it at your URL for your SyncThing and have it tell you what this API can do.
Chris: Because this is where it really exceeds what Dropbox can do for you.
Chris: And it becomes a complete system that you have observability over that doesn't
Chris: have a cloud, doesn't have any big tech login.
Chris: And I think so many times we deploy things like Home Assistant,
Chris: Plex, Jellyfin, and SyncThing.
Chris: And then a lot of us, and I'm definitely guilty of this for a long time,
Chris: just ignore that there is this beautiful API that allows us to do everything
Chris: we could do through the UI with our Clanker or OpenCode or whatever through this API.
Chris: And so you can start writing little scripts or automations or check on the sync status of things.
Wes: Think of the poor developers who implemented these APIs. They want you to call them.
Chris: They do.
Wes: Don't let them atrophy just there on the side.
Chris: Take advantage of those APIs. Because I stopped thinking of sync thing as just
Chris: this thing I self-host and it moves my files.
Chris: And now a full system I can manage.
Wes: It's a sync architecture.
Chris: It's a framework.
Brent: Nice.
Chris: That's what it is. That's what it is. And it's way better than any stuff you
Chris: could get with the commercial stuff. And really it's taking advantage of that API to its fullest.
Wes: And you can craft it to fit in with whatever you need it to be.
Brent: Well, as usual, we got a little feedback this week. Alex sends in a little note.
Brent: Hey, Chris, Wes, and Brent.
Chris: I'm Alex.
Brent: I'm Alex. That's A-L-E-K-S for, you know, that's a much cooler way to do it. I'm Alex.
Chris: Alex with a K?
Chris: I don't know, an X is pretty cool, dude.
Brent: Well, that's true.
Chris: X is pretty cool. K is different. I like it because it's unique.
Brent: Yeah, yeah, yeah.
Chris: I mean, it's pretty rare to get an X in your name.
Brent: So Alex with a K says, I'm a solar developer working on Local Comet,
Brent: an open source Windows-first local AI app built with Llama CPP, Tori, and Rust.
Chris: All right. Oh, you don't know.
Brent: It gives people a simple desktop way to run local models with more privacy and
Brent: control without depending on a cloud service.
Brent: I'm actively developing it and would be grateful if you'd consider mentioning
Brent: this on this here podcast and sends a little repository.
Chris: Well, this is right up our alley.
Wes: Yeah, we got Python, TypeScripts, Felt, and yes, Rust. Neat,
Wes: a private local first Windows AI assistant.
Chris: Yeah, and you don't see Windows getting a lot of love in this camera.
Wes: That's what I was going to say. I'm not about to go make myself an EXE,
Wes: but it's clearly out there.
Wes: And the people, I mean, if we're going to be using these models to help us with
Wes: our sync thing, I mean, as we already talked about on the show,
Wes: do you want to be touching Windows or do you want your model to do it?
Chris: I mean, honestly, why not, right? And then there's probably somebody out there
Chris: listening to this show that has WSL and has all this running and then they use
Chris: Windows MCP and all of that.
Chris: But if you're just getting started, something like Local Comet could actually
Chris: probably help you get WSL up and going. I mean, I don't know what these things
Chris: can do on a Windows platform, but it is really nice to see it.
Wes: And maybe it's an easy way, like if you have a Windows gaming machine that does
Wes: have a GPU, then maybe it's an easy way to go experiment with some local models too.
Chris: Yeah, I bet a lot of Windows, you know, that's a great point.
Wes: Don't lock them away in the Windows world. You can still get to play with the fun tech.
Chris: They've invested in those GPUs after all, right?
Wes: That's right.
Chris: So they might as well use them. Yeah. Local Comet. So there you go.
Chris: We'll put a link to that in the show notes. If you got a Windows box with a
Chris: Goopoo and you want to play around with it, you'll find it at linuxunplugged.com slash 681.
Chris: Oh, our first boost comes from the one, the only Greg, the lawyer.
Chris: He was in our mumble room for a little bit earlier today. He's often on the
Chris: live streams and frequently a big supporter of the show.
Chris: And he comes in with a baller boost of 33,333 Satoshis.
Chris: And Greg says, inspired by the release of Oma Archie Atra or Quattro,
Chris: I have been clanking my own Nix OS Senate.
Brent: Wait, wait, wait. I got to stop you there. clanking as a.
Chris: Verb i've.
Brent: Not seen till now i like it.
Chris: I do like it uh i've converted from kde to hyperland and quickshell nice making
Chris: my own desktop so much more fun oh,
Chris: this is i love seeing reports like this greg um we are entering a new era of customization on linux,
Chris: and uh hyperland is just so far it's so right i'll speaking of oma archie and
Chris: hyperland um an oma archie foundation of types,
Chris: was launched with eight million dollars of backing by jack dorsey the ceo of
Chris: dell and many others whoa and um one of their first expenses as a foundation
Chris: if you can call that i'm not sure,
Chris: is they're backing vaxery the hyperland developer for the next three to five years.
Wes: Oh that's that's.
Chris: Incredible that's huge yeah at least the next couple years i mean i don't know
Chris: how much vaxery is going to take as a paycheck but um that's massive so he can
Chris: focus on the hyperland desktop which i think is going to be really good.
Wes: A quick shell i was looking like quite the yes toolkit kit as well so i put
Wes: a show note a link in the show notes to a great lwn write-up of that recently.
Chris: Very good thank you greg appreciate hearing from you appreciate the support too.
Wes: Tomato boots in with 22 222 sats,
Wes: Catching up with the missed episodes from this summer. Loved the backwards episode.
Wes: And how it broke your brains.
Wes: Regarding virtual machines in the kernel, I'll just lean into my recent NetBSD
Wes: trolling here and point out that kernel already has a Lua VM in it.
Wes: It's honestly pretty fun.
Chris: That is fun. That does sound fun.
Wes: Great show on entropy and the cold card mess. I tried looking into their firmware
Wes: before moving my BTC to it and found the whole thing overly complicated and
Wes: confusing, so I decided I didn't trust it.
Chris: Nice dodge.
Wes: Despite having bought one, my coins are still on a Sparrow-created wallet on an offline machine.
Chris: Yep, turned out to be a better call.
Wes: Wish I'd shared my discomfort publicly, and I hope all the JB listeners got
Wes: their coins out in time. Especially in this new era of clanker-assisted hacking,
Wes: those of us in free software who have the expertise to autocode should really
Wes: speak up if something smells funny.
Chris: Yeah, there's something to think about there.
Brent: That's a great point.
Chris: Definitely worth checking your own stuff, too. And at least checking the stuff
Chris: you're going to use, even if you don't go file a clanker report.
Chris: Although, if you find something serious, you probably should consider it.
Chris: Yeah. Thank you, sir. Appreciate that boost.
Brent: Turt Ferguson also sent in a row of McDuck's 22,222 sets.
Brent: I enjoy the peek into the future of IT defense, but you did not answer a key question that I have.
Brent: Do you think AI security tools are actually useful for regular Linux admins
Brent: yet, or are they still too unreliable to trust?
Chris: I think that depends, maybe, because you experimented with a meta model that
Chris: did hallucinate some of the reporting, which maybe would be like,
Chris: okay, that's an example of when you can't fully trust.
Wes: Yeah, I think it's kind of like a lot of things, you know, defense in depth.
Wes: So you probably want to have, that's where you like a diversity of models or
Wes: harnesses or processes so that you do have some way to have a,
Wes: like, you know, grounding is very important here.
Wes: And so if you have a way that you can actually do some fact checks or you have
Wes: a council of several models, each doing it with different unrelated contexts,
Wes: things like that to try to be able to figure out when that's happening.
Chris: I think the biggest barrier right now to getting actually good results out of
Chris: these tools would be paying the cost for the higher-end uncensored models that
Chris: you've got to pay kind of per token for.
Chris: So that's the gating factor right now, right? For them to do anything more advanced, really, I think.
Wes: And that, yeah, that kind of pattern, too. You can have a grunt model and then
Wes: another model that sort of checks on things, does its due diligence,
Wes: and has a little more intelligence but doesn't have to do all of the original work.
Wes: Nathan from Brissy Boosin, 4,102 cents.
Wes: Come on, Lions. Hat trick time.
Chris: Yeah, that probably works. You're getting a few people thinking about it.
Chris: That puts it in a good direction, I think.
Chris: All right. Thank you, Nathan. Appreciate you. Right, you want to take Monty there?
Brent: Yeah, Monty sent in a row of ducks.
Chris: Ah.
Brent: As for the Frigate system overview, I have been running Frigate for several
Brent: years, and I haven't touched the config in a very long while.
Brent: So very interested in some of the newer features, including how to drain your own detections.
Chris: Yeah. And Friggin has recently added more detection capabilities in the freer
Chris: stuff. That's something I'll play around with, you know, and maybe we'll do
Chris: an updated segment. I'll dig into that.
Chris: One of the things I have recently done is I've expanded beyond person identification
Chris: to animal identification. Because again, keep an eye out for coyotes and yeah,
Chris: the birds and stuff like that. And that's been interesting.
Chris: So, all right. Thank you, Monty. That's good feedback.
Wes: Quick interruption.
Chris: Yes, sir.
Wes: I think you'll love this. You should go pull up our live chat. Our Matrix.
Brent: I'm going over there right now.
Wes: Because check out what Greg the lawyer just did.
Chris: While I was listening about SingThing, I clanked a QuickShell SingThing widget for my system.
Brent: That's so great.
Chris: I clanked it, he says.
Wes: And look at this lovely little widget, right? It shows devices connected.
Wes: It's got some folders there. Web UI rescan, refresh stop buttons.
Chris: QuickShell is so cool.
Wes: And clearly it is quick.
Chris: Yeah. And it's not going to break with Hyperland desktop update,
Chris: which is nice, too. So you can build that and it's going to work for a while.
Chris: Well done, Greg. That's why you dropped out of the mumble room.
Chris: You had to go work on that. Plank that up.
Chris: I'll take Thomas. Thomas English comes in with 16,345 Satoshis.
Chris: What are the best methods to get my agent to do my entire NixOS setup for me? Because I am clueless.
Chris: I've only ever Vive slopped Linux servers together, but going to take the plunge
Chris: and smash out this Windows box.
Chris: NixOS for me. Any tips for a sailor that can't swim? I've never seen a line of code in my life.
Chris: Agent first NixOS setup. What say you? I'm going to paste in your YoloPie and close my eyes.
Brent: Oh, my goodness.
Chris: I'm going to paste it into YoloPie.
Wes: It's like a prompt request via Boost.
Brent: It sure is.
Chris: I mean, you could, I mean, I don't know if you'd want to have it,
Chris: you could point a clinker at Hypervibe.
Wes: I was going to say, and we've had some, the whole collection of listeners who
Wes: have shared in past show notes.
Wes: So you could perhaps the first prompt is go get it to harvest some links from that.
Brent: Yeah.
Wes: And then from there, it could go build out a config for you.
Chris: I think that's a good starting spot. I would, I'd start probably in a web chat
Chris: with one of the free models to just sort of build out a plan.
Chris: talk about, you know, kind of ramble about your intentions, build out a plan, link it to Hypervibe.
Chris: I think there's a re-spin of Fedora that looks really good. You can even point
Chris: it at Oma Archie and see if there's some, you know, design tips you want to
Chris: pull from there and application ideas because it can look at all that stuff
Chris: and sort of build you a master list.
Chris: And then you could have it from there, produce a master prompt that you then
Chris: put into the final clanker on your machine that does the work.
Chris: And you probably get pretty good results with that.
Wes: Yeah. And, you know, the main kind of things too is once you get like the installing
Wes: Nix OS part It can be just pretty much like installing a normal,
Wes: any old Linux distribution.
Wes: It's just getting the, once you have it running and you have a way for a clinker
Wes: to control it, then you're off to the races.
Chris: You are. You are indeed.
Brent: I think I would recommend a very different approach, actually.
Brent: I think you should definitely use a clinker to help you.
Brent: But point it at some of the documentation and then just have it almost like
Brent: interview you about what you want the system to do.
Brent: what kind of things you want in it, and have it explain what it's doing.
Brent: I think the long-term benefit of understanding what's happening under the hood
Brent: for NixOS is really good, even though you don't have to worry about the implementation
Brent: details, like all the syntax and stuff like that. It's really good to understand what's possible.
Brent: And I think starting with some of the documentation, or at least projects that
Brent: are well-regarded, is a good way to go.
Chris: Two more tips. When it's done or when you're at a stopping point,
Chris: have it create like an agents MD that explains your system so that the next
Chris: fresh LM context, when it starts up, it kind of knows your system.
Chris: So a basic layout of where your config's at, where your preferences are,
Chris: the desktop choices, the things you prefer the clanker do, you put all that
Chris: into like an agents MD or some sort of description file about your system.
Chris: And then when you start a new session, you have a point at that.
Chris: So that way it's not rediscovering your next config where it's at every single time and all of that.
Wes: It knows the rebuild command. It knows all the, yeah.
Chris: Yeah, so I think that's probably maybe the most important tip is just have it build one of those.
Chris: And like, yeah, the other thing that would maybe be good for you to consider is,
Chris: second tip, you could ask it to document in a human reviewable format decisions
Chris: that are being made and just kind of it could append and update a markdown file
Chris: or a text file for you that you could review at the end.
Chris: You could have another LLM review, something you could have it also produce
Chris: two documents if you want.
Chris: You could have it produce a technical document for an LLM with no context,
Chris: a fresh LLM session with no context, a technical document that explains everything,
Chris: a fresh LLM with no context needs to get started and work from the beginning
Chris: to modify, change, upgrade, anything to my system.
Chris: And then a second document that's for a human audience that needs to understand
Chris: the decisions that are made and the design choices that were made.
Chris: And it will just continue to update both of those documents for you as it works in that session.
Wes: As long as you maintain the consistency and it sees that every session.
Chris: So remember that.
Wes: Yeah, a devlog is one word you can use for some of those concepts.
Chris: Ooh, okay.
Brent: I have two more quick tips that I think are good for someone starting out. If you are...
Brent: comfortable enough to install things, uh, like in open code,
Brent: having an MCP, uh, is really helpful here.
Brent: So there's MCP dash Nix OS, which is a really great program just to help like
Brent: query what's even available in Nix OS. And it saves a lot of time and mistakes
Brent: and just like querying the internet.
Brent: So, uh, I would recommend that. And then if you are up for it,
Brent: writing yourself just a little skill for NixOS with like,
Brent: common patterns that you particularly like once you get down the road is really
Brent: really helpful or at least has been for me.
Chris: Yeah and if it's not obvious what they might be they will become obvious to
Chris: you as you use it they will become.
Wes: Obvious if you keep having to tell it the same thing yeah.
Chris: There you go if you tell it three times.
Brent: Then you should put it in a skill.
Chris: Moon Knight comes in with two thank you very much for the boost good luck let
Chris: us know how it goes Moon Knight comes in with a boost to 2500 sats and he says
Chris: to all the new Fiat boosters you now have boost power,
Chris: nice i like that that's true thank you moon and i thanks for the sound clip too.
Wes: Uh pabby boosts in with 7108 sats,
Wes: time travel.
Chris: Boost episode 671 love hearing from the time travelers.
Wes: Thanks for the ulanzi smart pixel watch tip i've been looking for one to integrate
Wes: into Home Assistant since I'm not getting around to building the Autrix kit I've had for years.
Wes: Also, on 1.5 speed, Forescan sure sounded a lot like Forescan,
Wes: which had me quite confused for the segment. Had to look at the show notes.
Chris: Yeah, not a great name.
Brent: It's the Ford scanner.
Wes: But, you know, that is the risk you take with the 1.6X.
Chris: I think it's, what could else, yeah, that's right. What else could,
Chris: what did they do? Ford would come after them if they put the D in there.
Chris: They can't put the D in there, so they just had to go with Forescan.
Wes: Forescan? No, that doesn't help.
Chris: Yeah, it doesn't work either. yeah it doesn't work thank you though appreciate it pappy.
Brent: Well zach attack sent in 6 000 sets,
Brent: Loved the episode on hacking yourself.
Brent: I did dislike the episode where everything was in reverse, though.
Brent: Need to still watch your Clanker Therapy video? Yes, that's linked at every
Brent: episode recently, right at the top of the links. For now.
Chris: For now. Almost done linking that, too.
Brent: Something you could take a look at is Rockbox. I'm running it on a SureFans
Brent: F20 for podcast and music. It has better audio than the phone and increased privacy, too.
Chris: That's true. I haven't heard of Rockbox in a hundred years. That's really cool.
Brent: This sounds vaguely familiar. Is that a piece of software you can throw on some
Brent: MP3 devices? I believe so.
Wes: Yeah.
Chris: Yeah.
Brent: I think I might've tried to play with this a long time ago. Long,
Brent: long time. It's still around. There you go.
Chris: Zach, thanks for the memories. Appreciate that. Appreciate that.
Chris: Thank you very much. Sir, I'm here with a row of ducks.
Chris: It's pretty fitting. Thanks for breaking our brains with the reverse episode.
Chris: I was about to blame my podcast app, But how dare you skip the hello,
Chris: friends, and welcome back intro. Anyways, here's some value for the show.
Wes: Thanks.
Chris: Hey, if it got a boost, maybe it was worth it after all. But I think we're done
Chris: doing that. I think we're done doing that.
Wes: Our pal Magnolia, ma'am, comes in with a double row at ducks.
Chris: Hey-oh.
Wes: Two of the same message, but it says, just going off of the title here on 680
Wes: Go Hack Yourself, but I just got started with syncing myself after a Git workflow of mine failed.
Wes: Can't wait to see what the show's about. Ah, that must have been to the pending.
Chris: Yeah, the live one.
Brent: Live boost.
Chris: Well done. Well done. Boosting one before we even started the show,
Chris: man. Well done. Yeah, let us know how it goes with your sync thing set up.
Chris: That should be very interesting.
Chris: Blackhost comes in with a fiat boost of $1,000?
Chris: What? What?
Brent: Wow. What?
Chris: What?
Chris: When did that get in there?
Wes: I don't know. I'm just seeing it.
Chris: When did that come in?
Wes: 10, 21 a.m. this morning.
Chris: Wow, we were already on the air.
Wes: That is crazy.
Chris: Wow, Blackhost. Thank you. That's really, that's...
Brent: I got to pick up my jaw here this week. Yeah, thank you.
Chris: No ads placeholder.
Wes: Thank you, Blackhost. That's so sweet.
Chris: That is really amazing. Wow. Wow, wow, wow. Really do appreciate that,
Chris: and it definitely will make a difference. Thank you.
Wes: David from Burlington comes in with $10 fiat Hey.
Chris: You know what? We have a boost sound for that now, right? Should we roll it
Chris: up? Let's try it What do you think? You ready? See if it feels good,
Chris: That's not bad It's not bad Hey folks.
Wes: It occurred to me that while y'all have extensive tech backgrounds I don't know
Wes: if I've ever heard where it all came from Oh boy Can you guys give a quick summary
Wes: of your past work history?
Chris: Oh boy,
Chris: Oh, boy.
Brent: It's basically a virus that infected all of us.
Chris: Could we do like some special on that? Or is that worth it?
Wes: Like in a members we put in the show? I don't know.
Chris: I don't know.
Wes: We'll have to think about that. Thank you for asking, though.
Chris: Yeah.
Wes: And for the support.
Chris: The reason why we don't have time to do it now is because we're old.
Chris: And so there's a lot to it.
Wes: We only remember so much.
Chris: I remember before Linux. I was alive.
Chris: Yeah. So, yeah, that's a great, that could be a great topic.
Chris: We haven't really recapped that in a long time. I suppose some people have some
Chris: questions on that. Thank you, David.
Chris: From Burlington? My Burlington?
Wes: Yeah, which Burlington?
Brent: There's one in Ontario.
Wes: Maybe a zip code boost in there.
Chris: Yeah. I'm not getting beers with them then. But all right, we have some member
Chris: boosts that we'd like to get to. Our buddy Anonymous is back.
Chris: Do you guys really use Firefox? You talk about a record number of tabs.
Chris: Have we mentioned that? I don't know that.
Chris: But I don't believe it. I've used it on and off for years, and I keep having
Chris: to bail back to Chrome. I'm trying again, but after a day, too, it just hangs on me.
Chris: I have to kill all Firefox. I got 90 gigs of RAM, a 22-core Xeon,
Chris: 50 tabs really shouldn't hang.
Brent: Wow.
Wes: I never have it hang. And it's probably open the entire time my laptop is online.
Chris: I have in the past had issues, but it's been years.
Wes: Yeah, interesting.
Brent: For the question of if we, you know, do we actually use Firefox,
Brent: I'd say it's my permanent first choice.
Brent: I mostly only use Chromium when something goes wrong and the website's not built
Brent: for Firefox or for the show because I need multiple browsers.
Brent: And I would say I probably have the most tabs and windows open out of all of
Brent: us, probably out of you guys combined.
Brent: So there's that. But I only really used to have issues on low-end hardware where
Brent: I didn't have enough RAM.
Brent: But never experienced the slowness or anything like that unless I was filling
Brent: up the RAM. So not sure what your issue is. Hopefully you can figure it out.
Chris: I love this. On my home desktop, I do a boot, Garuda, and Arch for daily driver
Chris: and Windows 11 for occasional work applications that, well, I couldn't port to wine.
Chris: My 15-year-old plays video games, Minecraft, Steam on Garuda.
Chris: Yesterday, my six-year-old came up to me while I was working and said,
Chris: Dad, can you boot into Linux? Windows is boring.
Brent: That's amazing. Nice.
Wes: Aw.
Chris: He says, with all of the success with the reverse episode, how about a randomness
Chris: episode? You could do the segments in random order.
Brent: Whoa.
Chris: Good luck.
Brent: Yeah, that won't be harder at all.
Wes: Curse on the show. I don't know. You really want to melt our brains.
Chris: It'd be fun, too, if we just literally rolled a dice or something and we didn't
Chris: even know it just totally ran out. There's something to that.
Wes: Or each MP3 that gets generated is a unique order.
Chris: Different order. Boy, if we could pull that off.
Brent: Why do we do this to ourselves? Dynamic segments.
Wes: Anton member boosts in. For the Web Boost anonymous username issue,
Wes: the quickest fix might be a small confirmation pop-up when the username field
Wes: is left blank or default and the user clicks the member boost or checkout button.
Wes: the pop-up could confirm that they meant to send it anonymously.
Chris: I don't love a pop-up, but that is not a bad idea.
Wes: Thank you for thinking there, Anton.
Brent: Well, Jimmy's here with a free members boost, which you could do at boost.jupiterbroadcasting.com
Brent: if you're so interested.
Brent: How do you reconcile the privacy considerations of AI? It can be amazingly helpful,
Brent: of course, but they get a lot of sensitive personal info.
Brent: That's what's holding me back. A local model would fix that,
Brent: but of course the VRAM required for good models is quite prohibitive.
Brent: How can you be running graphene, but pumping configs into the AI machine?
Chris: I guess it, I mean, do you consider Nix configs to be confidential information?
Wes: I generally don't.
Chris: No, I don't either.
Wes: Especially for something I might consider putting on.
Chris: I post them on GitHub, yeah.
Brent: You guys don't keep your bank account information in your Nix OS config?
Wes: Not anymore.
Chris: No, now I don't know. Declarative banking.
Chris: So, like, if I were putting in personal information, I suppose that would be one thing.
Wes: Yeah, you know, health stuff, addresses, account number, you know.
Wes: So there can be ways you want to have concerns about that. And some information
Wes: will leak, right, if you ask it questions about how do I set up X thing or I'm
Wes: trying to solve this problem.
Wes: But the question is what risks – how does that fit into your model of – Right.
Chris: In my model, while that is a risk, it's actually less of a risk than like,
Chris: you know, data brokers that are monitoring all of my transactions or,
Chris: you know, government surveillance or Google surveillance.
Chris: or, you know, like those are actually more prevalent in my day-to-day life that I already live with.
Chris: I actually wouldn't mind if they did train on my Nix configs and they got better at it.
Chris: So in a way, I kind of want them to, I don't think the models I'm using are
Chris: using my data for training, but if they wanted to become better at Nix,
Chris: I really would not have any problem with that.
Wes: I think of it too as sort of like a controller agency thing.
Wes: Like for me, I mean, yes, the Graphene OS sort of privacy stuff,
Wes: is great, but i really like that it makes me actually feel like i control the
Wes: android phone that's it and that's where it's like with the ai
Wes: i might make mistakes or whatever um or be lazy but it's ultimately my choice
Wes: of like do i engage with it what do i put into that prompt before i hit enter
Wes: and stock android didn't really feel like that.
Brent: I have uh hmm yeah various categories like you say chris um nix west configs,
Brent: totally good to send those out i think they're,
Brent: not private really at all. Um, but I do hesitate on some topics.
Brent: Like there's a couple topics that I have that like accounting automations and
Brent: stuff that I have sitting in the queue for when I feel like,
Brent: there's some great option for like privacy, local AI stuff. So I can't wait.
Brent: But the other thing I would say is, um, there are a couple of things you could do about that.
Brent: Like I, I don't trust myself completely to to keep myself safe so i put a couple
Brent: layers like there's uh an mcp that you can add i'm trying to see it here that
Brent: i've added to my open code just to like
Brent: pull out um secrets and bank out and credit card numbers so it just goes through and,
Brent: sees if you've accidentally you know put anything like that in there and so
Brent: there's a couple things you can do just for peace of mind maybe before.
Chris: It hits the model.
Brent: Yeah uh.
Chris: Wes has talked about also like at the injector level if you have an injector
Chris: sitting between your client and the cloud service you can have something that's,
Chris: looks like it's the it looks like the api to the client but it's actually sitting
Chris: between the cloud model and the client and it's looking for sensitive information yeah.
Wes: We've talked about a few of these uh key g proxies another one there's a few
Wes: options out there now and you could also consider this is a lot more technical
Wes: setup but if you do uh entertain something like renting a cloud GPU and then
Wes: running your own self-hosted models, that has a lot of nice privacy implications too.
Chris: And maybe if you only need it for certain batch work or you want to build a
Chris: certain thing and you're not going on day-to-day stuff, you're not running an
Chris: agent or something like that, you're just using it for that type of work,
Chris: a rented GPU with a self-hosted model, well, then you've got full control.
Chris: You've got out of it. So that's a good question. Thank you, Jimmy.
Chris: Pabby comes in with a free time travel member boost.
Chris: So great. This is now possible, Pabby writes. Thank you, Pabby,
Chris: and I love the time travel boost. But you've got to say what episode you're
Chris: at when you time travel. Otherwise, we don't know from what time you're at.
Wes: Digital Jolt comes in with a member boost.
Chris: All right.
Wes: I'm a bit late, but just wanted to say I really enjoyed the Reverso episode.
Wes: It took me a minute to remember that you'd mentioned it a few weeks ago,
Wes: but I smiled every time I heard one of the jingles appear in the wrong place.
Chris: Good.
Wes: All right.
Chris: I'm glad.
Wes: Love that.
Brent: Well, Paul member boosted in. Where are you on the ad block is privacy topic.
Brent: I don't like giving money to corporate overlords. This is the main thing holding
Brent: me back from YouTube Premium right now, for instance.
Brent: I prefer to support a few selected creators. My willingness to spend money is
Brent: limited, leaving many creators out.
Chris: Yeah, that's a tricky one because the ad blocking, at least in the browser, it's not just about ads.
Chris: It's also about performance and readability, obnoxiousness with the pop-ups.
Chris: So there's other things now that are also a factor there. So it's not just like
Chris: a statement on trying to get the content for free.
Chris: And I do tend to agree. I think I kind of align with you, Paul.
Chris: Like I think Pharaonics is a great membership. LWN in our space is a great membership.
Chris: We try to make the Jupiter signal and unplugged core contribution a,
Chris: you know, really as much high signal as we can worth it membership.
Chris: But I have over the years decided because I have a family that watches YouTube,
Chris: I did subscribe for YouTube premium to avoid that. and the YouTube experience is notably better.
Wes: For sure.
Chris: When you sit down on an account that doesn't have it, I cannot believe how much
Chris: ads they're slamming in there now. And I just don't want my kids seeing all that.
Wes: As for the sort of, is it piracy or not? I think it's one of those,
Wes: like, you know, it's sort of a spectrum.
Wes: The world exists where both people are dependent on ads and,
Wes: one, local control of what your computer does is important.
Wes: And, you know, ad blockers are a known technology that exists.
Wes: So I think it is true that choosing that perhaps has a moral dimension, but...
Wes: The idea that, you know, you only need to make one choice for every single creator
Wes: doesn't seem right to me either.
Chris: Yeah, it is a nuanced problem.
Brent: I think for me, this is a really good one. I think about this all the time.
Brent: I've kind of landed on ad blocking everything because I feel like very naively,
Brent: actually, I feel like we could find a better way to fund things instead of using ads.
Brent: So if they're not making money on ads, maybe they can find a better way.
Brent: but I'm sure it would just be more draconian.
Chris: The problem is it's always kind of eventually a race to the bottom.
Chris: You always have to add more ads. It's just the way it goes.
Brent: Yeah. I would say one option is, let's say you're using Adblock Plus or something
Brent: like that, you could disable it on websites that you want to get the revenue.
Chris: I do that on Photopea.
Brent: Yeah.
Chris: They ask, and I think it's worth it. Photopea, they deserve my ad revenue.
Chris: That's another one I should probably...
Wes: It does give you a chance to find out, too, like how far did they go?
Wes: Are these like horrible ads or is it a reasonable sort of ad placement that
Wes: isn't actually that bad in practice?
Chris: Good question.
Brent: The last thing I would add is like the internet is completely unusable without
Brent: it. So there's that, right?
Chris: Yeah, it's bad. All right. I'll wrap us up. Senior Smile comes in with a free
Chris: member boost. Papa John was correct. I forgot to put my username last time.
Chris: I said a USD boost referencing my previous anonymous boost about K8s.
Chris: But you read them in reverse order. Oh, sorry.
Brent: We didn't catch that.
Wes: We don't always get that right.
Chris: Yeah. Sorry, Senior Smile. And then Bearded Zero wraps us up.
Chris: I enjoyed the backwards episode, and I got to say, I think doing the boost and
Chris: feedback may have been a good option.
Chris: It lets it bring the feedback into the show and kind of feels like a bookend
Chris: for the last episode. Just a thought.
Chris: I also felt it was kind of nice to warm up with the feedback and the boost.
Chris: And I think if I were going to always do the show in the order that felt the
Chris: best, that might be the order I'd do it in.
Chris: But if you think about it, you see that, you know, it's like if you're new to
Chris: the show and it takes us 30 minutes or whatever to get to the actual content.
Wes: Not everyone loves that.
Chris: Not a good impression. But I liked it too, Bearded Zero. I liked it too.
Chris: Thank you, everybody, and everybody who boosted below the 2,000 sat cutoff and all of that.
Chris: We really appreciate you. We really had a banger. So we had 124,000 sat sent in.
Chris: But then when you bring in that fiat boost, I mean, we should take a moment
Chris: again. I mean, that was really something. That was really something.
Chris: And $1,000 came into the show today from one single contributor, and we needed it.
Chris: It has been tight behind the scenes. It was.
Brent: That works out to be 1.29 million Satoshis.
Chris: Wow. Look at you doing the math.
Brent: No, it wasn't me. It was hybrid sarcasm. Thank you, hybrid sarcasm in the chat.
Chris: I mean, that's got to put Blackhost on the leaderboard for the year.
Brent: I mean, there you go.
Chris: Right there.
Wes: Yeah, maybe we'll have to do like a Boosties preview sometime soon.
Chris: Yeah. So when you bring it all together and you take the fiat boost and you
Chris: convert them to the current Satoshi value and you just smash it all to one line
Chris: item, we had 16 of your stream stats.
Chris: That's 27,485 Satoshis. When you combine it all together, it gives us,
Chris: I mean, a very impressive grand total of 1,458,305 Satoshis.
Wes: That's crazy.
Brent: What?
Chris: If you'd like to support the show, you can become a member, put it on autopilot.
Chris: You get the bootleg or the ad-free version of the show, and you get a free boost
Chris: every single week. Or support us one-off.
Chris: Like a particular show, want to get your thought on here, just support us.
Chris: Go to boost.jupiterbroadcasting.com. You can send a boost with Fiat or Sats.
Chris: It's crazy. Wow!
Chris: All right. And thank you, everyone, who supported the show. We really appreciate
Chris: it all. With that, it's time for some screaming picks.
Chris: All right. Buckle up, boys, because this week we're talking about SSH Clipboard,
Chris: and this could change your life.
Chris: A peer-to-peer Clipboard sync server over SSH. Copy anything on one machine, paste it on another.
Chris: A picture, a block of code, it doesn't matter.
Chris: You copy here, you paste there. Rich text content doesn't matter.
Chris: It keeps the native formats intact. Native formats, not just text,
Chris: preserving images, rich content, things like screenshots, something you might
Chris: want to put into a ticket maybe. You want to take it from one machine and put it on the other.
Chris: Maybe you've got a clanker that's working on something and doesn't have to save
Chris: it to the file system anymore. It can actually just put it right in your freaking clipboard. Or...
Chris: Or if you put it on multiple machines, the Clanker could put it on the machine's
Chris: clipboard you tell it to.
Wes: You're saying I should be using this instead of netcatting things around.
Chris: This is actually an option to reduce your netcat dependency.
Wes: It looks pretty new, huh?
Chris: It is very new, I should mention.
Wes: From Justin Schroeder and Andrew Boyer, the contributors so far.
Chris: Yeah, yeah, yeah. It is very new. It is very new.
Wes: But I think there's been several attempts at various tools like these over the
Wes: years. I think when I was first getting into Linux, I remember trying to roll something myself.
Wes: It's nice to see something that looks like a structured, well-thought-out approach.
Chris: Yeah, I went through it. I dug through it, and I was very impressed.
Chris: It's early days, but very, very, very impressed with what they've done so far.
Chris: And, of course, if you've got SSH keys set up, just you reuse those,
Chris: so it handles all of that for you.
Chris: It's cross-platforms. You can move stuff between Mac and Linux.
Chris: You can reuse the copy-paste mechanism there.
Wes: It won't break your other clipboard managers. That's good. Right,
Wes: it's using Rust and Ratatouille, which is just turning out to be such a great library.
Brent: Yes.
Chris: I think it'll be packaged in multiple ways right now. Like, they're kind of
Chris: the recommended path is NPM.
Chris: I did look into it. You could probably package this in NIC. It's pretty straightforward.
Chris: So it is very doable. And it is MIT licensed. MIT licensed.
Wes: Well, that has been Blackhost Pick of the Week.
Chris: Yeah. Brought to you by Blackhost.
Chris: I had another one.
Wes: Oh, a sneaky pic, you didn't even put it in the dog.
Chris: I didn't put it in the dog, so I don't remember what it was.
Wes: Though.
Wes: So you're saying you have a pic for next week?
Brent: It's in his clipboard.
Wes: It's been a long time since we've done it.
Chris: Yeah, you should have set it up. It probably is in my clipboard.
Chris: Oh my god, that's hilarious.
Wes: It's been a while since we've had a single pic pic.
Chris: I know. Well, that is a banger of a pic. I just can't remember what my other
Chris: pic was. I can't remember what it was.
Brent: It's probably a single pic.
Chris: No, I was pretty proud of it. It's funny that I can't remember it now.
Chris: That's what I get for editing a text editor first. So we'll sneak it in the next episode.
Chris: And if you've got a banger pick, you can always let us know too.
Chris: We love it when we feature an audience pick. We'll put it in the feedback or
Chris: we might even put it in the main pick segment.
Chris: So you can boost.jupiterbroadcasting.com that or go to linuxunplugged.com slash contact for that.
Chris: All right. Well, that wraps us up. Links to everything we talked about today
Chris: over at the website, linuxunplugged.com slash 681 or jupiterbroadcasting.com
Chris: for the links. and all the shows.
Chris: And if you want to just, you know, chew away at some extra stuff,
Chris: you want to know what we talked about, anything like that, tell them,
Chris: Wes. Tell them about these fancy metadatas that we got.
Wes: Yeah, well, the first level is Cloud Chapters. Yeah, that's right,
Wes: a JSON file with all the information about where our segments are and what they are.
Wes: And you know what? If you look in that, you get custom Drew-made chapters.
Chris: They're great.
Wes: Using all that editor cleverness.
Chris: They are great.
Wes: But if you need more details, check out the VTT file, or if you must, the SRT file.
Wes: We have subtitles, transcripts, right? Available in the RSS feed itself.
Wes: And if you want to see our ugly mugs, there might even be an MP4.
Chris: There could be with screencast stuff and all that of what we're talking about.
Chris: I'll just say again, thank you to Blackhost. You know, it strikes me.
Chris: I was thinking about it this week, like all four of us, the three of us here
Chris: on the show and also Drew, you know, we really have shaped our lives around
Chris: the show. It's that level of dedication. It really drives our life.
Chris: It's the defining clock and heartbeat of our day-to-day life.
Chris: And so when you see support by folks, like we saw this week,
Chris: it really does validate putting that kind of effort and energy and building
Chris: a life around the show to make it possible. So that way we're here every single
Chris: week going on like, what, 13 or something years?
Wes: Crazy.
Chris: Not missing a beat. So we really do appreciate the support. Thank you, everybody.
Chris: Like I said, links at the website. And of course, we'd love it if you joined
Chris: us live. Make it a live vibe on a Sunday.
Chris: Get past the Sunday blues by making it a Tuesday on a Sunday over at jblive.tv
Chris: in your local time at jupyterbroadcasting.com slash calendar or stream live
Chris: in your podcasting 2.0 app of choice.
Chris: And, of course, the Mumble Room's going. We've got details on the website.
Chris: The Mumble Room gets a low latency, high quality Opus stream.
Chris: It's pretty nice. Shout out to our members for making the show possible and
Chris: keeping us going. Everybody who supports us at boost.jupyterbroadcasting.com.
Chris: But that's it for this week's episode. Thank you so much for joining us,
Chris: and we'll see you back here next Tuesday, as in Sunday.