WEBVTT

00:00:11.489 --> 00:00:16.029
<v Chris>Hello, friends, and welcome back to your weekly Linux talk show. My name is Chris.

00:00:16.189 --> 00:00:16.789
<v Wes>My name is Wes.

00:00:17.009 --> 00:00:17.809
<v Brent>And my name is Brent.

00:00:18.429 --> 00:00:23.009
<v Chris>Hello, gentlemen. Well, coming up on episode 626 of your Unplugged program,

00:00:23.009 --> 00:00:27.569
<v Chris>we're digging into a Butterfest bug that is biting people in the wild right now.

00:00:27.729 --> 00:00:30.149
<v Chris>So we want to get the word out there as fast as possible.

00:00:30.529 --> 00:00:36.089
<v Chris>Then, over the week, I discovered how to actually get whole home audio that's working.

00:00:36.449 --> 00:00:40.489
<v Chris>And it's working like a charm. And there's a little extra twist in there,

00:00:40.549 --> 00:00:41.769
<v Chris>too. So I'll tell you about that.

00:00:41.849 --> 00:00:45.249
<v Chris>Then we'll round the show out with some great boos, killer picks, and a lot more.

00:00:45.449 --> 00:00:50.909
<v Chris>So before we get into all of that, let's say time of props to our virtual lug. Hello, Mumble Room.

00:00:51.869 --> 00:00:55.049
<v Wes>Hey, Chris, here we're St. Hello, Brent. Hello, gang.

00:00:55.289 --> 00:00:58.149
<v Chris>Hello, everybody in the Mumble Room. Thank you very much for joining us over

00:00:58.149 --> 00:01:02.029
<v Chris>there. And shout out to everybody who's joining us live and shout out to all

00:01:02.029 --> 00:01:03.269
<v Chris>the members that make it possible.

00:01:04.389 --> 00:01:09.649
<v Chris>And go check out Managed Nebula at define.net slash unplugged.

00:01:10.429 --> 00:01:14.349
<v Chris>Decentralized VPN built on open source software that we love,

00:01:14.709 --> 00:01:17.769
<v Chris>the Nebula platform, which you've been playing around with, Wes.

00:01:17.949 --> 00:01:18.409
<v Wes>I have.

00:01:18.729 --> 00:01:21.529
<v Chris>Yeah, you're getting me really jealous. We'll talk about that more at some other

00:01:21.529 --> 00:01:26.609
<v Chris>point. But Nebula is optimized for speed, simplicity, and it has industry-leading security.

00:01:26.789 --> 00:01:32.289
<v Chris>And the way this manifests for day-to-day life, less battery usage on your mobile

00:01:32.289 --> 00:01:36.569
<v Chris>devices, less network traffic, and less load on your servers.

00:01:36.949 --> 00:01:42.289
<v Chris>It's really light it's really light and unlike traditional vpns nebula's decentralized design,

00:01:42.849 --> 00:01:45.789
<v Chris>means that you can build out your own network as you

00:01:45.789 --> 00:01:50.409
<v Chris>like and make it as resilient as you need so great for a home lab great for

00:01:50.409 --> 00:01:53.989
<v Chris>a global enterprise whether you're using their managed product or the self-hosted

00:01:53.989 --> 00:01:58.269
<v Chris>option which chef's kiss is top-notch one of the reasons why we love it so much

00:01:58.269 --> 00:02:02.109
<v Chris>they don't hold back on the self-hosted product that's what they build on top

00:02:02.109 --> 00:02:03.829
<v Chris>of and That's what's available for all of us.

00:02:04.309 --> 00:02:07.649
<v Chris>And they're starting to roll out desktop clients.

00:02:07.809 --> 00:02:12.269
<v Chris>It's going to get even easier to use Nebula, which I'm really excited about.

00:02:12.429 --> 00:02:15.349
<v Chris>Because anything that just opens us up to more users is going to be fantastic.

00:02:15.749 --> 00:02:19.469
<v Chris>And Nebula takes advantage of things like the noise protocol framework for key

00:02:19.469 --> 00:02:22.849
<v Chris>exchange and symmetric encryption. So you know they're using good stuff in there.

00:02:23.129 --> 00:02:26.189
<v Chris>And whether you want to self-host the entire infrastructure or you want to check

00:02:26.189 --> 00:02:28.909
<v Chris>out their managed product, which makes it really straightforward,

00:02:29.729 --> 00:02:33.949
<v Chris>you got an option. In fact, if you go to define.net slash unplugged, you support the show.

00:02:34.529 --> 00:02:38.569
<v Chris>You can also get started with 100 hosts absolutely free. No credit card required.

00:02:38.809 --> 00:02:42.089
<v Chris>Go see why we're really excited about Nebula by playing around with it first

00:02:42.089 --> 00:02:44.469
<v Chris>at define.net slash unplugged.

00:02:46.926 --> 00:02:50.586
<v Chris>We've got a little bit of exciting housekeeping. This is approaching really fast.

00:02:51.006 --> 00:02:54.966
<v Wes>Yes, I'll be giving a talk at Nix Vegas at DEF CON this year.

00:02:55.106 --> 00:02:58.366
<v Wes>So if you'll be going to, you can find me there.

00:02:58.526 --> 00:03:02.586
<v Wes>Come check it out. We'll be talking about mesh sidecars for NixOS services.

00:03:03.086 --> 00:03:06.466
<v Wes>And then an attempt to find other people in the chaos, if anyone's interested,

00:03:06.706 --> 00:03:11.426
<v Wes>I think I'll try to post up at the Casbar Lounge on Saturday around 6.30 for

00:03:11.426 --> 00:03:13.866
<v Wes>an hour or so, just for a sort of office hours.

00:03:14.166 --> 00:03:18.666
<v Chris>Nice, simple meetup right there. no meetup page required just where is it?

00:03:19.066 --> 00:03:22.426
<v Wes>The Caspar Lounge in the Sahara Hotel at what time? 6.30.

00:03:22.426 --> 00:03:24.626
<v Chris>On Saturday so if you're in the area or.

00:03:24.626 --> 00:03:26.026
<v Wes>You can I don't know ping me on Matrix.

00:03:26.026 --> 00:03:28.386
<v Chris>You don't even have to be at DEFCON if you're just in the Vegas area that's

00:03:28.386 --> 00:03:32.446
<v Chris>true too come say hi to West Payne absolutely Brent and I can attest he's a

00:03:32.446 --> 00:03:35.126
<v Chris>delight to hang out with at a bar or at dinner,

00:03:35.826 --> 00:03:40.106
<v Chris>wherever it is so do recommend you go check it out and where do they go for

00:03:40.106 --> 00:03:44.906
<v Chris>details is it nix.vegas is that do they have a website that's right yeah nix.vegas

00:03:44.906 --> 00:03:47.066
<v Chris>slash schedule There's a dot Vegas?

00:03:47.126 --> 00:03:48.246
<v Wes>Of course there's a dot Vegas.

00:03:50.958 --> 00:03:54.558
<v Chris>So we wanted to talk about a file system corruption issue that is affecting

00:03:54.558 --> 00:04:01.978
<v Chris>ButterFS users on more recent kernels like 6.16 and 6.15.3.

00:04:02.478 --> 00:04:06.938
<v Chris>And we are now at a point with ButterFS adoption where there's enough leading-edge

00:04:06.938 --> 00:04:11.058
<v Chris>kernels and users out there and distributions that are actually deploying and

00:04:11.058 --> 00:04:14.018
<v Chris>testing these things kind of right as they ship.

00:04:14.238 --> 00:04:17.498
<v Chris>And thanks to them, we've discovered there's a bit of a problem going on that

00:04:17.498 --> 00:04:19.038
<v Chris>has left some systems unbootable.

00:04:19.038 --> 00:04:22.418
<v Wes>Yeah, it turns out there's a lot of root file systems out there now,

00:04:22.418 --> 00:04:27.258
<v Wes>too, which is a great thing. We know how much we like it, except if,

00:04:27.318 --> 00:04:30.138
<v Wes>you know, your root file system won't mount.

00:04:30.278 --> 00:04:30.918
<v Chris>That's not good.

00:04:31.198 --> 00:04:31.358
<v Wes>No.

00:04:31.738 --> 00:04:35.658
<v Chris>Yeah, so the bug itself didn't do any damage to the data, but it did prevent

00:04:35.658 --> 00:04:37.898
<v Chris>the system from properly mounting the root device.

00:04:38.078 --> 00:04:42.278
<v Wes>Yeah. Okay, so copy-on-write file system. We're also aware, generally,

00:04:42.458 --> 00:04:44.738
<v Wes>right, like ext4 is a journaling file system.

00:04:44.738 --> 00:04:50.058
<v Wes>And these journals are something like what's known as a write-ahead log in the

00:04:50.058 --> 00:04:55.358
<v Wes>database world or ZFS has the intent log which is somewhat different but very similar also.

00:04:56.158 --> 00:05:02.478
<v Wes>The idea is for data consistency and crash consistency you can write the things

00:05:02.478 --> 00:05:06.338
<v Wes>that you're doing especially because with copy and write let's say you're updating

00:05:06.338 --> 00:05:09.418
<v Wes>our show notes a markdown file well what happens?

00:05:09.418 --> 00:05:14.138
<v Wes>You make a new copy of that so you can do snapshotting so you have all the features we love

00:05:14.538 --> 00:05:18.218
<v Wes>but there's also this tree structure and you have to kind of go update all the

00:05:18.218 --> 00:05:21.178
<v Wes>tree to make sure that like all the way at the root when you go to like you

00:05:21.178 --> 00:05:25.098
<v Wes>know ls that directory it actually points at that updated copy and not the old copy,

00:05:25.658 --> 00:05:29.078
<v Wes>but the snapshot version has it so there's a bunch of bookkeeping and updates

00:05:29.078 --> 00:05:30.478
<v Wes>you need to make just for that write,

00:05:31.318 --> 00:05:34.798
<v Wes>and what happens if you crash in the middle so the idea is you make a note that

00:05:34.798 --> 00:05:36.338
<v Wes>you're like I'm going to do this update,

00:05:36.838 --> 00:05:42.238
<v Wes>and then you flush that to disk and then you can move that out of the log but

00:05:42.238 --> 00:05:46.358
<v Wes>if you crash in the middle you can see from the log oh i hadn't finished that

00:05:46.358 --> 00:05:48.078
<v Wes>and then you can kind of check to see like do i need,

00:05:49.175 --> 00:05:52.675
<v Wes>Can I fix it? Can I just replay that? Most of the time, if you have an unclean

00:05:52.675 --> 00:05:56.015
<v Wes>shutdown, it just automatically replays that journal from the log,

00:05:56.715 --> 00:06:00.515
<v Wes>brings your file system back to a clean state. You don't even really need to know about it.

00:06:00.655 --> 00:06:04.635
<v Chris>So a handful of users, like on CacheOS and Fedora, where they're getting pretty

00:06:04.635 --> 00:06:08.535
<v Chris>current kernels and they're using ButterFS on the root, they experienced some

00:06:08.535 --> 00:06:11.895
<v Chris>sort of crash, and then when they reset their system, they couldn't boot.

00:06:12.035 --> 00:06:15.395
<v Wes>Yep, you just get an error that it can't replay the log, which means it sees

00:06:15.395 --> 00:06:16.715
<v Wes>that there is data there.

00:06:16.835 --> 00:06:19.875
<v Wes>For some reason, your file system, whether it was like a total shutdown,

00:06:20.075 --> 00:06:24.275
<v Wes>forced shutdown, or maybe something just happened on the fly before as it was

00:06:24.275 --> 00:06:27.135
<v Wes>shutting down, you have stuff in that replay log.

00:06:27.455 --> 00:06:31.615
<v Wes>And so, of course, it doesn't want to just drop that because you could have,

00:06:31.795 --> 00:06:34.915
<v Wes>basically, the amount of data loss you could have if you didn't replay that

00:06:34.915 --> 00:06:38.155
<v Wes>is roughly constrained by how

00:06:38.155 --> 00:06:41.515
<v Wes>often you're doing these background commits sort of flushing to the disk.

00:06:41.635 --> 00:06:44.575
<v Wes>Usually it's like 30 seconds, maybe it could be a couple of minutes if you have

00:06:44.575 --> 00:06:47.975
<v Wes>some configuration or doing tons of fsyncs or lots of disk I.O. or something.

00:06:49.215 --> 00:06:52.955
<v Wes>And so what the fix ends up being in terms of just like I want to get my stuff

00:06:52.955 --> 00:06:59.155
<v Wes>going again is you run butterfs rescue which is a whole sub command for rescue commands,

00:06:59.875 --> 00:07:05.355
<v Wes>and then 0-log and that pretty much does what it says clear the tree log and.

00:07:05.355 --> 00:07:08.355
<v Chris>You essentially do that from a live environment because if you were to say to

00:07:08.355 --> 00:07:11.955
<v Chris>boot into a live environment you wouldn't be able to just mount that file system because of this.

00:07:11.955 --> 00:07:14.895
<v Wes>Problem no anytime you mount it it's just going to complain with the same thing

00:07:14.895 --> 00:07:17.815
<v Wes>I think there is also a mount option you can do to say like skip replay.

00:07:19.095 --> 00:07:24.755
<v Chris>And so now this is sort of splitting hairs, but this is technically not a ButterFS

00:07:24.755 --> 00:07:30.095
<v Chris>bug so much as it is something kind of related to a series of other patches

00:07:30.095 --> 00:07:32.395
<v Chris>that kind of led to this issue. Am I following this right?

00:07:32.595 --> 00:07:34.035
<v Wes>No, so it is ButterFS.

00:07:34.275 --> 00:07:34.415
<v Chris>Okay.

00:07:34.495 --> 00:07:36.715
<v Wes>It's just, it kind of has an interesting history.

00:07:37.035 --> 00:07:40.615
<v Chris>Yeah, okay. That's what I was trying to follow, and that's a piece I wasn't getting. Yeah.

00:07:40.835 --> 00:07:46.695
<v Wes>So it all goes back to 2018, actually. A commit called fix warning when replaying

00:07:46.695 --> 00:07:49.255
<v Wes>log after fsync of a temp file.

00:07:49.575 --> 00:07:52.235
<v Wes>And the thing with temp files is they're, they generally like they're meant

00:07:52.235 --> 00:07:57.135
<v Wes>to be discarded at reboot, right? And so they don't actually like have any extents on disks.

00:07:57.275 --> 00:08:00.255
<v Wes>They just sort of have like the inode for tracking and they can live in memory

00:08:00.255 --> 00:08:02.755
<v Wes>and the cache and all that, but they don't actually make it to the disk.

00:08:02.755 --> 00:08:04.935
<v Wes>Right. And so they were running into some warnings.

00:08:05.135 --> 00:08:09.895
<v Wes>So they went to go patch this up. And to do that, they made essentially two changes.

00:08:10.875 --> 00:08:13.995
<v Wes>One, because you basically have two sides of this. When you're going to do an

00:08:13.995 --> 00:08:17.615
<v Wes>operation, you write into the log to say, like, I'm about to do X,

00:08:17.715 --> 00:08:19.875
<v Wes>right? I'm about to delete this temp file.

00:08:20.706 --> 00:08:23.466
<v Chris>So before I take the action, I log that I'm going to take the action.

00:08:23.586 --> 00:08:27.666
<v Wes>Exactly. And then you have the state where you're booting up and you're mounting

00:08:27.666 --> 00:08:30.606
<v Wes>the file system and you're replaying that log if there's stuff in there that

00:08:30.606 --> 00:08:32.346
<v Wes>hasn't been synced with the disk yet.

00:08:33.386 --> 00:08:35.746
<v Wes>So in this case, what they did was they said, all right, well,

00:08:35.746 --> 00:08:38.526
<v Wes>we don't really care about these temp files in the log anyway,

00:08:38.526 --> 00:08:41.886
<v Wes>so we'll stop adding new stuff to the log going forward.

00:08:43.006 --> 00:08:47.166
<v Wes>And when we're in replay mode, we'll just skip these because they're going to

00:08:47.166 --> 00:08:50.366
<v Wes>end up getting deleted anyway. We don't care about them. the problem,

00:08:50.606 --> 00:08:52.686
<v Wes>which was not apparent at the time, because basically...

00:08:52.686 --> 00:08:53.246
<v Chris>Back in 2018.

00:08:53.486 --> 00:08:54.126
<v Wes>Back in 2018.

00:08:54.406 --> 00:08:55.086
<v Chris>When they're writing this patch.

00:08:55.226 --> 00:08:59.006
<v Wes>There's multiple different stages you can be in replay. And so there's one called

00:08:59.006 --> 00:09:02.826
<v Wes>replay inodes, and that's where they sort of did this skipping.

00:09:03.066 --> 00:09:05.126
<v Wes>But there's another one called replay all.

00:09:05.406 --> 00:09:09.986
<v Wes>And then it turns out that the way this change was made, it didn't apply during that stage.

00:09:11.206 --> 00:09:14.106
<v Wes>But because they were also making the change at the same time where they were

00:09:14.106 --> 00:09:17.226
<v Wes>just not going to add that stuff to the log anymore, the only way you could

00:09:17.226 --> 00:09:21.686
<v Wes>trigger it back then was if you were somehow, like you had upgraded your kernel

00:09:21.686 --> 00:09:25.506
<v Wes>and had an unclean, or you were mounting something unclean shutdown from an older kernel.

00:09:25.706 --> 00:09:29.806
<v Chris>Okay. So this is the bit. So then later on, there was like an additional patch

00:09:29.806 --> 00:09:30.946
<v Chris>that compounded this problem?

00:09:31.086 --> 00:09:36.406
<v Wes>Yeah. So just in May, it started and got picked up and added to 6.15.3 and 6.16.

00:09:36.726 --> 00:09:38.586
<v Chris>Uh-huh. So that's why it's in the most recent kernels.

00:09:38.766 --> 00:09:42.226
<v Wes>Yeah. So then we noticed a problem that it turns out by not,

00:09:43.206 --> 00:09:46.806
<v Wes>just by no longer putting this, like these temp file stuff in the log at all,

00:09:46.806 --> 00:09:50.246
<v Wes>it meant that we could have a problem where we actually

00:09:50.246 --> 00:09:52.966
<v Wes>left them undeleted like they were still kind of hanging

00:09:52.966 --> 00:09:56.026
<v Wes>around in like when you did have an unclean shutdown

00:09:56.026 --> 00:09:58.826
<v Wes>the accounting kind of was broken the

00:09:58.826 --> 00:10:02.826
<v Wes>other way where instead of like losing stuff it would keep stuff you were trying

00:10:02.826 --> 00:10:07.346
<v Wes>to delete around okay so not a huge issue but it's like incorrect behavior according

00:10:07.346 --> 00:10:11.826
<v Wes>to how do you expect the file kind of crufty over time yeah uh exactly if we

00:10:11.826 --> 00:10:16.746
<v Wes>f-sync a file that has no more hard links because while a process had a file descriptor open on it,

00:10:16.886 --> 00:10:17.966
<v Wes>the file's last hard link was

00:10:17.966 --> 00:10:20.866
<v Wes>removed, and then the process did an fsync against the file descriptor,

00:10:21.066 --> 00:10:25.466
<v Wes>after a power failure or crash, the file still exists after replaying the log.

00:10:25.586 --> 00:10:25.726
<v Chris>Okay.

00:10:26.046 --> 00:10:29.546
<v Wes>Right? So they'll fix this by not ignoring inodes with zero hard links.

00:10:29.906 --> 00:10:33.706
<v Wes>So now we're putting that stuff back into the log again.

00:10:33.906 --> 00:10:34.106
<v Chris>Okay.

00:10:34.426 --> 00:10:38.686
<v Wes>And that means suddenly this issue, which had actually technically sort of been

00:10:38.686 --> 00:10:41.046
<v Wes>there since 2018, can now be hit again.

00:10:41.907 --> 00:10:44.747
<v Wes>Fairly easily it turns out i mean you still kind of need something to happen

00:10:44.747 --> 00:10:48.207
<v Wes>weird with the file system where you have stuff to replay it has to be triggered.

00:10:48.207 --> 00:10:49.087
<v Chris>By some kind of event.

00:10:49.087 --> 00:10:49.807
<v Wes>Yeah it.

00:10:49.807 --> 00:10:52.707
<v Chris>Doesn't just happen if you've got butterfs and it's running and you have 616.

00:10:52.707 --> 00:10:56.387
<v Wes>You basically need to be in a state where it finds stuff in that replay journal

00:10:56.387 --> 00:10:59.047
<v Wes>when it's booting up and mounting the file system my.

00:10:59.047 --> 00:11:03.467
<v Chris>System upstairs by the way is on linux 616 and it is butterfs on root so that's

00:11:03.467 --> 00:11:06.767
<v Chris>why i was curious if yeah that's it has to be i'd have to crash or something.

00:11:06.767 --> 00:11:09.607
<v Wes>Yeah crash or yeah maybe i don't know Some application dumps.

00:11:09.607 --> 00:11:13.807
<v Wes>Yeah, kernel has a problem and it forces the file system offline without doing it correctly.

00:11:13.907 --> 00:11:15.547
<v Chris>Out of memory kills something. I mean, you never know.

00:11:15.867 --> 00:11:19.407
<v Wes>So it gets like, this gets thrown out there sometime in May in the Linux ButterFS

00:11:19.407 --> 00:11:25.027
<v Wes>lists and then eventually gets pulled in for 6.16. And then once that kind of

00:11:25.027 --> 00:11:28.847
<v Wes>gets, was pushing along, it also gets backported to 6.15.3.

00:11:29.067 --> 00:11:34.227
<v Wes>And you can see it's just this one commit from there that like as 6.15.3 started

00:11:34.227 --> 00:11:37.147
<v Wes>rolling out pretty much like by a week later or so,

00:11:37.147 --> 00:11:41.667
<v Wes>kind of the end of june 26 27th cache eos users were some of the first folks

00:11:41.667 --> 00:11:46.787
<v Wes>who really started running into these things and then by like july 7th or so

00:11:46.787 --> 00:11:51.407
<v Wes>uh a kernel mailing list thread gets going and people start trying to it's a

00:11:51.407 --> 00:11:54.627
<v Wes>good collective effort the cache folks arch folks they're all kind of like.

00:11:54.627 --> 00:11:56.147
<v Chris>And butterfest developers.

00:11:56.147 --> 00:11:59.247
<v Wes>Yeah they're reaching out to the butterfest devs who are kind of like well we

00:11:59.247 --> 00:12:03.887
<v Wes>really need d message output or like more than just you know sort of like the the one error they.

00:12:03.887 --> 00:12:07.087
<v Chris>Had to work out like a standard operating procedure to get these systems and

00:12:07.087 --> 00:12:08.927
<v Chris>these file systems back up so they could pull logs.

00:12:09.107 --> 00:12:12.127
<v Chris>Like, they had to come up with procedures and processes they could communicate to people.

00:12:12.227 --> 00:12:14.947
<v Wes>And then kind of by proxy, right, where, like, the distro folks are talking

00:12:14.947 --> 00:12:17.507
<v Wes>with the kernel folks and getting what the kernel folks needs and then reaching

00:12:17.507 --> 00:12:20.287
<v Wes>out to the users they're doing end-user support for to, like,

00:12:20.447 --> 00:12:22.647
<v Wes>tell them how to get this data and shepherd it around.

00:12:23.267 --> 00:12:28.267
<v Wes>But eventually enough stuff throughout July kind of comes together that there's

00:12:28.267 --> 00:12:33.527
<v Wes>now a fix out there that correctly does this skipping and replay of these temp

00:12:33.527 --> 00:12:36.527
<v Wes>files in all the stages that are actually necessary.

00:12:37.007 --> 00:12:39.647
<v Chris>Okay, so I basically just need to keep an eye out for a kernel update.

00:12:39.927 --> 00:12:46.207
<v Wes>Yeah, but that got proposed to the Linux ButterFS list, so presumably that will get...

00:12:47.021 --> 00:12:51.861
<v Wes>Pulled into 6.17 and then presumably backported as well, but that's all going to take time.

00:12:51.921 --> 00:12:55.601
<v Chris>And in the meantime, if this were to happen to you, you could go into a live

00:12:55.601 --> 00:12:58.741
<v Chris>environment, run that rescue command, and you'd be alright.

00:12:58.921 --> 00:13:01.801
<v Wes>Yeah. The only data loss you should be risking is, like we were saying,

00:13:02.021 --> 00:13:07.121
<v Wes>is just whatever had been sort of not flushed to disk yet as the unclean shutdown was happening.

00:13:07.601 --> 00:13:10.981
<v Brent>Do we have a sense of how many users have been affected up to now?

00:13:11.141 --> 00:13:15.081
<v Brent>And I would imagine because of the fixes we're not going to see too many more

00:13:15.081 --> 00:13:16.301
<v Brent>users being affected as well?

00:13:16.401 --> 00:13:18.241
<v Chris>My sense is it's less than a thousand but i don't know.

00:13:18.241 --> 00:13:21.401
<v Wes>Yeah i don't know because it's kind of like you have to have some event that

00:13:21.401 --> 00:13:28.101
<v Wes>triggers it and you have to be on 6 16 or 6 6 15 3 or newer and.

00:13:28.101 --> 00:13:30.001
<v Chris>With butterfest's root yeah yeah.

00:13:30.001 --> 00:13:34.281
<v Wes>I mean it could be on a not it could affect a non-root file system just it wouldn't break your boot you.

00:13:34.281 --> 00:13:37.301
<v Chris>Know who i think it's happening to the most is users that game and then the

00:13:37.301 --> 00:13:38.501
<v Chris>game crashes their system.

00:13:38.501 --> 00:13:40.701
<v Wes>There's been multiple reports of that yeah that's.

00:13:40.701 --> 00:13:44.641
<v Chris>The seemingly the most affected because that seems to be what crashes linux the most i don't.

00:13:44.641 --> 00:13:48.221
<v Wes>Know definitely folks who've self-reported also having sort of known sketchy

00:13:48.221 --> 00:13:50.721
<v Wes>power supplies or sorts or situations sure that'll.

00:13:50.721 --> 00:13:51.181
<v Chris>Do it yeah.

00:13:51.181 --> 00:13:54.281
<v Wes>Uh it is worth calling out here uh this has all been uh

00:13:54.281 --> 00:13:56.961
<v Wes>the initial issue as part of fixing a bunch

00:13:56.961 --> 00:14:00.161
<v Wes>of other stuff this has all been the same person's work across the

00:14:00.161 --> 00:14:03.281
<v Wes>years uh felipe manana from seuss

00:14:03.281 --> 00:14:05.981
<v Wes>has been oh responsible for fixing up all

00:14:05.981 --> 00:14:08.721
<v Wes>kinds of butterfs issues and working on the file system and it

00:14:08.721 --> 00:14:11.721
<v Wes>just works out that like he was the person who made the

00:14:11.721 --> 00:14:14.561
<v Wes>change in 2018 and the one that

00:14:14.561 --> 00:14:17.381
<v Wes>made it more apparent now and is the person who figured all

00:14:17.381 --> 00:14:20.301
<v Wes>of that out and made an excellent explanation and commit

00:14:20.301 --> 00:14:23.161
<v Wes>in the fix to patch it all up so props there for

00:14:23.161 --> 00:14:27.901
<v Wes>sure and it's

00:14:27.901 --> 00:14:31.301
<v Wes>exactly these kind of things that make file systems you know so hard

00:14:31.301 --> 00:14:34.001
<v Wes>to debug i mean he he's been

00:14:34.001 --> 00:14:38.061
<v Wes>proposing tests you know test cases and stuff as going for all of these stuff

00:14:38.061 --> 00:14:41.481
<v Wes>so it's not like they aren't testing it's just especially with situations of

00:14:41.481 --> 00:14:45.161
<v Wes>power failure and like there's all kinds of failure points where you can't have

00:14:45.161 --> 00:14:49.321
<v Wes>happen to the file system and take cases edge cases that are just hard to test

00:14:49.321 --> 00:14:51.801
<v Wes>so it's a community effort to squash those bugs.

00:14:51.941 --> 00:14:55.341
<v Chris>Thank you for digging into that I was aware of just bits and pieces of that

00:14:55.341 --> 00:14:58.341
<v Chris>and I wanted to get that info out at the top of the show because like my system

00:14:58.341 --> 00:15:01.541
<v Chris>just updated 6.16 Fedora users are updating so it's something to be aware of.

00:15:02.141 --> 00:15:03.641
<v Wes>ButterFS is still safe to use.

00:15:03.641 --> 00:15:07.081
<v Chris>So let us know does this give you any pause using ButterFS.

00:15:07.521 --> 00:15:10.541
<v Wes>Did it bite you? Maybe I hope not. But if so, please tell us.

00:15:10.701 --> 00:15:14.401
<v Chris>I'm kind of also just generally interested in file system war stories.

00:15:14.521 --> 00:15:17.181
<v Chris>It really seems like it hasn't been a problem as much these days,

00:15:17.341 --> 00:15:19.561
<v Chris>but maybe I'm wrong or maybe you have one from the past.

00:15:20.261 --> 00:15:24.181
<v Chris>Boosted, it's a great way to support the show and we love reading those messages.

00:15:24.861 --> 00:15:28.301
<v Wes>It is kind of funny. Just, you know, of course it's the mechanism in the file

00:15:28.301 --> 00:15:31.881
<v Wes>system to be robust that is in fact causing the problem.

00:15:32.161 --> 00:15:36.021
<v Chris>It is. And it also just shows you how complicated kernel development is and

00:15:36.021 --> 00:15:40.441
<v Chris>as things get integrated into the kernel there's a lot of other second order

00:15:40.441 --> 00:15:42.081
<v Chris>effects is what I'm trying to say It's.

00:15:42.081 --> 00:15:44.381
<v Wes>Not just what you change, it's all that changes around you.

00:15:48.297 --> 00:15:52.757
<v Chris>1password.com slash unplugged. Take the first step to better security for your

00:15:52.757 --> 00:15:55.797
<v Chris>team by securing credentials and protecting every application,

00:15:56.077 --> 00:15:58.577
<v Chris>even unmanaged shadow IT.

00:15:58.797 --> 00:16:03.277
<v Chris>Go learn more at 1password.com slash unplugged. That's the number 1password.com

00:16:03.277 --> 00:16:06.117
<v Chris>slash unplugged, all lowercase.

00:16:06.737 --> 00:16:09.477
<v Chris>Okay, if you're a security IT professional, you know this problem already.

00:16:09.797 --> 00:16:13.397
<v Chris>You have more and more assets to protect as time goes on.

00:16:13.737 --> 00:16:19.057
<v Chris>More devices, more identities, and more applications, and it becomes a mountain

00:16:19.057 --> 00:16:20.377
<v Chris>of security risks over time.

00:16:20.737 --> 00:16:24.877
<v Chris>But fortunately, you can conquer that mountain of security risks with 1Password

00:16:24.877 --> 00:16:27.377
<v Chris>Extended Access Management.

00:16:27.997 --> 00:16:32.357
<v Chris>The reality is that when surveyed, over half of IT pros say securing their SaaS

00:16:32.357 --> 00:16:34.237
<v Chris>apps is actually their biggest challenge now.

00:16:34.677 --> 00:16:38.477
<v Chris>And I kind of get it. You know, you've got SaaS sprawl essentially happening,

00:16:38.537 --> 00:16:41.877
<v Chris>and it creates the shadow IT where users sign up to things and use things because,

00:16:42.277 --> 00:16:44.297
<v Chris>quote unquote, it makes their job easier or better.

00:16:44.577 --> 00:16:48.537
<v Chris>So it's kind of understandable, but it's a hard problem to fix.

00:16:49.037 --> 00:16:56.737
<v Chris>Thankfully, Trellica by 1Password can discover and secure access to all your apps, managed or not.

00:16:57.277 --> 00:17:01.517
<v Chris>See, Trellica by 1Password inventories every app in use at your company.

00:17:01.837 --> 00:17:07.257
<v Chris>Then pre-populated app profiles assess SaaS risk, letting you manage access,

00:17:07.657 --> 00:17:12.537
<v Chris>optimize spend, and enforce security best practices across every app your employees use.

00:17:12.737 --> 00:17:17.697
<v Chris>Yes, even the shadow IT stuff. You can securely onboard and off-board employees

00:17:17.697 --> 00:17:20.097
<v Chris>and meet compliance goals.

00:17:20.777 --> 00:17:24.777
<v Chris>See, Trelica by 1Password provides a complete solution for SaaS access governance.

00:17:24.957 --> 00:17:28.777
<v Chris>And it's just one of the ways Extended Access Management helps teams strengthen

00:17:28.777 --> 00:17:30.497
<v Chris>compliance and security.

00:17:30.717 --> 00:17:33.717
<v Chris>You know 1Password and their award-winning password manager.

00:17:33.877 --> 00:17:37.097
<v Chris>Now go check out Extended Access Management and Trelica.

00:17:37.297 --> 00:17:40.897
<v Chris>You can take the first steps to better security for your team by securing credentials

00:17:40.897 --> 00:17:44.797
<v Chris>and protecting every application, even the unmanaged ones.

00:17:44.997 --> 00:17:50.337
<v Chris>Go find out more and support the show. You go to 1password.com slash unplugged.

00:17:50.397 --> 00:17:54.557
<v Chris>That is the number 1password.com slash unplugged.

00:17:57.177 --> 00:18:01.617
<v Brent>Well, Chris, you teased about whole home audio, and when I've been in Lady Joops,

00:18:01.957 --> 00:18:07.637
<v Brent>that has typically been a proprietary system, but something tells me things have changed.

00:18:07.637 --> 00:18:10.337
<v Chris>I am so, so...

00:18:11.373 --> 00:18:16.773
<v Chris>So ashamed of my love for the HomePods. And my whole home audio solution has

00:18:16.773 --> 00:18:20.613
<v Chris>been HomePod audio for years now. And it's embarrassing.

00:18:21.133 --> 00:18:22.533
<v Wes>I mean, you're a Big Mac guy.

00:18:24.113 --> 00:18:28.473
<v Chris>Yeah, it's really great because of the ecosystem. I love the ecosystem, Wes.

00:18:28.833 --> 00:18:32.473
<v Chris>It's just, it has been a shame of mine. And I wanted to solve for this for a while.

00:18:32.693 --> 00:18:37.193
<v Chris>I've been looking for a long time for a whole home audio solution where I can

00:18:37.193 --> 00:18:41.353
<v Chris>put speakers anywhere in my house using whatever device i want and then be able

00:18:41.353 --> 00:18:45.813
<v Chris>to play synchronized audio across all of them and then ideally create subgroups

00:18:45.813 --> 00:18:49.513
<v Chris>for like the living room and you know different rooms where maybe you want different

00:18:49.513 --> 00:18:51.333
<v Chris>content or you don't want music playing.

00:18:51.333 --> 00:18:53.713
<v Wes>Right yeah your wife's having a nap in the back.

00:18:53.713 --> 00:18:54.193
<v Chris>Room you're.

00:18:54.193 --> 00:18:56.853
<v Wes>Trying to get some work done you wouldn't mind listening to a podcast while you do it.

00:18:56.853 --> 00:19:00.933
<v Chris>Actually that is that is one of the number one use cases wes you'd be surprised

00:19:00.933 --> 00:19:05.473
<v Chris>and so i have dabbled in a lot of different things we have dabbled in a lot

00:19:05.473 --> 00:19:08.633
<v Chris>of different things but i've never felt confident enough to come on air and say,

00:19:08.793 --> 00:19:11.893
<v Chris>this is the solution that you should implement in your home.

00:19:12.173 --> 00:19:16.033
<v Chris>Because I've wanted something that was really flexible and nothing really matched it.

00:19:16.353 --> 00:19:20.373
<v Chris>I finally this week gave Music Assistant a go.

00:19:20.673 --> 00:19:25.413
<v Chris>And it's a library manager for your offline and online music sources.

00:19:25.433 --> 00:19:26.733
<v Chris>I'll get more to that here in a second.

00:19:26.893 --> 00:19:32.413
<v Chris>And then you can stream to a wide range of supported players and you can combine them and mix them.

00:19:32.653 --> 00:19:36.813
<v Chris>And then as the name kind of implies if you want though it's optional you can

00:19:36.813 --> 00:19:40.653
<v Chris>also integrate it into home assistant and i'll talk about more of that later

00:19:40.653 --> 00:19:44.273
<v Chris>so music assistant itself has the music assistant server,

00:19:44.813 --> 00:19:49.973
<v Chris>it also has what it calls music providers so you can import from various sources

00:19:49.973 --> 00:19:55.533
<v Chris>it has player providers so you can play music on a wide different range of ecosystem

00:19:55.533 --> 00:19:59.253
<v Chris>from like dlna devices sonos devices,

00:19:59.873 --> 00:20:02.073
<v Chris>Chromecast devices, AirPlay devices,

00:20:03.213 --> 00:20:05.673
<v Chris>Home Assistant voice preview devices. It goes on and on.

00:20:05.793 --> 00:20:09.593
<v Chris>And then they also have plugins, so you can add additional functionality in there.

00:20:11.033 --> 00:20:15.393
<v Chris>And the thing that I think I didn't appreciate about Music Assistant until I

00:20:15.393 --> 00:20:21.773
<v Chris>started using it is how well it integrates local sources with remote sources.

00:20:22.313 --> 00:20:26.573
<v Chris>Calling it Plex for your media is way underselling what this thing is capable

00:20:26.573 --> 00:20:28.773
<v Chris>of. Plex wishes it could do this.

00:20:30.099 --> 00:20:35.739
<v Chris>So first of all, I have a Tidal account. It was exceptionally easy to integrate it into Tidal.

00:20:35.979 --> 00:20:39.459
<v Chris>If you're logged into your Tidal account, it's just a would you like to authorize app.

00:20:39.759 --> 00:20:43.139
<v Chris>It synchronizes all your playlists. So this is nice because my wife creates

00:20:43.139 --> 00:20:43.959
<v Chris>playlists pretty frequently.

00:20:44.179 --> 00:20:47.859
<v Chris>And then they are immediately available on the music assistant within the next time she goes to use it.

00:20:47.899 --> 00:20:48.359
<v Wes>That's great.

00:20:48.819 --> 00:20:53.499
<v Chris>It supports the high quality playback. But it also easily integrates my local flax.

00:20:54.179 --> 00:21:00.959
<v Chris>And then another source that is exceptionally useful is it supports audio bookshelf servers directly.

00:21:01.379 --> 00:21:01.919
<v Wes>That's killer.

00:21:02.059 --> 00:21:04.419
<v Chris>So all my audio books are in there. And I'll get more to that in a moment.

00:21:04.579 --> 00:21:07.219
<v Wes>Were you able to just do like a point it at a folder? How does the,

00:21:07.299 --> 00:21:08.819
<v Wes>for your flax, like that source?

00:21:08.859 --> 00:21:12.839
<v Chris>Yes. So you have a lot of options there. You can point it at a SambaShare directly

00:21:12.839 --> 00:21:15.579
<v Chris>or just a local. And in my case, it was just on that local box.

00:21:15.579 --> 00:21:16.579
<v Chris>It was just a local file system.

00:21:16.699 --> 00:21:16.859
<v Wes>Sure.

00:21:17.059 --> 00:21:18.459
<v Chris>That I just brought into the Docker container.

00:21:18.679 --> 00:21:18.839
<v Wes>Nice.

00:21:19.079 --> 00:21:23.059
<v Chris>So there's lots of options for your speaker endpoints with Music Assistant too.

00:21:23.639 --> 00:21:26.639
<v Chris>And one of them that a lot of us have out there is Chromecast.

00:21:27.319 --> 00:21:31.159
<v Chris>And it works really well with Chromecast. And it can be anything like one of

00:21:31.159 --> 00:21:34.639
<v Chris>those stupid little Google, you know, assistant display boxes,

00:21:34.639 --> 00:21:37.339
<v Chris>or it can be a Chromecast hanging off your TV.

00:21:37.519 --> 00:21:40.219
<v Chris>Or if you're lucky enough to have a Chromecast audio, that works.

00:21:40.599 --> 00:21:43.999
<v Wes>Yeah, I got it going on my Google TV. It took a little bit.

00:21:44.119 --> 00:21:47.259
<v Wes>There was some discovery process. I toggled it on and off once.

00:21:47.359 --> 00:21:49.159
<v Wes>But then since then, it's been just fine.

00:21:49.299 --> 00:21:52.079
<v Chris>It does. Music Assistant does rely on a lot of like that, you know,

00:21:52.319 --> 00:21:56.139
<v Chris>DNS discovery stuff. So you have to make sure that's getting passed through to the application.

00:21:56.299 --> 00:21:58.559
<v Chris>So if you've got a firewall or if you're running a container,

00:21:58.579 --> 00:22:00.479
<v Chris>you have to make sure a lot of that stuff gets passed through.

00:22:00.659 --> 00:22:03.359
<v Wes>I guess they had two, like, multiple. They had, like, a newer,

00:22:03.739 --> 00:22:05.579
<v Wes>better way that they were doing the casting.

00:22:05.879 --> 00:22:09.339
<v Wes>So, like, once I, at first I tried that and it wasn't, I tried the old one that

00:22:09.339 --> 00:22:11.239
<v Wes>worked and then the new one works totally fine.

00:22:11.359 --> 00:22:14.419
<v Wes>So I don't know. It probably was all just still booting up because I just got to go.

00:22:14.519 --> 00:22:17.719
<v Chris>It does take a minute to discover. I think the other one that works really well

00:22:17.719 --> 00:22:20.619
<v Chris>is Sonos. Have not tested that, but I guess the Sonos integration is pretty tight.

00:22:21.499 --> 00:22:24.099
<v Chris>Fully kiosk browser, I thought was an interesting one because this is what I

00:22:24.099 --> 00:22:26.399
<v Chris>use on my tablets for my home assistant displays around the house.

00:22:26.699 --> 00:22:31.299
<v Chris>So I think, you know, you could in theory have a speaker plugged into the headphone

00:22:31.299 --> 00:22:34.639
<v Chris>out port on that tablet and then turn that into a player.

00:22:35.562 --> 00:22:36.102
<v Chris>Pretty neat.

00:22:36.442 --> 00:22:38.582
<v Wes>I mean, you could just see that for, yeah, I don't know. You have some sort

00:22:38.582 --> 00:22:42.422
<v Wes>of small office setup too, and those are just on there for displaying calendars

00:22:42.422 --> 00:22:44.442
<v Wes>already. Now you've got a PA system.

00:22:44.602 --> 00:22:47.662
<v Chris>Why not? Yeah, it does, you know, announcements and stuff like that too.

00:22:48.342 --> 00:22:51.982
<v Chris>AirPlay. So any AirPlay target that's not the Apple TV.

00:22:52.222 --> 00:22:54.782
<v Chris>The Apple TV is not currently supported because there's like a two-way pairing

00:22:54.782 --> 00:22:57.862
<v Chris>process, but, you know, anything else that's an AirPlay target.

00:22:58.482 --> 00:23:02.482
<v Chris>And this is a really interesting one. Any supported Home Assistant media player,

00:23:02.602 --> 00:23:07.182
<v Chris>like the Voice Preview Edition hardware, which is a little puck with an iPod-like

00:23:07.182 --> 00:23:09.942
<v Chris>wheel on it you can use for volume control and it has an aux out port.

00:23:10.862 --> 00:23:13.362
<v Chris>It's really nice. So that was the direction I thought I would go.

00:23:13.742 --> 00:23:17.662
<v Chris>I thought I would buy a handful of these at a time because they're 50 bucks-ish

00:23:17.662 --> 00:23:19.122
<v Chris>a pop so buy a couple, set them up.

00:23:19.162 --> 00:23:23.342
<v Chris>I already have two so I would only need a couple more and then I would use these

00:23:23.342 --> 00:23:26.562
<v Chris>as streaming endpoints for my music assistant server.

00:23:28.022 --> 00:23:31.202
<v Chris>And while you could definitely do that and I think it would work really well

00:23:31.202 --> 00:23:34.182
<v Chris>because it's all an open ecosystem, the prices are pretty good,

00:23:35.222 --> 00:23:41.802
<v Chris>what I discovered is that the open source community has so successfully reverse

00:23:41.802 --> 00:23:48.062
<v Chris>engineered AirPlay that the AirPlay supported Music Assistant is as if it was

00:23:48.062 --> 00:23:49.382
<v Chris>a native open source protocol.

00:23:49.562 --> 00:23:50.522
<v Wes>That's so killer.

00:23:50.702 --> 00:23:55.342
<v Chris>It's bonkers how good it is. And so it sort of was just, well,

00:23:55.442 --> 00:23:57.342
<v Chris>I already have these AirPlay devices. And it's- I mean.

00:23:57.422 --> 00:23:58.782
<v Wes>That's best case for you, really.

00:23:58.902 --> 00:24:03.022
<v Chris>It's kind of unbelievable because I bought these stupid expensive HomePods. They do sound good.

00:24:03.482 --> 00:24:06.682
<v Wes>Just go check the Coder Radio backlog for years on this.

00:24:06.842 --> 00:24:08.782
<v Chris>This has been years now, so I think the investment may have,

00:24:08.882 --> 00:24:10.282
<v Chris>quote unquote investment, may have paid off.

00:24:10.762 --> 00:24:14.402
<v Chris>But you would not think I would have gotten this much mileage out of some stupid

00:24:14.402 --> 00:24:17.002
<v Chris>proprietary speakers that only have a power cord coming out of them.

00:24:17.342 --> 00:24:21.342
<v Chris>It really is bonkers. And it's crazy how good it works with Music Assistant.

00:24:22.042 --> 00:24:26.102
<v Chris>So I just went with the practical route and I'm actually, I'm mostly just using AirPlay.

00:24:26.742 --> 00:24:30.342
<v Chris>But I do have some, based on some of the testing I did, I have some quick recommendations.

00:24:30.442 --> 00:24:34.162
<v Chris>If you just want easy speakers that work, i

00:24:34.162 --> 00:24:38.202
<v Chris>don't know how you say this but ikea makes these symphonisk speakers

00:24:38.202 --> 00:24:43.742
<v Chris>what kea speakers and they're wi-fi speakers that actually have a sonos control

00:24:43.742 --> 00:24:48.342
<v Chris>board in them but they're like you know 100 200 cheaper than an equivalent sonos

00:24:48.342 --> 00:24:53.162
<v Chris>speaker and they also support airplay and they support dlna and they support

00:24:53.162 --> 00:24:54.222
<v Chris>the sonos streaming protocol.

00:24:55.447 --> 00:24:59.847
<v Chris>And they sound decent. And the home assistant community and the music assistant

00:24:59.847 --> 00:25:01.707
<v Chris>community rave about these.

00:25:01.867 --> 00:25:05.907
<v Chris>So they range, depending on where you buy them, between $99 and $120 US dollars.

00:25:06.427 --> 00:25:08.447
<v Chris>So these are not super cheap for the sound.

00:25:09.227 --> 00:25:14.407
<v Chris>But the functionality, people. So my wife is buying a couple of these for her studio, for her clinic.

00:25:15.227 --> 00:25:17.807
<v Chris>And so I'll have kind of a real world report on these pretty soon.

00:25:18.107 --> 00:25:20.927
<v Chris>But these are, you plug them into your network. You plug them in,

00:25:21.267 --> 00:25:22.107
<v Chris>hook them up to your network.

00:25:22.587 --> 00:25:26.067
<v Chris>And then music assistant will automatically discover them. and they just work.

00:25:26.227 --> 00:25:29.367
<v Chris>And you can play them in pair synced up. It's really nice.

00:25:29.907 --> 00:25:37.267
<v Chris>Obviously, the HomePods, those work well, but you can also buy standalone AirPlay

00:25:37.267 --> 00:25:39.107
<v Chris>adapters that just have audio outports.

00:25:39.507 --> 00:25:42.327
<v Wes>Oh, now that is something.

00:25:42.607 --> 00:25:45.087
<v Chris>So you can use big speakers or hook it up to whatever you want.

00:25:45.327 --> 00:25:49.347
<v Chris>And those range from $90 to $110 US dollars.

00:25:49.847 --> 00:25:53.727
<v Chris>And then, of course, the Chromecast. I found a two-pack of the Chromecast audio

00:25:53.727 --> 00:25:55.347
<v Chris>streamer for $30 on eBay.

00:25:55.847 --> 00:25:55.967
<v Wes>Nice.

00:25:56.127 --> 00:25:56.307
<v Chris>Yeah.

00:25:56.847 --> 00:25:59.387
<v Wes>I also noticed that they had Snapcast support.

00:25:59.467 --> 00:25:59.947
<v Chris>Yes, sir.

00:26:00.107 --> 00:26:02.847
<v Wes>So if you have that or want to build out that infrastructure,

00:26:02.887 --> 00:26:05.887
<v Wes>you can just sort of target it with the music assistant.

00:26:06.127 --> 00:26:09.447
<v Chris>And that really seems like something I should look into in the future.

00:26:09.627 --> 00:26:15.847
<v Chris>I would love the listeners' feedback on Snapcast or if anybody has experimented with this stuff.

00:26:16.527 --> 00:26:20.987
<v Chris>But I want to tell you how I made this sort of spousal approval factor high,

00:26:21.227 --> 00:26:24.867
<v Chris>family approval factor high. because having music assistant and having one interface,

00:26:25.147 --> 00:26:27.967
<v Chris>which is a progressive web app to all your music and your audio books from all

00:26:27.967 --> 00:26:29.987
<v Chris>your different, you can put Apple music in there.

00:26:30.087 --> 00:26:33.847
<v Chris>You can put Spotify in there. You can put title in there. You can put sound cloud in there.

00:26:34.027 --> 00:26:36.707
<v Chris>You know, these are like, there's dozens you can add searches.

00:26:37.087 --> 00:26:40.607
<v Wes>I think it's the iTunes index, but it searches that for podcasts or you can

00:26:40.607 --> 00:26:42.227
<v Wes>just put in RSS feeds manually.

00:26:42.287 --> 00:26:46.447
<v Chris>Yes too. So you can do that as well. So while that is nice to have and then

00:26:46.447 --> 00:26:48.447
<v Chris>manage all of that from one interface, I,

00:26:49.151 --> 00:26:52.731
<v Chris>Where I think it really kicks it up and improves the family approval factor

00:26:52.731 --> 00:26:55.151
<v Chris>is if you can integrate it in with Home Assistant.

00:26:55.811 --> 00:27:01.031
<v Chris>And the Home Assistant integration allows you to create simple dashboard interfaces

00:27:01.031 --> 00:27:03.951
<v Chris>to actually play things and cue things up.

00:27:04.411 --> 00:27:08.031
<v Chris>So I've designed, we have a media card, we have a media dashboard,

00:27:08.071 --> 00:27:10.951
<v Chris>I'm sorry, in Home Assistant, where we have remote controls for our TVs.

00:27:11.351 --> 00:27:15.991
<v Chris>And now I've added another, you could call, it's like the size of a large badge

00:27:15.991 --> 00:27:17.871
<v Chris>or like a large thumbnail image.

00:27:18.011 --> 00:27:20.351
<v Chris>It takes up about that much space. But what it is, it's a media player control

00:27:20.351 --> 00:27:24.731
<v Chris>with an interface for selecting our favorite playlists and our favorite audio

00:27:24.731 --> 00:27:28.871
<v Chris>books and selecting which speakers it plays on. And I'll have links to these in the show notes.

00:27:29.051 --> 00:27:31.831
<v Chris>But this is how the family interfaces with this, right? They're not loading

00:27:31.831 --> 00:27:32.851
<v Chris>the music assistant tab.

00:27:33.071 --> 00:27:34.471
<v Wes>Oh, entirely through.

00:27:34.811 --> 00:27:36.971
<v Chris>They're going to the tablet they already managed stuff in the home with,

00:27:37.111 --> 00:27:38.971
<v Chris>going to the media tab, and then just selecting the playlist.

00:27:39.091 --> 00:27:40.331
<v Chris>And it's just, it plays for them.

00:27:40.411 --> 00:27:43.111
<v Wes>I'm going to have to check that out. Because so far I did get like the very

00:27:43.111 --> 00:27:48.511
<v Wes>default auto discovered, like it can detect and control the music assistant

00:27:48.511 --> 00:27:51.391
<v Wes>state in home assistant side. But I hadn't gone past that.

00:27:51.571 --> 00:27:57.291
<v Chris>So when you install the music assistant integration into home assistant, they find each other.

00:27:57.351 --> 00:28:02.031
<v Chris>In fact, you can, if you have a Hass OS music assistant, it is just an add on.

00:28:02.131 --> 00:28:04.591
<v Chris>You can just install from within Hass OS and it all just connects up.

00:28:04.891 --> 00:28:09.111
<v Chris>But I run on a separate machine and they discover each other over the DNS broadcast

00:28:09.111 --> 00:28:13.851
<v Chris>stuff. and then every speaker that's in Music Assistant shows up as a speaker

00:28:13.851 --> 00:28:15.331
<v Chris>in Home Assistant and vice versa.

00:28:15.671 --> 00:28:19.731
<v Chris>And so now Home Assistant can change the volume, it can do playback and pause,

00:28:19.771 --> 00:28:24.391
<v Chris>it can cue up new songs and then there's lots of community dashboard tools to

00:28:24.391 --> 00:28:25.991
<v Chris>build easy playback controls,

00:28:27.267 --> 00:28:31.967
<v Chris>A couple of, you know, you choose a few boxes. And then where I took it to the

00:28:31.967 --> 00:28:36.067
<v Chris>next level, and I'm very proud of this, it's that audio bookshelf integration.

00:28:36.607 --> 00:28:41.587
<v Chris>Having the audio book integration that syncs book playback between all my devices

00:28:41.587 --> 00:28:45.087
<v Chris>because the audio bookshelf server keeps track of that. So our mobile devices get synced.

00:28:45.307 --> 00:28:48.567
<v Chris>The web interface and now music assistant.

00:28:48.727 --> 00:28:54.067
<v Chris>And I have added Z-Wave buttons in our bedroom on both sides of the bed.

00:28:54.067 --> 00:28:57.667
<v Chris>You push a button, and I've labeled it book, and you hit that button.

00:28:57.667 --> 00:28:58.287
<v Wes>The book button.

00:28:58.407 --> 00:29:03.987
<v Chris>Yep. And that resumes our current audio book at our last left off position on

00:29:03.987 --> 00:29:07.747
<v Chris>one speaker in the bedroom, preset to just the right nighttime volume.

00:29:08.367 --> 00:29:14.167
<v Chris>It then plays for 25 minutes, and then using a new integration that I installed

00:29:14.167 --> 00:29:19.567
<v Chris>that allows you to gradually change the music. It's called gradual volume control.

00:29:20.167 --> 00:29:24.107
<v Chris>For the last 25 seconds of that 25 minute period while the book plays for the

00:29:24.107 --> 00:29:29.687
<v Chris>last 25 seconds i have it gradually turn the audio down to zero percent over

00:29:29.687 --> 00:29:34.287
<v Chris>25 seconds so the book fades out quietly and then it stops playback.

00:29:34.287 --> 00:29:39.327
<v Wes>That's that's wow this is bonkers i want to take a nap of your ass.

00:29:39.327 --> 00:29:44.847
<v Chris>It is so nice because it takes something that is rather complicated right and

00:29:44.847 --> 00:29:48.027
<v Chris>it just automates it with a push of a button because the automation is happening

00:29:48.027 --> 00:29:50.687
<v Chris>in home assistant i've told home assistant, when this button is pressed,

00:29:50.927 --> 00:29:53.507
<v Chris>go play that audio book at its last position.

00:29:53.907 --> 00:29:57.927
<v Chris>Audio bookshelf server is remembering the position. So when music assistant

00:29:57.927 --> 00:30:02.027
<v Chris>requests that audio bookshelves handling that aspect of like the metadata and whatnot.

00:30:02.807 --> 00:30:08.507
<v Chris>But what's really neat about it is none of my devices are in use.

00:30:08.927 --> 00:30:13.727
<v Chris>So I no longer need to bring the phone to the bedroom to listen to the audio

00:30:13.727 --> 00:30:16.787
<v Chris>book. I can leave the phone out in the charging area.

00:30:17.864 --> 00:30:22.244
<v Chris>I don't have to interface with any screen. I don't have to blast light in my face.

00:30:22.964 --> 00:30:26.784
<v Chris>I don't have any of those. I just push a button and the backend infrastructure.

00:30:26.784 --> 00:30:28.304
<v Chris>So I can be rebooting my phone.

00:30:28.444 --> 00:30:31.604
<v Chris>I could be not using my phone. My phone could be off, whatever.

00:30:32.004 --> 00:30:36.544
<v Chris>It doesn't matter because the server system is handling the book playback and

00:30:36.544 --> 00:30:39.244
<v Chris>then Home Assistant is handling the time and the fade out.

00:30:39.464 --> 00:30:42.364
<v Chris>And then it pauses through Music Assistant.

00:30:42.924 --> 00:30:49.664
<v Chris>And the end experience is so simple and so nice. I just, I love it for this one feature alone.

00:30:50.044 --> 00:30:53.744
<v Chris>But we also now have these, you know, with this dashboard, we have these buttons

00:30:53.744 --> 00:30:57.484
<v Chris>for playlists. And so some of our favorite playlists are one button away.

00:30:57.824 --> 00:31:01.344
<v Chris>And then it starts playing on all the speakers in the home in sync.

00:31:01.924 --> 00:31:04.824
<v Chris>And there's just something really fun about that. When you're doing some chores

00:31:04.824 --> 00:31:08.104
<v Chris>and you didn't have to even open up anything, you didn't have to go dig through

00:31:08.104 --> 00:31:10.264
<v Chris>anything. It's just right there. It's all ready. Boom. Click. Good place.

00:31:10.444 --> 00:31:12.304
<v Wes>You get that dump in tanks playlist.

00:31:12.524 --> 00:31:16.004
<v Chris>You sure could. You know, or like I'm on the way home. I open up the home assistant

00:31:16.004 --> 00:31:18.644
<v Chris>app on the phone. and I hit the playlist so the wife knows I'm coming home.

00:31:19.684 --> 00:31:23.304
<v Chris>There's a lot of fun you could have with it too. And then you can also integrate announcements.

00:31:23.784 --> 00:31:27.184
<v Chris>So you could have it announce the upcoming track, but you could also slip in

00:31:27.184 --> 00:31:31.244
<v Chris>sort of like house-wide time for dinner or whatever you want.

00:31:31.244 --> 00:31:34.284
<v Chris>You can slip in house-wide announcements because it supports that as well.

00:31:35.484 --> 00:31:40.064
<v Chris>It's so much fun playing with this stuff. It's like tinkering with a system

00:31:40.064 --> 00:31:41.824
<v Chris>that builds on top of itself.

00:31:41.964 --> 00:31:45.124
<v Chris>So first you get the home assistant running, then you get Z-Wave going,

00:31:45.264 --> 00:31:46.504
<v Chris>and then you get buttons figured out.

00:31:46.824 --> 00:31:49.904
<v Chris>And then you start stacking the stuff. So then you get Music Assistant going

00:31:49.904 --> 00:31:53.124
<v Chris>and you add, now I know how buttons work and I know how Home Assistant works.

00:31:53.244 --> 00:31:57.704
<v Chris>And you can just complete this whole stack of stuff that works really, really well together.

00:31:57.944 --> 00:32:01.044
<v Chris>And while you can completely use Music Assistant on its own,

00:32:01.144 --> 00:32:03.544
<v Chris>it's just like a container somewhere or whatever you want on a system.

00:32:04.124 --> 00:32:08.044
<v Chris>If you connect it to Home Assistant, it really kicks things up.

00:32:08.144 --> 00:32:09.764
<v Chris>And it's so much fun to play around with.

00:32:10.904 --> 00:32:14.584
<v Chris>Even if you don't care about turning on lights on and off, this kind of stuff

00:32:14.584 --> 00:32:18.424
<v Chris>is really choice. And the fact that you have this wide range of hardware it's

00:32:18.424 --> 00:32:20.904
<v Chris>compatible with means you could likely just get started right now.

00:32:21.044 --> 00:32:25.344
<v Wes>That's what's so impressive to me. Like the scope and the amount of integrations

00:32:25.344 --> 00:32:28.664
<v Wes>that already exist and are just kind of all ready to go.

00:32:29.404 --> 00:32:32.324
<v Wes>I was doing it on Nix. And so it's a little more apparent where,

00:32:32.724 --> 00:32:34.784
<v Wes>like I was looking at some of the packaging for it, right?

00:32:34.844 --> 00:32:39.684
<v Wes>So you can tell some plugins, you know, they need some extra packages to be

00:32:39.684 --> 00:32:41.104
<v Wes>included in the whole build as well.

00:32:41.104 --> 00:32:45.624
<v Wes>So like if you want the YouTube music integration, which is just such a pain,

00:32:45.764 --> 00:32:49.984
<v Wes>uh, cause you have to go get some cookie for YouTube DLP, but you need,

00:32:50.064 --> 00:32:53.224
<v Wes>you know, YouTube DLP is the, uh, as part of the packages, but,

00:32:53.925 --> 00:32:56.685
<v Wes>It's just neat because like some of them, they just, I think like titles,

00:32:56.845 --> 00:33:00.325
<v Wes>they don't even need any extra Python packages or anything. They're just ready to go.

00:33:00.525 --> 00:33:03.625
<v Chris>Yeah. I think there's a couple of caveats people should be aware of.

00:33:03.705 --> 00:33:07.525
<v Chris>I think where music assistant struggles is if you mix protocols.

00:33:07.965 --> 00:33:15.445
<v Chris>So if you mix AirPlay and Chromecast or Home Assistant Cast and AirPlay or whatever.

00:33:15.945 --> 00:33:19.525
<v Wes>Even just trying to mix sometimes like playback in the browser and.

00:33:19.525 --> 00:33:25.425
<v Chris>Yeah, or even sometimes if you're combining network speakers that are on different

00:33:25.425 --> 00:33:30.525
<v Chris>major versions of their local OS, there can be issues where it loses sync.

00:33:30.885 --> 00:33:36.125
<v Chris>And so where it seems to really nail it is if you have the same type of protocol

00:33:36.125 --> 00:33:38.505
<v Chris>for every speaker in that group.

00:33:38.585 --> 00:33:41.785
<v Chris>So if you stick to one thing, maybe it's the Home Assistant Preview hardware,

00:33:41.845 --> 00:33:45.025
<v Chris>maybe you're a Sonos person, or maybe you've already got AirPlay or Chromecast.

00:33:45.025 --> 00:33:49.345
<v Chris>If you stick to one streaming technology for all the speakers you're playing

00:33:49.345 --> 00:33:53.305
<v Chris>to, it seems to just really nail synced audio, especially with AirPlay.

00:33:53.825 --> 00:33:56.825
<v Chris>But when you mix and match, you get mixed results.

00:33:57.145 --> 00:34:00.485
<v Chris>Now, if you're in a space where you cannot hear the other speakers,

00:34:00.485 --> 00:34:03.905
<v Chris>you just want to be able to have the same content like a podcast or streaming

00:34:03.905 --> 00:34:06.005
<v Chris>radio. Streaming internet radio is also supported.

00:34:06.325 --> 00:34:10.205
<v Chris>And, you know, if you have a couple of second overlap when you're in the garage

00:34:10.205 --> 00:34:12.525
<v Chris>versus the kitchen, maybe you don't care.

00:34:12.985 --> 00:34:15.845
<v Chris>But if you're if you're standing in a space where you can hear all

00:34:15.845 --> 00:34:18.865
<v Chris>of the speakers at once the synchronization really matters

00:34:18.865 --> 00:34:21.945
<v Chris>then and that's where you want to keep it to the same protocol and that's

00:34:21.945 --> 00:34:25.325
<v Chris>where i think it's worth just going with one stack and i

00:34:25.325 --> 00:34:28.205
<v Chris>put the stuff that i feel like works the best in the

00:34:28.205 --> 00:34:31.285
<v Chris>show notes but having probably used home assistant in

00:34:31.285 --> 00:34:34.285
<v Chris>this entire stack for five years or so

00:34:34.285 --> 00:34:37.945
<v Chris>somewhere around there maybe six um music assistant

00:34:37.945 --> 00:34:41.365
<v Chris>really feels like i've taken it up to the next level now and it's

00:34:41.365 --> 00:34:44.685
<v Chris>so much fun to be able to just hit a button and set a tone

00:34:44.685 --> 00:34:48.385
<v Chris>set a vibe and then anybody that has the home assistant app can also pull it

00:34:48.385 --> 00:34:51.465
<v Chris>up and pause or change it or anybody that goes up to a tablet or if you use

00:34:51.465 --> 00:34:55.145
<v Chris>the voice control too you can do it that way it's really it's it's it's really

00:34:55.145 --> 00:34:58.805
<v Chris>a great experience it's not the best podcast player i will make that disclosure

00:34:58.805 --> 00:35:03.765
<v Chris>true so if you add a podcast like ours or any podcast that has years of back catalog.

00:35:05.205 --> 00:35:10.425
<v Chris>It sorts it and wants to play the oldest episode first, which maybe you want to do that.

00:35:10.505 --> 00:35:13.405
<v Chris>Or maybe you just imported your favorite podcast in your current and you want

00:35:13.405 --> 00:35:14.325
<v Chris>to play the current episode.

00:35:15.125 --> 00:35:18.085
<v Chris>Well, if they have hundreds of episodes, you have to manually go through one

00:35:18.085 --> 00:35:19.825
<v Chris>by one and mark them listened to.

00:35:20.705 --> 00:35:23.485
<v Chris>Unless maybe you're some sort of database hacker and you could go in there and

00:35:23.485 --> 00:35:28.285
<v Chris>fix it that way. But there is no native way to mark a whole bunch of podcasts listened to.

00:35:28.725 --> 00:35:30.465
<v Chris>And that was a real bummer because

00:35:30.465 --> 00:35:33.885
<v Chris>playing podcasts around the house is a great use of Music Assistant.

00:35:34.345 --> 00:35:36.705
<v Chris>So if you get the time, check it out. You can use it on its own.

00:35:37.045 --> 00:35:39.825
<v Chris>Or if you really like your peanut butter and your jelly or chocolate or whatever

00:35:39.825 --> 00:35:41.565
<v Chris>it is, I don't know. You mix it with Home Assistant.

00:35:44.799 --> 00:35:48.719
<v Chris>Unraid.net slash unplugged. Unleash your hardware.

00:35:48.999 --> 00:35:56.859
<v Chris>Self-hosters and home labbers unite. If you can believe it, Unraid is turning 20, 20 amazing years.

00:35:57.039 --> 00:36:02.919
<v Chris>And from August 7th to August 26th, you can join the 20-day birthday bash packed

00:36:02.919 --> 00:36:08.779
<v Chris>with 20% off select licenses and 20% off all new merch. No purchase necessary.

00:36:09.039 --> 00:36:10.939
<v Chris>Giveaways are throughout the celebration.

00:36:11.539 --> 00:36:15.159
<v Chris>It's all going to lead up to the grand finale on August 30th.

00:36:15.319 --> 00:36:20.499
<v Chris>There will be a live virtual event with Unraid's founder and also several others

00:36:20.499 --> 00:36:23.879
<v Chris>from Unraid where they're going to look back at 20 years of Unraid and also

00:36:23.879 --> 00:36:28.219
<v Chris>unveil a new short film and maybe even give a peek at the future of Unraid.

00:36:28.399 --> 00:36:33.599
<v Chris>So you can learn more and RSVP now to celebrate two decades of self-hosting

00:36:33.599 --> 00:36:35.179
<v Chris>innovation with the Unraid community.

00:36:35.359 --> 00:36:38.459
<v Chris>Go to unraid.net slash unplug.

00:36:38.659 --> 00:36:42.999
<v Chris>20 years if you can believe it. And Unraid continues to get better and better.

00:36:43.859 --> 00:36:47.379
<v Chris>Around the corner, not too far out, will be Unraid 7.2.

00:36:48.419 --> 00:36:53.599
<v Chris>7.1 is a blowaway release. But 7.2 is going to bring things that people that

00:36:53.599 --> 00:36:56.899
<v Chris>use the web dashboard or are looking for a full-featured API,

00:36:57.179 --> 00:37:00.619
<v Chris>no spoilers, but people that might be interested in those things,

00:37:00.799 --> 00:37:05.279
<v Chris>or maybe even NTFS support and Extended 4 support in your pools,

00:37:05.779 --> 00:37:08.079
<v Chris>those things, I don't know, Little Birdie tells me they might be coming.

00:37:08.239 --> 00:37:11.379
<v Chris>I mean, Unraid really goes from strength to strength. and they've been doing it for 20 years.

00:37:11.559 --> 00:37:15.159
<v Chris>They have a straightforward model. It's a really simple licensing structure,

00:37:15.159 --> 00:37:21.059
<v Chris>and if you go to unraid.net slash unplugged, you can try it for free for 30 days.

00:37:21.199 --> 00:37:23.699
<v Chris>If you like it, you pick it up, you support the development,

00:37:23.759 --> 00:37:25.579
<v Chris>and they keep going for another 20 years.

00:37:25.679 --> 00:37:30.199
<v Chris>So don't forget, it runs from the 7th to the 26th. You can join the 20-day birthday

00:37:30.199 --> 00:37:32.979
<v Chris>bash, and I have reason to believe you might want to.

00:37:33.359 --> 00:37:36.859
<v Chris>Get started at unraid.net slash unplugged.

00:37:39.669 --> 00:37:44.349
<v Brent>Well, like last week, we do have some shout-outs for new members of Jupiter Party.

00:37:44.649 --> 00:37:50.909
<v Brent>We want to say hi to the party to Kyle, Quinn, Athlon, Linus,

00:37:51.129 --> 00:37:57.869
<v Brent>John, and Scott, who all joined either the core contributors or Jupiter.party. Welcome to the party.

00:37:59.769 --> 00:38:02.269
<v Chris>Yes, thank you for the support. Shout-out to our new members.

00:38:02.389 --> 00:38:05.229
<v Chris>I hope you take advantage of either the ad-free version of the show.

00:38:05.469 --> 00:38:09.289
<v Chris>A little bit tighter runtime, but still with all Editor Drew's nice touches.

00:38:09.669 --> 00:38:13.649
<v Chris>Or get the raw bootleg version that's clocking in usually around double the

00:38:13.649 --> 00:38:15.909
<v Chris>length of the regular show, but lots of content in there.

00:38:16.249 --> 00:38:19.509
<v Chris>And also it's a way to support the show directly. And on top of that,

00:38:19.669 --> 00:38:22.809
<v Chris>it lets us continue to be extremely choosy about who we work with.

00:38:23.149 --> 00:38:28.469
<v Chris>And it's, I think, 100% why we don't have dynamic ads at this point, I would say.

00:38:28.669 --> 00:38:31.249
<v Wes>This week you might get tire talk.

00:38:31.489 --> 00:38:34.569
<v Chris>Yeah, if you listen to the bootleg, you did get a little tire talk.

00:38:34.669 --> 00:38:36.529
<v Chris>That's true. But we really appreciate you guys.

00:38:36.789 --> 00:38:40.449
<v Chris>Thank you for keeping the show going through the bad times. And thank you for

00:38:40.449 --> 00:38:45.549
<v Chris>also letting us say no to dynamic advertising or weird VPNs and weird other

00:38:45.549 --> 00:38:47.309
<v Chris>kind of, you know, another one that's been really.

00:38:47.369 --> 00:38:49.069
<v Wes>I don't even know what kind of mattress you have.

00:38:49.209 --> 00:38:52.609
<v Chris>I'm not. Yeah, that's true. I'm not even anti-CBDs, but like we've been getting

00:38:52.609 --> 00:38:54.449
<v Chris>hit up by a lot of CBD makers.

00:38:55.089 --> 00:38:59.589
<v Chris>I mean, not a lot, but in the last month, I've heard from three different CBD

00:38:59.589 --> 00:39:01.569
<v Chris>makers who want to sell their product directly.

00:39:01.709 --> 00:39:05.129
<v Chris>I don't like this. It's like it's like it's really nice to just be in the position

00:39:05.129 --> 00:39:08.449
<v Chris>to say no, thank you. So that goes to our members, and same with the dynamic

00:39:08.449 --> 00:39:09.929
<v Chris>ads, and also to our boosters, too.

00:39:10.009 --> 00:39:12.769
<v Chris>Thank you, everybody. We really appreciate you. Another round of applause to those new members!

00:39:15.469 --> 00:39:16.989
<v Chris>We got some emails, too, Wes.

00:39:17.749 --> 00:39:21.669
<v Wes>Sebastian wrote in, long-time listener, writing in with some rare critical feedback.

00:39:22.729 --> 00:39:27.889
<v Wes>Zotero deserves way more love. It's one of the most important open-source projects,

00:39:28.049 --> 00:39:30.729
<v Wes>especially in the closed world of academic publishing.

00:39:31.289 --> 00:39:35.809
<v Wes>Zotero 7 is fantastic. Citation management, PDF sync via web dev,

00:39:36.469 --> 00:39:41.789
<v Wes>annotations across devices, even figure extraction, all open source.

00:39:42.389 --> 00:39:44.729
<v Wes>It's a self-hosted essential in my book.

00:39:44.889 --> 00:39:45.069
<v Chris>Okay.

00:39:45.289 --> 00:39:48.909
<v Wes>I'm a cardiologist practicing in Germany, so if there's anything in my field

00:39:48.909 --> 00:39:51.829
<v Wes>I can do for you over from the ocean, let me know.

00:39:52.029 --> 00:39:54.749
<v Wes>Also, if Brent ever ends up in the southern part of Germany and wants to check

00:39:54.749 --> 00:40:00.209
<v Wes>out rock climbing or mountain biking, also let me know. Oh, that's so sweet. Thank you, Sebastian.

00:40:00.209 --> 00:40:04.349
<v Brent>Yes, please. Thanks, Sebastian. And I'm assuming that invite for mountain biking

00:40:04.349 --> 00:40:06.249
<v Brent>is open to these two boys as well?

00:40:06.729 --> 00:40:07.429
<v Chris>Who, us?

00:40:07.769 --> 00:40:10.949
<v Brent>Maybe, if you make it over there on the other side of the ocean.

00:40:11.329 --> 00:40:12.329
<v Wes>Southern Germany meetup?

00:40:12.489 --> 00:40:16.769
<v Chris>Charles also sent us an email. He says, hi, guys. I enjoyed the Terminal TUI challenge.

00:40:17.049 --> 00:40:20.589
<v Chris>But you missed a browser. But maybe it hadn't been released at the time you were trying it.

00:40:20.829 --> 00:40:29.109
<v Chris>Anyways, check out Chawan, the graphical TUI browser with JavaScript support, C-H-A-W-A-N.net.

00:40:30.290 --> 00:40:34.830
<v Wes>It's a text mode, web browser, and pager for Unix-like systems with a focus

00:40:34.830 --> 00:40:38.310
<v Wes>on implementing modern web standards while remaining self-contained,

00:40:38.510 --> 00:40:40.990
<v Wes>easy to understand, and extensible.

00:40:40.990 --> 00:40:44.510
<v Wes>It includes functionality like CSS, inline images inside the terminal,

00:40:44.730 --> 00:40:47.850
<v Wes>and JavaScript through a small, independent browser engine.

00:40:48.150 --> 00:40:53.790
<v Wes>It's written from scratch in the memory-safe NIMH programming language. Cool.

00:40:54.050 --> 00:40:54.770
<v Chris>That's really neat.

00:40:54.970 --> 00:40:59.070
<v Wes>It's in Homebrew, NixOS, and the AUR, and it looks like there's also an app

00:40:59.070 --> 00:41:01.210
<v Wes>image. so go check it out neat.

00:41:01.210 --> 00:41:03.570
<v Brent>But i thought only rust was memory safe.

00:41:03.570 --> 00:41:07.850
<v Chris>I love to he sent us some geeking out about emacs too oh yeah.

00:41:07.850 --> 00:41:09.850
<v Wes>Okay it does sixles or the kitty protocol so.

00:41:09.850 --> 00:41:10.410
<v Chris>She'll play nice.

00:41:10.410 --> 00:41:11.010
<v Wes>With kitty too.

00:41:11.010 --> 00:41:11.870
<v Chris>Well that's good to hear.

00:41:11.870 --> 00:41:15.030
<v Brent>Well matthew sent us a note here too hey regarding

00:41:15.030 --> 00:41:18.330
<v Brent>the ask for a note taking and to-do list tools i'd

00:41:18.330 --> 00:41:21.570
<v Brent>like to recommend a silver bullet it's very customizable if

00:41:21.570 --> 00:41:25.890
<v Brent>you want it to be has built-in queryable database of tasks works on mobile and

00:41:25.890 --> 00:41:30.170
<v Brent>desktop syncs between everything and is all marked down i found it suits my

00:41:30.170 --> 00:41:34.490
<v Brent>brain very well indeed it allows me to have separate work and personal workflows

00:41:34.490 --> 00:41:39.970
<v Brent>without causing friction or forcing me to work in a particular way i highly recommend it.

00:41:39.970 --> 00:41:44.470
<v Wes>Oh i'd seen some other folks kind of chatting about this more generally so i've

00:41:44.470 --> 00:41:49.750
<v Wes>been on my list to try but um yeah now with a little feedback from our smart audience yeah.

00:41:49.750 --> 00:41:53.510
<v Chris>I appreciate that matthew silverbullet.md is the website gotta.

00:41:53.510 --> 00:41:55.050
<v Wes>Love a queryable database of tests.

00:41:55.050 --> 00:41:58.710
<v Chris>So i you know this isn't quite the same level but

00:41:58.710 --> 00:42:01.570
<v Chris>what i decided to do after the show is i'm

00:42:01.570 --> 00:42:05.730
<v Chris>giving obsidian one more go for task management it has this concept of a daily

00:42:05.730 --> 00:42:10.670
<v Chris>note and then i found some plugins that allow me to pull tasks forward and render

00:42:10.670 --> 00:42:14.630
<v Chris>them in kind of like this visual way and uh you kind of create the task with

00:42:14.630 --> 00:42:19.990
<v Chris>a pretty simple markdown syntax so it might work for me we'll see it did work okay for me last week,

00:42:21.211 --> 00:42:24.511
<v Chris>I don't think I'm quite in the zone yet, though. But thank you,

00:42:24.571 --> 00:42:26.411
<v Chris>Matthew. Silverbullet.md. I may

00:42:26.411 --> 00:42:28.771
<v Chris>keep that in the back of my mind if the Obsidian stuff doesn't work out.

00:42:29.391 --> 00:42:33.591
<v Wes>Joe wrote in, answering the distro choice question in a FlatHub,

00:42:33.791 --> 00:42:39.671
<v Wes>Flatpak-only world, I would still consider update and upgrade frequencies to be pretty key.

00:42:40.111 --> 00:42:44.271
<v Wes>Fedora is six months and normally pretty solid, but I've seen some issues with

00:42:44.271 --> 00:42:48.951
<v Wes>lesser-used hardware, that kind of thing, and so I might still prefer a longer LTS.

00:42:49.691 --> 00:42:53.851
<v Wes>Also, I like flat packs for most desktop apps, but I think they could be a nightmare

00:42:53.851 --> 00:42:55.291
<v Wes>for server-side services.

00:42:55.671 --> 00:42:59.791
<v Wes>Apache, Samba, NFS, SFTP, they can be deployed a lot of ways,

00:42:59.791 --> 00:43:03.991
<v Wes>and even the Docker files will often need to get updates or get specific packages.

00:43:04.591 --> 00:43:08.891
<v Wes>File permissions and keyboard input are still edge cases, and it'd be sad to

00:43:08.891 --> 00:43:10.951
<v Wes>lose all development to traditional package types.

00:43:11.451 --> 00:43:14.691
<v Chris>Very true. Yeah, that I don't think can ever really fully go away,

00:43:15.391 --> 00:43:17.131
<v Chris>because some of this is based on all of that.

00:43:17.131 --> 00:43:23.091
<v Chris>But to your point about Fedora being six months and being pretty solid,

00:43:23.131 --> 00:43:29.351
<v Chris>but sometimes a little edgy with certain hardware, I wonder if,

00:43:29.451 --> 00:43:31.691
<v Chris>Joe, you would find Helium OS interesting.

00:43:31.931 --> 00:43:38.511
<v Chris>So they just released version 10, and Helium OS is an atomic distro that uses

00:43:38.511 --> 00:43:43.791
<v Chris>CentOS Stream and Alma Linux and kind of combines it together for a long-term

00:43:43.791 --> 00:43:45.891
<v Chris>support plasma atomic desktop.

00:43:47.333 --> 00:43:50.473
<v Chris>That could be pretty fascinating. I don't know if you get access to DNF still

00:43:50.473 --> 00:43:51.733
<v Chris>or those types of things.

00:43:51.913 --> 00:43:55.933
<v Chris>Because to your point, Flatpak isn't really a solution for server-side applications.

00:43:57.973 --> 00:44:02.733
<v Chris>But containers, snaps, you know, modifying the base.

00:44:02.833 --> 00:44:06.433
<v Chris>Like if this is, for example, if you had a server that was image-based,

00:44:07.133 --> 00:44:11.913
<v Chris>you know, that's cloud native, you would probably just modify your base image

00:44:11.913 --> 00:44:16.573
<v Chris>to include the Apache, Samba, NFS, and SFTP you needed. and then you would just

00:44:16.573 --> 00:44:17.913
<v Chris>ship your image with that stuff baked in.

00:44:18.633 --> 00:44:21.233
<v Wes>And for at least as long as we keep building things that way,

00:44:21.333 --> 00:44:23.513
<v Wes>where we do have these base images built from individual packages,

00:44:23.513 --> 00:44:26.433
<v Wes>I imagine we'll have to keep developing on those too.

00:44:26.533 --> 00:44:31.193
<v Chris>For sure. And we did get some boosts we wanted to shout out this week.

00:44:31.313 --> 00:44:35.193
<v Chris>A lot of people supported the show, but one of them stood out as our baller

00:44:35.193 --> 00:44:36.653
<v Chris>booster, no doubt about it.

00:44:36.753 --> 00:44:41.373
<v Chris>And that is Blackhost coming in with 100,000 sats.

00:44:55.673 --> 00:45:01.053
<v Chris>Thank you, Blackhost. This is just helping you help me help us all.

00:45:01.193 --> 00:45:04.553
<v Chris>Because, of course, that's the value for value cycle.

00:45:08.013 --> 00:45:13.153
<v Chris>That came in live as we are going. So thank you, Blackhost, for the live boost and being our baller.

00:45:14.593 --> 00:45:18.193
<v Chris>We really appreciate that that's going to turn around this episode right there

00:45:18.193 --> 00:45:22.413
<v Chris>how about that injecting some last minute energy into the show as we go out

00:45:22.413 --> 00:45:25.013
<v Chris>thank you black host really appreciate that.

00:45:25.013 --> 00:45:28.833
<v Wes>Turd ferguson comes in with 64 000 cents.

00:45:28.833 --> 00:45:29.273
<v Chris>There he is.

00:45:32.818 --> 00:45:38.338
<v Wes>Duckstation dev is dropping linux support oh yeah the duck the duckstation dev

00:45:38.338 --> 00:45:43.878
<v Wes>is dropping linux support and blocking arch builds fed up with quote headaches

00:45:43.878 --> 00:45:49.078
<v Wes>and hacks for a two percent user base for something i don't use says to grip

00:45:49.078 --> 00:45:52.398
<v Wes>the source for wayland to understand yeah.

00:45:52.398 --> 00:45:56.678
<v Chris>I saw this i also saw something about their they said their license explicitly

00:45:56.678 --> 00:45:58.018
<v Chris>prohibits linux packaging.

00:45:58.598 --> 00:46:01.738
<v Wes>I looked at that very briefly. It seems like patching at least.

00:46:01.918 --> 00:46:05.178
<v Wes>I don't know if it didn't actually say packaging to my read.

00:46:05.418 --> 00:46:08.558
<v Chris>Oh, I thought they did say packaging. Maybe I misread that. Maybe I did.

00:46:09.018 --> 00:46:11.338
<v Chris>You know, they do ship an app image.

00:46:11.438 --> 00:46:17.218
<v Wes>But I'm not an expert on the license. But it just kind of shows you the intent going on here.

00:46:17.318 --> 00:46:21.138
<v Chris>Well, and maybe a bit of the friction point that we were getting to last week too as well.

00:46:21.758 --> 00:46:24.658
<v Chris>And this developer is getting quite fed up. You know, they're talking about

00:46:24.658 --> 00:46:28.598
<v Chris>dropping support because it's, quote, 2% of the user base. I feel like we should

00:46:28.598 --> 00:46:31.718
<v Chris>send people over to DuckStation, get more Linux users using it,

00:46:31.898 --> 00:46:34.598
<v Chris>bring that number up, and maybe provide some helpful input.

00:46:34.878 --> 00:46:38.978
<v Chris>They ship a Windows binary, a Mac binary, and an app image, and I think that's

00:46:38.978 --> 00:46:41.318
<v Chris>probably their way of having you run it.

00:46:41.798 --> 00:46:47.218
<v Chris>And we have seen other projects get frustrated by downstream packaging and adding certain patches to it.

00:46:47.318 --> 00:46:49.558
<v Wes>A perennial Linux problem, unfortunately.

00:46:50.038 --> 00:46:56.858
<v Chris>Yeah, but at the end of the day, it's end users that kind of suffer when these things happen.

00:46:57.018 --> 00:46:59.938
<v Chris>So it is unfortunate to see the Duck Station developer do that. Thank you, Turd.

00:47:00.078 --> 00:47:04.718
<v Wes>It is also just, I don't know, I sympathize with devs having to provide support,

00:47:05.298 --> 00:47:09.098
<v Wes>but there's also just, you know, I like free and open source software,

00:47:09.138 --> 00:47:14.758
<v Wes>and the code flows, and that means people are going to ship it in all kinds of ways.

00:47:15.018 --> 00:47:15.818
<v Chris>True, true.

00:47:16.518 --> 00:47:20.398
<v Brent>Well, A-Ron comes in with 50,000 sats.

00:47:26.875 --> 00:47:30.615
<v Brent>On the network, has there been any talk or consideration around small embedded

00:47:30.615 --> 00:47:35.515
<v Brent>systems like using ESP32s with Home Assistant or similar niche projects?

00:47:35.515 --> 00:47:40.815
<v Brent>I was curious if you all had ever talked about doing a deep dive in this area

00:47:40.815 --> 00:47:42.175
<v Brent>specifically for an episode.

00:47:42.355 --> 00:47:46.195
<v Brent>Maybe best devices, use cases, gotchas, soldering techniques,

00:47:46.435 --> 00:47:47.835
<v Brent>best practices, all of that.

00:47:47.955 --> 00:47:50.755
<v Brent>I'd gladly appreciate it. i bought a

00:47:50.755 --> 00:47:56.095
<v Brent>kit to build a lightsaber from the kyber temple and it is way more soldering

00:47:56.095 --> 00:47:59.935
<v Brent>than i realized i know a little bit about this area but would like to know more

00:47:59.935 --> 00:48:04.115
<v Brent>it's taking me probably 10 times longer to put it together than i probably should

00:48:04.115 --> 00:48:09.695
<v Brent>but i'm having fun doing it a new hobby has been unlocked i'm sure my wife won't mind at all.

00:48:09.695 --> 00:48:14.355
<v Wes>Well depending on where you are we could just send our roving brent technician to you.

00:48:14.355 --> 00:48:15.995
<v Brent>This is quite true you.

00:48:15.995 --> 00:48:19.935
<v Chris>Know that would work Although it may take a little longer than you expect,

00:48:20.075 --> 00:48:21.015
<v Chris>but it's going to be done great.

00:48:21.455 --> 00:48:25.395
<v Brent>And probably, you know, watch your budget for the food portion of the project.

00:48:25.615 --> 00:48:30.555
<v Chris>That's true. That's true. So as you probably know, we did dive into some ESP

00:48:30.555 --> 00:48:33.775
<v Chris>use cases in Linux Unplugged episode 620.

00:48:34.335 --> 00:48:37.915
<v Brent>Yeah. And that episode called Brent Loves Building Things, because occasionally I do.

00:48:38.455 --> 00:48:41.275
<v Brent>We did explore, I think, a little bit more of the

00:48:41.275 --> 00:48:44.295
<v Brent>software side of things and how to integrate things

00:48:44.295 --> 00:48:47.115
<v Brent>into home assistant and get your esp going with

00:48:47.115 --> 00:48:50.075
<v Brent>some configurations but what i hear you saying here more is

00:48:50.075 --> 00:48:52.775
<v Brent>kind of the physical part of it

00:48:52.775 --> 00:48:55.475
<v Brent>as well like soldering you mentioned quite a bit

00:48:55.475 --> 00:49:01.195
<v Brent>which is um i would say a set of skills that i have been developing only recently

00:49:01.195 --> 00:49:08.995
<v Brent>as well and we certainly thanks to our dear producer jeff have a bunch of collective

00:49:08.995 --> 00:49:12.855
<v Brent>knowledge about sort of shortcuts, what to use for connectors,

00:49:13.175 --> 00:49:16.555
<v Brent>what to use as a great cheap soldering iron to get you started.

00:49:16.895 --> 00:49:22.555
<v Brent>All these tips and tricks that I think a lot of us have been interested in as a shared hobby.

00:49:22.755 --> 00:49:25.915
<v Brent>And of course, we should have expected that listeners would get interested too.

00:49:26.095 --> 00:49:28.215
<v Brent>So that's a great idea for another episode in the future.

00:49:28.555 --> 00:49:34.315
<v Chris>I will say the next one kind of on my radar, Aeron, is an ultrasonic sensor

00:49:34.315 --> 00:49:36.775
<v Chris>that I can use to measure how full a tank is.

00:49:37.295 --> 00:49:39.895
<v Chris>This is a great use for an ESP here and a little ultrasonic sensor.

00:49:40.175 --> 00:49:44.995
<v Chris>But yeah, this is an idea we'll kick around some more because there is so much

00:49:44.995 --> 00:49:48.675
<v Chris>to this ecosystem and I think probably just people fully wrapping their heads

00:49:48.675 --> 00:49:50.835
<v Chris>around it is what's needed to really appreciate the value there.

00:49:51.295 --> 00:49:56.495
<v Chris>Because you can buy things today that are built around the ESP32 platform and

00:49:56.495 --> 00:50:00.035
<v Chris>it's inside the product. You don't even know it, but it can be pretty nice to

00:50:00.035 --> 00:50:01.095
<v Chris>know what's actually in there.

00:50:01.355 --> 00:50:04.435
<v Chris>So thank you very much. Appreciate it, Aaron. Nice to hear from you.

00:50:06.335 --> 00:50:07.515
<v Chris>And I hope we'll hear from you again,

00:50:09.815 --> 00:50:12.775
<v Chris>Marcel boosted in with 20,000 sats.

00:50:17.124 --> 00:50:24.264
<v Chris>Way back in Linux Unplugged 4.28, you featured my project, RUT's message of

00:50:24.264 --> 00:50:28.264
<v Chris>the day in the pics, R-U-T-S-M-O-T-D in the pics.

00:50:28.444 --> 00:50:31.864
<v Chris>Well, version 2.0 was just released and it's better than ever.

00:50:32.064 --> 00:50:37.104
<v Chris>The new feature that I am the most excited about is showing the status of your Docker Compose stacks.

00:50:37.424 --> 00:50:41.744
<v Chris>Not only individual containers, it would be great to have some listeners go

00:50:41.744 --> 00:50:44.364
<v Chris>and test on it for me and feature requests are welcome.

00:50:44.544 --> 00:50:48.564
<v Chris>It's available in Nix and it's also in the AUR. So this is a neat idea.

00:50:48.664 --> 00:50:52.924
<v Chris>So you'll log into your box on the terminal and you get, amongst other things

00:50:52.924 --> 00:50:55.584
<v Chris>now, Docker Compose stats.

00:50:56.884 --> 00:50:58.704
<v Chris>Brilliant idea, Marcel. Good to hear.

00:50:59.004 --> 00:51:00.724
<v Wes>Rust message of the day.

00:51:00.924 --> 00:51:01.724
<v Chris>Oh, was that a typo?

00:51:01.764 --> 00:51:02.144
<v Wes>That's right.

00:51:02.344 --> 00:51:06.404
<v Chris>Oh, it was a typo. Oh, okay. That was weird. There you go. Rust message of the

00:51:06.404 --> 00:51:09.244
<v Chris>day. There we go. Now it's official.

00:51:09.284 --> 00:51:14.964
<v Wes>And it is indeed written in Rust. Well, okay. It's 98.8% Rust because 1.2% is next.

00:51:17.384 --> 00:51:21.124
<v Chris>Building it with nicks too i love it thank you marcel appreciate that.

00:51:23.604 --> 00:51:25.744
<v Wes>Doornail 7887 comes in with a row of ducks,

00:51:26.991 --> 00:51:31.371
<v Wes>With the slow shift to immutable, I wonder about offline and sensitive networks.

00:51:31.671 --> 00:51:36.671
<v Wes>With RHEL packaging and Apple, we get one-shot approvals, easy ingress mirrors,

00:51:36.891 --> 00:51:38.311
<v Wes>patching, all that kind of stuff.

00:51:38.551 --> 00:51:41.991
<v Wes>If flatpacks are the future, it seems like it will complicate things for this

00:51:41.991 --> 00:51:44.211
<v Wes>kind of use case, enterprise usage.

00:51:44.571 --> 00:51:48.891
<v Wes>Another downside might be storage, and scientific computing application packages

00:51:48.891 --> 00:51:51.611
<v Wes>add up quick. Am I missing an easy answer?

00:51:51.611 --> 00:51:56.511
<v Chris>Well, I think the one thing that gives me some confidence that this won't be

00:51:56.511 --> 00:52:02.071
<v Chris>a huge problem doornail is that these technologies could all be pointed at local repositories.

00:52:02.951 --> 00:52:06.571
<v Chris>So if you use Flatpaks and you ship it with Flathub as default,

00:52:07.031 --> 00:52:08.411
<v Chris>there's no reason why somebody couldn't.

00:52:08.571 --> 00:52:11.311
<v Chris>If they were shipping their own internal image, they could modify that base

00:52:11.311 --> 00:52:14.971
<v Chris>image to point to their own local repository for Flatpaks and their own local

00:52:14.971 --> 00:52:17.511
<v Chris>repository for brew even or whatever it might be.

00:52:17.731 --> 00:52:20.551
<v Chris>Now, your point is well taken that, well, that eats up a lot of storage.

00:52:20.551 --> 00:52:25.371
<v Chris>But you could probably argue that you could probably whittle it down to just

00:52:25.371 --> 00:52:29.551
<v Chris>the applications actually needed and the ones that are standard you would just

00:52:29.551 --> 00:52:33.131
<v Chris>bake into the image anyway so there would be less packages that people probably

00:52:33.131 --> 00:52:36.971
<v Chris>dynamically pull down because if it's a if it's a standard suite of applications

00:52:36.971 --> 00:52:39.291
<v Chris>everybody's using you're just going to build that into the image anyways and

00:52:39.291 --> 00:52:40.371
<v Chris>the user won't have to install that.

00:52:40.371 --> 00:52:43.531
<v Wes>Well i mean flat pack does have some facilities for

00:52:43.531 --> 00:52:46.451
<v Wes>that right in terms of relying on base images that other things yeah

00:52:46.451 --> 00:52:49.291
<v Wes>right so if you are building your own and like they you can't actually take

00:52:49.291 --> 00:52:52.451
<v Wes>advantage of that i do think you know uh to

00:52:52.451 --> 00:52:55.271
<v Wes>the side where you're not necessarily missing something easy i think

00:52:55.271 --> 00:52:58.111
<v Wes>is the side of just there probably is a fair amount of work in

00:52:58.111 --> 00:53:01.931
<v Wes>this kind of thing anytime you get new formats that just all the

00:53:01.931 --> 00:53:05.491
<v Wes>all the extra tooling on top right like the nexus

00:53:05.491 --> 00:53:09.831
<v Wes>proxies the approval systems the integrations with other systems those may just

00:53:09.831 --> 00:53:13.811
<v Wes>have to be stuff that people do work on as enough there's enough pressure to

00:53:13.811 --> 00:53:18.951
<v Wes>to use these new systems but at least a lot of it is built on oci stuff and

00:53:18.951 --> 00:53:24.051
<v Wes>so there's already some pressure there in terms of storage and deduple and um you know tools around it.

00:53:24.051 --> 00:53:30.291
<v Brent>Well wine eagle has a series of boosts here totaling nine thousand and one satoshis,

00:53:33.206 --> 00:53:36.426
<v Brent>Well, last week, my laptop's battery suddenly died.

00:53:36.686 --> 00:53:42.746
<v Brent>When I booted again, my ButterFS plus LVM Luxe Partition was totally fried.

00:53:43.106 --> 00:53:45.106
<v Chris>Whoa, whoa. I wonder if he hit this bug.

00:53:45.226 --> 00:53:47.366
<v Wes>Maybe. Although totally fried sounds worse.

00:53:47.746 --> 00:53:50.966
<v Chris>Yeah, I mean, but it does look totally fried because it's unmountable and it

00:53:50.966 --> 00:53:55.306
<v Chris>won't boot. So you could consider it totally fried. Oh, Wine Eagle. Oh, no. Let us know.

00:53:56.226 --> 00:54:01.106
<v Brent>Eagle continues here. Fearing a repeat, I formatted my laptop.

00:54:01.106 --> 00:54:04.126
<v Brent>Up gaming pc and nas

00:54:04.126 --> 00:54:06.906
<v Brent>to zfs on route thanks to nixos for making

00:54:06.906 --> 00:54:10.626
<v Brent>it that easy but i made a huge mistake i

00:54:10.626 --> 00:54:15.766
<v Brent>forgot not all my data on the server was in the pool many services kept data

00:54:15.766 --> 00:54:22.006
<v Brent>in slash var slash lib whoops lesson learned on that one don't forget to set

00:54:22.006 --> 00:54:27.366
<v Brent>data directories i almost gave my wife a heart attack regarding bitwarden thankfully

00:54:27.366 --> 00:54:30.446
<v Brent>we were still logged in for the export Oof.

00:54:30.646 --> 00:54:33.126
<v Wes>Well, I'm glad to hear this didn't go as poorly as it could have.

00:54:33.226 --> 00:54:35.806
<v Wes>And thank you for telling us about it. I mean, I'm sorry to hear.

00:54:36.166 --> 00:54:40.166
<v Chris>I should have made this point earlier in the show. I would like to point out

00:54:40.166 --> 00:54:42.106
<v Chris>that you are actually more at

00:54:42.106 --> 00:54:46.526
<v Chris>risk using ZFS with a kernel module that is developed outside the kernel.

00:54:46.766 --> 00:54:50.246
<v Chris>Now, in this particular case, it was ButterFS that was impacted.

00:54:50.666 --> 00:54:57.346
<v Chris>But if you think about it, a system that is properly integrated with the kernel had an entire team,

00:54:59.186 --> 00:55:03.446
<v Chris>you know, I mean, dozens and dozens of people that were able to work on this

00:55:03.446 --> 00:55:06.246
<v Chris>problem from the kernel side, the file system side, the distribution side,

00:55:06.346 --> 00:55:10.226
<v Chris>the user side, they were all able to collectively work together to solve this

00:55:10.226 --> 00:55:12.946
<v Chris>problem because it's GPL code and it's integrated into the kernel.

00:55:13.266 --> 00:55:16.986
<v Chris>If it's an external module that is getting linked to the kernel,

00:55:17.226 --> 00:55:21.286
<v Chris>it is actually more likely that internal changes to the kernel will break it.

00:55:21.506 --> 00:55:24.426
<v Chris>And when it does break, which it is more likely to

00:55:24.426 --> 00:55:27.746
<v Chris>do there will be less people that can collectively work

00:55:27.746 --> 00:55:34.746
<v Chris>on it to solve the problem so you're not really safer with root on zfs now that

00:55:34.746 --> 00:55:37.786
<v Chris>said zfs is a great file system the people that work on the kernel driver and

00:55:37.786 --> 00:55:40.786
<v Chris>modules are very very professional and they try to do a great job and the kernel

00:55:40.786 --> 00:55:44.426
<v Chris>team does everything they can to try to prevent breakage same is also true for

00:55:44.426 --> 00:55:45.246
<v Chris>the butterfest developers,

00:55:46.788 --> 00:55:51.088
<v Chris>So just being aware that when it's an external, if it's a proprietary driver

00:55:51.088 --> 00:55:56.528
<v Chris>like for NVIDIA or a file system that has wonky drivers because Oracle,

00:55:56.928 --> 00:56:01.848
<v Chris>or I mean wonky licenses, you are actually at more risk of this problem that bit ButterFS recently.

00:56:02.028 --> 00:56:05.268
<v Chris>It just happens to be this time ButterFS was unlucky.

00:56:05.548 --> 00:56:10.208
<v Wes>To Wineagle's point about setting data directories, that's just a handy thing

00:56:10.208 --> 00:56:15.408
<v Wes>that tons of NixOS services have where you can just define where do you want

00:56:15.408 --> 00:56:18.328
<v Wes>to store your data, and a lot of them default to slash var slash lib.

00:56:18.628 --> 00:56:22.108
<v Wes>You can also like have that whole directory be a mount point somewhere else

00:56:22.108 --> 00:56:25.928
<v Wes>if you want to. Lots of good ways to sort of make sure you put that data where you want.

00:56:26.068 --> 00:56:29.848
<v Chris>It just sounds like that was a massive hassle. I mean, you know, scaring the wife.

00:56:30.788 --> 00:56:34.128
<v Chris>Why do you go, I like that part I feel for you. Like I said just a moment ago,

00:56:34.248 --> 00:56:35.828
<v Chris>I was picking up Obsidian again.

00:56:36.208 --> 00:56:41.248
<v Chris>And so I opened up Obsidian and I opened up all, I opened up the extensions and I said update all.

00:56:41.968 --> 00:56:44.988
<v Chris>And then I closed the settings window and all my notes were gone.

00:56:45.548 --> 00:56:53.548
<v Chris>And i had a heart attack like i felt my heart sank and i and i and i i took a i took a beat,

00:56:54.328 --> 00:56:57.348
<v Chris>and i closed the window and i realized it's okay it's it

00:56:57.348 --> 00:57:00.648
<v Chris>should be marked down on the file system still so i i very quickly navigated

00:57:00.648 --> 00:57:03.508
<v Chris>to the directory and i verified all of the markdown files were there and i very

00:57:03.508 --> 00:57:08.768
<v Chris>quickly tarred them up set it aside and then opened up obsidian again and they

00:57:08.768 --> 00:57:14.928
<v Chris>were all back but that that that feeling that gut punch of losing data it oh i hate that so much.

00:57:15.068 --> 00:57:17.708
<v Chris>Sorry to hear you went through that, Wineagle. And thank you for the boost.

00:57:18.908 --> 00:57:21.168
<v Wes>Jordan Bravo comes in with a row of ducks.

00:57:22.680 --> 00:57:26.780
<v Wes>Hey, guys, I wanted to share that I made my first contribution to Nix packages.

00:57:27.100 --> 00:57:27.300
<v Chris>Hey!

00:57:27.660 --> 00:57:28.340
<v Wes>Nicely done.

00:57:28.560 --> 00:57:29.000
<v Chris>Congratulations.

00:57:29.460 --> 00:57:33.940
<v Wes>You can now find that package. It's called Rust Dress in Nix packages unstable.

00:57:34.600 --> 00:57:40.880
<v Wes>It's a lightning address server written in Rust that also provides Noster NIP 5 name verification.

00:57:41.460 --> 00:57:43.820
<v Wes>Shout out to Nitesh, the author of Rust Dress.

00:57:43.920 --> 00:57:46.080
<v Chris>Hey, this is exactly what we need.

00:57:46.240 --> 00:57:48.420
<v Wes>It does seem like kind of exactly what we need.

00:57:48.520 --> 00:57:50.340
<v Chris>This is exactly what you and I were talking about this week.

00:57:50.340 --> 00:57:51.960
<v Wes>Uh-huh. We'll have to check that out.

00:57:52.080 --> 00:57:52.580
<v Chris>That's amazing.

00:57:52.580 --> 00:57:53.560
<v Wes>Now with Jordan's package.

00:57:54.000 --> 00:57:57.640
<v Chris>Jordan, thank you. And we needed it in Nix, too, because we have Bitcoin Nix nodes.

00:57:57.880 --> 00:58:01.320
<v Chris>That's something we're working on the back end this week. And name verification

00:58:01.320 --> 00:58:04.800
<v Chris>using NIP5 is, that's really the chef's kiss there.

00:58:05.260 --> 00:58:08.300
<v Chris>Thank you for letting us know, Jordan. And thank you for the value. Appreciate it.

00:58:08.960 --> 00:58:11.960
<v Brent>Well, Outdoor Geek boosted in 5,000 sats.

00:58:13.700 --> 00:58:19.460
<v Brent>From Fosse in Portland, Oregon. AI generated reports of non-existent bugs are

00:58:19.460 --> 00:58:21.700
<v Brent>clogging up bug bounty programs.

00:58:22.360 --> 00:58:30.120
<v Brent>The talk on this plausible AI slop starts at about 6.43.00 in the schedule.

00:58:30.420 --> 00:58:34.800
<v Brent>Really good layouts of the problem and an abstract. Any links to the recording?

00:58:35.080 --> 00:58:38.720
<v Chris>Thank you, Outdoor Geek. I watched this this morning as we were preparing the show.

00:58:39.460 --> 00:58:44.880
<v Chris>And I think my takeaway would be is possibly skip this talk and just go read

00:58:44.880 --> 00:58:49.120
<v Chris>the Curl developer's complaints directly. because a lot of the talk this gal

00:58:49.120 --> 00:58:51.900
<v Chris>gives is predicated on the fact that the curl developer is concerned.

00:58:52.120 --> 00:58:56.180
<v Chris>So therefore she is concerned. She actually says that. And then I think actually

00:58:56.180 --> 00:58:59.040
<v Chris>Outdoor Geek, she kind of builds a lot of straw man cases.

00:58:59.280 --> 00:59:02.900
<v Chris>Well, I read on Reddit or I saw this blog post about one person complaining.

00:59:04.160 --> 00:59:07.400
<v Chris>And I think it's not a bad talk and it is a problem.

00:59:08.500 --> 00:59:12.780
<v Chris>But it feels like one of those things that we're getting really worked up about

00:59:12.780 --> 00:59:14.880
<v Chris>that will come to a resolution.

00:59:15.240 --> 00:59:19.960
<v Chris>And so I don't necessarily love the fear-mongering. I mean, the reality is it

00:59:19.960 --> 00:59:23.080
<v Chris>creates an issue today, but the tools will get better over time,

00:59:23.080 --> 00:59:25.240
<v Chris>and the tools on the other side will get better over time.

00:59:25.620 --> 00:59:28.200
<v Chris>And one thing I think we should keep in mind, and I'm not saying this is the

00:59:28.200 --> 00:59:32.080
<v Chris>direction it's going to go, but I think this is just something to keep in the back of our mind.

00:59:32.200 --> 00:59:35.920
<v Chris>In the free software and open source community, we're constantly saying we want

00:59:35.920 --> 00:59:39.000
<v Chris>to bring in new people. We want the new generation to come in,

00:59:39.080 --> 00:59:41.440
<v Chris>the next generation. We need more people contributing.

00:59:41.700 --> 00:59:45.860
<v Chris>Well, maybe this is how new people contribute. Because I don't necessarily buy

00:59:45.860 --> 00:59:50.480
<v Chris>into this sort of unspoken theory that's being floated that there's all these

00:59:50.480 --> 00:59:55.380
<v Chris>bots out there that are auto-generating patches and bug reports and then somehow

00:59:55.380 --> 00:59:56.560
<v Chris>submitting them to projects.

00:59:56.620 --> 01:00:00.880
<v Chris>I don't buy the idea that LLMs are spinning up on their own and then bots are

01:00:00.880 --> 01:00:02.940
<v Chris>using them autonomously to generate all of this.

01:00:02.940 --> 01:00:05.700
<v Chris>I think it's more likely that it's people that are

01:00:05.700 --> 01:00:08.960
<v Chris>seeking kind of glory they're looking for recognition

01:00:08.960 --> 01:00:13.820
<v Chris>they have a favorite project they want to contribute to and this is their perhaps

01:00:13.820 --> 01:00:18.020
<v Chris>they don't speak English these types of things and this is their way to now

01:00:18.020 --> 01:00:22.100
<v Chris>have a tool and an avenue to contribute for the first time and they're doing

01:00:22.100 --> 01:00:25.220
<v Chris>so and the results I'm not saying that's all of it some of it probably is bought

01:00:25.220 --> 01:00:26.360
<v Chris>traffic but that is costly.

01:00:27.579 --> 01:00:32.599
<v Chris>What I'm saying is we may, in our effort, to hate everything that's AI,

01:00:32.659 --> 01:00:39.499
<v Chris>because what that gal proposes is some sort of global ban list where you start banning people.

01:00:39.519 --> 01:00:43.259
<v Chris>And if I ban them, then you automatically ban them and we all ban them together.

01:00:43.699 --> 01:00:47.039
<v Chris>I think before we start getting into some sort of global ban list,

01:00:47.039 --> 01:00:50.499
<v Chris>we need to consider that who we might be banning are the very new contributors

01:00:50.499 --> 01:00:51.979
<v Chris>that we want to come join us.

01:00:52.079 --> 01:00:55.859
<v Chris>And the reality is these tools are likely going to get better.

01:00:55.859 --> 01:00:59.979
<v Chris>And when they do get better, these people may be contributing something of value.

01:01:00.099 --> 01:01:02.539
<v Chris>Maybe it's only one out of three, but that's something.

01:01:02.679 --> 01:01:04.899
<v Chris>And then if the tools on the other side get better to manage it,

01:01:05.099 --> 01:01:07.339
<v Chris>there actually may be a decent middle ground here.

01:01:07.439 --> 01:01:10.839
<v Chris>But if we are getting all worked up about it now while these tools are still

01:01:10.839 --> 01:01:14.719
<v Chris>nascent and the social issues haven't been worked out yet and the culture hasn't

01:01:14.719 --> 01:01:19.659
<v Chris>been established yet, but we take these sort of dramatic hyperbolic reactions

01:01:19.659 --> 01:01:21.359
<v Chris>with global ban lists and things like that,

01:01:21.479 --> 01:01:26.619
<v Chris>I think we may actually be blocking future contributors to free software in

01:01:26.619 --> 01:01:27.939
<v Chris>our attempt to reduce spam.

01:01:28.219 --> 01:01:32.199
<v Chris>And so it's a very complicated topic, and I don't think it really got the full justice it deserves.

01:01:32.319 --> 01:01:35.639
<v Chris>And I think what I would recommend is people just go look up the curl developers

01:01:35.639 --> 01:01:36.859
<v Chris>concerned about AI slop.

01:01:37.539 --> 01:01:42.159
<v Chris>And keep in mind that it's an evolving problem. But I really appreciate that

01:01:42.159 --> 01:01:44.339
<v Chris>because it was a great video to watch this morning. Thank you, Outdoor Geek.

01:01:45.979 --> 01:01:49.619
<v Chris>Well, look, it's PJ there, and he comes in with a big old double,

01:01:50.999 --> 01:01:54.239
<v Chris>I guess, duck boosts, which would be an Aflac.

01:01:55.339 --> 01:01:56.499
<v Chris>4,444 sats.

01:01:56.759 --> 01:01:57.999
<v Wes>Cornish game head boost?

01:01:58.179 --> 01:02:01.419
<v Chris>Mmm, we got to get that. And he just says one thing and says it boldly.

01:02:02.419 --> 01:02:04.099
<v Chris>Thank you, PJ. Appreciate that.

01:02:04.359 --> 01:02:08.979
<v Wes>And I pulled up Brewer Seth just because why not 500 sats to us just to say

01:02:08.979 --> 01:02:11.019
<v Wes>enjoyed this show this week for 625.

01:02:11.299 --> 01:02:15.679
<v Chris>Well, thank you. Appreciate that. Thank you, everybody who boosts.

01:02:15.759 --> 01:02:19.639
<v Chris>We do have a 2,000-sat cutoff for airtime, but every now and then we pull a

01:02:19.639 --> 01:02:23.439
<v Chris>few forward and we appreciate everybody who boosts or stream sats as they listen,

01:02:23.539 --> 01:02:25.899
<v Chris>which is really neat too. You're just streaming those over a peer-to-peer network

01:02:25.899 --> 01:02:26.739
<v Chris>as you listen to the show.

01:02:27.399 --> 01:02:31.579
<v Chris>18 of you did that, and we stacked 18,670 sats that way.

01:02:31.779 --> 01:02:40.479
<v Chris>Now, thanks to our baller boosters, we stacked a healthy 276,602 sats.

01:02:48.870 --> 01:02:52.230
<v Chris>Thank you, everybody. Appreciate that. Of course, our members who set that support

01:02:52.230 --> 01:02:56.210
<v Chris>on Autopilot and those of you who took the time to set up a boost and send it in.

01:02:56.790 --> 01:03:00.230
<v Chris>Fountain.fm really makes it easy because they host the wallet for you and really

01:03:00.230 --> 01:03:02.170
<v Chris>just walk you through the process of connecting it to, like,

01:03:03.190 --> 01:03:06.590
<v Chris>Strike or Stripe or other fiat systems as well.

01:03:06.750 --> 01:03:10.390
<v Chris>But, of course, there's an entire self-hosted stack out there, all open source.

01:03:10.690 --> 01:03:15.610
<v Chris>That journey starts at podcastapps.com. Thank you so much for supporting episode

01:03:15.610 --> 01:03:21.010
<v Chris>626 of the Unplugged program. It means the world to us.

01:03:23.170 --> 01:03:26.830
<v Chris>Now we have a few good picks this week. A few good picks, Wes.

01:03:27.390 --> 01:03:29.270
<v Chris>And I bet you're a little surprised by this one.

01:03:30.310 --> 01:03:32.950
<v Chris>I don't think we've ever really had one in this category before.

01:03:33.410 --> 01:03:38.790
<v Chris>It's called Plex Ripper, and it is a cross-platform Plex media downloader that

01:03:38.790 --> 01:03:43.230
<v Chris>seamlessly adds media from your friends' Plex servers to your own.

01:03:43.230 --> 01:03:47.210
<v Wes>Okay, so we've gone past the part where we rip the DVDs into Plex.

01:03:47.370 --> 01:03:47.470
<v Chris>Right.

01:03:47.610 --> 01:03:50.150
<v Wes>Now we rip out of Plex.

01:03:50.230 --> 01:03:50.690
<v Chris>Yeah, buddy.

01:03:50.870 --> 01:03:51.510
<v Wes>Or get back into another.

01:03:51.650 --> 01:03:54.930
<v Chris>You got a buddy who's been backing up your media for you, I guess?

01:03:55.590 --> 01:03:59.290
<v Chris>It's a neat idea because you establish a Plex content network,

01:04:00.010 --> 01:04:03.510
<v Chris>which is, I think, the core feature that Plex still offers over Jellyfin for some people.

01:04:03.810 --> 01:04:07.510
<v Chris>And this thing has a really nice interface. You can run it as a container if you like.

01:04:07.970 --> 01:04:12.430
<v Chris>And you can even log into multiple Plex accounts. And it has a back end that's

01:04:12.430 --> 01:04:14.690
<v Chris>multi-threaded. and resumes downloads.

01:04:14.910 --> 01:04:16.270
<v Wes>It's ready for some big rips.

01:04:16.390 --> 01:04:20.130
<v Chris>Yeah, dude. I mean, it's a fun idea. I will be honest with you.

01:04:21.230 --> 01:04:25.370
<v Chris>Plex and I are no longer together, and I've been happily seeing Jellyfin now

01:04:25.370 --> 01:04:27.990
<v Chris>for a while, but if I still ran Plex...

01:04:28.801 --> 01:04:30.701
<v Chris>I mean, this is a pretty cool idea.

01:04:31.001 --> 01:04:34.521
<v Wes>Oh, there's also guides for Unraid and Synology. So you don't have to have a

01:04:34.521 --> 01:04:36.621
<v Wes>crazy self-hosted super setup or anything.

01:04:36.821 --> 01:04:40.561
<v Chris>It's GPL3, but they say they prohibit downloading content from servers without

01:04:40.561 --> 01:04:44.681
<v Chris>proper authorization. They say if you use a tool to bypass copyright, you can't do that.

01:04:45.261 --> 01:04:49.421
<v Chris>Or you can't use it to circumvent restrictions. So it won't download Plex Pass stuff.

01:04:49.741 --> 01:04:53.181
<v Wes>Yeah, that makes sense. Written in C Sharp. Interesting. Cool.

01:04:53.941 --> 01:04:57.781
<v Chris>GPL3. Then this next one, it's just for us Plasma users out there.

01:04:57.781 --> 01:05:02.621
<v Chris>But Wes and I came across a couple of handy Plasma add-ons recently.

01:05:02.981 --> 01:05:07.281
<v Chris>This first one, I'm a little embarrassed that I'm recommending because it is

01:05:07.281 --> 01:05:09.081
<v Chris>inspired by macOS a little bit.

01:05:09.161 --> 01:05:12.421
<v Chris>And I generally don't go that way. It's KDE Control Station.

01:05:12.781 --> 01:05:15.481
<v Chris>And it's a widget you can add to your desktop or to your toolbar.

01:05:15.481 --> 01:05:20.281
<v Chris>And it gives you a menu to really easily toggle things like do not disturb,

01:05:21.001 --> 01:05:26.481
<v Chris>Wi-Fi, Bluetooth, display brightness, system volume, night dark mode,

01:05:26.821 --> 01:05:31.901
<v Chris>night mode, battery status, log in, log out, all in one little menu.

01:05:32.241 --> 01:05:36.761
<v Chris>Where Plasma sort of separates those out into about four different menus right

01:05:36.761 --> 01:05:38.481
<v Chris>now, this is all in one spot.

01:05:38.481 --> 01:05:43.301
<v Chris>But it is using a Mac-inspired UI, which I'm not a fan of, but the functionality

01:05:43.301 --> 01:05:46.061
<v Chris>is good enough that I'm going to suck that up, and I'm going to recommend it

01:05:46.061 --> 01:05:48.161
<v Chris>to you this week. So it's KDE Control Station.

01:05:48.801 --> 01:05:51.941
<v Chris>If you're already running Plasma, you can actually just go add a widget and

01:05:51.941 --> 01:05:55.161
<v Chris>go where you can search widgets and just put in KDE Control Station,

01:05:55.181 --> 01:05:55.901
<v Chris>and it'll come up for you.

01:05:56.121 --> 01:05:58.561
<v Wes>Yeah, I just thought that, I mean, it seemed handy enough. The look,

01:05:58.781 --> 01:06:01.841
<v Wes>okay, do what you will with that, but it is nice functionality.

01:06:02.101 --> 01:06:03.621
<v Chris>And now how about this one?

01:06:05.701 --> 01:06:11.121
<v Chris>Kairpods native airpod integration for the plasma 6 desktop with real-time battery

01:06:11.121 --> 01:06:14.761
<v Chris>monitoring noise control and a panel widget,

01:06:17.391 --> 01:06:21.331
<v Chris>I was always shocked that you could pair AirPods at all to a Linux desktop,

01:06:21.391 --> 01:06:23.151
<v Chris>but you missed all these features.

01:06:23.571 --> 01:06:27.771
<v Chris>Now, with this widget, you can toggle things like their sound transparency mode,

01:06:27.771 --> 01:06:29.891
<v Chris>or you can turn on active noise blocking.

01:06:30.131 --> 01:06:32.711
<v Chris>You get individual pod battery level.

01:06:33.831 --> 01:06:38.391
<v Chris>This really is kind of that last piece that was missing on the Linux desktop.

01:06:38.571 --> 01:06:43.251
<v Chris>And I'm bringing this up again because I know AirPods are very popular,

01:06:43.251 --> 01:06:45.951
<v Chris>and there's probably some of you out there that have this. And also,

01:06:46.051 --> 01:06:48.031
<v Chris>this is a GPL add-on as well, GPL3.

01:06:48.151 --> 01:06:48.771
<v Wes>Written in Rust.

01:06:49.011 --> 01:06:51.691
<v Chris>Oh, it is. Well, there you go. You got one more in there.

01:06:53.471 --> 01:07:00.431
<v Chris>Now, you have a pair of Sony AirPods, right? Yeah. Do those pair okay with the Linux desktop?

01:07:00.491 --> 01:07:02.351
<v Wes>They do, especially under PipeWire. Really no issues.

01:07:02.591 --> 01:07:06.291
<v Chris>I have the PixelPods. That's what I use. And they also pair with the Linux desktop.

01:07:06.431 --> 01:07:09.651
<v Chris>But I don't have any control for the noise stuff. You probably don't with the

01:07:09.651 --> 01:07:10.551
<v Chris>Sonys either, I would imagine.

01:07:10.731 --> 01:07:11.691
<v Wes>No, that's true.

01:07:11.791 --> 01:07:12.811
<v Chris>That's a nice feature.

01:07:12.951 --> 01:07:13.131
<v Wes>Yeah.

01:07:13.251 --> 01:07:14.151
<v Chris>That is really nice to see.

01:07:14.171 --> 01:07:14.351
<v Wes>Sophisticated.

01:07:14.531 --> 01:07:15.071
<v Chris>Mm-hmm.

01:07:15.071 --> 01:07:19.511
<v Wes>What a weird world. Suddenly, AirPods might be a great option for your Linux desktop.

01:07:19.771 --> 01:07:22.471
<v Chris>Yeah, yeah. Well, there you go. Links to all of that in the show notes,

01:07:22.471 --> 01:07:26.031
<v Chris>which you will find over at linuxunplugged.com slash 626.

01:07:26.851 --> 01:07:30.731
<v Chris>Remember, too, we also want to know if you've been spooked off of ButterFS or

01:07:30.731 --> 01:07:34.911
<v Chris>if you have any file system war stories that you'd like to share with us,

01:07:35.071 --> 01:07:38.611
<v Chris>recent or not. It's funny. I was just saying we haven't heard very many file system issues.

01:07:38.951 --> 01:07:41.991
<v Chris>And then, there we go.

01:07:42.731 --> 01:07:44.031
<v Chris>Oh, wow, that's crazy.

01:07:44.031 --> 01:07:48.191
<v Wes>But we love hearing those things, war stories, and your success stories.

01:07:48.311 --> 01:07:51.471
<v Chris>That's true. That's true. And also, if you're crazy enough to run ButterFS en

01:07:51.471 --> 01:07:54.511
<v Chris>route, of course, over here, Wes Payne's running BcacheFS.

01:07:54.751 --> 01:07:55.211
<v Wes>This is true.

01:07:55.391 --> 01:07:55.571
<v Chris>Yeah.

01:07:59.511 --> 01:08:02.531
<v Chris>Yeah. Why not join us live? Make it a Tuesday on a Sunday.

01:08:03.091 --> 01:08:06.371
<v Chris>We start the show at Sunday, 10 a.m. Pacific, 1 p.m.

01:08:06.691 --> 01:08:09.831
<v Chris>Eastern, in your local time over at jupiterbroadcasting.com slash calendar.

01:08:09.831 --> 01:08:14.251
<v Chris>Anything that supports an audio stream an icecast stream just plug in jblive.fm

01:08:14.251 --> 01:08:20.111
<v Chris>and tune in or get a podcasting 2.0 app because you also get that's your part,

01:08:21.111 --> 01:08:26.611
<v Chris>transcripts chapters you do do I have to remind you now no we gotta remind them well.

01:08:26.611 --> 01:08:27.231
<v Wes>Yeah we do gotta remind.

01:08:27.231 --> 01:08:29.771
<v Chris>Them and the live stream it's all in there cloud.

01:08:29.771 --> 01:08:30.751
<v Wes>Chapters no less.

01:08:30.751 --> 01:08:31.311
<v Chris>Okay you.

01:08:31.311 --> 01:08:32.191
<v Wes>Don't undersell these.

01:08:32.191 --> 01:08:33.511
<v Chris>Chapters super fancy both.

01:08:33.511 --> 01:08:36.791
<v Wes>Baked into the file and JSON on the cloud.

01:08:36.791 --> 01:08:39.971
<v Chris>Boom thank you for join us. See you right back here next Sunday.

