mastodon.sdf.org is part of the decentralized social network powered by Mastodon.
"I appreciate SDF but it's a general-purpose server and the name doesn't make it obvious that it's about art." - Eugen Rochko

Administered by:

Server stats:

2.6K
active users

Learn more

JulianCalaby<p>Cursed homelab update:</p><p>So server #2 is humming along nicely, however continuing to use the disk that nearly scuttled my repartition / recovery effort was not a good idea.</p><p>Me: creates a Ceph OSD on a known faulty hard disk<br>Faulty hard disk: has read errors causing a set of inconsistent PGs<br>Me: Surprised Pikachu face</p><p>Thankfully this was just read errors, no actual data has been lost.</p><p>So for a brief, glorious moment, I had just under 50TB of raw storage and now it's just under 49TB.</p><p>And for me now, the big question is: do I do complicated partition trickery to work around the bad spots (it's a consecutive set of sectors) or do I junk the disk and live with 1 less TB of raw storage?</p><p>In other news, I know understand a little bit more about Ceph and how it recovers from errors: PGs don't get "fixed" until they are next (deep) scrubbed, which means that if your PGs get stuck undersized, degraded or inconsistent (or any other state) it could be that they're not getting scrubbed.</p><p>So taking the broken OSD on the bad HDD offline immediately caused all but 2 of the inconsistent PGs to get fixed, and the remaining 2 just wouldn't move, so I smashed out a trivial script to deep scrub all PGs and last night, a couple of days after this all went down, one got fixed. Now hopefully the other will get sorted out soon.</p><p>ceph pg ls | awk '{print $1}' | grep '^[[:digit:]]' | xargs -l1 ceph pg deep-scrub</p><p>So read errors -&gt; scrub errors -&gt; inconsistent PGs.</p><p>Then: inconsistent PGs -&gt; successful scrub -&gt; recovery</p><p>What this also means is that while I stopped the latest phase of the Big Copy to (hopefully) protect my data, I think I can start it again with some level of confidence.</p><p><a href="https://social.treehouse.systems/tags/homelab" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>homelab</span></a> <a href="https://social.treehouse.systems/tags/cursed" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>cursed</span></a> <a href="https://social.treehouse.systems/tags/cursedhomelab" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>cursedhomelab</span></a> <a href="https://social.treehouse.systems/tags/ceph" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ceph</span></a></p>
Pete Keen<p><a href="https://hachyderm.io/tags/HomeLab" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>HomeLab</span></a> update for today:</p><p>- tore down and recreated Talos VMs a couple times</p><p>- set up a wrapper for talhelper that pulls secrets from 1Password</p><p>- set up a wrapper for helmfile that also pulls secrets from 1Password</p><p>- got a workable cluster running with three control plane nodes and one worker with the above setup</p><p>I'm gonna tear it down and rebuild it from scratch at least once more before I'm ready to start deploying anything real. Have to wait for the storage machine to get here anyway.</p>
Miroslav Stankic<p>Time for <a href="https://mastodon.social/tags/PatchSaturday" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>PatchSaturday</span></a>.<br><a href="https://mastodon.social/tags/ProxMox" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ProxMox</span></a> and a lot of <a href="https://mastodon.social/tags/Docker" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Docker</span></a> containers. <br>Gonna be a fun night. </p><p><a href="https://mastodon.social/tags/HomeLab" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>HomeLab</span></a> <a href="https://mastodon.social/tags/SelfHosted" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>SelfHosted</span></a> <a href="https://mastodon.social/tags/SelfHosting" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>SelfHosting</span></a></p>
Jeff<p><span>Found this txt file titled "Important-reminder" in the BIOS folder on Bee Link's support site. Helpful!<br><br></span><a href="https://fedia.social/tags/HomeLab" rel="nofollow noopener noreferrer" target="_blank">#HomeLab</a></p>
Zak :1password:<p>I've noticed that a lot of people that I've talked to on the subject of backups, including in response to this comment, do not back up their TV and movies. Which makes sense considering how much storage those things tend to take up. It just seems like a shame to be risking all of it to disk failure.</p><p>Backing all of that up to something like Backblaze would be incredibly expensive, and even purchasing an additional set of drives just for backups would be quite a pain. There doesn't seem to be a great solution other than "just replace it all when your disks fail." Either that, or spend the money.</p><p>(I'm set on everything else. Documents and photos are in hosted cloud storage, and my music will always be in at least one other place.)</p><p><a href="https://infosec.exchange/tags/homelab" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>homelab</span></a> <a href="https://infosec.exchange/tags/selfhosting" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>selfhosting</span></a></p>
Benny<p>Another upgrade for the <a href="https://mastodontech.de/tags/Homelab" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Homelab</span></a>: <a href="https://mastodontech.de/tags/3Dprinted" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>3Dprinted</span></a> a mount for the <a href="https://mastodontech.de/tags/ComputeBlade" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ComputeBlade</span></a> to get it onto the <a href="https://mastodontech.de/tags/IKEA" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>IKEA</span></a> <a href="https://mastodontech.de/tags/Skadis" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Skadis</span></a> board. Simple, but solid for now!</p><p><a href="https://mastodontech.de/tags/RaspberryPi" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>RaspberryPi</span></a> <a href="https://mastodontech.de/tags/3Dprint" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>3Dprint</span></a></p>
Tao of Mac<p>The Chuwi Larkbox S</p><p>It’s been a little while since I last looked at regular mini-PCs, and considering that the market is flooded with cheap Intel N100 and N150 variants, you would think there isn’t a (...)</p><p><a href="https://mastodon.social/tags/chuwi" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>chuwi</span></a> <a href="https://mastodon.social/tags/corei3" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>corei3</span></a> <a href="https://mastodon.social/tags/desktop" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>desktop</span></a> <a href="https://mastodon.social/tags/hardware" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>hardware</span></a> <a href="https://mastodon.social/tags/homelab" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>homelab</span></a> <a href="https://mastodon.social/tags/intel" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>intel</span></a> <a href="https://mastodon.social/tags/minipc" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>minipc</span></a> <a href="https://mastodon.social/tags/review" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>review</span></a></p><p><a href="https://taoofmac.com/space/blog/2025/04/05/1800" rel="nofollow noopener noreferrer" target="_blank"><span class="invisible">https://</span><span class="ellipsis">taoofmac.com/space/blog/2025/0</span><span class="invisible">4/05/1800</span></a></p>
Charlotte<p>niche product desire: small form factor computer that supports ECC memory, so I can use it as a tiny storage server attached to a big disk array. Checksumming filesystems will defend against bad ram, but I'd still like ECC without the requisite electricity bill that comes with the associated platforms.</p><p>I guess I could build a little micro-ATX Ryzen system or something, unsure if those boards support ECC though.</p><p><a href="https://toot.dusepo.co.uk/tags/homelab" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>homelab</span></a></p>
Rachel<p>As I plan out the new MiniLab, here is the <a href="https://mikrotik.social/@mikrotik" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@mikrotik@mikrotik.social</a><span> wishlist:<br><br>* A CRS310-8G+2S+IN variant with PoE, to support newer access points and SBCs that have PoE and 2.5gb NICs<br>* A Half-rack sized switch with 8ish 10gb ports and a mix of RJ45/SFP+<br><br>Bonus round<br><br>* 0.5u half rack switches that can stack alongside RB5009U routers, something like 8g PoE with either 2.5 or 10gb uplink, or other variations<br></span><a href="https://transitory.social/tags/MikroTik" rel="nofollow noopener noreferrer" target="_blank">#MikroTik</a> <a href="https://transitory.social/tags/HomeLab" rel="nofollow noopener noreferrer" target="_blank">#HomeLab</a> <a href="https://transitory.social/tags/Networking" rel="nofollow noopener noreferrer" target="_blank">#Networking</a></p>
Rachel<p><span>I'm always looking at new shiny tech, previously I was looking at cheap hardware I could learn to learn on, ended up with some dells with plenty of ram and dual cpus<br><br>Last year I started working towards lowering power usage<br><br>Now I'm looking at building something even lower power usage that I can fit in a suitcase or put on a passenger seat because I feel like the safe option will be to gtfo </span><a href="https://transitory.social/tags/Homelab" rel="nofollow noopener noreferrer" target="_blank">#Homelab</a> <a href="https://transitory.social/tags/Networking" rel="nofollow noopener noreferrer" target="_blank">#Networking</a> <a href="https://transitory.social/tags/Tech" rel="nofollow noopener noreferrer" target="_blank">#Tech</a></p>
Shrirang Kahale<p><a href="https://fosstodon.org/tags/homelab" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>homelab</span></a></p>
Mike<p>I'm decommissioning my Proxmox server in favour of running libvirt on my main server. I only had one VM I wanted to keep: the Windows 2000 domain controller. I looked into how to convert the VM disk and other stuff. But, there's another way more fun option: setup a new VM as a secondary domain controller and then promote it to primary.</p><p><a href="https://mas.to/tags/homelab" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>homelab</span></a> <a href="https://mas.to/tags/retrocomputing" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>retrocomputing</span></a> <a href="https://mas.to/tags/Windows2000" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Windows2000</span></a></p>
Paul Buetow<p>I had some fun with <a href="https://fosstodon.org/tags/FreeBSD" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>FreeBSD</span></a>, <a href="https://fosstodon.org/tags/Bhyve" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Bhyve</span></a> and <a href="https://fosstodon.org/tags/Rocky" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Rocky</span></a> <a href="https://fosstodon.org/tags/Linux" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Linux</span></a>. Not just for fun, also for science and profit!</p><p><a href="https://foo.zone/gemfeed/2025-04-05-f3s-kubernetes-with-freebsd-part-4.html" rel="nofollow noopener noreferrer" target="_blank"><span class="invisible">https://</span><span class="ellipsis">foo.zone/gemfeed/2025-04-05-f3</span><span class="invisible">s-kubernetes-with-freebsd-part-4.html</span></a></p><p><a href="https://fosstodon.org/tags/homelab" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>homelab</span></a> <a href="https://fosstodon.org/tags/selfhosting" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>selfhosting</span></a> <a href="https://fosstodon.org/tags/RockyLinux" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>RockyLinux</span></a></p>
Jeff Sikes<p>Tried a <a href="https://mastodon.social/tags/TDARR" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>TDARR</span></a> setup tonight in my <a href="https://mastodon.social/tags/homelab" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>homelab</span></a> with limited success.</p><p>ProxMox &gt; Portainer &gt; Tdarr (Web + Node)</p><p>Passing through the NAS mount is always fun, but I had done it before and took good notes.</p><p>This time, I ran into issues setting up Quick Sync, which is supposed to make transcoding faster. It will require a BIOS change. Another night.</p><p>All to rip our aged collection of DVDs....</p>
Rachel<p><span>Hmmmm, anyone working with NVMeoF in their homeland? <br><br></span><a href="https://transitory.social/tags/Homelab" rel="nofollow noopener noreferrer" target="_blank">#Homelab</a> <a href="https://transitory.social/tags/BadIdeas" rel="nofollow noopener noreferrer" target="_blank">#BadIdeas</a></p>
Zak :1password:<p>Those of you who have your own home media servers, do you back up your media? If so, how?</p><p><a href="https://infosec.exchange/tags/homelab" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>homelab</span></a> <a href="https://infosec.exchange/tags/selfhosting" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>selfhosting</span></a></p>
Salearlyman<p>Hey fellow sysadmin cosplay nerds, does anyone here use VyOS? I use VyOS as the main router for my home network and I just found out that a job for uploading backups to an off-site location is strangling my upstream bandwidth.</p><p>I want to create a QOS / traffic shaping policy to treat this as bulk traffic - take up as much bandwidth as available but give priority to all other traffic.</p><p>Is there a simple way to do this? I don't want to allocate a fixed bandwidth for it.</p><p><a href="https://cloudisland.nz/tags/HomeLab" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>HomeLab</span></a> <a href="https://cloudisland.nz/tags/vyos" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>vyos</span></a> <a href="https://cloudisland.nz/tags/sysadmin" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>sysadmin</span></a></p>
♡ Eva Winterschön ♡<p>💝 Systems Hardware Friday 💝</p><p>Yet another afternoon stroll on over to the storage unit, locating spares and system parts from former projects. These will be allocated to three systems in my homelab (the rack at home this time): </p><p>1. Raptor Talos II box, needs a pair of Optane NVMe boot drives ✔️<br>2. Supermicro, EPYC Zen4, an additional Nvidia GPU for CUDA meta-port dev ✔️<br>3. Supermicro, E3/v6 Xeon, a second QAT 8960 for OpenZFS + OpenSSL comp/decomp/crypto offload ✔️<br>~. Napatech 4x10G SmartNIC, this is for my TRex load generators, so it's going to the colo ✔️</p><p>Should be a relaxing evening, mostly focused on the installs, and maybe some project planning too. </p><p><a href="https://mastodon.bsd.cafe/tags/homelab" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>homelab</span></a> <a href="https://mastodon.bsd.cafe/tags/hardware" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>hardware</span></a> <a href="https://mastodon.bsd.cafe/tags/systems" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>systems</span></a> <a href="https://mastodon.bsd.cafe/tags/freebsd" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>freebsd</span></a> <a href="https://mastodon.bsd.cafe/tags/linux" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>linux</span></a> <a href="https://mastodon.bsd.cafe/tags/engineering" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>engineering</span></a> <a href="https://mastodon.bsd.cafe/tags/nvidia" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>nvidia</span></a> <a href="https://mastodon.bsd.cafe/tags/intel" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>intel</span></a> <a href="https://mastodon.bsd.cafe/tags/fridayfun" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>fridayfun</span></a></p>
beyondwatts<p>scaling back up now works too. I forgot about suspending CronJobs but that should be an easy add. </p><p>Now all I need to do is label some workloads - deployments , statefulsets, clusters etc… although I did get somewhat distracted tonight wondering if I can infer this from the Kubernetes API details. Ended up writing some code to draw network graphs with cytoscape and mermaid. Oops!</p><p><a href="https://beyondwatts.social/tags/homelab" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>homelab</span></a> <a href="https://beyondwatts.social/tags/upsScaler" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>upsScaler</span></a></p>
ItzTrain<p>Ok So it took me a while and I still feel like I'm not fully sure of <a href="https://hachyderm.io/tags/seaweedfs" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>seaweedfs</span></a> . I got 3 master servers ( which I don't think I need). I have 1 filer and one s3 gateway and one volume server ( right now). I'm uploading to it right now and using rclone ( which is pretty dope) to mount them to my services. Right now it's just Linux ISO's. </p><p><a href="https://hachyderm.io/tags/homelab" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>homelab</span></a> <a href="https://hachyderm.io/tags/selfhosted" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>selfhosted</span></a> <a href="https://hachyderm.io/tags/selfhosting" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>selfhosting</span></a></p>