Tiernan's Comms Closet

Geek, Programmer, Photographer, network egineer…

Day 53 of #100daysofhomelab

March 10, 2023
0 Comments

Day 53 of and It’s been a busy week… ish… I’ve been battling with Vertigo on and off this week, so haven’t don’t a lot. I did, however, fix some issues with the network, set up a proper failover WAN connection using SmoothWAN and my Quad 2.5Gb Box, and have started making major changes to GodBoxV3.

Originally, GodBoxV3 had all spinning disks (8 8Tb drives shucked from WD My Book 8TBs or 8TB Seagate IronWolf) in a single RAID 5 pool in Windows Storage Spaces. Then the NVMe drives were a second pool (5 of them, 4 Force MP510 480Gb NVMe SSDs on a Hyper M.2 x16 card and a 5th unbranded one of a 1X PCI-E add-in card) and a third pool of 2 960Gb IronWolf SSDs.

I deleted the RAID5 and NVMe arrays, and now, for testing, I have spun up a TrueNAS Core VM on the 2 SSDs and passed the NVMe and HDDs into that VM. Windows can still “see” them, but they are marked as offline, but Hard Disk Sentinel and CrystalDiskInfo can both see the SMART status of them (TrueNAS cant, weirdly…). Then, I have 7 of the 8 drives added to a single pool (one is failing so I left it out, this is for testing currently, anyway) and then the 5 NVMes are added to a second pool (the plan is to use the 4 Force MP510s or replacement drives as a single pool, then the other NVMe (or maybe even 2) as a Cache or Log for the Spinning disk pool).

So far, doesn’t matter if I am using the NVMe or HDD pool, but speeds from my Mac (with a 10Gb Thunderbolt adapter) are around the same… Might be a config issue, might be the odd NVNe drive slowing the rest down… but I am happy with the speeds so far… I have seen 3-400Mb/s Writes and 900+ Reads on both NVMe and HDD… Most of that is probably cached… the VM has 64Gb RAM given to it, and the test file was only 5Gb (BlackMagic Disk Speed Test). More testing is required though.

ZFS over multiple DVD/BD-R images

March 7, 2023
0 Comments

A couple of days back, I started thinking about archiving and backup software. I kind of have backups “sorted”, with my MacBook Pro using BackBlaze to backup to the cloud, Time Machine backing it up to my Synology, my VMs on Proxmox being backed up to Proxmox Backup Server off-site, my Synology and QNAPs being backup to B2 and Hetzner and some other bits and bobs… But for the Archiving stuff, I am not really set up… So, I went looking for archiving software. Couldn’t find anything, so asked on r/DataHoarder. Still no options, at the time of posting, but someone did reply with the idea of using DVDs (or Blu Rays) for ZFS...

Ok, that’s just crazy, but in a kind of a good way… kind of like the floppy RAID stuff I have seen… It does help with the storage of data, plus allows for potential loss of data… but it needs some automation to get it fully perfect…

Assuming you are using this for archiving, you could automate building 5 ISOs, just shy of 100Gb each, once a month, create ZFS ZRAID 2 or 3 (depending on how paranoid you are) and then write your data to it. ZRAID lets you lose 1 disk, giving you around 400GB of usage. Z2 brings that up to 2 losable disks, and 300Gb and Z3 is 3 disks and 200GB. I think Z2 would be your best bet, especially if you are using something like MDisk and are storing them safely.

Once finished, unmount and send an email saying you need to write the ISOs to disk. Label each disk with a unique serial number (this is where the archiving software would be handy) plus the set details and number (so, March 2023 Disk 1/5).

If you need something from that backup you stick it in the drives… You can do it with multiple drives, so with 5 disks and ZRAID, you need to mount a minimum of 4 of them. ZRAID2 needs 3 and ZRAID3 needs a minimum of 2… Ideally, you would want 5 of them, allowing you to check all disks (ZFS Scrub) and then get your files off.

A year of archiving would require 5 drives (say 100 quid a pop, USB makes things easier... Internal is possibly cheaper) and 60 disks (I Found 25 100Gb MDisks disks on Amazon for around 500 EUR) costing a total of maybe 2k, with 15 extra disks…

Follow-up questions:

  • Does ZFS allow the mounting of read-only?
  • Could you do this with Rewritable BluRay disks? Could they be mounted directly and written to? Leave them in the drives for the month, let writes do their thing and then archive them once a month? It’s archived, so it doesn’t need to be fast…

Day 52 of #100daysofhomelab

March 5, 2023
0 Comments

Day 52 of and I have been out of the lab most of the day. Still looking through all the docker containers on docker box 1, which has been running for many years now, and trying to figure out how to move everything is going to be fun… looks like I have an iSCSI volume added to the box. It’s shared between a few docker containers… then there are NFS mounts too… This might take longer than expected… So… more digging through YAML…

Day 51 of #100daysofhomelab

March 4, 2023
1 Comment

Day 51 of and I am planning my move of some of my Docker instances in the house to new machines… GodBoxV3 is currently running Windows Server 2022 with a couple of HyperV VMs on it. One runs docker containers and the other USIP from Ubiquiti for managing my EdgeSwitches. I am trying to move these VMs off that machine and do a clean-up, and the plan is to either install Proxmox with TrueNAS as a VM with disks passed directly into it, plus some other VMs or TrueNAS direct with VMs on there… Suggestions? Anyway, as part of the clean-up, I put my custom WordPress Container up on GitHub and it builds new builds nightly. The move is going to be fun, so my weekend will be busy… So, other than that, some links.

Day 50 of #100daysofhomelab

March 2, 2023
0 Comments

Day 50 of (this was stuck in a draft folder, so this is a couple of weeks old… I decided to recycle this as day 50, but it was originally day 37 or something…).

Just about 13 days ago: After running ZFS on my Mac for a few hours, I removed it and installed a trial of the SoftRAID software… I am not sure what was going on, but with ZFS installed, my machine just kept crashing… less than an hour and bang… So, I installed SoftRAID, and the speed ok… Not massive speeds, but not 100% sure I am using the right cables… More testing with cables soon…But in reality, this is software RAID 5 over 5 spinning disks. 270Mb/s read ain’t bad… 115Mb/s write ain’t great, but it’s RAID 5…

Cut to today: The trial of SoftRaid is just about up, and I am not sure I am going to buy it… I have been thinking of installing Proxmox or TrueNAS on GodBoxV3, which already has 8 8Tb Spinning drives, 7 NVMe drives (2 in RAID 0 for boot, and 4 in RAID 5 (ish, Windows Storage Spaces) one not usable for some reason, along with 2 960Gb SSDs). If I use the 5-bay enclosure with GodBoxV3, I can use that as one pool (External) the 8 Spinning disks inside as a second pool, the MVMEs as a third, and the SSDs either as a cache to the 8 internal disks, or possibly a more different pool… But this is something I am still thinking about… Anyway, links to random stuff are below…

UntitledImage

Day 49 of #100daysofhomelab

March 1, 2023
0 Comments

Day 49 of , and I missed this yesterday, but it’s only going to be a link dump… And todays link dump is mostly Mikrotik gear! Some of these are a little cringy (looking at your Solid Rack video) But hilarious nonetheless! Some of these are so new, they don’t even seem to have product pages, just videos announcing them…

Day 48 of #100daysofhomelab

February 27, 2023
1 Comment

Day 48 of and I have been out of commission for the last couple of days… Havnt been well… cold and flu-like symptoms, but luckily, not Covid… Haven’t done a major amount, so it’s mostly links for today, but I did try a few projects and installed them. Links for those are below.

Day 47 of #100daysofhomelab

February 21, 2023
0 Comments

Day 47 of and i have missed a few days due to, well, a mix of laziness and being busy… I was working from my new home office today… still need to get some cabling and other stuff sorted, but we are getting close… nearly a year since it was installed! More on that at a later stage… for today, some links.

Day 46 of #100daysofhomelab

February 19, 2023
0 Comments

Day 46 of and I haven’t had much time to work on the homelab this weekend, but have had some time using it, somewhat indirectly… Plex, Netflix and Disney Plus streaming, etc. Internet is more stable (but not 100%… more messing on that part soon) and the RB5009 is definitely more stable (IPv6 BGP is currently off, and only using 5 of my 14ish BGP sessions I could use… I think 1Gb RAM is struggling, or my filters are wrong… Hopefully, it’s a filter thing, that way I can sort it out without new hardware.

day 45 of #100daysofhomelab

February 17, 2023
0 Comments

Day 45 (little late) of , and still quite busy with $DayJob stuff… Mostly monitoring of the network and the like… Some links are below:

  • My Mastodon Instance

    Buy Me A Coffee

  • Recent Posts

  • Archives

    • DevDmBootstrap4 Created by Danny Machal (DevDm.com)