++ ATOT official NEF thread part IV ++

[bigboxes]

It's actually good news that the enterprise drives work. I spent enough cash on them that they better work. Fun hobby. Yay.

 

[bigboxes]

Where's everybody this evening?

 

[bigboxes]

I am due for a HDD upgrade myself at some point.

Most of the drives in here are like 1TB:

Seems like a waste of electricity and very noisy. Did you get these dirt cheap? I'm sure we've discussed this a little before. I'd rather have a few big drives than a bank of smaller drives. What exactly is your total capacity?

 

[Red Squirrel]

Seems like a waste of electricity and very noisy. Did you get these dirt cheap? I'm sure we've discussed this a little before. I'd rather have a few big drives than a bank of smaller drives. What exactly is your total capacity?

The only reason the drives are so small is that it's what I have, so as they die I tend to go with bigger drives. When I built that box I think 3-4TB may have been the biggest drive you could get and the 1TB drives were already in an array on another server so they just got moved. I have one small raid array with 3TB drives and then another with 2TB drives I got for cheap, then the oldest array is the raid 5 with 1TB drives. It's dated like 2008 lol.

As they die or as I require more expansion I'll swap out drives with larger ones. This is the current capacity, not that much by today's standards though:

Filesystem            Size  Used Avail Use% Mounted on
/dev/md0              5.4T  3.5T  1.7T  69% /volumes/raid1
/dev/md1              6.3T  5.3T  693G  89% /volumes/raid2
/dev/md3              7.2T  1.7T  5.2T  25% /volumes/raid3

raid2 is mostly just used for backups of the other arrays, then I have a separate backup job that just backs up to stand alone drives. I run it once in a while and bring a drive to work.

Interestingly that server does not draw all that much power, when I originally built it it was only drawing 75w. I have not tested with all the drives though. It's almost full. Have a few spare slots.

I don't use the SAN enclosures though, they actually use way more power and have less capacity. The drives are like 400GB lol.

 

[Red Squirrel]

Raid info:

[root@isengard ~]# mdadm --detail /dev/md0
/dev/md0:
        Version : 1.2
  Creation Time : Thu Sep  5 00:19:01 2013
     Raid Level : raid10
     Array Size : 5860270080 (5588.79 GiB 6000.92 GB)
  Used Dev Size : 2930135040 (2794.39 GiB 3000.46 GB)
   Raid Devices : 4
  Total Devices : 4
    Persistence : Superblock is persistent

    Update Time : Tue Oct 17 00:10:35 2017
          State : clean 
 Active Devices : 4
Working Devices : 4
 Failed Devices : 0
  Spare Devices : 0

         Layout : near=2
     Chunk Size : 512K

           Name : isengard.loc:0  (local to host isengard.loc)
           UUID : 2e257e19:33dab86c:2e112e06:b386598e
         Events : 429

    Number   Major   Minor   RaidDevice State
       0      65       32        0      active sync set-A   /dev/sds
       1      65       48        1      active sync set-B   /dev/sdt
       2      65       64        2      active sync set-A   /dev/sdu
       3      65       16        3      active sync set-B   /dev/sdr
[root@isengard ~]# 
[root@isengard ~]# 
[root@isengard ~]# 
[root@isengard ~]# mdadm --detail /dev/md1
/dev/md1:
        Version : 0.90
  Creation Time : Sat Sep 20 02:15:28 2008
     Raid Level : raid5
     Array Size : 6837319552 (6520.58 GiB 7001.42 GB)
  Used Dev Size : 976759936 (931.51 GiB 1000.20 GB)
   Raid Devices : 8
  Total Devices : 8
Preferred Minor : 1
    Persistence : Superblock is persistent

    Update Time : Tue Oct 17 00:10:38 2017
          State : active 
 Active Devices : 8
Working Devices : 8
 Failed Devices : 0
  Spare Devices : 0

         Layout : left-symmetric
     Chunk Size : 64K

           UUID : 11f961e7:0e37ba39:2c8a1552:76dd72ee
         Events : 0.2206891

    Number   Major   Minor   RaidDevice State
       0       8      144        0      active sync   /dev/sdj
       1      65        0        1      active sync   /dev/sdq
       2       8      176        2      active sync   /dev/sdl
       3       8      160        3      active sync   /dev/sdk
       4       8      224        4      active sync   /dev/sdo
       5       8      128        5      active sync   /dev/sdi
       6       8      208        6      active sync   /dev/sdn
       7       8      192        7      active sync   /dev/sdm
[root@isengard ~]# 
[root@isengard ~]# 
[root@isengard ~]# 
[root@isengard ~]# mdadm --detail /dev/md3
/dev/md3:
        Version : 1.2
  Creation Time : Mon Jul 28 23:31:31 2014
     Raid Level : raid10
     Array Size : 7813531648 (7451.56 GiB 8001.06 GB)
  Used Dev Size : 1953382912 (1862.89 GiB 2000.26 GB)
   Raid Devices : 8
  Total Devices : 8
    Persistence : Superblock is persistent

    Update Time : Tue Oct 17 00:10:40 2017
          State : active 
 Active Devices : 8
Working Devices : 8
 Failed Devices : 0
  Spare Devices : 0

         Layout : near=2
     Chunk Size : 512K

           Name : isengard.loc:3  (local to host isengard.loc)
           UUID : 99f0389f:dbf75cb3:c841340e:33f62841
         Events : 383

    Number   Major   Minor   RaidDevice State
       0       8       48        0      active sync set-A   /dev/sdd
       1       8       32        1      active sync set-B   /dev/sdc
       2       8       16        2      active sync set-A   /dev/sdb
       3       8        0        3      active sync set-B   /dev/sda
       4       8      112        4      active sync set-A   /dev/sdh
       5       8       96        5      active sync set-B   /dev/sdg
       6       8       80        6      active sync set-A   /dev/sdf
       7       8       64        7      active sync set-B   /dev/sde
[root@isengard ~]#

 

[Red Squirrel]

I had one of the 1TB drives die a while back and replaced it with a spare. Don't have any spares left, so next one that dies I will probably just consider retiring that array and building a larger raid 10. May even look into ZFS. I think I should be able to run ZFS and mdadm raid on same box, then I can slowly migrate stuff. When I first set this up ZFS on Linux was kind of a new thing, but now I think it's considered production worthy.

 

[Red Squirrel]

I also want to look into Gluster FS though, I was thinking it would be cool to have a VM environment where instead of a traditional setup with multiple VM servers and one SAN/NAS, each VM server is also storage but with high availability for both VMs and storage. I'm pretty sure this is possible with GFS and Quemu, just not easy to setup at all... so I'd probably want to automate it and build a custom distro or something. Would be awesome to setup a turn key VM environment with full HA and everything that is based on open source.

 

[Red Squirrel]

That way you don't need expensive HBAs for the drives, as each server would just have like 8 drives that can use the motherboard ports. With the size of drives now you could setup a fairly large raid 10 with just 8 drives.

 

[bigboxes]

For those wondering...

H: Music
I: Other Video (Cartoons, Sports, Music, Personal, etc.)
J: TV Shows 1
K: TV Shows 2
L: TV Shows 3
M: Movies 1
N: Movies 2
Q: Adult

On my main rig...
F: Temp music
O: Temp video (software, Android backups, TV Shows, books, training)
P: Temp video (movies, music disc images)

Looks like I'm going to merge the data on Discs K & L onto one new drive. When I put together the dual Xeon rig, that's where you'll see the bank of 10 He enterprise drives. That's my end goal for the home server project. The 8TB drives will become my backup drives after I complete the dual Xeon rig.

 

[bigboxes]

nite moo

 

[Red Squirrel]

No raid? I can't imagine that many drives without raid. Even with backups, still a pita to deal with a drive failure.

 

[bigboxes]

No raid? I can't imagine that many drives without raid. Even with backups, still a pita to deal with a drive failure.

Not really. For home, JBOD is probably the least problematic. I realize you are the ultimate geek when it comes to this. If a drive dies I just replace the drive and copy the data from the corresponding backup drive. Not to mention it's a money thing. I do plan on implementing a RAID or equivalent in the home server. However. RAID is for uptime, not backup.

 

[Red Squirrel]

Yeah I guess so, but I prefer being able to just slap a new drive in and not even have to worry about touching the data or digging up backups. Then having to reset file permissions and all that jazz. Like some stuff my require different permissions than others. Like web stuff has to be owned by apache some stuff has to be chmodded a certain way etc. so I'd hate having to deal with that if a drive dies. Basically I don't want to have to spend more time than I have to if a drive dies.

I did suffer a partial raid failure once though, but managed to recover it without needing to go to backups. Basically a raid 5 with 2 drives failing at once after a long power outage. It was actually what prompted me to setup my big UPS.

 

[bigboxes]

Yeah I guess so, but I prefer being able to just slap a new drive in and not even have to worry about touching the data or digging up backups. Then having to reset file permissions and all that jazz. Like some stuff my require different permissions than others. Like web stuff has to be owned by apache some stuff has to be chmodded a certain way etc. so I'd hate having to deal with that if a drive dies. Basically I don't want to have to spend more time than I have to if a drive dies.

I did suffer a partial raid failure once though, but managed to recover it without needing to go to backups. Basically a raid 5 with 2 drives failing at once after a long power outage. It was actually what prompted me to setup my big UPS.

Are we talking work or home? Permissions? You don't realize how much work you actually put into that to maintenance that. I suspect that you have trouble leaving your work at work. LOL. In the end, what we all have is overkill. We do it because we like to do it. Why? Because we can.

I just have backup images of the OS drive and how I have everything setup. That makes recovery simple. Just restore the image. Like you said, who wants to spend all that time getting everything back to the way you like it. But there has to be a sane limit. You just like to do all that cuz you do it every day. Not that it's necessary in a home. Nobody I know has a setup like mine either. That's why we are here on ATOT discussing our setups.

 

[Pick2]

Good Morning Planet Earth !

 

[sdifox]

Good Morning Planet Earth !

It's night for a lot of people

 

[bigboxes]

It's night for a lot of people

The sky is blue.

 

[Bubbleawsome]

Nef

 

[sdifox]

Ni

 

[Pick2]

So now it's down to following the new boxer around with a plastic bag until he decides to shit ... stool samples ... the joys of dog ownership

 

[sdifox]

So now it's down to following the new boxer around with a plastic bag until he decides to shit ... stool samples ... the joys of dog ownership

 

[bigboxes]

So now it's down to following the new boxer around with a plastic bag until he decides to shit ... stool samples ... the joys of dog ownership

My cats deposits them in a box. No following them around.

 

[bigboxes]

lunch moo

 

[bigboxes]

moo

 

[Red Squirrel]

Nef

I finally booked an appointment to get a will done. Had been thinking about that before but just never actually remembered to do it.

Now I'm probably going to die.