SmartOS – Mirroring your Zones pool

If you performed your initial SmartOS configuration with a single disk your Zones pool will have no fault-tolerance. If that disk dies you would have lost all your data.

 

I used a single WD Red 2TB disk when I first setup SmartOS. A zpool list -v shows the single disk (c0t3d0) in my zones pool.

# zpool list -v 
NAME         SIZE  ALLOC   FREE  EXPANDSZ    CAP  DEDUP  HEALTH  ALTROOT 
zones       1.81T   484G  1.34T         -    26%  1.00x  ONLINE  - 
  c0t3d0    1.81T   484G  1.34T         -

 

A few weeks after the inital setup I added several other disks to the system. A second 2TB WD Red drive and two old OCZ Vertex2 SSD drives I had lying around. The SSD drives will be used as cache and log drives later. First I will create a mirror to establish some very basic fault tolerance by adding the second WD red drive to the zones pool as a mirror of the existing drive.

 

Below I use the format commando to generate a list of disks devices in my system. I exit format by pressing CTRL+C.

# format 
Searching for disks...done 

AVAILABLE DISK SELECTIONS: 
       0. c0t0d0 <ATA-OCZ-VERTEX2-1.37-55.90GB> 
          /pci@0,0/pci1849,1c02@1f,2/disk@0,0 
       1. c0t1d0 <ATA-OCZ-VERTEX2-1.37-55.90GB> 
          /pci@0,0/pci1849,1c02@1f,2/disk@1,0 
       2. c0t2d0 <ATA-WDC WD20EFRX-68A-0A80-1.82TB> 
          /pci@0,0/pci1849,1c02@1f,2/disk@2,0 
       3. c0t3d0 <ATA-WDC WD20EFRX-68A-0A80-1.82TB> 
          /pci@0,0/pci1849,1c02@1f,2/disk@3,0 
Specify disk (enter its number): ^C

 

A cfgadm -l -v also shows your disk devices.

# cfgadm -l -v
Ap_Id                          Receptacle   Occupant     Condition  Information
When         Type         Busy     Phys_Id
sata0/0::dsk/c0t0d0            connected    configured   ok         Mod: OCZ-VERTEX2 FRev: 1.37 SN:
unavailable  disk         n        /devices/pci@0,0/pci1849,1c02@1f,2:0
sata0/1::dsk/c0t1d0            connected    configured   ok         Mod: OCZ-VERTEX2 FRev: 1.37 SN:
unavailable  disk         n        /devices/pci@0,0/pci1849,1c02@1f,2:1
sata0/2::dsk/c0t2d0            connected    configured   ok         Mod: WDC WD20EFRX-68AX9N0 FRev: 80.00A80 SN:
unavailable  disk         n        /devices/pci@0,0/pci1849,1c02@1f,2:2
sata0/3::dsk/c0t3d0            connected    configured   ok         Mod: WDC WD20EFRX-68AX9N0 FRev: 80.00A80 SN:
unavailable  disk         n        /devices/pci@0,0/pci1849,1c02@1f,2:3
sata0/4                        empty        unconfigured ok
unavailable  sata-port    n        /devices/pci@0,0/pci1849,1c02@1f,2:4
sata0/5                        empty        unconfigured ok
unavailable  sata-port    n        /devices/pci@0,0/pci1849,1c02@1f,2:5
[..additional output removed for brevity..]

The above two lists show disk c0t3d0 which I know to be already part of the zones pool, a new WD disk called c0t2d0 plus two new OCZ Vertex2 disks called c0t0d0 and c0t1d0.

 

After learning the device ids of the new disks it is time to create the mirror in the zones pool by “attaching” the new WD device (c0t2d0) to the existing device c0t3d0 in the zones zpool.

# zpool attach zones c0t3d0 c0t2d0

Refer to man zpool for more info about the zpool attach command.  A quick zpool list -v to confirm the result is as expected:

# zpool list -v 
NAME         SIZE  ALLOC   FREE  EXPANDSZ    CAP  DEDUP  HEALTH  ALTROOT 
zones       1.81T   484G  1.34T         -    26%  1.00x  ONLINE  - 
  mirror    1.81T   484G  1.34T         - 
    c0t3d0      -      -      -         - 
    c0t2d0      -      -      -         -

A new mirror consisting of c0t3d0 and c0t2d0 had been created.

The new mirror starts rebuilding (resilvering) immediately. Zpool status -v shows the progress.

# zpool status -v 
  pool: zones 
 state: ONLINE 
status: One or more devices is currently being resilvered.  The pool will 
        continue to function, possibly in a degraded state. 
action: Wait for the resilver to complete. 
  scan: resilver in progress since Mon Sep 30 20:45:59 2013 
    427M scanned out of 484G at 71.2M/s, 1h55m to go 
    423M resilvered, 0.09% done 
config: 

        NAME        STATE     READ WRITE CKSUM 
        zones       ONLINE       0     0     0 
          mirror-0  ONLINE       0     0     0 
            c0t3d0  ONLINE       0     0     0 
            c0t2d0  ONLINE       0     0     0  (resilvering) 

errors: No known data errors

 

You can monitor the progress by repeatedly querying the status. When resilvering is done the status will change accordingly.

# zpool status -v
  pool: zones
 state: ONLINE
  scan: resilvered 484G in 2h32m with 0 errors on Mon Sep 30 23:18:47 2013
config:

        NAME        STATE     READ WRITE CKSUM
        zones       ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            c0t3d0  ONLINE       0     0     0
            c0t2d0  ONLINE       0     0     0

errors: No known data errors