last updated: Apr. 2005
Examine and understand the output of the following commands:
sfdisk -l /dev/sdb cat /etc/raidtab cat /proc/mdstat mdadm -E /dev/hda1 cat /proc/partitions cat /etc/fstab cat /proc/mounts cat /etc/mtab df /etc/diskcheck.conf
choosing between raidtools and mdadm
raidtools has been the standard software RAID management package for Linux since the inception of the software RAID driver. Over the years, raidtools have proven cumbersome to use, mostly because they rely on a configuration file (/etc/raidtab) that is difficult to maintain, and partly because its features are limited. In August 2001, Neil Brown, a software engineer at the University of New South Wales and a kernel developer, released an alternative. His mdadm (multiple devices admin) package provides a simple, yet robust way to manage software arrays. mdadm is now at version 1.0.1 and has proved quite stable over its first year of development. There has been much positive response on the Linux-raid mailing list and mdadm is likely to become widespread in the future. This article assumes that you have at least some familiarity with software RAID on Linux and that you have had some exposure to the raidtools package.
create an /etc/mdadm.conf - in general, it's best to create an /etc/mdadm.conf file after you have created an array and update the file when new arrays are created.
mdadm --detail --scan > /etc/mdadm.conf echo "DEVICE /dev/hda0 /dev/hdb0 /dev/hda1 /dev/hdb1 /dev/hda2 \ /dev/hdb2 /dev/hda3 /dev/hdb3" >> /etc/mdadm.conf
check that the array can be stopped and started successfully:
stop the array: mdadm -S /dev/md0 If you created an /etc/mdadm.conf you can automatically start an array listed there with the following command: start the array: # mdadm -As /dev/md0
destroying the partition ; /dev/hdc
- Erasing the contents of a disk drive - unmount all of the disk drive's partitions, - issue the following command as root: badblocks -ws /dev/hdb
Keep in mind that badblocks is actually writing four different data patterns to every block on the disk drive. For large disk drives, this process can take a long time quite often several hours.
parted print parted /dev/hdb
Format the Partition The partition still does not have a file system. Create the file system: /sbin/mkfs -t ext3 /dev/hdb0 Label the Partition e2label /dev/hda3 /work
mkfs does not actually do the work of writing the file-system-specific information onto a disk drive; instead it passes control to one of several other programs that actually create the file system.
umount /dev/hda2
umount /home
A partition can only be unmounted if it is not currently in use.
If the partition cannot be unmounted while at the normal runlevel,
boot into rescue mode and remove the partition's /etc/fstab entry.
When using swapoff to disable swapping to a partition, you must
specify the device file name representing the swap partition:
swapoff /dev/hda4
rebuild the raid array:
mkraid /dev/md0
http://www.cse.unsw.edu.au/~neilb/source/mdadm/ mdadm (multiple devices admin) package provides a simple, yet robust way to manage software arrays. mdadm is now at version 1.0.1 and has proved quite stable over its first year of development. tar xvf ./mdadm-1.0.1.tgz # cd mdadm-1.0.1.tgz # make install mdadm has five major modes of operation. The first two modes, Create and Assemble, are used to configure and activate arrays. Manage mode is used to manipulate devices in an active array. Follow or Monitor mode allows administrators to configure event notification and actions for arrays. Build mode is used when working with legacy arrays mdadm [mode][options] Creating an Array create a RAID-0 at /dev/md0 made up of /dev/sdb1 and /dev/sdc1: # mdadm --create --verbose /dev/md0 --level=0 --raid-devices=2 /dev/sdb1 /dev/sdc1 mdadm: chunk size defaults to 64K mdadm: array /dev/md0 started. The --level option specifies which type of RAID to create in the same way that raidtools uses the raid-level configuration line. Valid choices are 0,1,4 and 5 for RAID-0, RAID-1, RAID-4, RAID-5 respectively. Linear (--level=linear) is also a valid choice for linear mode. The --raid-devices option works the same as the nr-raid-disks option when using /etc/raidtab and raidtools. the short form of each option but is identical to the example I showed above. # mdadm -Cv /dev/md0 -l0 -n2 -c128 /dev/sdb1 /dev/sdc1 -C selects Create mode, and I have also included the -v option here to turn on verbose output. -l and -n specify the RAID level and number of member disks. Users of raidtools and /etc/raidtab can see how much easier it is to create arrays using mdadm. You can change the default chunk size (64KB) using the --chunk or -c option. In this previous example I changed the chunk size to 128KB. mdadm also supports shell expansions, so you don't have to type in the device name for every component disk if you are creating a large array. In this example, I'll create a RAID-5 with five member disks and a chunk size of 128KB: # mdadm -Cv /dev/md0 -l5 -n5 -c128 /dev/sd{a,b,c,d,e}1 mdadm: layout defaults to left-symmetric mdadm: array /dev/md0 started. This example creates an array at /dev/md0 using SCSI disk partitions /dev/sda1, /dev/sdb1, /dev/sdc1, /dev/sdd1, and /dev/sde1. Notice that I have also set the chunk size to 128 KB using the -c128 option. When creating a RAID-5, mdadm will automatically choose the left-symmetric parity algorithm, which is the best choice. Use the --stop or -S command to stop running array: # mdadm -S /dev/md0 ---------------------------------------------------- /etc/mdadm.conf /etc/mdadm.conf is mdadms' primary configuration file. Unlike /etc/raidtab, mdadm does not rely on /etc/mdadm.conf to create or manage arrays. Rather, mdadm.conf is simply an extra way of keeping track of software RAIDs. Using a configuration file with mdadm is useful, but not required. Having one means you can quickly manage arrays without spending extra time figuring out what array properties are and where disks belong. For example, if an array wasn't running and there was no mdadm.conf file describing it, then the system administrator would need to spend time examining individual disks to determine array properties and member disks. Unlike the configuration file for raidtools, mdadm.conf is concise and simply lists disks and arrays. The configuration file can contain two types of lines each starting with either the DEVICE or ARRAY keyword. Whitespace separates the keyword from the configuration information. DEVICE lines specify a list of devices that are potential member disks. ARRAY lines specify device entries for arrays as well as identifier information. This information can include lists of one or more UUIDs, md device minor numbers, or a listing of member devices. A simple mdadm.conf file might look like this: DEVICE /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1 ARRAY /dev/md0 devices=/dev/sda1,/dev/sdb1 ARRAY /dev/md1 devices=/dev/sdc1,/dev/sdd1 In general, it's best to create an /etc/mdadm.conf file after you have created an array and update the file when new arrays are created. Without an /etc/mdadm.conf file you'd need to specify more detailed information about an array on the command in order to activate it. That means you'd have to remember which devices belonged to which arrays, and that could easily become a hassle on systems with a lot of disks. mdadm even provides an easy way to generate ARRAY lines. The output is a single long line, but I have broken it here to fit the page: # mdadm --detail --scan ARRAY /dev/md0 level=raid0 num-devices=2 \ UUID=410a299e:4cdd535e:169d3df4:48b7144a If there were multiple arrays running on the system, then mdadm would generate an array line for each one. So after you're done building arrays you could redirect the output of mdadm --detail --scan to /etc/mdadm.conf. Just make sure that you manually create a DEVICE entry as well. Using the example I've provided above we might have an /etc/mdadm.conf that looks like: DEVICE /dev/sdb1 /dev/sdc1 ARRAY /dev/md0 level=raid0 num-devices=2 \ UUID=410a299e:4cdd535e:169d3df4:48b7144a --------------------------------------- mdadm --detail --scan > /etc/mdadm.conf echo "DEVICE /dev/hda0 /dev/hdb0 /dev/hda1 /dev/hdb1 /dev/hda2 \ /dev/hdb2 /dev/hda3 /dev/hdb3" >> /etc/mdadm.conf -------------------------------------------- ################ DISK MIRROR (RAID1) SETUP ###################### #part raid.3 --size=100 --ondisk=hda #part raid.7 --size=100 --ondisk=hdb #part raid.4 --size=5000 --ondisk=hda #part raid.8 --size=5000 --ondisk=hdb #part raid.5 --size=5000 --ondisk=hda #part raid.9 --size=5000 --ondisk=hdb #part swap --size=500 --ondisk=hda #part swap --size=500 --ondisk=hdb #part raid.6 --size=1 --grow --ondisk=hda #part raid.10 --size=1 --grow --ondisk=hdb #raid / --fstype ext3 --level=RAID1 raid.4 raid.8 #raid /boot --fstype ext3 --level=RAID1 raid.3 raid.7 #raid /var --fstype ext3 --level=RAID1 raid.5 raid.9 #raid /home --fstype ext3 --level=RAID1 raid.6 raid.10 ------------------------------------------------- Starting an Array Assemble mode is used to start an array that already exists. If you created an /etc/mdadm.conf you can automatically start an array listed there with the following command: # mdadm -As /dev/md0 mdadm: /dev/md0 has been started with 2 drives. The -A option denotes assemble mode. You can also use --assemble. The -s or --scan option tells mdadm to look in /etc/mdadm.conf for information about arrays and devices. If you want to start every array listed in /etc/mdadm.conf, don't specify an md device on the command line. If you didn't create an /etc/mdadm.conf file, you will need to specify additional information on the command line in order to start an array. For example, this command attempts to start /dev/md0 using the devices listed on the command line: # mdadm -A /dev/md0 /dev/sdb1 /dev/sdc1 Since using mdadm -A in this way assumes you have some prior knowledge about how arrays are arranged, it might not be useful on systems that have arrays that were created by someone else. So you may wish to examine some devices to gain a better picture about how arrays should be assembled. ----------------------------------------------------------- The examine options (-E or --examine) allows you to print the md superblock (if present) from a block device that could be an array component. mdadm -E /dev/hda1 Version : 00.90.00 UUID : d28e2193:85cec5af:3243ce3d:baa1fbe2 Creation Time : Sat Jul 31 10:26:23 2004 Raid Level : raid1 Device Size : 102208 (99.81 MiB 104.66 MB) Raid Devices : 2 Total Devices : 2 Preferred Minor : 1 Update Time : Sun Aug 1 09:24:14 2004 State : dirty, no-errors Active Devices : 2 Working Devices : 2 Failed Devices : 0 Spare Devices : 0 Checksum : 7089803a - correct Events : 0.5 Number Major Minor RaidDevice State this 0 3 1 0 active sync /dev/hda1 0 0 3 1 0 active sync /dev/hda1 1 1 3 65 1 active sync /dev/hdb1 mdadm's examine option displays quite a bit of useful information about component disks. In this case we can tell that /dev/sdc1 belongs to a RAID-0 made up of a total of four member disks. What I want to specifically point out is the line of output that contains the UUID. A UUID is a 128-bit number that is guaranteed to be reasonably unique on both the local system and across other systems. It is a randomly generated using system hardware and timestamps as part of its seed. UUIDs are commonly used by many programs to uniquely tag devices. See the uuidgen and libuuid manual pages for more information. When an array is created, the md driver generates a UUID for the array and stores it in the md superblock. You can use the UUID as criteria for array assembly. In the next example I am going to activate the array to which /dev/sdc1 belongs using its UUID. # mdadm -Av /dev/md0 --uuid=84788b68:1bb79088:9a73ebcc:2ab430da /dev/sd* This command scans every SCSI disk (/dev/sd*) to see if it's a member of the array with the UUID 84788b68:1bb79088:9a73ebcc:2ab430da and then starts the array, assuming it found each component device. mdadm will produce a lot of output each time it tries to scan a device that does not exist. You can safely ignore such warnings. ----------------------------------------------- Managing Arrays Using Manage mode you can add and remove disks to a running array. This is useful for removing failed disks, adding spare disks, or adding replacement disks. Manage mode can also be used to mark a member disk as failed. Manage mode replicates the functions of raidtools programs such as raidsetfaulty, raidhotremove, and raidhotadd. For example, to add a disk to an active array, replicating the raidhotadd command: # mdadm /dev/md0 --add /dev/sdc1 Or, to remove /dev/sdc1 from /dev/md0 try: # mdadm /dev/md0 --f ail /dev/sdc1 --remove /dev/sdc1 Notice that I first mark /dev/sdc1 as failed and then remove it. This is the same as using the raidsetfaulty and raidhotremove commands with raidtools. It's fine to combine add, fail, and remove options on a single command line as long as they make sense in terms of array management. So you have to fail a disk before removing it, for example. Monitoring Arrays Follow, or Monitor, mode provides some of mdadm's best and most unique features. Using Follow/Monitor mode you can daemonize mdadm and configure it to send email alerts to system administrators when arrays encounter errors or fail. You can also use Follow mode to arbitrarily execute commands when a disk fails. For example, you might want to try removing and reinserting a failed disk in an attempt to correct a non-fatal failure without user intervention. The following command will monitor /dev/md0 (polling every 300 seconds) for critical events. When a fatal error occurs, mdadm will send an email to sysadmin. You can tailor the polling interval and email address to meet your needs. # mdadm --monitor --mail=sysadmin --delay=300 /dev/md0 When using monitor mode, mdadm will not exit, so you might want to wrap it around nohup and ampersand: # nohup mdadm --monitor --mail=sysadmin --delay=300 /dev/md0 & Follow/Monitor mode also allows arrays to share spare disks, a feature that has been lacking in Linux software RAID since its inception. That means you only need to provide one spare disk for a group of arrays or for all arrays. It also means that system administrators don't have to manually intervene to shuffle around spare disks when arrays fail. Previously this functionality was available only using hardware RAID. When Follow/Monitor mode is invoked, it polls arrays at regular intervals. When a disk failure is detected on an array without a spare disk, mdadm will remove an available spare disk from another array and insert it into the array with the failed disk. To facilitate this process, each ARRAY line in /etc/mdadm.conf needs to have a spare-group defined. DEVICE /dev/sd* ARRAY /dev/md0 level=raid1 num-devices=3 spare-group=database \ UUID=410a299e:4cdd535e:169d3df4:48b7144a ARRAY /dev/md1 level=raid1 num-device=2 spare-group=database \ UUID=59b6e564:739d4d28:ae0aa308:71147fe7 In this example, both /dev/md0 and /dev/md1 are part of the spare group database. Just assume that /dev/md0 is a two-disk RAID-1 with a single spare disk. If mdadm is running in monitor mode (as I showed earlier), and a disk in /dev/md1 fails, mdadm will remove the spare disk from /dev/md0 and insert it into /dev/md1. mdadm has many other options that I haven't covered here. I strongly recommend reading its manual page for further details. Remember, you don't have to switch to mdadm. raidtools is still in development, and it has the benefit of many years of development. But, I find that mdadm is a worthy replacement. It is both feature rich and intuitive, and there's no harm in trying out alternatives. ---------------------------------------------------------- http://www.linuxdevcenter.com/pub/a/linux/2002/12/05/RAID.html NAME mdadm - manage MD devices aka Linux Software Raid. SYNOPSIS mdadm [mode] [options] DESCRIPTION RAID devices are virtual devices created from two or more real block devices. This allows multiple devices (typically disk drives or parti- tions there-of) to be combined into a single device to hold (for exam- ple) a single filesystem. Some RAID levels include redundancy and so can survive some degree of device failure. Linux Software RAID devices are implemented through the md (Multiple Devices) device driver. Currently, Linux supports LINEAR md devices, RAID0 (striping), RAID1 (mirroring), RAID4 and RAID5. Recent kernels (2002) also support a mode known as MULTIPATH. mdadm only provides limited support for MULTIPATH as yet. mdadm is a program that can be used to create, manage, and monitor MD devices. As such it provides a similar set of functionality to the raidtools packages. The key differences between mdadm and raidtools are: Ā· mdadm is a single program and not a collection of programs. Ā· mdadm can perform (almost) all of its functions without having a configuration file. Also mdadm helps with management of the con- figuration file. Ā· mdadm can provide information about your arrays (through Query, Detail, and Examine) that raidtools cannot. MODES mdadm has 6 major modes of operation: Assemble Assemble the parts of a previously created array into an active array. Components can be explicitly given or can be searched for. mdadm checks that the components do form a bona fide array, and can, on request, fiddle superblock information so as to assemble a faulty array. Build Build a legacy array without per-device superblocks. Create Create a new array with per-device superblocks. Manage This is for doing things to specific components of an array such as adding new spares and removing faulty devices. Misc This mode allows operations on independent devices such as exam- ine MD superblocks, erasing old superblocks and stopping active arrays. Follow or Monitor Monitor one or more md devices and act on any state changes. OPTIONS Available options are: -A, --assemble Assemble a pre-existing array. -B, --build Build a legacy array without superblocks. -C, --create Create a new array. -Q, --query Examine a device to see (1) if it is an md device and (2) if it is a component of an md array. Information about what is dis- covered is presented. -D, --detail Print detail of one or more md devices. -E, --examine Print content of md superblock on device(s). -F, --follow, --monitor Select Monitor mode. -h, --help Display help message or, after above option, mode specific help message. -V, --version Print version information for mdadm. -v, --verbose Be more verbose about what is happening. -b, --brief Be less verbose. This is used with --detail and --examine. -f, --force Be more forceful about certain operations. See the various modes of the exact meaning of this option in different contexts. -c, --config= Specify the config file. Default is /etc/mdadm.conf. -s, --scan scan config file or /proc/mdstat for missing information. In general, this option gives mdadm permission to get any missing information, like component devices, array devices, array iden- tities, and alert destination from the configuration file: /etc/mdadm.conf. One exception is MISC mode when using --detail or --stop in which case --scan says to get a list of array devices from /proc/mdstat. For create or build: -c, --chunk= Specify chunk size of kibibytes. The default is 64. --rounding= Specify rounding factor for linear array (==chunk size) -l, --level= Set raid level. Options are: linear, raid0, 0, stripe, raid1, 1, mirror, raid5, 4, raid5, 5, multipath, mp. Obviously some of these are synonymous. Only the first 4 are valid when Building. -p, --parity= Set raid5 parity algorithm. Options are: left-asymmetric, left- symmetric, right-asymmetric, right-symmetric, la, ra, ls, rs. The default is left-symmetric. --layout= same as --parity -n, --raid-devices= --layout= same as --parity -n, --raid-devices= number of active devices in array. -x, --spare-devices= number of spare (eXtra) devices in initial array. Spares can be added and removed later. -z, --size= Amount (in Kibibytes) of space to use from each drive in RAID1/4/5. This must be a multiple of the chunk size, and must leave about 128Kb of space at the end of the drive for the RAID superblock. If this is not specified (as it normally is not) the smallest drive (or partition) sets the size, though if there is a variance among the drives of greater than 1%, a warning is issued. For assemble: -u, --uuid= uuid of array to assemble. Devices which donāt have this uuid are excluded -m, --super-minor= Minor number of device that array was created for. Devices which donāt have this minor number are excluded. If you create an array as /dev/md1, then all superblocks will contain the minor number 1, even if the array is later assembled as /dev/md2. -f, --force Assemble the array even if some superblocks appear out-of-date -R, --run Attempt to start the array even if fewer drives were given than are needed for a full array. Normally if not all drives are found and --scan is not used, then the array will be assembled but not started. With --run an attempt will be made to start it anyway. For Manage mode: -a, --add hotadd listed devices. -r, --remove remove listed devices. They must not be active. i.e. they should be failed or spare devices. -f, --fail mark listed devices as faulty. --set-faulty same as --fail. For Misc mode: -R, --run start a partially built array. -S, --stop deactivate array, releasing all resources. -o, --readonly mark array as readonly. -w, --readwrite mark array as readwrite. --zero-superblock If the device contains a valid md superblock, the block is over- written with zeros. With --force the block where the superblock would be is over-written even if it doesnāt appear to be valid. For Monitor mode: -m, --mail Give a mail address to send alerts to. -p, --program, --alert Give a program to be run whenever an event is detected. -d, --delay Give a delay in seconds. mdadm polls the md arrays and then waits this many seconds before polling again. The default is 60 seconds. ASSEMBLE MODE Usage: mdadm --assemble device options... Usage: mdadm --assemble --scan options... This usage assembles one or more raid arrays from pre-existing compo- nents. For each array, mdadm needs to know the md device, the identity of the array, and a number of component-devices. These can be found in a number of ways. The md device is either given before --scan or is found from the config file. In the latter case, multiple md devices can be started with a single mdadm command. The identity can be given with the --uuid option, with the --super- minor option, can be found in in the config file, or will be taken from the super block on the first component-device listed on the command line. Devices can be given on the --assemble command line or from the config file. Only devices which have an md superblock which contains the right identity will be considered for any device. The config file is only used if explicitly named with --config or requested with --scan. In the later case, /etc/mdadm.conf is used. If --scan is not given, then the config file will only be used to find the identity of md arrays. Normally the array will be started after it is assembled. However if --scan is not given and insufficient drives were listed to start a com- plete (non-degraded) array, then the array is not started (to guard against usage errors). To insist that the array be started in this case (as may work for RAID1 or RAID5), give the --run flag. BUILD MODE Usage: mdadm --build device --chunk=X --level=Y --raid-devices=Z devices This usage is similar to --create. The difference is that it creates a legacy array without a superblock. With these arrays there is no dif- ference between initially creating the array and subsequently assem- bling the array, except that hopefully there is useful data there in the second case. The level may only be 0, raid0, or linear. All devices must be listed and the array will be started once complete. CREATE MODE Usage: mdadm --create device --chunk=X --level=Y --raid-devices=Z devices This usage will initialise a new md array, associate some devices with it, and activate the array. As devices are added, they are checked to see if they contain raid superblocks or filesystems. They are also checked to see if the vari- ance in device size exceeds 1%. If any discrepancy is found, the array will not automatically be run, though the presence of a --run can override this caution. To create a "degraded" array in which some devices are missing, simply give the word missing in place of a device name. This will cause mdadm to leave the corresponding slot in the array empty. For a RAID4 or RAID5 array at most one slot can be missing. For a RAID1 array, only one real device needs to be given. All of the others can be missing. The General Management options that are valid with --create are: --run insist of running the array even if some devices look like they might be in use. --readonly start the array readonly - not supported yet. MANAGE MODE Usage: mdadm device options... devices... This usage will allow individual devices in an array to be failed, :
Recovering from a disk failure with Linux-raid mirrored disks
by Mark FosterMarch 13th, 2003
This article presents the steps I used to recover from a hard drive failure while running software RAID1 (mirrored drives) on Red Hat Linux 7.1.
Environment
- host1 was Red hat 7.1 on a Compaq Proliant ML370
- 2 SCSI drives (/dev/sda, /dev/sdb)
- 1 mirror device (/dev/md0) aka RAID1
- lilo for the boot record
Situation
The first drive sda has failed and the software raid is running in "degraded" mode, as shown in the output of/proc/mdstat.
There is one mirrored partition (sda5 -=- sdb5) and a swap partition on each drive (sda1 and sdb1).
Preparation
Read up on software RAID (see References below)Since sdb was identical to sda, we can use it's partition table as reference.
You'll need a boot floppy - use mkbootdisk(8)
Examine and understand the output of the following commands:
sfdisk -l /dev/sdb cat /etc/raidtab cat /proc/mdstat
Process
Dump the partition table to a file suitable to feed back to sfdisk.[root@host1 ~]# sfdisk -d /dev/sdb > /tmp/sdb.out [root@host1 ~]# mv /tmp/sdb.out /tmp/sda.inEdit the file, changing sdb to sda in the first column.
[root@host1 ~]# vi /tmp/sda.inNow let's re-create the partition table on sda.
[root@host1 ~]# sfdisk /dev/sda < /tmp/sda.inAdd sda5 back into the mirror device md0.
[root@host1 ~]# raidhotadd /dev/md0 /dev/sda5Check out the result in
/proc/mdstat, you should see that recovery is underway.
[root@host1 ~]# cat /proc/mdstatReactivate our swap space.
[root@host1 ~]# mkswap /dev/sda2 [root@host1 ~]# swapon -aVerify the swap partition is back online
[root@host1 ~]# swapon -sOnce recovery is complete, run lilo to recreate the MBR on /dev/sda
[root@host1 ~]# /sbin/lilo -vNote that this will likely not work on Red Hat 7.2 and above since the default boot loader was made grub... but if you upgraded from 7.1 your system might still be using lilo.
Voila! Reboot if you like although this is not strictly necessary
[root@host1 ~]# reboot
Conclusion
The key to a successful recovery is keeping a level head and do not panic. Thesfdisk -d command is wonderful for recreating the partition layout. It should prove just as useful for a RAID5 recovery, but I cannot attest to that personally.
References
man pages:mkbootdisk(8) sfdisk(8) swapon(8) mkswap(8) raidtab(5)
- The Software-RAID HOW-TO
- http://unthought.net/Software-RAID.HOWTO/
- Penguin Magazine Article
- http://www.penguinmagazine.com/Magazine/This_Issue/0011/1
- Home-brew High Availability: Booting Linux from a RAID-1 Device
- http://www.samag.com/documents/s=1155/sam0101g/0101g.htm
IDE Drives Most home & SOHO systems will probably use IDE disks They do have some limitations. o The total length of an IDE cable can only be a few feet long which generally limits their use to small home systems. o IDE drives do not "hot swapping". You cannot replace them while your system is running. o Only two devices can be attached per controller. o The performance of the IDE bus can be degraded by the presence of a second device on the cable. o The failure of one drive on an IDE bus often causes the malfunctioning of the second device. This can be fatal if you have two IDE drives of the same RAID set attached to the same cable. It is for these reasons that it is recommended to use only one IDE drive per controller when using RAID, especially in a corporate environment. In a home or SOHO setting, IDE based software RAID may be adequate. SCSI Drives SCSI hard disks have a number of features that make them more attractive for RAID use. o SCSI controllers are more tolerant of disk failures. The failure of a single drive is less likely to disrupt the remaining drives on the bus. o SCSI cables can be several meters long, making them suitable for data center applications. o Much more than two devices may be connected to a SCSI cable bus. o Some models of SCSI devices support "hot swapping" which allows you to replace them while the system is running. However SCSI drives tend to be more expensive than IDE drives which may make them less attractive for home use.
> How does one rebuild a raid array? I lost a disk in a power event last > week, and finally secured a replacement drive (knew I should have bought > an extra). If it md raid (software raid) you need to create partitions on the new drive and add it to the rain array/ for example: /sbin/raidhotadd /dev/md0 /dev/hdc5
> Any tips as to what I should > look for when recreating the image, if necessary? Include the appropriate modules. man mkinitrd. Okay. lilo.conf uses an initrd image for both the 2.4.9-34 kernel, as > well as the 2.4.20-18.7 copy. You could check if the initrd for 2.4.20 actually loads the appropriate raid modules (compare it with the contents of the old one). You should know the initrd.img is a gzipped file system image. To check its contents do the following: cp /boot/initrd.img /root/tmp/initrd.img.gz gzip -d /root/tmp/initrd.img.gz mount -o loop [-t ext2] /root/tmp/initrd.img /mnt/tmp Now you can check it's contents under /mnt/tmp. See if the file system contains the appropriate modules, and whether they are actually loaded (linuxrc). > > What happens if you run raidstart /dev/md1 by hand? Getting the > > same errors? > > Can I do this with the / partition already mounted? Good question, because it probably answers why you are having problems in the first place. Since you are using / raid you need a initrd.img that holds and loads the (new) raid modules before / is mounted. Check your lilo.conf or grub.conf to verify that you did use an initrd before. Check man mkinitrd for details on how to create an initrd image.
One of my other operating systems seems to have overwritten the boot track on my RedHat 7.3 hard drive. When I try and boot I just get a No OS message. What's the best way to reinstall the boot track leaving the rest of the installation intact. boot from the Redhat installation CD into a rescue mode. If only the bootrack has been damaged, then the CD will find the RH installation on your harddrive and will mount it. Then do a chroot to "log" into the Redhat tree, go to /etc/lilo.conf see if there is something wrong and if everything is fine, issue lilo -v to overwrite the MBR once again. Hope that helps.
Make a boot disk
check the kernel version:
# Insert a disk in /dev/fd0. # Any information on the disk will be lost. mount /dev/fd0 /mnt/floppy
uname -a # should see something like >>> devel.hostinghacks.net 2.4.20-8
make the boot disk:
mkbootdisk --device /dev/fd0 2.4.20-8 # >>> (or whatever kernel )
While the format is admittedly not very user-friendly, looking at /proc/mounts is the best way to be 100% sure of seeing what is mounted on your Red Hat Linux system, as the kernel is providing this information. Other methods can, under rare circumstances, be inaccurate. As with the other files under /proc/, the mounts "file" does not exist on any disk drive in your Red Hat Linux system. In fact, it is not even a file; instead it is a representation of system status made available (by the Linux kernel) in file form.
You do want to set up the raid software to tell you if a disk dies. Otherwise you might not notice--I once had a disk die and didn't notice until I got an automatic e-mail the next morning saying so. RH 8.0 includes mdadm that does this (among other things). I am running an older Red Hat without mdadm and so run an extremely simple homebrew cron job that compares /proc/mdstatus against a file holding the correct contents. You also want to run raid 1 for your swap partition so everything is covered for a running kernel and processes and your machine can keep running.
The following command will monitor /dev/md0 (polling every 300 seconds) for critical events. When a fatal error occurs, mdadm will send an email to sysadmin. You can tailor the polling interval and email address to meet your needs. # mdadm --monitor --mail=sysadmin --delay=300 /dev/md0 When using monitor mode, mdadm will not exit, so you might want to wrap it around nohup and ampersand: # nohup mdadm --monitor --mail=sysadmin --delay=300 /dev/md0 &
mirror drives /dev/hda and /dev/hdb by creating the same partitions on each drive, then creating a raidtab as following:
raiddev /dev/md0
raid-level 1
nr-raid-disks 2
nr-spare-disks 0
persistent-superblock 1
chunk-size 4
device /dev/hda
raid-disk 0
device /dev/hdb
raid-disk 1
then run mkraid /dev/md0 and it should sync. Removing one drive at a time, the computer will boot up fine on either drives.
man raidtab
raidtab(5) raidtab(5)
NAME
raidtab - configuration file for md (RAID) devices
DESCRIPTION
/etc/raidtab is the default configuration file for the raid tools
(raidstart and company). It defines how RAID devices are configured on
a system.
FORMAT
/etc/raidtab has multiple sections, one for each md device which is
being configured. Each section begins with the raiddev keyword. The
order of items in the file is important. Later raiddev entries can use
earlier ones (which allows RAID-10, for example), and the parsing code
isnāt overly bright, so be sure to follow the ordering in this man page
for best results.
Hereās a sample md configuration file:
#
# sample raiddev configuration file
# āoldā RAID0 array created with mdtools.
#
raiddev /dev/md0
raid-level 0
nr-raid-disks 2
persistent-superblock 0
chunk-size 8
device /dev/hda1
raid-disk 0
device /dev/hdb1
raid-disk 1
raiddev /dev/md1
raid-level 5
nr-raid-disks 3
nr-spare-disks 1
persistent-superblock 1
parity-algorithm left-symmetric
device /dev/sda1
raid-disk 0
device /dev/sdb1
raid-disk 1
device /dev/sdc1
raid-disk 2
device /dev/sdd1
spare-disk 0
Here is more information on the directives which are in raid configura-
tion files; the options are listen in this file in the same order they
should appear in the actual configuration file.
raiddev device
This introduces the configuration section for the stated device.
nr-raid-disks count
Number of raid devices in the array; there should be count raid-
disk entries later in the file. (current maximum limit for RAID
devices -including spares- is 12 disks. This limit is already
extended to 256 disks in experimental patches.)
nr-spare-disks count
Number of spare devices in the array; there should be count
spare-disk entries later in the file. Spare disks may only be
used with RAID4 and RAID5, and allow the kernel to automatically
build new RAID disks as needed. It is also possible to
add/remove spares runtime via raidhotadd/raidhotremove, care has
to be taken that the /etc/raidtab configuration exactly follows
the actual configuration of the array. (raidhotadd/raidhotremove
does not change the configuration file)
persistent-superblock 0/1
newly created RAID arrays should use a persistent superblock. A
persistent superblock is a small disk area allocated at the end
of each RAID device, this helps the kernel to safely detect RAID
devices even if disks have been moved between SCSI controllers.
It can be used for RAID0/LINEAR arrays too, to protect against
accidental disk mixups. (the kernel will either correctly
reorder disks, or will refuse to start up an array if something
has happened to any member disk. Of course for the āfail-safeā
RAID variants (RAID1/RAID5) spares are activated if any disk
fails.)
Every member disk/partition/device has a superblock, which car-
ries all information necessary to start up the whole array. (for
autodetection to work all the āmemberā RAID partitions should be
marked type 0xfd via fdisk) The superblock is not visible in the
final RAID array and cannot be destroyed accidentally through
usage of the md device files, all RAID data content is available
for filesystem use.
parity-algorithm which
The parity-algorithm to use with RAID5. It must be one of left-
asymmetric, right-asymmetric, left-symmetric, or right-symmet-
ric. left-symmetric is the one that offers maximum performance
on typical disks with rotating platters.
chunk-size size
Sets the stripe size to size kilobytes. Has to be a power of 2
and has a compilation-time maximum of 4M. (MAX_CHUNK_SIZE in the
kernel driver) typical values are anything from 4k to 128k, the
best value should be determined by experimenting on a given
array, alot depends on the SCSI and disk configuration.
device devpath
Adds the device devpath to the list of devices which comprise
the raid system. Note that this command must be followed by one
of raid-disk, spare-disk, or parity-disk. Also note that itās
possible to recursively define RAID arrays, ie. to set up a
RAID5 array of RAID5 arrays. (thus achieving two-disk failure
protection, at the price of more disk space spent on RAID5
checksum blocks)
raid-disk index
The most recently defined device is inserted at position index
in the raid array.
spare-disk index
The most recently defined device is inserted at position index
in the spare disk array.
parity-disk index
The most recently defined device is moved to the end of the raid
array, which forces it to be used for parity.
failed-disk index
The most recently defined device is inserted at position index
in the raid array as a failed device. This allows you to create
raid 1/4/5 devices in degraded mode - useful for installation.
Donāt use the smallest device in an array for this, put this
after the raid-disk definitions!
NOTES
The raidtools are derived from the md-tools and raidtools packages,
which were originally written by Marc Zyngier, Miguel de Icaza, Gadi
Oxman, Bradley Ward Allen, and Ingo Molnar.
SEE ALSO
raidstart(8), raid0run(8), mkraid(8), raidstop(8)
raidtab(5)
(END)