RAID6 구성에 4개의 디스크가 있는 서버에 정전이 발생한 후 /dev/md1은 부팅되지 않습니다(그러나 /dev/md0은 부팅됨).
mdadm --assemble --scan
mdadm: /dev/md/1 assembled from 3 drives - not enough to start the array while not clean - consider --force.
mdadm: No arrays found in config file or automatically
>: mdadm --assemble --force /dev/md1 /dev/sdb2 /dev/sdc2 /dev/sdd2 /dev/sde2 -v
mdadm: looking for devices for /dev/md1
mdadm: cannot open device /dev/sdc2/: Not a directory
mdadm: /dev/sdc2/ has no superblock - assembly aborted
>: mdadm --examine /dev/sdc2
Magic: a92b4efc
Raid Level: raid6
Raid devices: 4
...
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : 24c01b72 - correct
Array State : AAA
하지만 다른 raid md0을 보면 괜찮아 보입니다.
cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : Active (auto-read-only) raid6 sdb1[0] sde1[3] sdd1[2] sdc1[1]
124932095 blocks super 1.2 level 6, 512k chunk, algorithm 2 [4/4] [UUUU]
/dev/sdc1에서 mdadm --examine을 사용하면 /dev/sdc2와 거의 동일해 보이지만 분명히 손상되지는 않았습니다. 다음과 같이 /dev/sdc2에서 대체 슈퍼블록을 복원해 보았습니다.
>: e2fsck /dev/sdc2
ext2fs_open2: Bad magic number in super-block
e2fsck: Superblock invalid, trying backup blocks...
e2fsck: Bad magic number in super-block while trying to open /dev/sdc2
...
you might try running e2fsck with an alternate superblock:
e2fsck -b 8193 <device>
or
e2fsck -b 32768 <device>
>: e2fsck -b 8193
e2fsck: Bad magic number in super-block while trying to open /dev/sdc2
두 개의 대체 슈퍼블록 외에 잘못된 매직 넘버를 수정하는 다른 방법이 있습니까(둘 다 시도했습니다)? RAID 배열의 다른 3개 볼륨을 살펴보았는데 /dev/sdc2와 같은 내용이 나와 있었습니다. 모두 손상되었을 수는 없습니다. 그렇죠? 어떻게 복원하나요? 다음은 4개 드라이브 각각에 대한 파티션 테이블입니다.
fdisk -l /dev/sdc
Disk /dev/sdc: 2.7 TiB, xxxxxxx
Disklabel type: gpt
Device Start End Sectors Size Type
/dev/sdc1 xxxx xxxx xxxx 59.6G Linux RAID
/dev/sdc2 xxxx xxxx xxxx 2.7T Linux RAID
참고: SSH가 표시되지 않기 때문에 이전 스크린샷에서 입력한 내용입니다. 이제는 잘라내기/붙여넣기가 가능합니다.
md1의 첫 번째 드라이브:
>: mdadm -E /dev/sdb
/dev/sdb:
MBR Magic : aa55
Partition[0] : 4294967295 sectors at 1 (type ee)
>: fdisk -l /dev/sdb
Disk /dev/sdb: 2.7 TiB, 3000592982016 bytes, 5860533168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: D724DBDE-FC51-4BA1-AF65-01C21E6D1846
Device Start End Sectors Size Type
/dev/sdb1 2048 124999679 124997632 59.6G Linux RAID
/dev/sdb2 124999680 5860532223 5735532544 2.7T Linux RAID
md1 볼륨이 있는 파티션 2:
>: mdadm -E /dev/sdb2
/dev/sdb2:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 31731ada:b0804a8a:a69cbab4:505c2adf
Name : kvmhost4:1 (local to host kvmhost4)
Creation Time : Fri Nov 3 12:52:05 2017
Raid Level : raid6
Raid Devices : 4
Avail Dev Size : 5735270400 (2734.79 GiB 2936.46 GB)
Array Size : 5735270400 (5469.58 GiB 5872.92 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262056 sectors, after=0 sectors
State : active
Device UUID : ee3768a3:397f01ad:6086d77f:222b5e8d
Internal Bitmap : 8 sectors from superblock
Update Time : Wed Jul 29 15:24:04 2020
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : 190180d - correct
Events : 16450527
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 0
Array State : AAA. ('A' == active, '.' == missing, 'R' == replacing)
이제 raid 6의 두 번째 볼륨인 /dev/sdc입니다.
>: mdadm -E /dev/sdc
/dev/sdc:
MBR Magic : aa55
Partition[0] : 4294967295 sectors at 1 (type ee)
>: mdadm -E /dev/sdc1
/dev/sdc1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 3e7427b8:e82071a4:8224a7b7:f4dc6d0f
Name : kvmhost4:0 (local to host kvmhost4)
Creation Time : Fri Nov 3 12:51:44 2017
Raid Level : raid6
Raid Devices : 4
Avail Dev Size : 124932096 (59.57 GiB 63.97 GB)
Array Size : 124932096 (119.14 GiB 127.93 GB)
Data Offset : 65536 sectors
Super Offset : 8 sectors
Unused Space : before=65448 sectors, after=0 sectors
State : clean
Device UUID : dd82716f:49b276d7:9d383a43:06cf9206
Update Time : Fri Jul 10 06:20:40 2020
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : c2291539 - correct
Events : 37
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 1
Array State : AAAA ('A' == active, '.' == missing, 'R' == replacing)
md1 파티션이 위치한 파티션 2
>: mdadm -E /dev/sdc2
/dev/sdc2:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 31731ada:b0804a8a:a69cbab4:505c2adf
Name : kvmhost4:1 (local to host kvmhost4)
Creation Time : Fri Nov 3 12:52:05 2017
Raid Level : raid6
Raid Devices : 4
Avail Dev Size : 5735270400 (2734.79 GiB 2936.46 GB)
Array Size : 5735270400 (5469.58 GiB 5872.92 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262056 sectors, after=0 sectors
State : active
Device UUID : b5d3a4ac:e353b079:3c77d4fe:39cda55b
Internal Bitmap : 8 sectors from superblock
Update Time : Wed Jul 29 15:24:04 2020
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : 24c01b72 - correct
Events : 16450527
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 1
Array State : AAA. ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdd 볼륨의 드라이브 3:
>: mdadm -E /dev/sdd2
/dev/sdd2:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 31731ada:b0804a8a:a69cbab4:505c2adf
Name : kvmhost4:1 (local to host kvmhost4)
Creation Time : Fri Nov 3 12:52:05 2017
Raid Level : raid6
Raid Devices : 4
Avail Dev Size : 5735270400 (2734.79 GiB 2936.46 GB)
Array Size : 5735270400 (5469.58 GiB 5872.92 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262056 sectors, after=0 sectors
State : active
Device UUID : 5399392a:dfbadbbf:f6d4148d:ff796dc2
Internal Bitmap : 8 sectors from superblock
Update Time : Wed Jul 29 15:24:04 2020
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : dd88528c - correct
Events : 16450527
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 2
Array State : AAA. ('A' == active, '.' == missing, 'R' == replacing)
어레이의 4개를 드라이브합니다.
>: mdadm -E /dev/sde2
/dev/sde2:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x9
Array UUID : 31731ada:b0804a8a:a69cbab4:505c2adf
Name : kvmhost4:1 (local to host kvmhost4)
Creation Time : Fri Nov 3 12:52:05 2017
Raid Level : raid6
Raid Devices : 4
Avail Dev Size : 5735270400 (2734.79 GiB 2936.46 GB)
Array Size : 5735270400 (5469.58 GiB 5872.92 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262056 sectors, after=0 sectors
State : active
Device UUID : 5230e8e3:792ce23c:8431a3f1:99d5bfef
Internal Bitmap : 8 sectors from superblock
Update Time : Mon Jul 6 13:10:59 2020
Bad Block Log : 512 entries available at offset 72 sectors - bad blocks present.
Checksum : a5d7e1fa - correct
Events : 13500044
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 3
Array State : AAAA ('A' == active, '.' == missing, 'R' == replacing)
답변1
글쎄, 나는 다음과 같이 작동하게 만들었습니다.
mdadm --assemble --scan --force
mdadm: Marking array /dev/md/1 as 'clean'
mdadm: /dev/md/1 has been started with 3 drives (out of 4).
>: cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md1 : active raid6 sdb2[0] sdd2[2] sdc2[1]
5735270400 blocks super 1.2 level 6, 512k chunk, algorithm 2 [4/3] [UUU_]
[>....................] resync = 0.0% (765440/2867635200) finish=561.7min speed=85048K/sec
bitmap: 4/22 pages [16KB], 65536KB chunk
md0 : active (auto-read-only) raid6 sdb1[0] sde1[3] sdd1[2] sdc1[1]
124932096 blocks super 1.2 level 6, 512k chunk, algorithm 2 [4/4] [UUUU]
unused devices: <none>
이제 다시 동기화될 때까지 기다렸다가 재부팅해 보겠습니다. 그 사이에 볼륨이 마운트되고 데이터를 복구했습니다!