내 서버에는 2개의 raid 5 장치로 구성된 raid 0 장치인 raid 5+0 구성이 있습니다.
md1은 md10과 md11로 구성됩니다.
md1 : active raid0 md10[0] md11[1]
15627286528 blocks super 1.2 512k chunks
md10 : active raid5 sdh[3] sdf[0] sdg[1]
7813774336 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU]
bitmap: 0/30 pages [0KB], 65536KB chunk
md11 : active raid5 sdd[3] sda[1] sdb[4]
7813774336 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU]
bitmap: 0/30 pages [0KB], 65536KB chunk
며칠 전 정전이 발생하여 md11의 디스크 중 하나에 오류가 발생했습니다. 여유분이 있어서 교체했어요. 그런데 디스크를 교체하여 서버를 시작했더니 같은 어레이에 있던 다른 드라이브가 사라졌습니다. 알고 보니 케이블 문제였습니다(아마도 다른 드라이브를 교체하는 동안 케이블에 충격을 가한 것 같습니다). 다시 설치하고 다시 켜면 드라이브가 어레이에 다시 나타나고 새 교체 디스크로 다시 구축할 수 있었습니다.
몇 시간 후에는 모든 것이 정상이었으며 어레이가 정상적으로 나타났습니다. 그런데 md10과 md11로 구성된 raid 0 장치(md1)를 마운트하려고 하면 오류가 발생하며 실패합니다.
# mount /dev/md1 md1/
mount: /storage/md1: wrong fs type, bad option, bad superblock on /dev/md1, missing codepage or helper program, or other error.
# fdisk -l /dev/md1
Disk /dev/md1: 14.6 TiB, 16002341404672 bytes, 31254573056 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 524288 bytes / 1048576 bytes
# fsck.ext4 /dev/md1
e2fsck 1.44.1 (24-Mar-2018)
fsck.ext4: Attempt to read block from filesystem resulted in short read while trying to open /dev/md1
Could this be a zero-length partition?
어레이를 구축할 때의 원래 mkfs.ext4 명령을 저장했기 때문에 슈퍼블록 백업이 어디에 있는지 알고 마운트를 시도했지만 모두 실패했습니다.
이것은 원래 mkfs 명령 출력입니다.
# mkfs.ext4 /dev/md1
mke2fs 1.42.13 (17-May-2015)
Creating filesystem with 3906821632 4k blocks and 488353792 inodes
Filesystem UUID: ac6fef5c-54ea-4c79-be24-6feec0cc05fc
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
102400000, 214990848, 512000000, 550731776, 644972544, 1934917632,
2560000000, 3855122432
백업을 설치하려고 하면 이런 일이 발생합니다.
# mount -o sb=32768 /dev/md1 md1/
mount: /storage/md1: wrong fs type, bad option, bad superblock on /dev/md1, missing codepage or helper program, or other error.
# mount -o sb=98304 /dev/md1 md1/
mount: /storage/md1: wrong fs type, bad option, bad superblock on /dev/md1, missing codepage or helper program, or other error.
# mount -o sb=163840 /dev/md1 md1/
mount: /storage/md1: wrong fs type, bad option, bad superblock on /dev/md1, missing codepage or helper program, or other error.
등. 모든 백업을 시도했지만 동일한 결과를 얻었습니다.
# dumpe2fs /dev/md1 | grep -i superblock
dumpe2fs 1.44.1 (24-Mar-2018)
dumpe2fs: Attempt to read block from filesystem resulted in short read while trying to open /dev/md1
Couldn't find valid filesystem superblock.
다시 설치/온라인으로 시도해 볼 수 있는 것이 있나요?
감사해요
편집 1
# mdadm --misc --detail /dev/md11
/dev/md11:
Version : 1.2
Creation Time : Wed Apr 5 16:30:50 2023
Raid Level : raid5
Array Size : 7813774336 (7451.80 GiB 8001.30 GB)
Used Dev Size : 3906887168 (3725.90 GiB 4000.65 GB)
Raid Devices : 3
Total Devices : 3
Persistence : Superblock is persistent
Intent Bitmap : Internal
Update Time : Thu Aug 3 23:00:46 2023
State : clean
Active Devices : 3
Working Devices : 3
Failed Devices : 0
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 512K
Consistency Policy : bitmap
Name : fileserver:11 (local to host fileserver)
UUID : 2ccc1d61:79f0e11a:6d3bb224:797ca224
Events : 375478
Number Major Minor RaidDevice State
4 8 16 0 active sync /dev/sdb
1 8 0 1 active sync /dev/sda
3 8 48 2 active sync /dev/sdd
# mdadm --examine /dev/sd[abd]
/dev/sda:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 2ccc1d61:79f0e11a:6d3bb224:797ca224
Name : fileserver:11 (local to host fileserver)
Creation Time : Wed Apr 5 16:30:50 2023
Raid Level : raid5
Raid Devices : 3
Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB)
Array Size : 7813774336 (7451.80 GiB 8001.30 GB)
Used Dev Size : 7813774336 (3725.90 GiB 4000.65 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262056 sectors, after=688 sectors
State : clean
Device UUID : 88255f65:0cfbc0d7:01b3bb6b:a659a3be
Internal Bitmap : 8 sectors from superblock
Update Time : Thu Aug 3 23:00:46 2023
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : 4fa1243c - correct
Events : 375478
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 1
Array State : AAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdb:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x9
Array UUID : 2ccc1d61:79f0e11a:6d3bb224:797ca224
Name : fileserver:11 (local to host fileserver)
Creation Time : Wed Apr 5 16:30:50 2023
Raid Level : raid5
Raid Devices : 3
Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB)
Array Size : 7813774336 (7451.80 GiB 8001.30 GB)
Used Dev Size : 7813774336 (3725.90 GiB 4000.65 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262064 sectors, after=688 sectors
State : clean
Device UUID : 0116c972:b7e63a6f:ad991903:6d0da535
Internal Bitmap : 8 sectors from superblock
Update Time : Thu Aug 3 23:00:46 2023
Bad Block Log : 512 entries available at offset 24 sectors - bad blocks present.
Checksum : 2e49aad - correct
Events : 375478
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 0
Array State : AAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdd:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x9
Array UUID : 2ccc1d61:79f0e11a:6d3bb224:797ca224
Name : fileserver:11 (local to host fileserver)
Creation Time : Wed Apr 5 16:30:50 2023
Raid Level : raid5
Raid Devices : 3
Avail Dev Size : 7813775024 (3725.90 GiB 4000.65 GB)
Array Size : 7813774336 (7451.80 GiB 8001.30 GB)
Used Dev Size : 7813774336 (3725.90 GiB 4000.65 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262056 sectors, after=688 sectors
State : clean
Device UUID : c244b202:6c858ef1:170354cb:6d41f5db
Internal Bitmap : 8 sectors from superblock
Update Time : Thu Aug 3 23:00:46 2023
Bad Block Log : 512 entries available at offset 72 sectors - bad blocks present.
Checksum : 83ac05bd - correct
Events : 375478
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 2
Array State : AAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdb는 교체 디스크이고 /dev/sdd는 어레이에서 일시적으로 사라진 디스크입니다.