20개의 하드 드라이브 raid6 어레이가 ubuntu 설치에서 작동을 멈춘 이유를 진단하려고 합니다. 강제로 시작하려고 할 때 여러 드라이브가 더 이상 RAID에 전혀 나열되지 않고 내 컴퓨터에도 더 이상 드라이브로 나열되지 않는 것을 발견했습니다.
하드웨어 문제인 줄 알고 케이스를 열고 SATA 케이블을 만지작거렸습니다. 나는 공격을 중단하고 일부 전선을 뽑았다가 다시 연결했으며 컴퓨터가 새로운 SATA 장치를 확인하도록 강제하고 싶었습니다. 이 사이트를 검색한 결과 SATA 장치를 플래시하려면 다음 명령을 실행하라는 답변을 찾았습니다.
for host in /sys/class/scsi_host/*; do echo "- - -" | sudo tee $host/scan; ls /dev/sd* ; done
명령 출력에는 드라이브가 사라진 것으로 표시됩니다. (설명하자면 드라이브 sdb-sdu는 raid6 어레이의 일부이며 이 명령을 사용하면 sdo, sdp, sdq 및 sdr 4개가 사라집니다.)
- - -
/dev/sda /dev/sda2 /dev/sdc /dev/sde /dev/sdg /dev/sdi /dev/sdk /dev/sdm /dev/sdo /dev/sdq /dev/sds /dev/sdu
/dev/sda1 /dev/sdb /dev/sdd /dev/sdf /dev/sdh /dev/sdj /dev/sdl /dev/sdn /dev/sdp /dev/sdr /dev/sdt
- - -
/dev/sda /dev/sda2 /dev/sdc /dev/sde /dev/sdg /dev/sdi /dev/sdk /dev/sdm /dev/sdo /dev/sdq /dev/sds /dev/sdu
/dev/sda1 /dev/sdb /dev/sdd /dev/sdf /dev/sdh /dev/sdj /dev/sdl /dev/sdn /dev/sdp /dev/sdr /dev/sdt
- - -
/dev/sda /dev/sda2 /dev/sdc /dev/sde /dev/sdg /dev/sdi /dev/sdk /dev/sdm /dev/sdp /dev/sdr /dev/sdt
/dev/sda1 /dev/sdb /dev/sdd /dev/sdf /dev/sdh /dev/sdj /dev/sdl /dev/sdn /dev/sdq /dev/sds /dev/sdu
- - -
/dev/sda /dev/sda2 /dev/sdc /dev/sde /dev/sdg /dev/sdi /dev/sdk /dev/sdm /dev/sds /dev/sdu
/dev/sda1 /dev/sdb /dev/sdd /dev/sdf /dev/sdh /dev/sdj /dev/sdl /dev/sdn /dev/sdt
- - -
/dev/sda /dev/sda2 /dev/sdc /dev/sde /dev/sdg /dev/sdi /dev/sdk /dev/sdm /dev/sds /dev/sdu
/dev/sda1 /dev/sdb /dev/sdd /dev/sdf /dev/sdh /dev/sdj /dev/sdl /dev/sdn /dev/sdt
- - -
/dev/sda /dev/sda2 /dev/sdc /dev/sde /dev/sdg /dev/sdi /dev/sdk /dev/sdm /dev/sds /dev/sdu
/dev/sda1 /dev/sdb /dev/sdd /dev/sdf /dev/sdh /dev/sdj /dev/sdl /dev/sdn /dev/sdt
- - -
/dev/sda /dev/sda2 /dev/sdc /dev/sde /dev/sdg /dev/sdi /dev/sdk /dev/sdm /dev/sds /dev/sdu
/dev/sda1 /dev/sdb /dev/sdd /dev/sdf /dev/sdh /dev/sdj /dev/sdl /dev/sdn /dev/sdt
- - -
/dev/sda /dev/sda2 /dev/sdc /dev/sde /dev/sdg /dev/sdi /dev/sdk /dev/sdm /dev/sds /dev/sdu
/dev/sda1 /dev/sdb /dev/sdd /dev/sdf /dev/sdh /dev/sdj /dev/sdl /dev/sdn /dev/sdt
- - -
/dev/sda /dev/sda2 /dev/sdc /dev/sde /dev/sdg /dev/sdi /dev/sdk /dev/sdm /dev/sds /dev/sdu
/dev/sda1 /dev/sdb /dev/sdd /dev/sdf /dev/sdh /dev/sdj /dev/sdl /dev/sdn /dev/sdt
- - -
/dev/sda /dev/sda2 /dev/sdc /dev/sde /dev/sdg /dev/sdi /dev/sdk /dev/sdm /dev/sds /dev/sdu
/dev/sda1 /dev/sdb /dev/sdd /dev/sdf /dev/sdh /dev/sdj /dev/sdl /dev/sdn /dev/sdt
- - -
/dev/sda /dev/sda2 /dev/sdc /dev/sde /dev/sdg /dev/sdi /dev/sdk /dev/sdm /dev/sds /dev/sdu
/dev/sda1 /dev/sdb /dev/sdd /dev/sdf /dev/sdh /dev/sdj /dev/sdl /dev/sdn /dev/sdt
- - -
/dev/sda /dev/sda2 /dev/sdc /dev/sde /dev/sdg /dev/sdi /dev/sdk /dev/sdm /dev/sds /dev/sdu
/dev/sda1 /dev/sdb /dev/sdd /dev/sdf /dev/sdh /dev/sdj /dev/sdl /dev/sdn /dev/sdt
- - -
/dev/sda /dev/sda2 /dev/sdc /dev/sde /dev/sdg /dev/sdi /dev/sdk /dev/sdm /dev/sds /dev/sdu
/dev/sda1 /dev/sdb /dev/sdd /dev/sdf /dev/sdh /dev/sdj /dev/sdl /dev/sdn /dev/sdt
- - -
/dev/sda /dev/sda2 /dev/sdc /dev/sde /dev/sdg /dev/sdi /dev/sdk /dev/sdm /dev/sds /dev/sdu
/dev/sda1 /dev/sdb /dev/sdd /dev/sdf /dev/sdh /dev/sdj /dev/sdl /dev/sdn /dev/sdt
- - -
/dev/sda /dev/sda2 /dev/sdc /dev/sde /dev/sdg /dev/sdi /dev/sdk /dev/sdm /dev/sds /dev/sdu
/dev/sda1 /dev/sdb /dev/sdd /dev/sdf /dev/sdh /dev/sdj /dev/sdl /dev/sdn /dev/sdt
- - -
/dev/sda /dev/sda2 /dev/sdc /dev/sde /dev/sdg /dev/sdi /dev/sdk /dev/sdm /dev/sds /dev/sdu
/dev/sda1 /dev/sdb /dev/sdd /dev/sdf /dev/sdh /dev/sdj /dev/sdl /dev/sdn /dev/sdt
- - -
/dev/sda /dev/sda2 /dev/sdc /dev/sde /dev/sdg /dev/sdi /dev/sdk /dev/sdm /dev/sds /dev/sdu
/dev/sda1 /dev/sdb /dev/sdd /dev/sdf /dev/sdh /dev/sdj /dev/sdl /dev/sdn /dev/sdt
- - -
/dev/sda /dev/sda2 /dev/sdc /dev/sde /dev/sdg /dev/sdi /dev/sdk /dev/sdm /dev/sds /dev/sdu
/dev/sda1 /dev/sdb /dev/sdd /dev/sdf /dev/sdh /dev/sdj /dev/sdl /dev/sdn /dev/sdt
- - -
/dev/sda /dev/sda2 /dev/sdc /dev/sde /dev/sdg /dev/sdi /dev/sdk /dev/sdm /dev/sds /dev/sdu
/dev/sda1 /dev/sdb /dev/sdd /dev/sdf /dev/sdh /dev/sdj /dev/sdl /dev/sdn /dev/sdt
더 많은 드라이브가 사라지고 있는지 확인하기 위해 SATA 케이블을 뽑아 사라지는 드라이브의 연결을 물리적으로 확인했습니다. 이러한 문제 해결 과정을 통해 사라진 4개의 드라이브는 모두 제가 최소 4년 동안 문제 없이 사용했던 8포트 PCIE to SATA 카드에 연결되어 있다는 사실을 발견했습니다.
시스템을 몇 번 재부팅했고, 시작 시 몇 개의 하드 드라이브가 누락되었지만 Ubuntu 디스크 유틸리티를 연 후 경고음과 함께 다시 나타나는 것을 발견했습니다. 그러다 다시 위 명령을 내리자 4개의 드라이브가 다시 사라졌습니다.
raid6 어레이는 현재 연결된 드라이브 수와 마지막 변경 시간에 대해 서로 다른 값을 보고하는 일부 드라이브로 인해 혼란스러운 상태에 있습니다. 강제로 두 번 조립하려고 시도했는데 작동했고 재구축이 시작되었지만 2개 이상의 드라이브가 OS에서 완전히 사라지고 재구축이 중단되었습니다.
카드인가요? 교체해야 하나요?
편집: 문제의 카드는 다음과 같습니다.https://www.ebay.com/itm/Marvell-Chipset-8-Ports-SATA-3-0-6GB-PCI-Express-Card-NCQ-Port-Multiplier-FIS/191962272465?hash=item2cb1d7d2d1:g: -UMAAoxy4t1SmhAS
RAID 어레이를 중지한 후 다음 명령을 실행하여 SATA 장치를 확인할 때 dmesg에 있는 내용은 다음과 같습니다.
[ 65.849212] md: md0 stopped.
[ 76.462011] ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[ 76.463189] ata1.00: configured for UDMA/133
[ 76.791618] ahci 0000:06:00.0: FBS is disabled
[ 76.951538] ahci 0000:06:00.0: FBS is enabled
[ 76.952111] ata10.00: SATA link up 3.0 Gbps (SStatus 123 SControl 330)
[ 77.267083] ata10.01: SATA link down (SStatus 610 SControl 330)
[ 77.581891] ata10.02: SATA link down (SStatus 610 SControl 330)
[ 77.894936] ata10.03: SATA link down (SStatus 610 SControl 330)
[ 78.211968] ata10.04: SATA link down (SStatus 610 SControl 330)
[ 78.215129] ata10.00: configured for UDMA/133
[ 82.816960] ata10.01: SATA link down (SStatus 610 SControl 330)
[ 83.128896] ata10.02: SATA link down (SStatus 610 SControl 330)
[ 83.440843] ata10.03: SATA link down (SStatus 610 SControl 330)
[ 83.752802] ata10.04: SATA link down (SStatus 610 SControl 330)
[ 88.192093] ata10.01: SATA link down (SStatus 610 SControl 330)
[ 88.504051] ata10.02: SATA link down (SStatus 610 SControl 330)
[ 88.815986] ata10.03: SATA link down (SStatus 610 SControl 330)
[ 89.127946] ata10.04: SATA link down (SStatus 610 SControl 330)
[ 89.127970] ata10.01: disabled
[ 89.127988] ata10.02: disabled
[ 89.128003] ata10.03: disabled
[ 89.128018] ata10.04: disabled
[ 89.128332] ata10.01: detaching (SCSI 10:1:0:0)
[ 89.129036] sd 10:1:0:0: [sdo] Synchronizing SCSI cache
[ 89.129059] sd 10:1:0:0: [sdo] Synchronize Cache(10) failed: Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[ 89.129060] sd 10:1:0:0: [sdo] Stopping disk
[ 89.129066] sd 10:1:0:0: [sdo] Start/Stop Unit failed: Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[ 89.185725] ata10.02: detaching (SCSI 10:2:0:0)
[ 89.186427] sd 10:2:0:0: [sdp] Synchronizing SCSI cache
[ 89.186454] sd 10:2:0:0: [sdp] Synchronize Cache(10) failed: Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[ 89.186455] sd 10:2:0:0: [sdp] Stopping disk
[ 89.186461] sd 10:2:0:0: [sdp] Start/Stop Unit failed: Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[ 89.253718] ata10.03: detaching (SCSI 10:3:0:0)
[ 89.254504] sd 10:3:0:0: [sdq] Synchronizing SCSI cache
[ 89.254528] sd 10:3:0:0: [sdq] Synchronize Cache(10) failed: Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[ 89.254529] sd 10:3:0:0: [sdq] Stopping disk
[ 89.254535] sd 10:3:0:0: [sdq] Start/Stop Unit failed: Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[ 89.317679] ata10.04: detaching (SCSI 10:4:0:0)
[ 89.318503] sd 10:4:0:0: [sdr] Synchronizing SCSI cache
[ 89.318527] sd 10:4:0:0: [sdr] Synchronize Cache(10) failed: Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[ 89.318528] sd 10:4:0:0: [sdr] Stopping disk
[ 89.318534] sd 10:4:0:0: [sdr] Start/Stop Unit failed: Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[ 89.460199] ata11: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[ 89.462818] ata11.00: configured for UDMA/133
[ 89.796131] ata12: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[ 89.798858] ata12.00: configured for UDMA/133
[ 90.132065] ata13: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[ 90.133201] ata13.00: NCQ Send/Recv Log not supported
[ 90.134221] ata13.00: NCQ Send/Recv Log not supported
[ 90.134227] ata13.00: configured for UDMA/133
[ 90.472068] ata14: SATA link down (SStatus 0 SControl 300)
[ 90.808247] ata15: SATA link down (SStatus 0 SControl 300)
[ 91.147895] ata16: SATA link down (SStatus 0 SControl 300)
[ 91.484079] ata17: SATA link down (SStatus 0 SControl 300)
[ 91.824122] ata18: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[ 91.824750] ata18.00: configured for UDMA/66
[ 92.155995] ata2: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[ 92.157411] ata2.00: configured for UDMA/133
[ 92.491879] ata3: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[ 92.494534] ata3.00: configured for UDMA/133
[ 92.827218] ata4: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[ 92.828045] ata4.00: supports DRM functions and may not be fully accessible
[ 92.830499] ata4.00: supports DRM functions and may not be fully accessible
[ 92.832149] ata4.00: configured for UDMA/133
[ 93.847192] ata5: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[ 93.849644] ata5.00: configured for UDMA/133
[ 94.175633] ata6: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[ 94.177221] ata6.00: configured for UDMA/133
[ 94.664704] ata7: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[ 94.667911] ata7.00: configured for UDMA/133
[ 95.152651] ata8: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[ 96.732470] ata8.00: configured for UDMA/133
[ 97.236448] ata9: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[ 97.237568] ata9.00: configured for UDMA/133