클러스터 헤드 노드에서 linpack을 실행할 수 없습니다.

Question

마침내 문제를 발견하고 해결했습니다.

나는 rpislave1 2개의 IP 주소(192.168.50.1(rpislave1에 설정한 DHCP 서버에서 온 것임)와 또 다른 169.254.131.47)에 eth0을 제공하고 있음을 발견했습니다. 이로 인해 Open MPI가 mpirun 주소 중에 엉망이 되었습니다. 문제는 - 프로세스가 무엇입니까? rpislave1은 모든 Raspberry Pi의 내부 네트워크(범위 192.168.50.0/24) 및 외부 세계에 대한 연결을 위한 DHCP 서버이므로 net.ipv4.ip_forward=1이고 iptables라고 생각했습니다. 아니요, 몇 가지 테스트를 수행했습니다. 전혀 효과가 없습니다.

그런 다음 라즈베리 파이가 동일한 네트워크 인터페이스에 2개의 IP 주소를 갖는 이유를 조사하고 dhcpcd.service를 찾았습니다.

user@rpislave1:/etc $ sudo systemctl status  dhcpcd.service
● dhcpcd.service - DHCP Client Daemon
     Loaded: loaded (/lib/systemd/system/dhcpcd.service; enabled; vendor preset: enabled)
    Drop-In: /etc/systemd/system/dhcpcd.service.d
             └─wait.conf
     Active: active (running) since Fri 2023-07-28 16:22:45 +08; 16min ago
       Docs: man:dhcpcd(8)
    Process: 473 ExecStart=/usr/sbin/dhcpcd -w -q (code=exited, status=0/SUCCESS)
      Tasks: 2 (limit: 8755)
        CPU: 685ms
     CGroup: /system.slice/dhcpcd.service
             ├─564 wpa_supplicant -B -c/etc/wpa_supplicant/wpa_supplicant.conf -iwlan0
             └─765 dhcpcd: [master] [ip4] [ip6]

Jul 28 16:22:42 rpislave1 dhcpcd[473]: eth0: probing for an IPv4LL address
Jul 28 16:22:45 rpislave1 dhcpcd[473]: forked to background, child pid 765
Jul 28 16:22:45 rpislave1 systemd[1]: Started DHCP Client Daemon.
Jul 28 16:22:45 rpislave1 dhcpcd[765]: wlan0: leased 192.168.101.15 for 259200 seconds
Jul 28 16:22:45 rpislave1 dhcpcd[765]: wlan0: adding route to 192.168.101.0/24
Jul 28 16:22:45 rpislave1 dhcpcd[765]: wlan0: adding default route via 192.168.101.1
Jul 28 16:22:47 rpislave1 dhcpcd[765]: eth0: using IPv4LL address 169.254.131.47
Jul 28 16:22:47 rpislave1 dhcpcd[765]: eth0: adding route to 169.254.0.0/16
Jul 28 16:22:50 rpislave1 dhcpcd[765]: eth0: offered 192.168.50.1 from 192.168.50.1
Jul 28 16:22:51 rpislave1 dhcpcd[765]: eth0: no IPv6 Routers available

DHCP 요청을 보내기 전에 IPv4LL을 할당한 서비스를 찾으세요(그런데 동일한 장치에 있습니다). 이 IPv4LL은 동일한 서버에서 실행되는 DHCP 서버로부터 IP 주소를 얻기 전에 IP 주소와 경로를 설정합니다. 그러나 169.254.0.0/16에 대한 경로는 제거되지 않는 것 같습니다(rpislave.local에 재사용되는 것으로 의심되지만 이는 내 목적과 관련이 없습니다).

dhcpcd.service를 살펴보면서 이 옵션을 찾았습니다.

Local Link configuration
If dhcpcd failed to obtain a lease, it probes for a valid IPv4LL address (aka ZeroConf, aka APIPA). Once obtained it restarts the process of looking for a DHCP server to get a proper address.

When using IPv4LL, dhcpcd nearly always succeeds and returns an exit code of 0. In the rare case it fails, it normally means that there is a reverse ARP proxy installed which always defeats IPv4LL probing. To disable this behaviour, you can use the -L, --noipv4ll option.

따라서 /etc/systemd/system/dhcpcd.service.d의 wait.conf 파일이 편집되었습니다. 이러한 모든 세부 정보는 위의 상태 코드에서 찾을 수 있습니다.

[Service]
ExecStart=
ExecStart=/usr/sbin/dhcpcd -w -q --noipv4ll

재부팅하고 dhcpcd를 다시 확인하세요.

user@rpislave1:/etc/systemd/system/dhcpcd.service.d $ sudo systemctl status  dhcpcd.service
● dhcpcd.service - DHCP Client Daemon
     Loaded: loaded (/lib/systemd/system/dhcpcd.service; enabled; vendor preset: enabled)
    Drop-In: /etc/systemd/system/dhcpcd.service.d
             └─wait.conf
     Active: active (running) since Fri 2023-07-28 16:44:15 +08; 3min 17s ago
       Docs: man:dhcpcd(8)
    Process: 472 ExecStart=/usr/sbin/dhcpcd -w -q --noipv4ll (code=exited, status=0/SUCCESS)
      Tasks: 2 (limit: 8755)
        CPU: 619ms
     CGroup: /system.slice/dhcpcd.service
             ├─568 wpa_supplicant -B -c/etc/wpa_supplicant/wpa_supplicant.conf -iwlan0
             └─775 dhcpcd: [master] [ip4] [ip6]

Jul 28 16:44:10 rpislave1 dhcpcd[472]: wlan0: adding route to 2001:f40:950:b164::/64
Jul 28 16:44:10 rpislave1 dhcpcd[472]: wlan0: requesting DHCPv6 information
Jul 28 16:44:10 rpislave1 dhcpcd[472]: wlan0: adding default route via fe80::101
Jul 28 16:44:15 rpislave1 dhcpcd[472]: wlan0: leased 192.168.101.15 for 259200 seconds
Jul 28 16:44:15 rpislave1 dhcpcd[472]: wlan0: adding route to 192.168.101.0/24
Jul 28 16:44:15 rpislave1 dhcpcd[472]: wlan0: adding default route via 192.168.101.1
Jul 28 16:44:15 rpislave1 dhcpcd[472]: forked to background, child pid 775
Jul 28 16:44:15 rpislave1 systemd[1]: Started DHCP Client Daemon.
Jul 28 16:44:19 rpislave1 dhcpcd[775]: eth0: offered 192.168.50.1 from 192.168.50.1
Jul 28 16:44:21 rpislave1 dhcpcd[775]: eth0: no IPv6 Routers available
user@rpislave1:/etc/systemd/system/dhcpcd.service.d $

그리고 "eth0: IPv4LL 주소 169.254.131.47 사용" 경로가 전혀 설정되지 않았습니다.

그래서 전체 클러스터를 테스트하기 시작했습니다.

user@rpislave1:/sharezone/ClusterProcessing/HPL $  time mpirun -hostfile ../host_file -np 16 /sharezone/xhpl/bin/xhpl |tee HPL.log
================================================================================
HPLinpack 2.3  --  High-Performance Linpack benchmark  --   December 2, 2018
Written by A. Petitet and R. Clint Whaley,  Innovative Computing Laboratory, UTK
Modified by Piotr Luszczek, Innovative Computing Laboratory, UTK
Modified by Julien Langou, University of Colorado Denver
================================================================================

An explanation of the input/output parameters follows:
T/V    : Wall time / encoded variant.
N      : The order of the coefficient matrix A.
NB     : The partitioning blocking factor.
P      : The number of process rows.
Q      : The number of process columns.
Time   : Time in seconds to solve the linear system.
Gflops : Rate of execution for solving the linear system.

The following parameter values will be used:

N      :   57600
NB     :     192
PMAP   : Row-major process mapping
P      :       4
Q      :       4
PFACT  :   Right
NBMIN  :       4
NDIV   :       2
RFACT  :   Crout
BCAST  :  1ringM
DEPTH  :       1
SWAP   : Mix (threshold = 64)
L1     : transposed form
U      : transposed form
EQUIL  : yes
ALIGN  : 8 double precision words

--------------------------------------------------------------------------------

- The matrix A is randomly generated for each test.
- The following scaled residual check will be computed:
      ||Ax-b||_oo / ( eps * ( || x ||_oo * || A ||_oo + || b ||_oo ) * N )
- The relative machine precision (eps) is taken to be               1.110223e-16
- Computational tests pass if scaled residuals are less than                16.0

================================================================================
T/V                N    NB     P     Q               Time                 Gflops
--------------------------------------------------------------------------------
WR11C2R4       57600   192     4     4            2558.14             4.9804e+01
HPL_pdgesv() start time Fri Jul 28 16:55:04 2023

HPL_pdgesv() end time   Fri Jul 28 17:37:42 2023

--------------------------------------------------------------------------------
||Ax-b||_oo/(eps*(||A||_oo*||x||_oo+||b||_oo)*N)=   3.39874554e-03 ...... PASSED
================================================================================

Finished      1 tests with the following results:
              1 tests completed and passed residual checks,
              0 tests completed and failed residual checks,
              0 tests skipped because of illegal input values.
--------------------------------------------------------------------------------

End of Tests.
================================================================================
[rpislave4:00682] PMIX ERROR: NO-PERMISSIONS in file ../../../../../../src/mca/common/dstore/dstore_base.c at line 237
[rpislave3:00688] PMIX ERROR: NO-PERMISSIONS in file ../../../../../../src/mca/common/dstore/dstore_base.c at line 237
[rpislave2:00696] PMIX ERROR: NO-PERMISSIONS in file ../../../../../../src/mca/common/dstore/dstore_base.c at line 237

real    43m33.445s
user    141m36.616s
sys     28m21.561s
user@rpislave1:/sharezone/ClusterProcessing/HPL $

그리고 효과가 있어요~! 이제 헤드를 포함한 4개 유닛 모두에 대해 전체 벤치마크를 할 수 있게 되었습니다~!

Answer 1