서버 중 하나가 갑자기 응답하지 않게 되었습니다. 강제로 다시 시작한 후에는 로그 파일에 액세스할 수 있습니다. 커널 버그라고 생각되는 것들이 가득합니다. plot명령을 실행하는 첫 번째 명령(사용자 프로그램)에 대한 오류는 다음과 같습니다. 그 후에는 다른 프로세스도 oopses를 트리거하지만 첫 번째 줄은 다음과 같습니다.
sshd: page allocation stalls for 57387ms, order:1, mode:0x16040c0(GFP_KERNEL|__GFP_COMP|__GFP_NOTRACK), nodemask=(null)
그뿐만 아니라 sshd,,,, crond등등. 초기 이벤트 후에는 무엇이든 "죄송합니다"를 유발할 수 있는 것 같습니다. 그들은 몇 초마다 로그 파일에 oops가 기록되도록 계속해서 oops를 트리거합니다.cpnrpeautomount
무엇이 잘못되었는지 알아낼 수 있나요? 왜 아야는 멈추지 않았습니까? 다시 시작한 이후로 서버는 제가 발견한 문제 없이 실행되고 있지만 plot지금까지는 프로그램이 실행되지 않습니다(왜냐하면 어떤 사용자가 먼저 실행했는지 모르기 때문입니다...).
plot: page allocation failure: order:0, mode:0x1411100(GFP_NOIO|__GFP_COLD|__GFP_NORETRY|__GFP_NOMEMALLOC), nodemask=(null)
plot cpuset=/ mems_allowed=0-1
CPU: 40 PID: 1340614 Comm: plot Not tainted 4.13.0-1.el6.elrepo.x86_64 #1
Hardware name: Supermicro X10DRi/X10DRI-T, BIOS 2.1 09/13/2016
Call Trace:
dump_stack+0x67/0x98
warn_alloc+0xe3/0x180
? drain_all_pages+0x1b9/0x1f0
__alloc_pages_slowpath+0xe83/0xee0
? ext4_map_blocks+0x325/0x540 [ext4]
? submit_bio+0x77/0x150
? __pagevec_lru_add_fn+0x16a/0x2f0
? ext4_mpage_readpages+0x504/0x910 [ext4]
? __radix_tree_create+0xaf/0x180
? workingset_update_node+0x19/0x70
? __radix_tree_lookup+0x92/0x110
__alloc_pages_nodemask+0x1af/0x2b0
? find_get_entry+0x43/0x140
? __add_to_page_cache_locked+0x124/0x1c0
alloc_pages_current+0x86/0x150
__page_cache_alloc+0xa6/0xd0
? add_to_page_cache_lru+0x92/0x100
? add_page_wait_queue+0x82/0xa0
cachefiles_read_backing_file+0x101/0x720 [cachefiles]
? _ext4_get_block+0x74/0x120 [ext4]
? vfs_statfs+0x1b/0xc0
? ext4_get_block+0x16/0x20 [ext4]
? generic_block_bmap+0x3f/0x50
? mapping_tagged+0x12/0x20
cachefiles_read_or_alloc_pages+0x2e7/0x3a0 [cachefiles]
? kmem_cache_alloc_trace+0x1ea/0x230
? wake_up_bit+0x41/0x50
? fscache_run_op+0x54/0x90 [fscache]
? __nfs_readpages_from_fscache+0xf0/0xf0 [nfs]
? nfs_fh_get_context+0x11/0x20 [nfs]
? fscache_alloc_retrieval+0xc6/0xf0 [fscache]
__fscache_read_or_alloc_pages+0x210/0x300 [fscache]
__nfs_readpages_from_fscache+0xba/0xf0 [nfs]
nfs_readpages+0xad/0x190 [nfs]
? __alloc_pages_nodemask+0x1af/0x2b0
? __radix_tree_lookup+0x92/0x110
read_pages+0x55/0x130
__do_page_cache_readahead+0x135/0x160
? filemap_map_pages+0x315/0x4d0
filemap_fault+0x439/0x510
? change_pte_range+0x1c8/0x3a0
? flush_tlb_func_common+0xa2/0x150
? pagevec_lru_move_fn+0xc5/0xe0
? find_next_bit+0xb/0x10
? cpumask_any_but+0x27/0x40
__do_fault+0x1e/0x70
do_fault+0x19e/0x2a0
handle_pte_fault+0x1ca/0x230
__handle_mm_fault+0x327/0x490
handle_mm_fault+0x10d/0x240
? find_vma+0x20/0x80
__do_page_fault+0x1b7/0x520
? exit_to_usermode_loop+0x58/0xd0
do_page_fault+0x2c/0x40
page_fault+0x28/0x30
RIP: 0033:0x43807a
RSP: 002b:00007ffcfbe17720 EFLAGS: 00010202
RAX: 000000004e90ea80 RBX: 0000000000000066 RCX: 0000000000000168
RDX: 000000000047c60b RSI: 0000000000000168 RDI: 000000004e90ea80
RBP: 00007ffcfbe17730 R08: 0000003aec78e188 R09: 0000000000000001
R10: 0000000000000010 R11: fffffffffffff000 R12: 0000000000000168
R13: 0000000000000065 R14: 0000000000000001 R15: 0000000000000001
Mem-Info:
active_anon:4147502 inactive_anon:476052 isolated_anon:0
active_file:2426 inactive_file:27709508 isolated_file:256
unevictable:0 dirty:7 writeback:0 unstable:0
slab_reclaimable:371384 slab_unreclaimable:65708
mapped:4313 shmem:525467 pagetables:18502 bounce:0
free:88956 free_pcp:21 free_cma:0
Node 0 active_anon:8812820kB inactive_anon:932012kB active_file:3872kB inactive_file:54201636kB unevictable:0kB isolated(anon):0kB isolated(file):768kB mapped:5208kB dirty:8kB writeback:0kB shmem:32kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 102400kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
Node 1 active_anon:7777188kB inactive_anon:972196kB active_file:5832kB inactive_file:56636396kB unevictable:0kB isolated(anon):0kB isolated(file):256kB mapped:12044kB dirty:20kB writeback:0kB shmem:2101836kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 38912kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
Node 0 DMA free:15884kB min:8kB low:20kB high:32kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15968kB managed:15884kB mlocked:0kB kernel_stack:0kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 1809 64269 64269
Node 0 DMA32 free:251108kB min:1264kB low:3116kB high:4968kB active_anon:433048kB inactive_anon:10372kB active_file:96kB inactive_file:1242292kB unevictable:0kB writepending:0kB present:1967292kB managed:1951368kB mlocked:0kB kernel_stack:20kB pagetables:664kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 0 62460 62460
Node 0 Normal free:43792kB min:43700kB low:107656kB high:171612kB active_anon:8379772kB inactive_anon:921640kB active_file:3728kB inactive_file:52958832kB unevictable:0kB writepending:8kB present:65011712kB managed:63959216kB mlocked:0kB kernel_stack:6116kB pagetables:28756kB bounce:0kB free_pcp:84kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0
Node 1 Normal free:45040kB min:45132kB low:111184kB high:177236kB active_anon:7777188kB inactive_anon:972196kB active_file:5832kB inactive_file:56636416kB unevictable:0kB writepending:20kB present:67108864kB managed:66056324kB mlocked:0kB kernel_stack:8728kB pagetables:44588kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 1*4kB (U) 1*8kB (U) 0*16kB 0*32kB 2*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (M) 3*4096kB (M) = 15884kB
Node 0 DMA32: 5882*4kB (UME) 5631*8kB (UME) 3549*16kB (UME) 2064*32kB (UME) 689*64kB (UME) 110*128kB (UME) 2*256kB (M) 0*512kB 1*1024kB (M) 0*2048kB 0*4096kB = 251120kB
Node 0 Normal: 344*4kB (UMEH) 2832*8kB (UMEH) 1133*16kB (UEH) 17*32kB (UEH) 7*64kB (H) 6*128kB (H) 1*256kB (H) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 44176kB
Node 1 Normal: 945*4kB (UMEH) 3129*8kB (UMEH) 922*16kB (UMEH) 55*32kB (UE) 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 45324kB
Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
Node 1 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Node 1 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
10421934 total pagecache pages
4010 pages in swap cache
Swap cache stats: add 2182308, delete 2178023, find 1238728/1661691
Free swap = 44475900kB
Total swap = 45257724kB
33525959 pages RAM
0 pages HighMem/MovableOnly
530261 pages reserved
0 pages HighMem/MovableOnly
530261 pages reserved
0 pages hwpoisoned