systemd-journald는 Docker를 실행하는 시스템에 로그를 유지하지 않습니다.

systemd-journald는 Docker를 실행하는 시스템에 로그를 유지하지 않습니다.

대략 매 시간마다 systemd-journald는 /var/log/journal/에 저장하는 대신 약 1시간이 지난 로그를 삭제합니다. 저는 Journal.conf에 Storage=pertant를 설정했습니다. 로그는 회전될 때마다 새로 고쳐지지 않습니다.

dmesg에 디버그 로그를 표시하도록 systemd-journald 재정의:

pi@######:/etc/systemd/system/systemd-journald.service.d$ cat override.conf
[Service]
Environment=SYSTEMD_LOG_LEVEL=debug

삭제 시 dmesg에 나타나는 로그는 다음과 같습니다.

[Tue Jun 14 11:13:33 2022] systemd-journald[2020]: /run/log/journal/e00bd6223a7245d0aa6e19e0ab12a024/system.journal: Journal header limits reached or header out-of-date, rotating.
[Tue Jun 14 11:13:33 2022] systemd-journald[2020]: Rotating...
[Tue Jun 14 11:13:33 2022] systemd-journald[2020]: Reserving 8448 entries in hash table.
[Tue Jun 14 11:13:33 2022] systemd-journald[2020]: Vacuuming...
[Tue Jun 14 11:13:33 2022] systemd-journald[2020]: Deleted archived journal /run/log/journal/e00bd6223a7245d0aa6e19e0ab12a024/system@d30583b2cbf04d2aab3e5927a175ef5e-0000000000065a39-0005e163dbe0ff99.journal (4.6M).
[Tue Jun 14 11:13:33 2022] systemd-journald[2020]: Vacuuming done, freed 4.6M of archived journals from /run/log/journal/e00bd6223a7245d0aa6e19e0ab12a024.

내가 이해하지 못하는 가장 중요한 점은 왜 저널ctl에 명령을 보낼 때 저널이 중지되었다고 저널ctl에 표시되는지입니다. 아래 날짜를 보면 중지되었다가 다시는 시작되지 않았음을 알 수 있습니다. 그래서 로그가 새로 고쳐지지 않는 것 같아요?

시스템 로그:

pi@#######:~$ date
Wed Jun 15 11:06:27 SAST 2022
pi@#######:~$ sudo journalctl -u systemd-journald | cat
-- Logs begin at Mon 2022-01-10 06:56:38 SAST, end at Wed 2022-06-15 11:06:34 SAST. --
Jan 10 06:56:39 ####### systemd-journald[2019]: Journal started
Jan 10 06:56:39 ####### systemd-journald[2019]: Runtime Journal (/run/log/journal/e00bd6223a7245d0aa6e19e0ab12a024) is 4.6M, max 37.1M, 32.4M free.
-- Reboot --
Jan 10 06:56:39 ####### systemd-journald[2011]: Journal started
Jan 10 06:56:39 ####### systemd-journald[2011]: Runtime Journal (/run/log/journal/e00bd6223a7245d0aa6e19e0ab12a024) is 4.6M, max 37.1M, 32.4M free.
Jan 10 06:56:39 ####### systemd-journald[2011]: Runtime Journal (/run/log/journal/e00bd6223a7245d0aa6e19e0ab12a024) is 4.6M, max 37.1M, 32.4M free.
-- Reboot --
Jan 10 06:56:39 ####### systemd-journald[2019]: Runtime Journal (/run/log/journal/e00bd6223a7245d0aa6e19e0ab12a024) is 4.6M, max 37.1M, 32.4M free.
Jun 13 12:16:53 ####### systemd-journald[2019]: Journal stopped
Jun 13 12:16:53 ####### systemd-journald[7765]: Journal started
Jun 13 12:16:53 ####### systemd-journald[7765]: Runtime Journal (/run/log/journal/e00bd6223a7245d0aa6e19e0ab12a024) is 18.5M, max 37.1M, 18.5M free.
Jun 13 12:16:53 ####### systemd-journald[7765]: Time spent on flushing to /var/log/journal/e00bd6223a7245d0aa6e19e0ab12a024 is 455.515ms for 8948 entries.
Jun 13 12:16:53 ####### systemd-journald[7765]: System Journal (/var/log/journal/e00bd6223a7245d0aa6e19e0ab12a024) is 24.0M, max 900.0M, 876.0M free.
Jun 13 12:19:39 ####### systemd-journald[7765]: Journal stopped
Jun 13 12:19:39 ####### systemd-journald[8468]: Journal started
Jun 13 12:19:39 ####### systemd-journald[8468]: Runtime Journal (/run/log/journal/e00bd6223a7245d0aa6e19e0ab12a024) is 4.6M, max 37.1M, 32.4M free.
Jun 13 12:19:39 ####### systemd-journald[8468]: Time spent on flushing to /var/log/journal/e00bd6223a7245d0aa6e19e0ab12a024 is 4.480ms for 15 entries.
Jun 13 12:19:39 ####### systemd-journald[8468]: System Journal (/var/log/journal/e00bd6223a7245d0aa6e19e0ab12a024) is 32.0M, max 900.0M, 867.9M free.
Jun 13 12:19:49 ####### systemd-journald[8468]: Journal stopped
Jun 13 12:19:49 ####### systemd-journald[8648]: Journal started
Jun 13 12:19:49 ####### systemd-journald[8648]: Runtime Journal (/run/log/journal/e00bd6223a7245d0aa6e19e0ab12a024) is 4.6M, max 37.1M, 32.4M free.
Jun 13 12:19:49 ####### systemd-journald[8648]: Time spent on flushing to /var/log/journal/e00bd6223a7245d0aa6e19e0ab12a024 is 6.845ms for 54 entries.
Jun 13 12:19:49 ####### systemd-journald[8648]: System Journal (/var/log/journal/e00bd6223a7245d0aa6e19e0ab12a024) is 32.0M, max 900.0M, 867.9M free.
Jun 13 12:22:28 ####### systemd-journald[8648]: Journal stopped
-- Reboot --
Jan 10 06:56:39 ####### systemd-journald[2014]: Journal started
Jan 10 06:56:39 ####### systemd-journald[2014]: Runtime Journal (/run/log/journal/e00bd6223a7245d0aa6e19e0ab12a024) is 4.6M, max 37.1M, 32.4M free.
Jan 10 06:56:39 ####### systemd-journald[2014]: Runtime Journal (/run/log/journal/e00bd6223a7245d0aa6e19e0ab12a024) is 4.6M, max 37.1M, 32.4M free.
Jun 13 15:57:22 ####### systemd-journald[2014]: Journal stopped
Jun 13 15:57:22 ####### systemd-journald[6463]: Journal started
Jun 13 15:57:22 ####### systemd-journald[6463]: Runtime Journal (/run/log/journal/e00bd6223a7245d0aa6e19e0ab12a024) is 27.8M, max 37.1M, 9.2M free.
Jun 13 15:57:22 ####### systemd-journald[6463]: Time spent on flushing to /var/log/journal/e00bd6223a7245d0aa6e19e0ab12a024 is 2.085184s for 16277 entries.
Jun 13 15:57:22 ####### systemd-journald[6463]: System Journal (/var/log/journal/e00bd6223a7245d0aa6e19e0ab12a024) is 48.0M, max 900.0M, 851.9M free.
Jun 13 15:57:42 ####### systemd-journald[6463]: Journal stopped
Jun 13 15:57:42 ####### systemd-journald[6532]: Journal started
Jun 13 15:57:42 ####### systemd-journald[6532]: Runtime Journal (/run/log/journal/e00bd6223a7245d0aa6e19e0ab12a024) is 4.6M, max 37.1M, 32.4M free.
Jun 13 15:57:42 ####### systemd-journald[6532]: Time spent on flushing to /var/log/journal/e00bd6223a7245d0aa6e19e0ab12a024 is 5.834ms for 19 entries.
Jun 13 15:57:42 ####### systemd-journald[6532]: System Journal (/var/log/journal/e00bd6223a7245d0aa6e19e0ab12a024) is 48.0M, max 900.0M, 851.9M free.
Jun 13 16:20:10 ####### systemd-journald[6532]: Journal stopped
-- Reboot --
Jun 14 13:06:17 ####### systemd-journald[2020]: Journal stopped
Jun 14 13:06:17 ####### systemd-journald[333498]: Journal started
Jun 14 13:06:17 ####### systemd-journald[333498]: Runtime Journal (/run/log/journal/e00bd6223a7245d0aa6e19e0ab12a024) is 37.1M, max 37.1M, 0B free.
Jun 14 13:06:17 ####### systemd-journald[333498]: Time spent on flushing to /var/log/journal/e00bd6223a7245d0aa6e19e0ab12a024 is 3.509167s for 26919 entries.
Jun 14 13:06:17 ####### systemd-journald[333498]: System Journal (/var/log/journal/e00bd6223a7245d0aa6e19e0ab12a024) is 82.0M, max 900.0M, 817.9M free.
Jun 14 13:06:30 ####### systemd-journald[333498]: Journal stopped
Jun 14 13:06:30 ####### systemd-journald[333538]: Journal started
Jun 14 13:06:30 ####### systemd-journald[333538]: Runtime Journal (/run/log/journal/e00bd6223a7245d0aa6e19e0ab12a024) is 4.6M, max 37.1M, 32.4M free.
Jun 14 13:06:30 ####### systemd-journald[333538]: Time spent on flushing to /var/log/journal/e00bd6223a7245d0aa6e19e0ab12a024 is 4.822ms for 15 entries.
Jun 14 13:06:30 ####### systemd-journald[333538]: System Journal (/var/log/journal/e00bd6223a7245d0aa6e19e0ab12a024) is 82.0M, max 900.0M, 817.9M free.
Jun 14 14:22:28 ####### systemd-journald[333538]: Journal stopped

읽기/쓰기를 보존하기 위해 SSD 대신 하드 드라이브에 로그를 저장할 수 있도록 기호 링크를 사용합니다.

pi@######:~$ ls -ltr /var/log
lrwxrwxrwx 1 root syslog 17 Jan 10 06:57 /var/log -> /mnt/usbdrive/log

machine-id는 채워지고 시작 시 변경되지 않습니다.

pi@######:~$ cat /etc/machine-id
e00bd6223a7245d0aa6e19e0ab12a024

/etc/systemd/journal.conf 파일은 다음과 같이 설정됩니다.

[Journal]
Storage=persistent
#Compress=yes
#Seal=yes
#SplitMode=uid
#SyncIntervalSec=5m
#RateLimitIntervalSec=30s
#RateLimitBurst=10000
SystemMaxUse=900M
#SystemKeepFree=
SystemMaxFileSize=50M
#SystemMaxFiles=100
#RuntimeMaxUse=
#RuntimeKeepFree=
#RuntimeMaxFileSize=
#RuntimeMaxFiles=100
#MaxRetentionSec=
#MaxFileSec=1month
ForwardToSyslog=no
#ForwardToKMsg=no
#ForwardToConsole=no
#ForwardToWall=yes
#TTYPath=/dev/console
#MaxLevelStore=debug
#MaxLevelSyslog=debug
#MaxLevelKMsg=notice
#MaxLevelConsole=info
#MaxLevelWall=emerg
#LineMax=48K
#ReadKMsg=yes

/var/log/journal 권한:

pi@######:/var/log$ ls -ltr journal
total 4
drwxr-xr-x 2 root systemd-journal 4096 Jun 14 13:06 e00bd6223a7245d0aa6e19e0ab12a024
pi@######:~$ ls -ld /var/log/journal/
drwxr-xr-x 3 root systemd-journal 4096 Jun 13 12:16 /var/log/journal/

도커 버전:

pi@######:~$ docker --version
Docker version 20.10.7, build 20.10.7-0ubuntu5~20.04.2
pi@######:~$ docker-compose --version
docker-compose version 1.25.0,

docker-compose.yml의 Docker 로깅 드라이버

    logging:
      driver: journald
      options:
        tag: "########"

하드웨어 플랫폼은 Odroid HC4(pi 사용자를 위한 레거시 제품)입니다. OS: Ubuntu 20.04

커널 버전:

pi@#######:~$ uname -r
4.9.277-83

관련 정보