Reading the system log

journalctl is your interface into a single machine’s journal/logging. All service files insert data into the systemd journal. There are a few helpful commands to read the journal:

Read the entire journal

$ journalctl

-- Logs begin at Fri 2013-12-13 23:43:32 UTC, end at Sun 2013-12-22 12:28:45 UTC. --
Dec 22 00:10:21 localhost systemd-journal[33]: Runtime journal is using 184.0K (max 49.9M, leaving 74.8M of free 499.0M, current limit 49.9M).
Dec 22 00:10:21 localhost systemd-journal[33]: Runtime journal is using 188.0K (max 49.9M, leaving 74.8M of free 499.0M, current limit 49.9M).
Dec 22 00:10:21 localhost kernel: Initializing cgroup subsys cpuset
Dec 22 00:10:21 localhost kernel: Initializing cgroup subsys cpu
Dec 22 00:10:21 localhost kernel: Initializing cgroup subsys cpuacct
Dec 22 00:10:21 localhost kernel: Linux version 3.11.7+ ([email protected]) (gcc version 4.6.3 (Gentoo Hardened 4.6.3 p1.13, pie-0.5.2)
...
1000s more lines

Read entries for a specific service

Read entries generated by a specific unit:

$ journalctl -u apache.service

-- Logs begin at Fri 2013-12-13 23:43:32 UTC, end at Sun 2013-12-22 12:32:52 UTC. --
Dec 22 12:32:39 localhost systemd[1]: Starting Apache Service...
Dec 22 12:32:39 localhost systemd[1]: Started Apache Service.
Dec 22 12:32:39 localhost docker[9772]: /usr/sbin/apache2ctl: 87: ulimit: error setting limit (Operation not permitted)
Dec 22 12:32:39 localhost docker[9772]: apache2: Could not reliably determine the server's fully qualified domain name, using 172.17.0.6 for ServerName

Read the user journal from the current user

It might be required to add a user different from core user to the systemd-journal group to read the user journal. It can be done with this Butane config:

variant: flatcar
version: 1.0.0
passwd:
  users:
    - name: flatcar
      groups:
        - systemd-journal

Then from flatcar login: journalctl --user.

Read entries since boot

Reading just the entries since the last boot is an easy way to troubleshoot services that are failing to start properly:

journalctl --boot

Tail the journal

You can tail the entire journal or just a specific service:

journalctl -f

journalctl -u apache.service -f

Read entries with line wrapping

By default journalctl passes FRSXMK command line options to less . You can override these options by setting a custom SYSTEMD_LESS environment variable with omitted S option:

SYSTEMD_LESS=FRXMK journalctl

Read logs without pager:

journalctl --no-pager

Debugging journald

If you’ve faced some problems with journald you can enable debug mode following the instructions below.

Enable debugging manually

mkdir -p /etc/systemd/system/systemd-journald.service.d/

Create a Drop-In /etc/systemd/system/systemd-journald.service.d/10-debug.conf with following content:

[Service]
Environment=SYSTEMD_LOG_LEVEL=debug

And restart systemd-journald service:

systemctl daemon-reload
systemctl restart systemd-journald
dmesg | grep systemd-journald

Enable debugging via a Butane Config

Define a Drop-In in a Butane Config :

variant: flatcar
version: 1.0.0
systemd:
  units:
    - name: systemd-journald.service
      dropins:
        - name: 10-debug.conf
          contents: |
            [Service]
            Environment=SYSTEMD_LOG_LEVEL=debug

More information

Getting Started with systemd Network Configuration with networkd