Centralise Linux Logs with systemd-journal-remote (Forward and Collect)

systemd journal remote - custom-jr-featured.png

When you have more than a handful of servers, logging into each host to run journalctl does not scale. systemd can forward journals to a collector over HTTPS while keeping
the familiar journal fields (unit name, priority, boot ID) intact.

This complements local troubleshooting in our journalctl guide and log retention with
journal vacuum scripts. For long-term metrics, still ship important events to
Prometheus and Grafana.

Upload vs Remote: Two Roles

systemd-journal-upload runs on each client and pushes logs. systemd-journal-remote runs on the collector and writes incoming streams under /var/log/journal/remote/. You need both packages on their respective hosts.

architecturejournal-remote.txt
  [app servers] --HTTPS--> [collector: systemd-journal-remote]
                                      |
                               journalctl on collector
Client and collector
Upload pushes, remote receives

Install Packages

Package names differ slightly by distribution; the services are systemd-journal-upload and systemd-journal-remote.

bashDebian / Ubuntu
sudo apt install -y systemd-journal-remote
bashenable receiver on collector
sudo systemctl enable --now systemd-journal-remote.socket
Packages
upload + remote

Configure the Collector Socket

The collector listens on a TCP port (often 19532). Put it on a management VLAN and firewall it; do not expose log ingestion to the public internet without TLS and auth.

ini/etc/systemd/journal-remote.conf (snippet)
[Remote]
Storage=auto
Compress=yes
bashfirewall (example)
sudo ufw allow from 10.0.0.0/24 to any port 19532 proto tcp
journal-remote.socket
Listen address

Point Clients at the Collector

On each sending host, configure systemd-journal-upload with the collector URL. Use HTTPS when you terminate TLS on the collector or behind a reverse proxy.

ini/etc/systemd/journal-upload.conf
[Upload]
URL=https://logs.internal.example:19532/upload
ServerKeyFile=/etc/ssl/private/journal-upload.key
ServerCertificateFile=/etc/ssl/certs/journal-upload.pem
bashenable on client
sudo systemctl enable --now systemd-journal-upload.service
journal-upload
URL and interval

TLS Between Client and Collector

Generate or import certificates for mutual TLS or server-auth TLS depending on your threat model. At minimum use a private CA and verify the collector hostname.

bashtest HTTPS endpoint
curl -vk https://logs.internal.example:19532/
TipFor lab setups you can start with HTTP on a private network only, then add TLS before production.
HTTPS forwarding
Certificates

Search Forwarded Logs on the Collector

Remote journals appear as separate files. Use journalctl with --directory or the MACHINE_ID / hostname fields to filter.

bashlist remote hosts
journalctl --directory=/var/log/journal/remote --list-boots
basherrors from one unit
journalctl --directory=/var/log/journal/remote -u nginx.service -p err --since today
journalctl on collector
Filter by host and unit

Retention and Disk on the Collector

Centralising logs concentrates disk use. Set SystemMaxUse in journald.conf on the collector and monitor free space.

ini/etc/systemd/journald.conf
SystemMaxUse=20G
MaxRetentionSec=30day
bashvacuum now
sudo journalctl --vacuum-size=15G
Disk limits
journald vacuum

Quick Reference

  • Clients: systemd-journal-upload; collector: systemd-journal-remote
  • Firewall ingestion port; prefer HTTPS
  • Query with journalctl --directory=/var/log/journal/remote

Related tutorials

Diagrams are original illustrations by Gnome IT Solutions. Tutorial text © Gnome IT Solutions.