Situation

When Login Docker Server, Use df -h Check Disk Usage Show :

root@docker-service:/var/lib# df
Filesystem                        1K-blocks      Used Available Use% Mounted on
udev                                8078356         0   8078356   0% /dev
tmpfs                               1625492    153072   1472420  10% /run
/dev/mapper/ubuntu--vg-ubuntu--lv 121962128 121945744         0 100% /
tmpfs                               8127448         0   8127448   0% /dev/shm
tmpfs                                  5120         0      5120   0% /run/lock
tmpfs                               8127448         0   8127448   0% /sys/fs/cgroup

How To Fix

The First Step, Fix Error & Make Sure All Services Work.

truncate -s 0 <container_log_path/container_log>.log

or You Can Keep Last Line 10,000

tail -n 10000 <container_log_path/container_log>.log > /tmp/latest.log
cat /tmp/latest.log > <container_log_path/container_log>.log 
rm /tmp/latest.log

The Second Step, Change Log Rotate Setting Or Centralized Logging

Local Setting

max-size Is The Max Size Per Log File, max-file Is How Many Files To Keep. daemon.json Is Plain JSON, So Don’t Paste # Comments Into It Or Docker Won’t Start.

In docker daemon setting Add These Conf, And Restart Docker Process

vim /etc/docker/daemon.json

{
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m",
    "max-file": "3"
  }
}

systemctl restart docker

Centralized Setting

In docker daemon setting Add These Conf, And Restart Docker Process

vim /etc/docker/daemon.json

{
  "log-driver": "fluentd",
  "log-opts": {
    "fluentd-address": "your-fluentd-server:24224"
  }
}

systemctl restart docker