Situation

When Login Docker Server, Use df -h Check Disk Usage Show :

root@docker-service:/var/lib# df
Filesystem                        1K-blocks      Used Available Use% Mounted on
udev                                8078356         0   8078356   0% /dev
tmpfs                               1625492    153072   1472420  10% /run
/dev/mapper/ubuntu--vg-ubuntu--lv 121962128 121945744         0 100% /
tmpfs                               8127448         0   8127448   0% /dev/shm
tmpfs                                  5120         0      5120   0% /run/lock
tmpfs                               8127448         0   8127448   0% /sys/fs/cgroup

How To Fix

The First Step, Fix Error & Make Sure All Services Work.

truncate -s 0 <container_log_path/container_log>.log

or You Can Keep Last Line 10,000

tail -n 10000 <container_log_path/container_log>.log > /tmp/latest.log
cat /tmp/latest.log <container_log_path/container_log>.log 
rm /tmp/latest.log

The Second Step, Change Log Rotate Setting Or Centralized Logging

Local Setting

In docker daemon setting Add These Conf, And Restart Docker Process

vim /etc/docker/daemon.json

{
  "log-driver": "json-file", #Default Log Driver
  "log-opts": {
    "max-size": "100m", #Max Size For Per Log File
    "max-file": "3" #Log File Count
  }
}

systemctl restart docker

Centralized Setting

In docker daemon setting Add These Conf, And Restart Docker Process

vim /etc/docker/daemon.json

{
  "log-driver": "fluentd", #Use Fluentd Agent
  "log-opts": {
    "fluentd-address": "your-fluentd-server:24224"  #Log Server URL
  }
}

systemctl restart docker