Docker - Logging

Docker logs are vital in maintaining and troubleshooting applications running within containers. They provide real-time insight into container behavior and performance to help identify problems and, thus, optimize performance. They capture information wide enough to cover errors, warnings, and informative messages that an application and Docker engine might have produced. This data is beneficial for debugging: developers could - with this - trace which events had happened and led up to a specific issue, understand the context, and apply fixes.

Moreover, Docker logs have great significance in monitoring and auditing. Logs should be collected and continuously analyzed to ensure the applications run smoothly and securely. It detects the occurrence of anomalies and unauthorized access, among other security threats, so that potential breaches can be responded to promptly.

Docker logs enable essential visibility into the system and control for any proper stewardship of systems to maintain the resilience and robustness of a system in the production environment where uptime and reliability are of critical importance.

In this chapter, let’s learn more about Docker logs and logging drivers.

How is Docker Logging Different?

Docker logs are different than traditional logs due to the containerized nature of Docker. Let’s have a look at the basic difference.

Centralization and Aggregation

Traditional logging often involves collecting logs from individual servers or applications, which tends to scale out and, with time, starts becoming cumbersome as the number of servers and applications continues to rise. In contrast, Docker logging often involves centralizing and aggregating logs from multiple containers running across various hosts.

In a centralized approach, this would simplify log management and make it easy for logs to be watched or analyzed from a single point, even in an environment that is complex and distributed.

Log Drivers and Plugins

Docker provides a broad set of log drivers and plugins for tailoring the means of collecting, storing, and processing logs. These log drivers will allow sending logging to different storage points - JSON files, Syslog, Fluentd, AWS CloudWatch, and others. This way, numerous options can be provided to tune the logging setup to meet specific needs and preferences and integrate seamlessly into existing logging and monitoring tools.

Ephemeral Nature of Containers

Containers are ephemeral; they are meant to have a short life span and to be paused, stopped, or removed. This transient characteristic appears to be a bottleneck for the traditional methods of logging that primarily rely on persistent native storage of the host system. Docker solves this problem by storing logs out of container life cycles.

Persistence is essential for maintaining a complete chronology of event history for critical diagnostic information that may be accessed even after the removal or replacement of containers. These subtle differences drive home one thing: Docker logging is designed for the dynamic and scalable property associated with containerized environments. It is reflected in the solutions around centralized log management, thus making log data available.

Docker Logging Strategies and Best Practices

Active logging is an essential process in managing or supporting your Dockerized applications. Log entries give incredible insight into the behavior of your applications, their performance, and issues, allowing managing proactively and troubleshooting quickly.

Docker provides several ways to manage logging, each with its benefits and suitable use cases. Let’s discuss each of them one by one.

Logging Through the Application

The simplest way to log from Dockerized applications is actually from the application itself. This can be done by setting it up so that it logs using standard output (stdout) and standard error (stderr). Docker collects these outputs, so for the logs, you can retrieve them easily using the docker logs command.

Advantages of Logging through the Application

Ease of Implementation − It is easy to carry out and requires no additional configuration.
Portability − Logs can easily be accessed natively by means of Docker's logging.
Compatibility − Works well with any app that can be configured to log out to stdout and stderr.

Best Practices

Structured Logging − Better parsing and analysis of logs, since it is in a structured format, i.e., JSON logs.
Log Rotation − Use log rotation by an application to limit log file bloat.
Log Levels − Set up appropriate log levels (e.g., debug, info, warning, error) to specify the verbosity of the logs.

Data Volumes Logging

Another approach is to have Docker data volumes where logs are stored in them. If you attach a volume to a directory inside the container that writes out logs, you can be confident that it will withstand the removal or restarting of the container.

Advantages of Data Volumes Logging

Persistence − The logs do not get lost by the destruction and recreation of the container.
Separation of Concerns − Keep log storage away from containerized applications.
Flexibility − External log management tools can access and process logs directly from the volume.

Best Practices

Volume Management − Monitor and manage log volume size to ensure disk space is not a problem.
Backup and Retention − Implement your organization's logging backup policies and their retention.
Access Controls − Protect the volume of logs from unauthorized access.

Logging with the Docker Logging Driver

Docker includes numerous built-in logging drivers, which give flexible options for sending the container logs to various destinations: syslog, journald, Fluentd, and AWS CloudWatch, among others. It is configurable at the daemon and container levels.

Advantages of Logging with the Docker Logging Driver

Centralized Logging − Easily collect logs from several hosts and containers.
CI/CD Integration − Includes integration with the existing infrastructure and tools for logging.
Scalability − It supports multiple log storage back-ends and services.

Best Practices

Picking a Driver − Choose a logging driver that goes well with your logging infrastructure and your needs.
Configuration − The Logging driver is configured to ensure that the delivered performance and reliability are at their best.
Monitoring − Observe the driver for errors or problems in logging continuously.

Logging with a Specific Logging Container

The logging container is dedicated, meaning it can do log collection from all other containers on the host. This usually means the container runs a logging agent, like Fluentd or Logstash, and aggregate and send logs to a central logging system.

Advantages

Isolation − The logging concerns remain in their context isolation from the application containers.
Scalability − It is independent of application containers.
Flexibility − Supports complex log processing and forwarding configurations.

Best Practices

Resource Allocation − Resource the logging container sufficiently to handle the log volume.
Network Configuration − Set up the networking for secure and effective log transfer between containers.
Redundancy − Introduce redundancy and failover features for the logging container.

Logging Through the Sidecar approach

The sidecar container runs alongside the main application container within the same pod in Kubernetes or Docker setup. The responsibilities of the sidecar container include the collection, processing, and forwarding of the application logs.

Benefits

Proximity − It ensures the collection of logs with low latency because the main container is placed close to the applications.
Modularity − Logging functionalities can be scaled and managed independently.
Consistency − Have the same logging configuration across all the environments.

Best Practices

Synchronization should be proper between the main container and the sidecar container.

Resource Allocation − Allocate sufficient resources to the sidecar container for log processing.
Observability − Observe and monitor the health and functioning of both the main and sidecar containers.

How to Work With Docker Container Logs Using the Docker Logs Command?

Managing and accessing logs are vital parts of working with Docker containers. The docker logs command provides a convenient way to access and view your running container logs. This guide will, therefore, take you through how to use the docker logs command with both practical examples and best practices.

Basic Usage of the Docker Logs Command

The docker logs command is useful for fetching a container's logs. The syntax is quite simple, for instance:

$ docker logs [OPTIONS] CONTAINER

Here, CONTAINER stands for the name or ID of the container of which you want to view the logs.

Viewing Logs

To view the logs of a container, we have to use the docker logs command and then the name or ID of the container. Example −

$ docker logs my-container

The command below would display all the logs produced by the container my-container.

Log Streaming in Real Time

The -f or --follow flags will allow you to see running logs, which is like doing a "tail -f" on a log within a Unix-like system. It will tail you logs right off the treadmill −

$ docker logs -f my-container

Tail Logs

If you want to see just the latest log records, use the --tail option: This comes in handy, especially when managing substantial log files.

$ docker logs --tail 100 my-container

This command will get up the last 100 lines of logs for my-container.

Timestamped Logs

Use the -t or --timestamps option for logs with timestamps. This option introduces a timestamp in each log entry to make it easier to correlate events −

$ docker logs -t my-container

Combining Options

Multiple options can then be combined to create a custom log view. For example, if you would like to see logs with entries that have timestamps and are limited to the last 50 entries, you can use −

$ docker logs -f -t --tail 50 my-container

Filtering Logs by Date

While the docker logs command does not support filtering the date from logs, this can be achieved with the help of the Unix grep command. Here's an example to demonstrate −

$ docker logs my-container | grep "2024-06-01"

This command filters logs from my-container to show only those that contain the 2024-06-01 date.

Saving Logs to a File

You can also save these logs into a file for subsequent analysis or for archival purposes −

$ docker logs my-container > my-container-logs.txt

This command logs all the logs from my-container to a file with the name my-container-logs.txt.

Log Rotation

Docker itself doesn't do log rotation by itself. So, log files might get way too large in size. The Docker logging driver options can configure the log rotation. For example, the limit of file size for log and the number of log files: Add the following in your `docker-compose.yml` file or `daemon.json` −

{
   "log-driver": "json-file",
   "log-opts": {
      "max-size": "10m",
      "max-file": "3"
   }
}

It's set to limit each log file size to 10 MB and retain a maximum of three log files.

What is a Logging Driver?

A Docker logging driver is a means of directing the log messages that originate from your Docker containers to a specific location and in a particular manner. By default, Docker captures logs from the standard output (stdout) and standard error (stderr) stream of each container.

A logging driver specifies where these logs are directed to: locally on the file, far away on the server, or could be a hook to a service external to the actual logging system. Docker also supports multiple logging drivers to have scalable and flexible log management solutions.

How to Configure the Docker Logging Driver?

You can configure the logging driver both at the Docker daemon level and on a per-container basis.

Configure Docker Logging Driver for Daemon

To set the default value, thereby controlling all the containers being managed on the Docker daemon, you can use the daemon.json configuration file. Typically, this file is at /etc/docker/daemon.json on Linux systems. For example, here's how you would set it to log to the 'JSON-file' driver −

{
   "log-driver": "json-file",
   "log-opts": {
      "max-size": "10m",
      "max-file": "3"
}

Then, to apply the changes, you must restart the Docker daemon by running −

$ sudo systemctl restart docker

Using a Logging Driver for Individual Containers

The default logging driver can be overridden when starting a container using the --logdriver option. For example, to use the syslog logging driver, the following line can be added −

$ docker run --log-driver=syslog my-container

The following can also be specified to select more logging options −

$ docker run --log-driver=syslog --log-opt syslog-address=udp://localhost:514 my-container

Where are Docker logs saved by default?

By default, Docker uses the log driver: json-file. Log files in this format will be dropped into a directory on the host at /var/lib/docker/containers/<container-id>/. Each container will have one of these directories. Logs are written into a file called <container-id>-json.log.

What Are Modes of Delivery?

Delivery modes only define how log messages are passed to their final destination in Docker. Two main delivery modes are supported for logging drivers −

Blocking (default)

In blocking mode, messages are delivered synchronously. The sending process within the container will be paused until the logging driver has acknowledged that the message to the log has been processed. It is reliable in the sense that the message escapes the container. However, that can be painful in terms of performance if a logging backend is slow or unavailable.

Non-blocking

In the non-blocking delivery, the container process does not wait for the log message processing by the logging driver, which increases performance at the expense that, in the case of a logging driver or its backend being incapable of coping with the data rate, logs may be lost. One way of activating the non-blocking mode is using the mode logging option.

Example of setting up non-blocking mode −

$ docker run --log-driver=json-file --log-opt mode=non-blocking my-container

Docker Logging Driver Options/Flags

Docker logging drivers provide numerous options or flags with which one can configure them per one's needs. Below are some of these common possibilities and popularly used logging drivers −

json-file

max-size − The maximum size of the log file before it is rotated.
max-file − Number of maximum log files to keep.

$ docker run --log-driver=json-file --log-opt max-size=10m 
   --log-opt max-file=3 my-container

syslog

syslog-address − The server address where syslog messages will be sent.
syslog-facility − The syslog facility to use (e.g., daemon, user).
syslog-tls-ca-cert − The path on the machine to the CA certificate for TLS.

$ docker run --log-driver=syslog --log-opt syslog-address=udp://localhost:514 
   --log-opt syslog-facility=daemon my-container

Fluentd

fluentd-address − Fluentd server address.
fluentd-async-connect − Connect to Fluentd asynchronously.
fluentd-buffer-limit − Set buffer limit for Fluentd.

$ docker run --log-driver=fluentd --log-opt fluentd-address=localhost:24224 
   --log-opt fluentd-async-connect=true my-container

AWS-Logs

awslogs-region − The region of AWS.
awslogs-group − Name of the CloudWatch Logs group.
awslogs-stream − Name of the CloudWatch Logs stream.

$ docker run --log-driver=awslogs --log-opt awslogs-region=us-east-1 
   --log-opt awslogs-group=my-log-group 
   --log-opt awslogs-stream=my-log-stream my-container

Conclusion

By using the diverse logging approach proposed by Docker, from application-level logging and data volumes to advanced logging drivers and popular sidecars, developers and system administrators can adjust the logging setup to the needs and operational requirements they are facing. Each of them has its advantages—ranging from simplicity and an easy way to scale to integration with external logging systems, hence providing comprehensive log management in heterogeneous environments.

FAQ

Q 1. What is the default logging driver in Docker, and how does it work?

In Docker, the json-file driver serves as the default logging mechanism. This driver captures the logs generated by your containers and stores them as JSON objects, with each container having its dedicated log file. This structured format is convenient for parsing and analyzing log data, but be mindful of potential storage issues if these files aren't managed effectively.

Q 2. Can I customize the way Docker handles logs?

Absolutely! Docker offers a variety of logging drivers to choose from, each with unique features and behaviors. You can opt for drivers that send logs to centralized logging services like Syslog or Fluentd for streamlined management and analysis. If none of the existing drivers perfectly fit your needs, you even have the option to create a custom logging driver tailored to your specific requirements.

Q 3. How do I view logs for a running Docker container?

Viewing logs for a running container is a simple task. You can use the docker logs command followed by the container's name or ID to display the logs. To monitor logs in real time, include the --follow option. If you're only interested in seeing the most recent entries, use the --tail option to limit the output. For more advanced filtering or analysis, you might want to explore third-party logging tools designed specifically for Docker.

Q 4. What are some best practices for logging in Docker?

Effective logging in Docker involves several key practices. Firstly, centralizing logs from multiple containers into a unified location makes management and analysis significantly easier. It's also crucial to select the right logging driver based on your infrastructure and specific requirements. This involves balancing factors like performance and the desired features of the driver.

Using a structured format like JSON simplifies log parsing and querying, while setting log rotation policies or using drivers that automatically manage log size prevents storage issues. Lastly, actively monitoring your logs helps you detect issues, security threats, or performance bottlenecks promptly.

Q 5. How can I troubleshoot common logging issues in Docker?

When encountering logging issues in Docker, start by reviewing your chosen logging driver and its configuration settings. Ensure that log files are being written to the correct location and that permissions are properly configured. If you're utilizing a remote logging driver, confirm that network connectivity is established.

In some cases, simply restarting the container or the Docker daemon can resolve the issue. If problems persist, refer to the documentation for your specific logging driver or seek assistance from the Docker community forums.

Print Page