Skip to content

Logs Concepts

Definition

Logs are textual records of events, actions, or messages generated by software applications, systems, or devices. They provide a chronological account of activities, errors, and diagnostics, offering insights into the behavior and performance of these entities.

Logs are vital for troubleshooting, monitoring, and auditing, helping developers and operators understand what has occurred within an application or system.

Formats

Logs are usually produced in either a human-readable (sometimes freeform) format, or a machine-readable structured format. Both are acceptable for our logs platform, but the more structured the format is, the easier searching and querying the data is later.

Common formats include, but are not limited to:

  • W3C format, used by some web servers
  • JSON format
  • logfmt, commonly used by Go applications

Storing and shipping

Most commonly logs are stored as flat files on disk, usually rotated when they reach a certain size or age. A common location on Unix systems is /var/log. These files must be parsed and shipped.

Other applications may transmit their logs to a remote system for processing. A common protocol is syslog. These applications may configured to send their logs to multiple destinations, or may require a central aggregator.