Logging, one of the most basic and powerful tool in software development, predates modern networking. It stores events occurring in a system while executing a process. Initially databases used logging to write changes in the data to make sure data integrity. Now it is used everywhere. From distributed applications to large monoliths, every system make use of log for different purposes. Example: an application uses logging to track flow of a process. In a more complex scenario, a distributed environment can use logging to trace a request’s journey among multiple services and databases.

Definition:

Logging is a practice that allows developers create a logical map of how a software executes its processes. Log is the record that stores the information of individual event. A log can record a single event in the form of debug, information, warning, error or critical record.

Logging is not same as printing strings in the console. At first it may seem printing string and logging record are same but they are very different from each other. A log can hold a lot more information that a simple print statement. Take an example of python,

print("list_products controller started executing with params limit: 10, offset: 0 search: rtx 5080")

# This will print
# list_products controller started executing with params limit: 10, offset: 0 search: rtx 5080

import logging

logger = logging.getLogger()

logging.basicConfig(
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
    level=logging.INFO
)

logger.info("list_products controller started executing with params limit: 10, offset: 0 search: rtx 5080")

# This will log
# 2025-09-17 13:26:49,773 - root - INFO - list_products controller started executing with params limit: 10, offset: 0 search: rtx 5080

This is just the beginning, logging libraries provide powerful features like writing logs in a file, rotating logs, log handlers, log formatters and many more.

Levels of Logging:

Universally there are five levels of logs, DEBUG, INFO, WARNING, ERROR and CRITICAL. DEBUG is used for recording detailed information of operation, typically used in development and test environments for debugging and troubleshooting. DEBUG level logs are used in production level. INFO is general messages that tracks the operation and application flow. WARNING is used to record potential issue. It may not be fatal but something to investigate about. ERROR is used to record harmful issue in the application. May be to record controlled errors that developers anticipates. It will not bring down the entire application but definitely affecting applications workflow. Example: application needs to call a third party API but it has failed. Developer already anticipated this already and add an error handler for this situation. CRITICAL log record stores information about errors that are unexpected. Example: application needs to access a specific environment variable to run or complete a process but it is missing resulting in application crash or process abortion.

Why You Need to Understand Logging?

Logging is not just storing millions of information so that developers can debug application. There are many crucial use cases of logging that makes a software resilient and stable. Example: write ahead logging and transactional logging plays a critical part in data integrity in database. These logs serves as a source of truth what data was inserted into the database and what changes were applied to it. Logging is heavily used for generating alerts. Information from log records can be aggregated to generate metrics. Metrics are then used to generate alerts that play a very crucial role in software maintainability. Modern applications are not centralized. Each application is trying to make best use of decentralized cloud native applications. An application can be distributed among many servers and functions in a single cloud provider or multiple cloud providers. Each pieces are continuously communicating with each other through messaging or event streaming platforms. In such scenario a centralized logging system that can track the life-cycle of a request becomes critical to the business.

Example: Ride Booking App

Let’s take the example of ride booking. User wants to go from place A to place B. User enters the address and clicks on search.

1. Initial Request & Correlation ID

• The mobile app makes an API call to the Ride Service. The API Gateway or the Ride Service itself generates a unique correlation ID (e.g., a UUID). This ID is the golden thread that links all log entries for this specific request. It should be included in the request headers and propagated to all downstream services.

• The Ride Service logs a single, concise entry at the INFO level to mark the beginning of the process. This entry should include the correlation ID and key data points.

  • INFO log: "msg=Ride request received", correlation_id="[ID]", user_id="[ID]", from_address="[address]", to_address="[address]"

2. API Calls & Business Logic

• The Ride Service calls the Estimation Service and passes the correlation ID.

• The Estimation Service performs its calculation and, if successful, logs a concise INFO message.

  • INFO log: "msg=Price estimation calculated", correlation_id="[ID]", ride_cost="[cost]"

• The Ride Service calls the Matching Service and passes the correlation ID.

• The Matching Service performs the driver search and logs an INFO message upon completion.

  • INFO log: "msg=Found nearby drivers", correlation_id="[ID]", drivers_found="[count]"

3. Asynchronous Messaging & Confirmation

• The Matching Service publishes a message to a queue. The message payload includes the correlation ID and details of the matched drivers.

• The Notification Service consumes this message. Its logging strategy mirrors the synchronous calls. It logs an INFO message upon receiving and processing the message.

  • INFO log: "msg=Processing driver notifications", correlation_id="[ID]", drivers_notified="[count]"

• When a driver accepts the ride, the mobile app calls the Ride Service to confirm. The Ride Service logs a final INFO message.

  • INFO log: "msg=Ride confirmed by driver", correlation_id="[ID]", driver_id="[ID]"

Above example is not an optimal solution for a distributed system but it paints the picture how each service can track its progress. At any point the if the service encounters any issue, it will log error.

4. Handling Errors

• Effective logging isn't just about success; it's about failure. If any service fails, it should log a WARN or ERROR message with the correlation ID and as much detail as possible, including the exact error message and a stack trace.

• For example, if the Estimation Service fails to connect to its database, it logs an ERROR message.

  • ERROR log: "msg=Failed to connect to database", correlation_id="[ID]", error="[error_message]"

Centralized Log Aggregation System:

These systems allow the developers see all the logs generated by different services at one place. Purpose of these systems is collecting, storing and analyzing the log data. In this series we will learn about Splunk. Splunk can collect data through various method like splunk forwarder, HTTP event collector and through cloud platforms like AWS, GCP or Azure.

There are other popular systems like ELK stack and DataDog.