Magento Logs: A Better Way To Understand Your Application And Take Better Decisions

1. How to View Magento 2 Logs
   1. Understanding Magento Log
   2. View Magento Logs
2. Types Of Magento Logs and How to Study them
   1. Magento PHP Logs
   2. Magento Apache Logs
   3. Magento NGINX Logs
3. Improve Store Performance with Magento Store Logs
4. Magento Log Monitoring Through Cloudways
   1. Magento PHP Logs
   2. Magento Apache Logs
   3. Magento NGINX Logs
   4. Magento Apache Error Logs
5. Conclusion

How to View Magento 2 Logs

Magento logs are really helpful in terms of troubleshooting the problems related to Magento commerce, as these Magento logs maintain all the records of Magento activity. But before getting into the details of the Magento logs, let’s first see what these Magento 2 logs are and the flow of information in these logs.

Understanding Magento Log

Let’s start with how the process works.

When a visitor generates a request, it is initially handled by the server (Apache), which communicates to the app (Magento/PHP). Magento then sends the query to the database (MySQL). During this flow, there are chances that error might occur. In such cases, the Magento log maintains a complete log of all the activities which help in error identification and resolution.

By default, Magento does not send error & exceptions messages and thus developers do not get notified if anything happens. Magento maintains a log of all errors in the Magento 2 error log. I recommended that developers and store owners continuously check Magento 2 logs and take timely action before the damage gets done.

View Magento Logs

• Log Directories: By default, all Magento logs are maintained in var/log directory which can be accessed via SSH/CLI or through FTP

Via FTP structure

/applications/[application folder]/public_html/var/log

To view a specific log, download it on a local drive.

Via CLI

Use the following command to access the Magento 2 log file:

cd public_html/var/log

Once in the folder, use the following command to view the Magento log file:

ls -l 

Project web UI: For Magento 1, you need to enable the Magento log from the Magento admin panel and if you would like to see the log in Magento 2 admin panel too then we have covered that in our detail on How To Debug A Magento 2 Store With Ease.

Types Of Magento Logs and How to Study them

Magento Access log contains a record of client requests. When a client generates any request, Apache records several parameters associated with the request. The list includes the IP address of the client, the parameters of the client request, the HTTP status code, and the timestamp of the request.

Magento PHP Logs

PHP logs contain detailed PHP request information such as files/routes, CPU and memory usage.

Magento 2 is a full-stack ecommerce platform that uses Monolog logging framework as its main logging functionality. This makes Magento comply with the PSR-3 logging implementation standards and namespaces.

Magento PHP builtin error logs are usually used to display information related to:

• Failure of a core function or code parsing (these errors are generally generated by the PHP engine)
• PHP custom errors which are triggered in the application, such errors are usually generated when wrong or incomplete information is provided
• Activities that are performed in the Magento application which may be helpful in the analysis of the problems later on.

When log_errors are enabled but are not properly defined, these Magento errors are logged in the web server error log which is maintained in the Apache and NGINX logs (that I would discuss later).

Magento PHP Logging Functions

Magento utilizes several core PHP error functions to report errors.

error_log()

This function is used to send a message to the recently configured error log. The most common use is:

error_log(‘Your message’);

An Apache server will display the error like this

[20-Jul-2020 18:20:02 Europe/Malta] Your message

If you wish, you can send the error to a different file. But this is not recommended as it would be confusing for all:

error_log(‘Invalid input on user login’, 3, ‘/var/www/example.com/log/error.log’);

trigger_error()

This function is a great way to deal with custom errors. In fact, the recommended practice is to use trigger_error() with a descriptive message at every point where the code could break. This function displays two parameters, the error message, and an error level.

syslog()

Another way of managing the error log is to send it directly to the system using the syslog() function. It takes the error level as the first parameter (LOG_INFO, LOG_DEBUG, and many other parameters), with the second parameter being the actual error message.

Example (using openlog() for the connection):

Aug 4 11:42:10 vaprobash php: User #5 is logged from three different locations.

Magento PHP Log Format

When logging data, the proper formatting convention should be followed. As such the log should contain essential fields such as date/time, filename, message, etc. This simplifies the analysis process and helps you better understand the data.

PHP Error Log Types

PHP provides different types of Magento error logs that define the severity of error for your Magento store. These error levels define which PHP statement can’t be compiled or parsed, or whether the Magento application could access all the required resources or the errors caused because of typos and related issues.

The PHP error levels are integer values where each error is assigned a constant. These constants make the code easy to read, understand and compatible enough that new error levels could be introduced. Here are some common error levels:

• E_ERROR – These are fatal run-time errors that stop code execution. A good example would be calling a function that does not exist.
• E_WARNING – These errors do not stop the execution of code, but due to these errors the code might not work as expected.
• E_PARSE – These are compile-time parse errors, like missing parentheses or semicolons.
• E_NOTICE – This indicates possible bugs, such as using a variable that doesn’t exist in an expression. This means that either the variable is a typo or is not assigned the right way.

Here is a detailed guide that describes PHP logs in detail.

Magento Apache Logs

Magento Apache error logs are further categorized into the following two subsections.

Magento Apache Error Logs

This is the place where Apache maintains a record of any errors that are encountered in processing requests received by the server. In case of a problem with the server, this contains the details of what went wrong (and, often how to fix it).

The format of Magento Apache logs is often in the following format:

[Sat Oct 10 11:45:39.902022 2011] [core:error] [pid 35708:tid 4328636416] [client 72.15.99.184] File does not exist: /usr/local/apache2/htdocs/favicon.ico

Reading the Error Entry

• The first item in the log entry is the date and time of the message: [Sat Oct 10 11:45:39.902022 2011]
• The next is the module producing the message and the severity level of that message: [core:error]
• This is followed by the process ID and, if appropriate, the thread ID, of the process that experienced the condition: [pid 35708:tid 4328636416]
• Next, is the client address that generated the request: [client 72.15.99.184] 
• And finally, the detailed error message: File does not exist: /usr/local/apache2/htdocs/favicon.ico

Magento Apache Access Logs

The server access log records all requests processed by the server. Storing the log is only the start of log management. The next step is to analyze this information to come up with an actionable correction course.

The format of the access log is highly configurable. The format is specified using a format string that looks much like a C-style printf(1) format string.

Common Log Format

A typical configuration for the access log might look as follows.

LogFormat “%h %l %u %t ”%r” %>s %b” common

The above configuration will record log entries in a format known as the Common Log Format (CLF). This standard format can be produced by many different web servers and read by many log analysis programs. The log file entries produced in CLF will look something like this:

127.0.0.1 – frank [10/Oct/2000:13:55:36 -0700] “GET /apache_pb.gif HTTP/1.0” 200 2326

Each part of this log entry is described below.

127.0.0.1 (%h)

This is the IP address of the client (remote host) which made the request to the server. It is best to use a log post-processor such as logresolve to determine the hostnames. These IP addresses are not always the IP of the user, particularly in cases when the user is behind a proxy server.

– (%l)

The “hyphen” in the output indicates that the requested piece of information is not available.

frank (%u)

This is the userid of the person requesting the document as determined by HTTP authentication.

[10/Oct/2000:13:55:36 -0700] (%t)

The time that the request was received. The format is:

[day/month/year:hour:minute:second zone]

“GET /apache_pb.gif HTTP/1.0″ (”%r”)

The request line from the client is given in double quotes. The request line contains a great deal of useful information. First, the method used by the client is GET. Second, the client requested the resource /apache_pb.gif, and third, the client used the protocol HTTP/1.0.

200 (%>s)

This is the status code that the server sends back to the client. This information is very important because it reveals whether the request resulted in a successful response (codes beginning in 2), a redirection (codes beginning in 3), an error caused by the client (codes beginning in 4), or an error in the server (codes beginning in 5).

2326 (%b)

The last part indicates the size of the object returned to the client, not including the response headers. If no content was returned to the client, this value will be “-“. To log “0” for no content, use %B instead.

Combined Log Format

Another commonly used format string is called the Combined Log Format. It can be used as follows.

LogFormat “%h %l %u %t ”%r” %>s %b ”%{Referer}i” ”%{User-agent}i”” combined

CustomLog log/access_log combined

This format is exactly the same as the Common Log Format, with the addition of two more fields. Each of the additional fields uses the percent-directive %{header}i, where the header can be any HTTP request header. The access log under this format will look like:

127.0.0.1 – frank [10/Oct/2000:13:55:36 -0700] “GET /apache_pb.gif HTTP/1.0” 200 2326 “http://www.example.com/start.html” “Mozilla/4.08 [en] (Win98; I ;Nav)”

Magento NGINX Logs

NGINX is a popular proxy and web server that is commonly used to speed up the response of the website.

NGINX Error Logs

When NGINX faces any issue, it will be recorded in the NGINX error log. These errors may appear due to issues in the configuration file. So if NGINX is unable to start or abruptly stops, you should check the NGINX error logs. Chances are that you may also find a few warnings in the error log. However, this does not mean that a problem has occurred; rather, this is an alert that this may point to a serious issue in the near future.

Example: error_log /var/log/nginx/error.log warn;

• Error_log: Defines the directive (error_log)
• Log_file: Defines the absolute path of your log file ( /var/log/nginx/error.log)
• Log_level: Defines the severity level for which you want error messages to be logged (warn)

When defining the severity level, NGINX will log all errors that are equal to or above the defined level.

NGINX Error Log Severity Levels

Several severity levels can be defined in the error_log directive. The following is a list of all severity levels ranging from low to high.

• debug – Useful debugging information to help determine where the problem lies.
• info – Informational messages that might add value for the developers.
• notice – Something normal happened that is worth noting.
• warn – Something unexpected happened, however, it is not a cause for concern.
• error – Something was unsuccessful.
• crit – There is a serious problem that needs your attention.
• alert – Prompt action is required.
• emerg – The system is in an unusable state and requires immediate attention.

What is an NGINX Access Log?

The NGINX access log is similar to the error log in that it logs information. However, NGINX maintains the log of every request made by the client. The access_log directive uses the following syntax:

access_log log_file log_format;

• Access_log: Defines the directive
• Log_file: Defines the location of the access.log file,
• Log_format: Can be defined using variables (default format is combined).

access_log /var/log/nginx/access.log log_file combined;

Improve Store Performance with Magento Store Logs

Like any other ecommerce platform, Magento performance is also very much affected by the size of the database. All Magento logs are stored in the store database. To improve website performance, these logs need to be cleared on a regular basis. This not only boosts performance but also assists in reducing the latency of query execution. Magento logs can occupy GBs of data so regular cleaning saves a lot of resources. Here’s how you can clear the logs and understand the impact on performance.

Magento Log Monitoring Through Cloudways

Cloudways offers a great Monitoring feature that includes a log component that users can leverage to identify problems in the application hosted on their managed Magento servers. The logs are available on the Cloudways Platform but there is no need to download the logs via FTP or view them through SSH access.

Here is the easy way to Monitor your Magento Logs:

Magento Apache Logs

All Magento Apache logs can be further filtered to help discover specific issues recorded in the Magento Apache logs.

Magento NGINX Logs

All Magento NGINX logs are displayed here without any need to get into your logs file or database. These logs can be further filtered to help find specific issues in Magento NGINX logs.

Magento PHP Logs

These logs can be further filtered to help us to find specific Magento PHP logs only.

Magento Apache Error Logs

Magento Apache error related logs can be further filtered to find specific Magento Apache error logs entries.

For further details about application log management, read the Cloudways log management guide.

Conclusion

Magento logs are very helpful to identify the activities related to the Magento store. These activities do also create Magento error logs that are displayed in their respective categories. If the logs are dynamic then it would be displayed in Magento Apache logs, if it’s related to static assets then in Magento NGINX logs, if it’s related to PHP detail information like files/routes, CPU and memory consumption then they are displayed in Magento PHP logs. 

These logs should not be ignored and are essential for developers & store owners to monitor the store and find the right solution. If there is a need then a custom log can also be created for better management. If these logs are properly managed and timely decisions are made then you can save your Magento store from multiple major issues.

Keep reading the article at The Official Cloudways Blog. The article was originally written by Syed Muneeb Ul Hasan on 2020-03-17 09:56:19.

The article was hand-picked and curated for you by the Editorial Team of WP Archives.