Logging Configuration

You have an ability to set up the logging process for collecting and recording access to the Web Service, requests and results of execution, etc.

There are three types of logs available in the scope of the Document Reader Web Service:

Access Logs — logs of accessing the Document Reader Web Service via HTTP
Application Logs — logs of operations during the application execution
Processing Results Logs — logs of requests and processing results in JSON format
Processing Error Logs - logs of the Document Reader Core errors

For better log files readability, you can label the /api/process requests, related to the same context, by adding the X-RequestID header. This header value will be included in the log file for all the labeled requests. It allows the client and server to correlate each HTTP request.

See the example request below.

curl --location --request POST '{url}/api/process' \
--header 'Content-Type: application/json' \
--header 'X-RequestID: b45ef1ac-95a7-12e4-b91f-53b189c5e8b5' \
--data-raw '....'

Access Logs

Access logs represent the logs of the accessing the Document Reader Web Service via HTTP.

To set up the access logs during the Web Service startup, use:

config.yaml

service:
  webServer:
    logging:
      level: "INFO"
      formatter: "text"

      access:
        enabled: true
        console: true
        path: "logs/docreader-access.log"
        format: '%(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s "%(f)s" "%(a)s"'

Parameter	Type	Default	Description
`level`	string	`"INFO"`	Logging level (severity). Defines the events that are recorded by their urgency. Available values: `"ERROR"` — to log only the issues, preventing the service from proper functioning `"WARN"` — to log errors and issues that may disturb the execution process, but not necessarily represent the failure `"INFO"` — to log errors, warnings, and any events that are crucial from the features standpoint `"DEBUG"` — to log errors, warnings, info, and additional verbose diagnostic data
`formatter`	string	`"text"`	Logs format. Possible values: `"text"` or `"json"`. (Some log collectors require logs in JSON format.)
`enabled`	boolean	`false`	Whether to save access logs to a file. Writing logs into a file, instead of stdout, may consume a significant amount of space (up to several dozen of Mbs). Consider enabling them wisely. For the access and applications log files, a day-based rotation occurs every midnight UTC. The service keeps the last 30 days of log files.
`console`	boolean	`true`	Whether to print access logs to a console.
`path`	string	`"logs/access/docreader-access.log"`	If `logging` → `access` → `enabled`, specifies the custom file path to save access logs.
`format`	string	`'%(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s "%(f)s" "%(a)s"'`	Customizable logs representation in accordance with Gunicorn access_log_format settings.

Administration instructions for managing the logging process:

DockerLinux

The Access logs are effectively the gunicorn server Access log file.

To enable the Document Reader Access logs, set the YAML parameter service → webServer → logging → access → enabled to true.

You can customize the access log file location via the service → webServer → logging → access → path YAML parameter.

To get access to the Access log file from the host machine:

Create a folder for logs and ensure the group is root and group members have the write & execute permissions over the created folder.
Mount the host folder to the application log folder inside the container:

mkdir ~/docreader-logs
chmod g+wx ~/docreader-logs
sudo chgrp root ~/docreader-logs

docker run -p host_port:8080 -v host_path_to_license_folder/regula.license:/app/extBin/unix/regula.license -v ~/docreader-logs:/app/logs -e regulaforensics/docreader

The Access logs are effectively the gunicorn server Access log file.

To enable the Document Reader Access logs, set the environment variable service → webServer → logging → access → enabled to true at the /opt/regula/document-reader-webapi/config.yaml file.

You сan customize the access log file location via the service → webServer → logging → access → path YAML parameter.

Note that the application should have the write permission on the folder where logs will be stored.

Application Logs

Application Logs are the logs of application execution, including errors and debug messages.

To set up the application logs during the Web Service startup, use:

config.yaml

service:
  webServer:
    logging:
      level: "INFO"
      formatter: "text"

      app:
        enabled: false
        console: true
        path: "logs/docreader-app.log"

Parameter	Type	Default	Description
`level`	string	`"INFO"`	Logging level (severity). Defines the events that are recorded by their urgency. Available values: `"ERROR"` — to log only the issues, preventing the service from proper functioning `"WARN"` — to log errors and issues that may disturb the execution process, but not necessarily represent the failure `"INFO"` — to log errors, warnings, and any events that are crucial from the features standpoint `"DEBUG"` — to log errors, warnings, info, and additional verbose diagnostic data
`formatter`	string	`"text"`	Logs format. Possible values: `"text"` or `"json"`. (Some log collectors require logs in JSON format.)
`enabled`	boolean	`false`	Whether to save application logs to a file. Writing logs into a file, instead of stdout, may consume a significant amount of space (up to several dozen of Mbs). Consider enabling them wisely. For the access and applications log files, a day-based rotation occurs every midnight UTC. The service keeps the last 30 days of log files.
`console`	boolean	`true`	Whether to print application logs to a console.
`path`	string	`"logs/app/docreader-app.log"`	If `logging.app.enabled`, specifies the custom file path to save access logs.

Administration instructions for managing the logging process:

DockerLinux

The Application logs are effectively the gunicorn server Application log file.

To enable the Document Reader Application logs, set the YAML parameter variable service → webServer → logging → app → enabled to true.

You can customize the application log file location via the service → webServer → logging → app → path YAML parameter.

To get access to the Application log file from the host machine:

Create a folder for logs and ensure the group is root and group members have the write & execute permissions over the created folder.
Mount the host folder to the application log folder inside the container:

mkdir ~/docreader-logs
chmod g+wx ~/docreader-logs
sudo chgrp root ~/docreader-logs

docker run -p host_port:8080 -v host_path_to_license_folder/regula.license:/app/extBin/unix/regula.license -v ~/docreader-logs:/app/logs -e regulaforensics/docreader

The Application logs are effectively the gunicorn server Application log file.

To enable the Document Reader Application logs, set the YAML parameter service → webServer → logging → app → enabled to true at the /opt/regula/document-reader-webapi/config.yaml file.

You can customize the application log file location via the service → webServer → logging → app → path YAML parameter.

Note that the application should have the write permission on the folder where logs will be stored.

Processing Results Logs

The Document Processing Results Logs keep the input requests and the execution results in JSON format.

To set up the processing results logs during the Web Service startup, use:

config.yaml

service:
  processing:
    results:
      audit: true
      saveResult: true
      location:
        bucket: "docreader-processing"
        container: "docreader-processing"
        folder: "output"
        prefix: "processing"

Parameter	Type	Default	Description
`audit`	boolean	`false`	Whether to replicate the transactions' data to the separate database table. If the `saveResult` property is enabled, both requests and results are saved, otherwise only requests are logged. To disable the saving of requests, remove the entire `processing` section, including all elements within it.
`saveResult`	boolean	`false`	Whether to save the results of the requests' processing. If disabled, only the requests are saved.
`bucket`	string	`docreader-processing`	Location (bucket) where to save processing results in the case of the Google Cloud Storage or Amazon S3.
`container`	string	`docreader-processing`	Location (container) where to save processing results in the case of the Microsoft Azure.
`folder`	string	`output`	Location (folder) where to save processing results in the case of the local file system. May represent an absolute path.
`prefix`	string	—	Single prefix line for all defined locations. When set, gives ability to keep the stored processing results in a more organized way regardless of the selected storage type.

Administration instructions for managing the logging process:

DockerLinux

The Document Reader Processing results logs are the artifacts of the processed request.

To enable the processing results logs, use any parameter of the processing section.

To enable the recording of both requests and results, set the YAML parameter service → processing → results → saveResult → enabled to true.

You can customize the processing results log folder path via the service → processing → results → location YAML parameter.

To get access to the processing results logs from the host machine:

Create a folder for logs and ensure the group is root and group members have the write & execute permissions over the created folder.
Mount the host folder to the application log folder inside the container:

mkdir ~/docreader-logs
chmod g+wx ~/docreader-logs
sudo chgrp root ~/docreader-logs

docker run -p host_port:8080 -v host_path_to_license_folder/regula.license:/app/extBin/unix/regula.license -v ~/docreader-logs:/app/logs -e regulaforensics/docreader

The Document Reader Processing results logs are the artifacts of the processed request.

To enable the processing results logs, use any parameter of the processing section.

To enable the recording of both requests and results, set the YAML parameter service → processing → results → saveResult → enabled to true.

You can customize the processing results log folder path via the service → processing → results → location YAML parameter.

Note that the application should have write permision on the folder where logs will be stored.

Processing Error Logs

During the document processing, errors may appers that Document Reader SDK returns.

To set up the error logging during the Web Service startup, use:

config.yaml

service:
  sdkErrorLog:
    enabled: true
    location:
      bucket: "docreader-processing"
      container: "docreader-processing"
      folder: "processing"
      prefix: "sdk"

Parameter	Type	Default	Description
`enabled`	boolean	`false`	Whether to enable error logging.
`bucket`	string	`docreader-processing`	Location (bucket) where to save error logs in the case of the Google Cloud Storage or Amazon S3.
`container`	string	`docreader-processing`	Location (container) where to save error logs in the case of the Microsoft Azure.
`folder`	string	`processing`	Location (folder) where to save error logs in the case of the local file system. May represent an absolute path.
`prefix`	string	—	Single prefix line for all defined locations. When set, gives ability to keep the stored error logs in a more organized way regardless of the selected storage type.

Next Steps

Regula Document Reader Web API