Real time monitoring with simple counters

Counters

Preliminaries

This note describes the counter module, how to enable it, how to retrieve real-time server statistics, its limitations, and security implications.

Webmasters often need to check the health of the server. Is it still up and running? Are there unexpected response codes being returned to users? Is the web site traffic unusually high or low? Is it being subjected to new hacking attempts?

The counter module can provide answers to these types of questions.

Each time the server receives a request and returns with a response, it increments an in-memory data structure that holds a set of counters. These counts can then be retrieved by the webmaster using a simple HTTP GET request to the "counters" page (a self-declared virtual resource). The response can be formatted as XML, JSON, or HTML.

Five types of header values are tracked in this way:

  1. method
  2. user-agent
  3. content-type
  4. content-length
  5. status-code

Each type (except content-length) is counted by value, so for example, the content-type header keeps track of how many requests were fulfilled for each distinct mime-type: text/html, application/js, image/x-icon, etc. Similarly, the status-code counter keeps track of how many responses were 200, 303, 404, etc. On the other hand, the content-length value contains the number of bytes sent in the response payload; this value is used to track the total bytes transferred for all requests.

When more than one hostname has been configured, separate counters will be used to track each of them.

Counters are initialized to zero when the server is started, and continue to increment until the server stops. Counters are not automatically saved to disk, so if that is desired, the webmaster should issue a request to the "counters" page via periodic cron job, or just before stopping the server.

The counter module provides simple monitoring capabilities only. The server's Logging module provides a much more sophisticated approach to traffic analysis.

Configuration

The counter module is not enabled by default. To use it, add a counters entry to the modules section and set its value to on.

As alluded to before, the "counters" page is a virtual resource. To make it visible, add an entry to the plugins/router section using a path-pattern of your choice, for example `/rwserve/counters`; declare a *methods=GET attribute; and declare a *plugin='rwserve-counters' attribute.

This configuration will enable the counters module to respond to your requests for current real-time statistics. If your website hostname is example.com, you can get XML, JSON, and HTML "counter" pages using these URLs:

https://example.com/rwserve/counters.xml
https://example.com/rwserve/counters.json
https://example.com/rwserve/counters.html

Restricting access to the counters page

Since the "counters" page is accessed using a standard GET request, it is visible to the general public by default. While there is nothing especially sensitive in the response or of concern to the website's security, it is understandable that many webmasters will want to restrict access to it. The best way to do that is using the RBAC module. Follow these steps:

  1. Enable the rbac module.
  2. Add an entry to the rbac/resources subsection having the same path-pattern that is defined in the plugins/router section; declare a *methods=GET attribute; and declare a *roles=devops attribute.
  3. Login to the website using the devops userid.

The third example in the Cookbook section below demonstrates this type of configuration.

EBNF

SP ::= U+20
CR ::= U+0D
ASTERISK ::= U+2A
APOSTROPHE ::= U+27
EQUALS-SIGN ::= U+3D
QUESTION-MARK ::= U+3F
LEFT-CURLY-BRACKET ::= U+7B
RIGHT-CURLY-BRACKET ::= U+7D
counters-module ::= 'counters' | ('on' | 'off')
rbac-module ::= 'counters' | ('on' | 'off')
modules-section ::= 'modules' SP LEFT-CURLY-BRACKET CR
counters-module
router-module
rbac-module
RIGHT-CURLY-BRACKET CR

Cookbook

Example 1: Counters off
server {
modules {
counters off
}
}
Example 2: Counters on, unsecured access
server {
modules {
counters on
}
plugins {
router {
`/rwserve/counters` *methods=GET *plugin='rwserve-counters'
}
}
}
Example 3: Counters on, devops access only
server {
modules {
counters on
rbac on
}
plugins {
router {
`/rwserve/counters` *methods=GET *plugin='rwserve-counters'
}
}
rbac {
roles `/etc/rwserve/roles` // the file created by the 'addrole' CLI utility
cipher-secret C#9fB$2gD@5zR*7e // secret used to encrypt the 'rw-roles' cookie
max-idle 1800 // number of seconds of inactivity before credentials expire
resources {
`/rwserve/counters` *methods=GET *role=devops
}
}
}

Review

Key points to remember:

  • The counters module provides quick real-time insight into the status of website traffic.
  • The logging module is an alternative that is better suited to in-depth statistical analysis.

Real time monitoring with simple counters