Accessing request/response data from a plugin

Work Orders

Preliminaries

This note provides guidance on how to make use of the data structures provided to plugins during the request/response cycle.

request/response WorkOrder object

The processingSequence method receives one argument, a WorkOrder object. It contains the incoming request headers and payload, and references to the outgoing response headers and payload.

Here is what it looks like, in a nutshell:

WorkOrder {        
    // Incoming headers and payload
    requestHeaders
    requestBody
    
    // Processed headers
    _resourcePath
    _queryString
    _parameterMap
    _cookieMap
    _formDataMap
    _multipartFormData
    _maskedReplacementPath
    userAgentCommonName
    userAgentGroups
    
    // Outgoing headers and body
    responseHeaders
    responseBody

    // Metadata
    configHost
    remoteAddress
    processID
    traceID
    pushTraceID
    outgoingType
    doNotFulfill

    // Dynamic plugins / BLUE PROCESSOR
    pushCandidate
    proxyCache
    proxyPath
}

The rest of this note provides details useful to anyone developing a server plugin.

Request Headers

`requestHeaders`

Upon entry to a plugin's processingSequence method , the requestHeaders object will have properties representing each HTTP request header, where the properties are identical to the HTTP request header names and their values are identical to the HTTP request header settings.

For example, a typical request might have an accept-encoding header, which the plugin could access like this:

processingSequence(workOrder) {
    var acceptableEncodings = workOrder.requestHeaders['accept-encoding'];
}

The HTTP/2 standard specifies that all lower-case letters be used for header names, so what was Accept-Encoding in HTTP/1.1 is now accept-encoding in HTTP/2.

No name mangling is performed on incoming or outgoing headers by the server. This means that the header accept-encoding cannot be accessed using code similar to workOrder.requestHeaders.acceptEncoding; instead, it must be accessed using JavaScript's square-bracket notation as workOrder.requestHeaders['accept-encoding'].

Request Body

When the request method is a GET, HEAD, OPTIONS or TRACE the requestBody will be null.

When the request method is POST, PATCH, PUT or DELETE the requestBody will contain the payload data, in the form specified by the request's content-type header.

If the request was compressed, the requestBody will be uncompressed by the server according to the content-encoding rules.

`requestBodyStr`

This property converts the raw requestBody Buffer into a UTF-8 string. Use this when the content-type is text data (this is common).

`requestBodyBytes`

This property is a reference to the raw Buffer holding the request body. Use this when the content-type is binary data (this is not common).

`requestBodyLength`

The length of the raw Buffer, in bytes.

Processed headers

Before the plugin is called, the request's :path and cookie headers are processed, resulting in the following work order properties.

Each of the underscore-prefixed properties is pseudo-private. Use the methods listed later in this note to access them safely.

`_resourcePath`

The _resourcePath contains the portion of the incoming request header's :path without any query-string. For example, if the :path header is . . .

:path: /translate/en/ja?english=hello%20world&japanese=%E3%81%93%E3%82%93%E3%81%AB%E3%81%A1%E3%81%AF%E4%B8%96%E7%95%8C

. . .the _resourcePath would be:

assert(workOrder._resourcePath, '/translate/en/ja');

`_queryString`

The _queryString contains the unparsed query-string portion of the request header's :path. Using the above example, the _queryString would be:

english=hello%20world&japanese=%E3%81%93%E3%82%93%E3%81%AB%E3%81%A1%E3%81%AF%E4%B8%96%E7%95%8C

`_parameterMap`

The _parameterMap is an ES6 map, where the keys and values are strings that have been obtained by decomposing the query-string portion of the request's :path header, and reversing any url-encoding that may have existed. Using the above example, the _parameterMap would have two entries:

assert(_parameterMap.get('english')  == 'hello world');
assert(_parameterMap.get('japanese') == 'こんにちは世界');

`_cookieMap`

The _cookieMap is an ES6 map, where the keys and values are strings that have been obtained by decomposing the cookie request header, and reversing any url-encoding that may have existed. For example, if the cookie header is . . .

cookie: belarusian=%E3%81%93%E3%82%93%E3%81%AB%E3%81%A1%E3%81%AF%E4%B8%96%E7%95%8C; swedish=Hej%20v%C3%A4rlden

. . . then the _cookieMap would have these two entries:

assert(_cookieMap.get('belarusian') == 'Прывітанне Сусвет');
assert(_cookieMap.get('swedish') == 'Hej världen');

`_formDataMap`

The _formDataMap is meaningful only when the request body has content-type application/x-www-form-urlencoded. It is an ES6 map, where the keys and values are strings that have been obtained by decomposing the request body and reversing any url-encoding that may have existed. For example, if the request body is . . .

afrikaans=Hello%20W%C3%AAreld&bosnian=zdravo%20svijet

. . . then the _formDataMap would have these two entries:

assert(_formDataMap.get('afrikaans') == 'Hello Wêreld');
assert(_formDataMap.get('bosnian') == 'zdravo svijet');

`_multipartFormData`

The _multipartFormData is meaningful only when the request body has content-type multipart/form-data. It is an array of MultipartEntry objects, one for each section of a multipart request body.

`_maskedReplacementPath`

The _maskedReplacementPath is an extra property that is present only in requests that have been processed by the Resource Mask handler. It contains the text resulting from the mask replacement algorithm. Using the example from the note on Resource Masks where the request pattern is . . .

/article/{id}/{seo}

. . . if the incoming request :path is . . .

:path: /article/42/the-answer-to-everything

. . . and the replacement string is . . .

/hitchhikers-guide-to-the-galaxy/{id}.html

. . . then the _maskedReplacementPath would be:

assert(workOrder._maskedReplacementPath == '/hitchhikers-guide-to-the-galaxy/42.html');

`userAgentCommonName`

The userAgentCommonName is an extra property that is present only in requests that have been processed by the User Agent module. It contains the text resulting from that pattern matching algorithm, possibly containing the common name of a crawler. For example, if the incoming request header user-agent is . . .

user-agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

. . . and the server configuration file contains this rule . . .

request {
    user-agent {
        Googlebot  *pattern=Googlebot  *groups=noserve
    }

. . . then the userAgentCommonName would be:

assert(workOrder.userAgentCommonName == 'Googlebot');

`userAgentGroups`

The userAgentGroups property is also present only in requests that have been processed by the User Agent module. It contains an Array of the names of groups that the request's userAgentCommonName is a member of. Reusing the previous example, the userAgentGroups would have a single entry in its array:

assert(workOrder.userAgentGroups.length == 1);
assert(workOrder.userAgentGroups[0] == 'noserve');

Methods for accessing processed headers

The properties whose name begins with an underscore should be accessed using the methods described here.

`hasMaskedReplacementPath()`

Returns true if the _maskedReplacementPath property is not null.

`getMaskedReplacementPath()`

Obtains the value of the _maskedReplacementPath property.

`getResourcePath()`

Obtains the value of the _resourcePath property.

`getQueryString()`

Obtains the value of the _queryString property.

`parameterMapSize()`

Obtains the number of entries in the _parameterMap.

`hasParameter(key)`

Returns true if the given key is a valid entry in the _parameterMap.

`getParameter(key)`

Obtains the value associated with the given key in the _parameterMap.

`cookieMapSize()`

Obtains the number of entries in the _cookieMap.

`hasCookie(key)`

Returns true if the given key is a valid entry in the _cookieMap.

`getCookie(key)`

Obtains the value associated with the given key in the _cookieMap.

`formDataMapSize()`

Obtains the number of entries in the _formDataMap.

`hasFormData(key)`

Returns true if the given key is a valid entry in the _formDataMap.

`getFormData(key)`

Obtains the value associated with the given key in the _formDataMap.

`multipartSize()`

Obtains the number of multipart entries in the _multipartFormData array.

`getMultipartEntry(index)`

Obtains a MultipartEntry object from the _multipartFormData array.

`hasUserAgentNopush()`

Returns true if the userAgentGroups array contains a "nopush" entry.

`hasUserAgentNoserve()`

Returns true if the userAgentGroups array contains a "noserve" entry.

Response headers and body

Each request/response work order contains two properties that are used to fulfill the request.

responseHeaders
responseBody

Plugin authors should treat these as private properties, and instead use one of the work order's methods to manipulate their values. This will provide an extra measure of safety to ensure that the overall state of the work order is consistent with the server's expectations.

The work order methods for response headers and body are:

setResponseBody(text)
setEmptyResponseBody()
addStdHeader(headerName, value)
setStatusCode(statusCode)

`setResponseBody() / setEmptyResponseBody()`

It is the responsibility of the plugin author to fulfill the request using either the setResponseBody or setEmptyResponseBody method.

When the request method is GET, the setResponseBody may be provided with a string value or a Buffer value.

When the request method is a HEAD, PUT, DELETE, OPTIONS or TRACE the setEmptyResponseBody should be called instead.

When the request method is POST or PATCH the body is determined by the plugin's author, and may be set or left empty.

`setStdHeader()`

Any standard response headers can be set using the setStdHeader(headerName, headerValue) method. This method will add a key-value pair to the work order's responseHeaders map, which will be used by the server when fulfilling the response.

The headerName argument must follow the naming rules for HTTP/2 headers, that is, it must consist of only the lowercase letters [a-z], the digits [0-9], and the HYPHEN.

The headerValue argument must contain only the ASCII characters from 0x20 through 0x7E.

`setStatusCode()`

The plugin author is responsible for setting the HTTP status code. It should be a JavaScript number, in integer format, between 100 and 599. Use the setStatusCode(statusCode) method to do this.

Metadata

These extra properties are managed by the server and should be treated as read-only.

getALPN()
configHost
remoteAddress
processID
traceID
pushTraceID
outgoingType
doNotFulfill

`getALPN()`

Which HTTP version is being used for the current request/response: either the value http/2 or http/1.1.

`configHost`

The configHost is a reference to the same data structure passed to the plugin's constructor. It is described in the plugins note.

`remoteAddress`

The IP address of the requestor is provided in remoteAddress. It is a string value using IPv4 dotted notation, like the pattern "xxx.xxx.xxx.xxx".

`processID`

The processID is the operating system's process identifier for the cluster worker that is handling the current request/response.

`traceID`

The traceID is a unique number which identifies the current request/response cycle. It is sequentially incremented for each incoming request.

`pushTraceID`

The pushTraceID is a sequentially incremented identifier of each speculative push request originating from the current request/response.

`outgoingType`

The outgoingType is an indicator of which type of outgoing response body has been provided.

`doNotFulfill`

When the statusCode is in the range 300 to 599, the work order's boolean flag doNotFulfill will be set to true, instructing the server that no further processing should be attempted during the current request/response cycle. This means that standard processing for content length, content encoding, etc. will be skipped. This flag is automatically set based on the value passed in any call to setStatusCode().

Dynamic plugins / BLUEPROCESSOR

These extra properties are used by plugins that generate HTML from templated resources. The BLUEPROCESSOR uses this to point to the HTML it generates. Dynamic plugins such as rwserve-interscribe use this to point to dynamically generated and privately cached HTML.

proxyCache
proxyPath
pushCandidate

`proxyCache`

The proxyCache variable contains the absolute directory to a dynamically generated file created from the originally requested file. Use of this allows the response handler to make full use of Etags, cache-control, and content-encoding (compression).

`proxyPath`

The proxyPath variable contains the local path and filename to a dynamically generated file created from the originally requested file. Used in connection with proxyCache.

`setProxyLocation(proxyCache, proxyPath)`

Use this function to instruct the server to perform Etags, cache-control, and content-encoding response processing on the file pointed to by these two variables.

`getProxyLocation()`

Use this function to get the fully qualified proxy location. It is the concatenation of proxyCache and proxyPath.

`pushCandidate`

The pushCandidate property is a boolean value that is set to true if the request may be a candidate for HTTP/2 speculative push.

Review

Key points for plugin authors to remember:

Treat the incoming headers and processed headers as read-only properties.
Set the payload using either the setResponseBody() OR setEmptyResponseBody() method.
Before exiting, call setStatusCode().

In-depth reading

Developing Plugins

Handling requests with application-specific code

This note describes how the server can be customized to handle application-specific needs using plugin modules written in JavaScript.

router, plugins, dynamic handler, custom handler, workOrder, requestHeaders, incomingPayload, remoteAddress, _queryString, _parameterMap, _cookieMap, outgoingPayload, outgoingFilename, doNotFulfill, responseHeaders

router, dynamic routing, static routing, custom handler, path-pattern, rw-router-missing-module, rw-router-syntax-error, rw-router-unknown-error, rw-router-not-forwarded

Customization

Making the server do things your way

These notes provide information on how the server can handle the request/response cycle in ways that go beyond the mimimum requirements of HTTP.

BLUEPHRASE, BLUE PROCESSOR, cross origin, dynamic modules, resource masks, push priority, raw path, router

Smart tech

READ WRITE TOOLS

Templates & Content

READ WRITE HUB

Rediscover HTML

BLUE PHRASE

Accessing request/response data from a plugin

Work Orders

request/response WorkOrder object

Request Headers

requestHeaders

Request Body

requestBodyStr

requestBodyBytes

requestBodyLength

Processed headers

_resourcePath

_queryString

_parameterMap

_cookieMap

_formDataMap

_multipartFormData

_maskedReplacementPath

userAgentCommonName

userAgentGroups

Methods for accessing processed headers

hasMaskedReplacementPath()

getMaskedReplacementPath()

getResourcePath()

getQueryString()

parameterMapSize()

hasParameter(key)

getParameter(key)

cookieMapSize()

hasCookie(key)

getCookie(key)

formDataMapSize()

hasFormData(key)

getFormData(key)

multipartSize()

getMultipartEntry(index)

hasUserAgentNopush()

hasUserAgentNoserve()