BLUE PROCESSOR

Preliminaries

This note describes how the server handles requests for documents with content type text/blue.

Compilation of BLUEPHRASE documents ahead of time can be done using many of Read Write Tools' specialized tools, but compilation just-in-time is also possible with the server's blue-processor.

Just-in-time compilation is advantageous when documents are actively undergoing revision, especially those documents that are assembled from shared external files. It is a straightforward process, and is simple to configure and manage.

Dynamically serving BLUEPHRASE documents this way also makes it possible to take advantage of HTTP/2 speculative push protocol.

The server handles requests for documents with content type text/blue using these steps:

• the requested document is dynamically parsed
• the source is compiled into HTML
• the compiled HTML is cached
• the dependency map, resource map, and link map are cached
• dependent resources are sent using HTTP/2 speculative push

Linkmaps

BLUEPHRASE encoding has three types of pragmas that use external files to assemble the final document: !include, !enclosure and !use. When authors create shared external files and employ them in their documents using these pragmas, they are in effect building a custom authoring system.

When any of these shared files change, all of the documents that make use of them must be recompiled. The server's blue-processor recognizes this situation through the use of dependency maps that it builds and maintains. Because the server handles this automatically, authors can modify-in-place any base document or external file without explicitly instructing the server to refresh the authoring system. Fresh, up-to-date HTML will be compiled and served when any dependency changes.

When a BLUEPHRASE source file is compiled, map files are built that contain references to files used by the base document. Three types of resources are catalogued: BLUEPHRASE dependency pragmas, HTML resource files, and HTML links. The cataloged references are saved to the dynamic cache, together with the compiled document itself, in files with the extension .dep, .res and .lnk.

Dependency maps are used to determine if a subsequent request can be pulled from the dynamic cache immediately or if it needs to be recompiled first.

Resource maps are used to by the automatic preload detection (APD) algorithm to determine which HTML resource files are candidates for HTTP/2 speculative push.

Link maps may be used by external utilities to check for broken external links.

Dynamic Cache

The dynamic cache stores files created by the blue-processor. The cache comprises the dynamically compiled HTML documents and their associated linkmap files. Structurally, the cache has the same directory hierarchy as the public document area. The location of the cache is determined by the server's configuration. It must be writable by the process running the server.

When a document is requested via HTTP, the server determines if it has been changed by the author since its last access, and compiles it just-in-time, if needed. The compiled document is immediately cached, using the server's dynamic-cache directory, any time a compilation occurs. Subsequent requests — by any requestor — will pull the compiled HTML document from the dynamic cache and send it in the HTTP response using the normal server processing steps.

Processor options

BLUEPHRASE encoding is flexible enough to be used as a source document for many types of markup languages and data structures. Compiler options determine the rules to follow when compiling source documents and emitting output files. These options can be set by adding query-string variables to the HTTP request.

Options are available for:

1. setting the source vocabulary
2. the allowed shorthand notation
3. the emitted output indentation style
4. the file content-type to emit
5. the HTML usage to target
6. whether graynotes should appear in the output
7. whether to honor doppelmarks

^† Default value

Speculative Push Processing

Each linkmap in the dynamic cache contains a list of resource files referenced by the base document: these are candidates for HTTP/2 speculative push (SP). The blue-processor assembles this list when parsing the original BLUE source document. The following HTML tags are examined and considered for this:

If the sourceref attribute has as URL with the same domain as the base document, it remains as a candidate for SP. If not, it is disqualified from further consideration.

Just prior to sending the base document to the requestor — while the HTTP/2 duplex stream is still open — the server examines the push-priority configuration, testing the candidates against the configured resource path-patterns, applying a stream weight to the candidate, and initiating an SP request. Candidates that do not match a resource path-pattern in the push-priority configuration, have a weight of zero and are not pushed.

Overriding the speculative push defaults

The SP protocol is only honored when the HTTP/2 stream has its pushAllowed flag set by the browser to true.

The server should not send resources using SP for requests that are made by crawlers if they are not ready to make use of it. User agents are supposed to set the stream's pushAllowed flag to true only if they can handle it. The server honors that flag and will refuse to push anything when it is false.

If desired, the server can be configured to make its own determination on this matter, and override any streams whose pushAllowed flag is true. This is done using pattern matching against the user-agent request header. Refer to the separate notes regarding the User Agent and Push Priority handlers.

Configuration

Configuring the server to process text/blue files is accomplished with three entries:

1. A dynamic-cache entry in the host section. This defines the absolute path where compiled output and linkmaps are stored.
2. A blue entry in the content-types section, to associate the blue filename extension with the MIME-type text/blue.
3. A `*.blue` entry in the /plugin/router section, where the resource path-pattern is setup to shunt requests to the built-in blue-processor plugin.

Push Configuration

In order for the server to take advantage of the SP protocol with BLUEPHRASE documents, the server needs to be configured with these additional entries:

4. A push-priority entry in the modules section. This is required in order to enable the push protocol handler, which by default is disabled.
5. A set of push entries in the push-priority section, where each entry comprises a resource path-pattern, and a weight. Weights are numbers between 1 (lowest priority) and 255 (highest priority) that prioritize the relative importance of files.

Information Headers

The following information headers may be issued when the blue-processor is used:

† Legal file system characters vary by platform.

†† Other host-section entries omitted for brevity.

Review

Key points to remember:

• The blue-processor compiles BLUE source documents just-in-time.
• The dynamic cache retains freshly compiled output and linkmaps.
• Linkmaps provide automatic support for:
1. Dynamic cache dependency checking.
2. Speculative push protocol candidates.
• The Push Priority handler maps resource path-patterns to speculative push weights.