Processing requests for blue-phrase documents
This note describes how the server handles requests for documents with content type text/blue.
Compilation of blue-phrase documents ahead of time can be done using many of Read Write Tools' specialized tools, but compilation just-in-time is also possible with the server's blue-processor.
Just-in-time compilation is advantageous when documents are actively undergoing revision, especially those documents that are assembled from shared external files. It is a straightforward process, and is simple to configure and manage.
Dynamically serving blue-phrase documents this way also makes it possible to take advantage of HTTP/2 speculative push protocol.
The server handles requests for documents with content type
text/blue using these steps:
- the requested document is dynamically parsed
- the source is compiled into HTML
- the compiled HTML is cached
- the dependency map, resource map, and link map are cached
- dependent resources are sent using HTTP/2 speculative push
Blue-phrase encoding has three types of pragmas that use external files to assemble the final document:
!use. When authors create shared external files and employ them in their documents using these pragmas, they are in effect building a custom authoring system.
When any of these shared files change, all of the documents that make use of them must be recompiled. The server's blue-processor recognizes this situation through the use of dependency maps that it builds and maintains. Because the server handles this automatically, authors can modify-in-place any base document or external file without explicitly instructing the server to refresh the authoring system. Fresh, up-to-date HTML will be compiled and served when any dependency changes.
When a blue-phrase source file is compiled, map files are built that contain references to files used by the base document. Three types of resources are catalogued: blue-phrase dependency pragmas, HTML resource files, and HTML links. The cataloged references are saved to the dynamic cache, together with the compiled document itself, in files with the extension
Dependency maps are used to determine if a subsequent request can be pulled from the dynamic cache immediately or if it needs to be recompiled first.
Resource maps are used to by the automatic preload detection (APD) algorithm to determine which HTML resource files are candidates for HTTP/2 speculative push.
Link maps may be used by external utilities to check for broken external links.
The dynamic cache stores files created by the blue-processor. The cache comprises the dynamically compiled HTML documents and their associated linkmap files. Structurally, the cache has the same directory hierarchy as the public document area. The location of the cache is determined by the server's configuration. It must be writable by the process running the server.
When a document is requested via HTTP, the server determines if it has been changed by the author since its last access, and compiles it just-in-time, if needed. The compiled document is immediately cached, using the server's
dynamic-cache directory, any time a compilation occurs. Subsequent requests — by any requestor — will pull the compiled HTML document from the dynamic cache and send it in the HTTP response using the normal server processing steps.
Blue-phrase encoding is flexible enough to be used as a source document for many types of markup languages and data structures. Compiler options determine the rules to follow when compiling source documents and emitting output files. These options can be set by adding query-string variables to the HTTP request.
Options are available for:
- setting the source vocabulary
- the allowed shorthand notation
- the emitted output indentation style
- the file content-type to emit
- the HTML usage to target
- whether graynotes should appear in the output
- whether to honor doppelmarks
|Query String||Allowed Values|
† Default value
Speculative Push Processing
Each linkmap in the dynamic cache contains a list of resource files referenced by the base document: these are candidates for HTTP/2 speculative push (SP). The blue-processor assembles this list when parsing the original BLUE source document. The following HTML tags are examined and considered for this:
|HTML tag||Sourceref attribute|
If the sourceref attribute has as URL with the same domain as the base document, it remains as a candidate for SP. If not, it is disqualified from further consideration.
Just prior to sending the base document to the requestor — while the HTTP/2 duplex stream is still open — the server examines the push-priority configuration, testing the candidates against the configured resource path-patterns, applying a stream weight to the candidate, and initiating an SP request. Candidates that do not match a resource path-pattern in the
push-priority configuration, have a weight of zero and are not pushed.
Overriding the speculative push defaults
The SP protocol is only honored when the HTTP/2 stream has its
pushAllowed flag set by the browser to
The server should not send resources using SP for requests that are made by crawlers if they are not ready to make use of it. User agents are supposed to set the stream's
pushAllowed flag to
true only if they can handle it. The server honors that flag and will refuse to push anything when it is
If desired, the server can be configured to make its own determination on this matter, and override any streams whose
pushAllowed flag is
true. This is done using pattern matching against the
user-agent request header. Refer to the separate notes regarding the User Agent and Push Priority handlers.
Configuring the server to process
text/blue files is accomplished with three entries:
dynamic-cacheentry in the
hostsection. This defines the absolute path where compiled output and linkmaps are stored.
blueentry in the
content-typessection, to associate the
bluefilename extension with the MIME-type
`*.blue`entry in the
/plugin/routersection, where the resource path-pattern is setup to shunt requests to the built-in blue-processor plugin.
In order for the server to take advantage of the SP protocol with blue-phrase documents, the server needs to be configured with these additional entries:
push-priorityentry in the
modulessection. This is required in order to enable the push protocol handler, which by default is disabled.
- A set of
pushentries in the
push-prioritysection, where each entry comprises a resource path-pattern, and a weight. Weights are numbers between 1 (lowest priority) and 255 (highest priority) that prioritize the relative importance of files.
The following information headers may be issued when the blue-processor is used:
| ||Only |
| ||The compiled HTML and linkmap were added to the dynamic cache.|
| ||The HTML document was recompiled because the BLUE source was newer.|
| ||The HTML document was recompiled because one of the BLUE file dependencies was newer.|
| ||The dynamic cache was used because the BLUE file and its dependencies are up-to-date.|
| ||A query-string |
|file-system-chars||::=||(ALPHA | DIGIT | †)*|
|dynamic-cache-entry||::=||'dynamic-cache' SP GRAVE-ACCENT file-system-chars* GRAVE-ACCENT CR|
|host-section||::=||'host' SP LEFT-CURLY-BRACKET CR|
dynamic-cache-entry | ††
† Legal file system characters vary by platform.
host-section entries omitted for brevity.
Example 1: Configuring the dynamic-cache
Example 2: Configuring the
Example 3: Routing
text/blue to the blue processor
`*.blue` *methods=GET,HEAD *plugin=rwserve-blue
Example 4: Enabling the Push Priority module
Example 5: Typical Push Priority settings
push `*/css/*.css` *weight=80
push `*/js/*.js` *weight=80
push `*/img/*.gif` *weight=20
push `*/img/*.png` *weight=20
push `*/img/*.jpg` *weight=20
Example 6: Forcing Googlebot crawler NOPUSH
Googlebot *pattern='Googlebot' *groups=nopush
Example 7: GET requests emitting other content-types
<a href='sample.blue?option.html-target=epub'>HTML for EPUBS</a>
<a href='sample.blue?option.vocabulary=unchecked&option.emit=gfmd'>Github Flavored Markdown</a>
<a href='sample.blue?option.vocabulary=unchecked&option.emit=text'>Plain text</a>
Key points to remember:
- The blue-processor compiles BLUE source documents just-in-time.
- The dynamic cache retains freshly compiled output and linkmaps.
- Linkmaps provide automatic support for:
- Dynamic cache dependency checking.
- Speculative push protocol candidates.