ESA Edge Routine HTMLStream API reference - Edge Security Acceleration

Overview

Edge Routine handles frontend scenarios where POPs send special data such as User-Agent headers, geographic locations, and IP addresses. You may need to modify HTML page flow in real time on POPs. Regex-based ad hoc parsers are error-prone and lack stream support. Open-source parsers like parse5 and htmlparser2 consume excessive memory. Edge Routine solves this with a built-in stream-processing parser for modifying HTML on POPs.

Note

The parser is built in Edge Routine and is not based on web standards.

Example

Scenario

To modify all anchor tags <a/> in an HTML page to link them to http://www.taobao.com, use the following Edge Routine code.

Sample code

async function handleRequest(request) {
  // 1. In this example, the HTML page that you want to modify is returned. 
  const response = await fetch("http://www.example.com");
  // 2. Configure the stream processing-based parser to manage HTML content. The parser supports multiple CSS selectors.
  // Specify the method for capturing the syntax and register a callback function for rewriting. 
  const htmlStream = new HTMLStream(
    response.body, // Specify the HTML flow that you want to modify.
    [[
      "a",         // The element selector. This specifies that all the anchor tags are selected. 
      {  
        // Register a callback function. The element callback function can be called in the anchor tags or in the element nodes of the Document Object Model (DOM) API. 
        // In the callback function, you can change the event object (e). 
        element: function(e) {
          // Modify the href attribute.
          e.setAttribute("href", "http://www.taobao.com");
        }
      }
    ]]);
  
  // 3. Return the modified request to the browser. HTMLStream is a readable stream.
  // You can use HTMLStream in all scenarios that support ReadableStream. 
  return new Response(htmlStream);
}

            export default {
  async fetch(request) {
    return handleRequest(request);
  }
};

Result analysis
The sample code modifies an HTML flow in real time with the HTMLStream API. HTMLStream works as follows:
- The Fetch API retrieves a flow expression for the request. Edge Routine may not have retrieved the response body yet, which reduces garbage collection from data buffering.
- HTMLStream is a flow that supports TransformStream. It calls rewrite callback functions to modify HTML pages in real time. Pass the raw data stream into the HTMLStream flow as shown in Step 2 of the sample code.
  - The first parameter is a stream representing the raw HTML data.
  - The second parameter is an array of rewriters. Each rewriter is a two-element array: a selector string and a callback object. In the example, ["a" , {....}] declares a rewriter where "a" selects all anchor tags. The callback object can contain these functions:
    - element: function(e). Called when matched elements are parsed.
    - comments: function(e). Called when comments nested inside the matched elements are parsed.
    - text: function(e). Called when text inside matched elements is parsed. May be called multiple times because HTMLStream processes text in chunks.
- HTMLStream does not buffer data or generate a DOM tree, unlike parse5 and htmlparser2. This reduces processing time and memory consumption, enabling high throughput and concurrency for HTML parsing.

Rewriter

A rewriter registers the target to rewrite. It is a two-element array.

The first element must be a string or null.
- String: an element selector that locates a specific element or tag.
- null: applies the rewriter to the entire document.
  
  Note
  A document-level rewriter cannot locate individual elements. Use it only when you need to process the entire document.
The second element must be a JavaScript object containing the registered callback functions.

With an element selector, this is the element callback object. With a document selector, this is the document callback object.

Note

An HTMLStream operation supports multiple element selectors but only one document selector.

Syntax of element selectors

The element selector syntax is a subset of CSS selectors. The programming language of an element selector may differ from that of a CSS selector. Supported patterns:

*: selects all elements.
div: selects the div tag. Supports HTML and custom tags.
E#id: selects element E with the specified id.
E.Class: selects element E with the specified Class.
E[attr]: selects element E that has the attr attribute.
Element attributes:
- E[attr="a"]: selects element E where attr equals a (case-sensitive).
- E[attr^="a"]: selects element E where attr equals a (case-insensitive).
- E[attr$="a"]: selects element E where attr ends with a.
- E[attr^="a"]: selects element E where attr starts with a.
- E[attr*="a"]: selects element E where attr contains a.
- E[attr|="a"]: selects element E where attr starts with a- (hyphen-separated values, such as en-ch, en-us).
Order between elements:
- E F: selects element F that is a descendant of element E.
- E > F: selects element F that is a direct child of element E.
E:not(S): selects element E only when selector S does not match.

Callback functions for element selectors

Element selectors support the following callback functions:

Callback function	Description	Callback function signature
element	A non-asynchronous callback function called after the matched elements are fully parsed.	The signature of the callback function is function(e). This signature is carried in the Element object. For more information, see Element.
comments	A non-asynchronous callback function called when comments exist in the matched elements.	The signature of the callback function is function(e). This signature is carried in the Comments object. For more information, see Comments.
text	A non-asynchronous callback function called when parsed text is returned. May be called multiple times due to chunked processing.	The signature of the callback function is function(e). This signature is carried in the TextChunk object. For more information, see TextChunk. Note This callback may be called multiple times. HTMLStream reads text in chunks, calling this function for each chunk. Merge all chunks to get the complete text.

Note

An element selector can omit all callback functions. In that case, matched elements are output without modification. Register only the callbacks you need.

Document selector

A document selector targets the entire document. Set the first element in the rewriter array to null. Only one document selector is allowed per HTMLStream operation.

Callback functions for document selectors

Document selectors support callbacks similar to element selectors:

Callback function	Description	Callback function signature
doctype	A non-asynchronous callback function called when the DOCTYPE declaration is parsed.	The signature of the callback function is function(e). This signature is carried in the Doctype object. For more information, see Doctype.
comments	A non-asynchronous callback function called when the document has comments.	The signature of the callback function is function(e). This signature is carried in the Comments object. For more information, see Comments.
text	A non-asynchronous callback function called when the document has text nodes.	The signature of the callback function is function(e). This signature is carried in the TextChunk object. For more information, see TextChunk. Note This callback may be called multiple times. HTMLStream reads text in chunks, calling this function for each chunk. Merge all chunks to get the complete text.
docend	A non-asynchronous callback function called after the document is fully parsed. Use this to append content such as debugging information as comments at the end of the HTML document.	The signature of the callback function is function(e). This signature is carried in the Docend object. For more information, see Docend.

Error handling

Edge Routine catches all JavaScript exceptions thrown by callback functions. HTMLStream stops processing and propagates the exception to the outer layers.

If triggered via the reader.read method in JavaScript, exceptions are re-thrown.
If reader.read is called while Edge Routine is running (for example, returning a response to a client), Edge Routine hides the exceptions and the response is interrupted. The client receives only a partial response because HTMLStream treats data as streams that may be interrupted before all data is returned. This behavior is similar to how TransformStream handles exceptions.

Callback parameters

Each callback function receives an object representing the selected HTML tags or related information. This topic describes callback parameter types: Element, TextChunk, and Comments.

Note

All parameters must be used within callback functions. Invoking methods or attributes of a parameter outside of a callback function throws JavaScript exceptions. To use parameter data outside a callback, copy it into other JavaScript objects or data structures.
The option parameter in the methods described below is an object. Set its HTML attribute to true for HTML content or false for text content. When set to false, HTMLStream calls the html encoding/escaping function.

Element

Definition

Returned when the Element callback function is called. Represents the selected HTML tags.
Attributes
- tagName(string): the tag name.
- attributes(iterator): returns an iterator over all attributes in [name, value] format.
- removed(bool): whether the element is deleted. Read-only. Use remove() to delete an element. Check this attribute to skip already-deleted elements.
- namespaceURI: the namespace URI of the element (for example, SVG or Script). Read-only.
Methods
- Modify attributes
  - getAttribute(name): gets an attribute value.
  - setAttribute(name, value): sets or modifies an attribute.
  - hasAttribute(name): checks whether an attribute exists.
  - removeAttribute(name): removes an attribute.
  Note
  Both attribute name and value must be strings.
- Modify content
  - before(data, option): inserts content before the element tag.
  - after(data, option): inserts content after the element tag.
  - prepend(data, option): inserts content after the opening tag. Example: <div>(prepend) |aaaa|(append)</div>.
  - append(data, option): inserts content before the closing tag. Example: <div>(prepend) |aaaa|(append)</div>.
  - replace(data, option): replaces the entire element, including tags and nested content.
  - setInnerContent(data, option): replaces the element content while preserving tags and attributes.
  - remove(): deletes the element. Sets removed to true.
  - removeAndKeepContent(): removes tags and attributes but preserves the content.

TextChunk

Definition

Returned when the Text callback function is called. Represents a chunk of the selected HTML text.
Attributes
- removed(bool): whether the element is deleted. Read-only. Use remove() to delete.
- text(string): the text content. Read-only. May be a partial chunk. An empty string indicates the last chunk — merge all chunks to get the full text.
- lastInTextNode(bool): whether this is the last chunk. Read-only. When true, the text attribute returns an empty string.
Methods
Modify content
- before(data, option): inserts content before the specified element (element tag).
- after(data, option): inserts content after the specified element (element tag).
- replace(data, option): replaces the entire element, including the tags and nested tags.
- remove(): deletes the specified element. After the element is deleted, the value of the removed attribute changes to true.

Comments

Definition

Returned when the Comments callback function is called. Represents comments in the selected HTML content.
Attributes
- removed(bool): whether the element is deleted. Read-only. Use remove() to delete.
- text(string): the comment text. Readable and writable — set to overwrite existing comments.
Methods
Modify content
- before(data, option): inserts content before the specified element (element tag).
- after(data, option): inserts content after the specified element (element tag).
- replace(data, option): replaces the entire element, including the tags and nested tags.
- remove(): deletes the specified element. After the element is deleted, the value of the removed attribute changes to true.

Doctype

Definition

Returned when the DOCTYPE callback function is called. Represents the DOCTYPE of the HTML content.
Attributes
- name(string): the DOCTYPE name. Read-only.
- publicId(string): the public identifier, or null if none exists. Read-only.
- systemId(string): the system identifier, or null if none exists. Read-only.

Docend

Definition

Returned when the Docend callback function is called. Represents the end of the HTML document.
Methods

append(string, option): appends content to the end of the HTML document.