All Products
Search
Document Center

API Gateway:Plug-in dataset

Last Updated:Aug 21, 2023

API Gateway provides some of its core capabilities through powerful plug-ins that support hot updates. Up to now, 12 such plug-ins are provided, and the number is growing. These plug-ins enrich the feature suite of API Gateway and makes metadata management on API Gateway flexible. However, in some large-scale business scenarios, these text plug-ins limit users' business performance. To resolve this issue, the configuration data of plug-ins is extracted as dataset objects and managed separately. This greatly enhances user experience and plug-in scalability.

I. Limitations of text plug-ins

API Gateway plug-ins have the following features:

  • Text-based configuration

  • Size limit

As a result, users experience trouble in using some plug-ins or even find themselves unable to use certain plug-ins. The following example shows the typical configuration of an IP address-based access control plug-in. This plug-in type is popular among users.

---
type: ALLOW 
items: 
- blocks:
  - 61.3.XX.XX/24
  - 192.168.34.XX/32
  - 192.168.158.XX/32
  appId: 219810
- blocks:
  - 79.11.XX.XX
  - 85.13.XX.XX
  - 72.152.XX.XX

The configuration seems fine for small-scale applications that involve only several CIDR blocks. But it causes big issues if the application is large in scale and involves scores or even over a hundred CIDR blocks. The following items list two major issues.

  • Error-prone management: A small change in a CIDR block leads to a forklift update of the entire plug-in.

  • Large size: Too many CIDR blocks cause the plug-in to exceed the size limit.

While neither issue can be eliminated based on text configuration, the plug-in dataset feature can solve both issues at the same time.

II. Extraction of configuration data as dataset objects

This section describes the functions of the plug-in dataset. In a word, configurations are extracted as dataset objects for plug-ins to reference whenever necessary. Changes of a dataset object take effect immediately in all plug-ins that reference this dataset object.

Based on dataset objects, the configuration in the preceding example can be simplified as:

---
type: ALLOW 
items: 
- blocksDatasetId: 87b65008e92541938537b1a4a236eda5
  appId: 219810
- blocksDatasetId: 87b65008e92541938537b1a4a236eda3

If you want to change your access control policy, you need only to change the referenced data entries in the plug-in dataset.

The plug-in dataset easily solves the management and size issues of text plug-ins.

  • The dataset is not subject to the size limit of plug-ins, which can reference as many objects as required.

  • You can change your policy by changing data entries in the dataset, so you do not need to update the entire plug-in. For example, if you want to add an IP address to a whitelist, you can simply add a data entry to the dataset.

Currently, the plug-in dataset feature is supported by JSON Web Token (JWT) authentication plug-ins. It will soon be supported by more plug-in types, such as IP address-based access control, parameter-based access control, and backend routing plug-ins.

III. Characteristics of plug-in datasets

The plug-in dataset allows you to dynamically modify the configuration of a plug-in. The plug-in dataset has the following characteristics.

  1. Changes made to data entries in a dataset take effect in all your dedicated instances within ten seconds.

  2. You can define a maximum of 100 datasets and add a maximum of 200 data entries to each dataset. If the quotas are insufficient for you, you can submit a ticket to increase them.

  3. A dataset can be referenced by multiple plug-ins, and changes made to the dataset are synchronized to all the plug-ins.

  4. Duplicate data is not allowed in a dataset.

  5. You can set an expiration time for each data entry, which automatically becomes invalid when the time comes. You can also set a data entry to be valid permanently.

  6. Dataset settings are valid only to groups in dedicated instances. As soon as the groups are migrated to shared instances, the settings become invalid.

IV. Configure a plug-in dataset for JWT authentication plug-ins

1. Configure the plug-in dataset

  • Create a dataset

In this step, you must set a name and select a type for the dataset. You can change the name of the dataset any time, but you cannot change the type of the dataset. For JWT authentication plug-ins, set the dataset type to JWT_BLOCKING.

  • Create data entries in the dataset

Click the dataset that you created to create data entries. The value of a data entry must be unique within the dataset. You can set an expiration time for each data entry, which automatically becomes invalid when the time comes.

  • Obtain the ID of the dataset

After the plug-in dataset is created, the system generates an ID for the dataset. You can view the ID in the dataset list. This ID is used to associate plug-ins with the dataset and does not change.

2. Configure a JWT authentication plug-in

The plug-in dataset is supported by JWT authentication plug-ins.

JWT authentication plug-ins are typically used to block requests sent from users who have been added to a blacklist but have obtained an official token. The plug-in dataset enables a JWT authentication plug-in to reject requests based on the claim parameters that are decrypted from the token. API Gateway also allows you to set custom responses for rejected requests. The following code provides an example about how to configure a JWT authentication plug-in. Note the parameter definitions that start with block:

---
parameter: Authorization   # The parameter from which the token is obtained. 
parameterLocation: header  # The location from which the token is obtained.
claimParameters:           # The claims to be converted into parameters. API Gateway maps JWT claims to backend parameters.
- claimName: aud           # The name of the JWT claim, which can be public or private.
  parameterName: X-Aud     # The name of the backend parameter, to which the JWT claim is mapped.
  location: header         # The location of the backend parameter, to which the JWT claim is mapped. Valid values: query, header, path, and formData.
- claimName: userId        # The name of the JWT claim, which can be public or private.
  parameterName: userId    # The name of the backend parameter, to which the JWT claim is mapped.
  location: query          # The location of the backend parameter, to which the JWT claim is mapped. Valid values: query, header, path, and formData.
blockClaimParameterName: userId # The location of the backend parameter, to which the JWT claim is mapped. Valid values: query, header, path, and formData.
blockByDataSet: 87 b65008e92541938537b1a4a236eda5 # The location of the backend parameter, to which the JWT claim is mapped. Valid values: query, header, path, and formData.
blockStatusCode: 403       # The status code of the response that is returned to a rejected request.
blockResponseHeaders:      # The header of the response that is returned to a rejected request.
  Content-Type: application/xml
blockResponseBody:         # The body of the response that is returned to a rejected request.
  <Reason>be blocked</Reason>
jwks:
- kid: O9fpdhrViq2zaaaBEWZITz         # kid must be set to different values for different JWKs.
  kty: RSA
  e: AQAB
  use: sig
  alg: RS256
  n: qSVxcknOm0uCq5v....

V. Summary

The plug-in dataset feature ushers plug-in configuration from a text era to a structured data era. Partial hot updates of data are supported, and a plug-in can contain much more settings than before. API Gateway supports over a million plug-in datasets for heavy-load scenarios. In addition, as plug-ins are no longer limited by a tiny size, users can do a lot more with API Gateway plug-ins.

Currently, this new feature is available only for JWT authentication plug-ins. It will soon support more plug-in types such as In the future, plug-ins, such as IP address-based access control, parameter-based access control, and backend routing plug-ins.