An API-first Development Practice Based on an API Gateway

By Pan Shengwei (Shimian)

Application programming interfaces (APIs) bridge applications in a digital ecosystem and are the key to improving enterprise operation efficiency and digitalizing enterprise business. The API-first approach is a new development approach that is being widely adopted and recognized. In the API-first approach, application systems are designed around APIs. Before applications or services are built and integrated, APIs are defined and designed with everything that they require to run considered. Microservices architectures are typical adopters of this approach. As more and more microservices develop in such an architecture, APIs grow exponentially in quantity and the benefits and importance of the API-first approach become increasingly self-evident.

What is API-first?

API-first prioritizes the development of APIs over that of other components to facilitate seamless system integration and efficient DevOps practices. In comparison, in the traditional code-first approach, APIs are added at the end of application development, which causes integration difficulty and low efficiency.

Here are some of the principles that the industry mentions for API-first practices:

1. Use of API design as the basis: In the API-first approach, APIs are where everything starts. Therefore, APIs must be carefully planned and designed to ensure they meet the needs of all stakeholders, such as developers, testers, partners, and end users.

Upstream and downstream stakeholders

2. Consistency and reusability: APIs must be designed to be consistent and reusable across projects and applications. This makes APIs standard, expedites development, and improves solution scalability.

3. Collaboration and documentation: The API-first approach emphasizes collaboration between development teams, business stakeholders, and external partners. Comprehensive documentation is essential to ensure that everyone understands how to use the API effectively.

4. Test-driven development: Rigorous tests must be performed on APIs at early stages to identify and resolve issues during development. This helps maintain high quality standards and reduces the risk of costly errors at a later stage.

The above-mentioned principles cover the most part of an API lifecycle. From these principles, it is obvious that API design and development are no easy thing. What benefits, then, does the API-first approach bring us? The following items list some of the benefits:

• A better development experience: Well-designed APIs and detailed documentation make it easier for developers to understand and implement APIs.

• Efficient development and collaboration: Based on API descriptions and mocked responses, developers can efficiently collaborate with each other, testing teams can prepare test cases in advance, front-end teams can design pages ahead of schedule, and back-end teams can develop their own interfaces simultaneously. In the case of microservices development, teams can work on different components in parallel to accelerate the overall development.

Collaborating teams do not block each other's development progress.

• Flexibility and ease of integration: Consistent, reusable, and scalable APIs enable services to quickly adapt to changing needs by smoothly integrating with third-party services, platforms, and applications. This in turn promotes a more coherent and integrated ecosystem. E-commerce and auto industries can leverage this to build capability opening platforms to make integration easier and less costly.

• Automation and innovation: Under certain control, complete operations and data can be revealed through APIs. On this basis, more automated tools can be developed to improve efficiency, and complete DevOps is made possible. In this sense, APIs are the cornerstone of automation and innovation.

• Security: Security is taken into consideration early in API design. Standard and consistent APIs help implement unified security policies. In addition, as APIs define the scopes and boundaries of provided capabilities, teams can better control the accessible functionality and data based on the principle of least privilege.

API-first Practices on Alibaba Cloud

In this section, let's talk about the API-first practice on Alibaba Cloud based on the policies in Alibaba Cloud's API Gateway service.

API Gateway has many policies.

As a service developed from API Gateway, Cloud-native API Gateway also has various policies on security, traffic governance, and API/operation usage. To effectively manage these policies, we designed a common way to abstract these policies to flexibly apply them to different entities (such as gateways, routes, services, and domain names).

Design APIs

Our solution is based on two models: the Policy model and the PolicyAttachment model. The Policy model defines the details and rules of all policies. You can use this model to add, delete, modify, and query policies. This model ensures flexibility and manageability.

Model design for policy management

API Gateway policies fall into two categories: security management policies and API/operation policies. Security management policies govern security capabilities, such as IP address blacklist/whitelist, consumer authentication, and global authentication policies. Examples of API/operation policies include timeout, retry, cross-origin resource sharing (CORS), and redirection policies. The following code snippet shows a sample policy:

{
  "policyId": "policy1",
"name": "IP address blacklist policy",
"description": "Prohibit access from specific IP addresses",
  "type": "security",
  "rules": {
    "blacklist": ["192.168.1.1", "10.0.0.1"]
  }
}

To apply these policies to specific entities, we introduce the PolicyAttachment model, which is used to bind defined policies to specific entities (such as gateways, routes, services, and domain names) so that the same policy can be applied in different contexts and scenarios, thus providing higher reusability and management efficiency. The following example binds a policy to a service:

{
  "attachmentId": "attachment1",
  "policyId": "policy1",
  "entityType": "service",
  "entityId": "serviceA"
}

Aside from the preceding models, we developed a set of RESTful API operations to allow users to create, obtain, update, delete, attach, and detach policies. The following code shows a sample blacklist policy:

{
  "name": "IPBlacklist",
  "type": "security",
  "rules": {
    "blacklist": ["192.168.1.1", "10.0.0.2"]
  }
}

The following example shows how to attach the policy to an API Gateway instance:

POST /api/policy

Content-Type: application/json

{
  "name": "IPBlacklist",
  "description": "gateway",
  "config": {
    "name": "IPBlacklist",
    "type": "security",
    "rules": {
      "blacklist": ["192.168.1.1", "10.0.0.2"]
    }
  },
  "attachResourceIds":[
    "gatewayId1",
  ],
  "attachResourceType": "Gateway",
  "environmentId": "environmentId1",
  "gatewayId": "gatewayId1"
}

Unified policy management systems such as this not only make your system more flexible and scalable, but also improve business security and manageability.

Based on this, we provide the basic API design methodology in API Gateway.

Build an API in API Gateway

OpenAPI Specification (OAS) is the standard format in which RESTful APIs are defined and described. The following example shows a policy management API model based on OAS:

openapi: 3.0.0
info:
  title: Policy Management API
  version: v1
servers:
  - url: /api
paths:
  /policies/{policyId}:
    get:
      summary: Get details of a specific policy.
      operationId: getPolicy
      parameters:
        - name: policyId
          in: path
          required: true
          description: The ID of the policy to retrieve.
          schema:
            type: string
      responses:
        '200':
          description: Successful response with the policy details.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Policy'
components:
  schemas:
    Policy:
      type: object
      properties:
        id:
          type: string
        name:
          type: string
        description:
          type: string
        createdAt:
          type: string
          format: date-time
        updatedAt:
          type: string
          format: date-time

After we design policy management APIs, we must be able to manage APIs throughout their lifecycles to help us implement the API-first approach. In this section, we will take Alibaba Cloud's Cloud-native API Gateway service as an example to walk you through a complete API-first development cycle.

Cloud-native API Gateway works around APIs. The first API to be created is the policy management API, which can be created by importing OAS- or Swagger-compliant data or manually in the console. Let's take a look at manual creation.

Create APIs in the Cloud-native API Gateway console

After the API is created, the console prompts you to create operations. In this example, we will create operations to add, delete, modify, and query resources for the policy management API.

Create operations in the Cloud-native API Gateway console

Since we have designed the API in advance, what we need to do during operation creation is only fill in the metadata according to our design.

Documentation is an indispensable part of the API-first approach. Therefore, each time we modify an operation, the corresponding documentation must be updated at the same time. Since we have entered the complete metadata of operations, the related documentation, SDKs, and specifications will be updated in real time. Take JSON Scheme in the request body for example, if it is not specified, documentation will be hard to understand as it generates the detailed content of API operation documentation. We can also see that JSON Scheme repeats itself with standard content. API Gateway provides a feature to help with the generation of Schema information.

Create operations in the Cloud-native API Gateway console

After filling in all the fields, click Add. Then, we repeat the action for all operations. The following figure shows the complete API:

Let's download an SDK and the documentation to see how pre-designed APIs help us accelerate the development process and improve the development experience.

SDKs and documentation generated in the Cloud-native API Gateway console

Check the result in the SDK for Java

API debugging is not performed in this example due to lack of a back end. However, collaborators have what they need for their own development work.

In addition, the console provides the mocking feature to facilitate development and debugging by collaborators.

API Mocking

The API mocking feature mocks responses for APIs. This way, you can verify whether your API works as expected before the whole business logic is developed. Only two steps are required: define your API operation and its response to requests and publish the mocked data.

Configure API mocking in the API Gateway Console

Publish the mocked data in the API Gateway console

After the publish, you can debug your API online in the console.

Debug an API in the API Gateway console

We find the operation to be debugged and debug it directly. The mocked data is returned. This greatly improves the efficiency of collaborative development.

Debug an API by using documentation and the mocking feature

With API mocking, frontend, backend, QA, and other dependent business teams can work in parallel for testing, client application, and server implementations.

Implement API Logic

We can export API data in the OAS 3.0 format and use Swagger Codegen to quickly generate a server-side code framework to accelerate development.

What is Swagger Codegen?

At the meantime, we need to implement API logic and integrate it with backend systems. Follow best practices for coding, security, and performance optimization. We will not go into details here.

API Testing and Verification

When your API development is completed, you can publish the API and its backend service to a test environment and switch the Mock service to your real backend service in the test environment. Using the test environment, you can test your API fully according to quality standards in terms of functionality, performance, and security with your QA, front-end, and other related parties.

We need to configure automated test tools for our API for their easy integration with our test system. This way, the correctness, performance, and security of our API operations can be ensured in routine.

We also need to configure security protection policies for APIs, configure throttling policies such as traffic protection according to the performance of our APIs, configure security authentication rules, and complete policy capability verification.

Configure appropriate traffic protection policies

You can authorize different consumers to access the current API.

After completing these preparations, we can publish and deploy the API.

API Policy Configuration

In the API management lifecycle, API security and traffic protection policies are also very important. APIs work not only as the communication ingress between systems, but also the first line of defense for security and traffic protection. Alibaba Cloud API Gateway provides a wide range of plug-ins and policies to help you enhance security and ensure API stability.

Security plug-ins of Cloud-native API Gateway

Configure a blacklist or a whitelist: You can configure a blacklist or whitelist on an API to effectively control the permissions of access requests. Blacklist/whitelist plug-ins allow you to configure blacklists and whitelists of client IP addresses at the gateway, domain name, and route levels to deny or allow access requests from specific IP addresses. Flexible configurations allow for fine-grained access control of APIs.

Authentication plug-ins of Cloud-native API Gateway

Fine-grained permission control: Fine-grained permission control is implemented on APIs to ensure that users can access only authorized resources. Cloud-native API Gateway supports global authentication, route configuration authentication, and consumer authentication to ensure API access control, security assurance, and policy management. An authentication plug-in used in Cloud-native API Gateway is shown below.

Various policies in Cloud-native API Gateway

Fine-grained traffic control: Cloud-native API Gateway supports traffic control and concurrency control specific to APIs and even operations. This allows you to effectively manage and monitor request traffic, thus preventing systems from being overloaded by bursty or malicious traffic. Granular management not only limits the request frequency of a single user and protects the performance of backend services, but also dynamically adjusts policies based on actual loads and optimizes resource allocation to ensure that API services can still run smoothly in high-concurrency scenarios.

API publishing, Deployment, and Monitoring

API Gateway provides canary release and end-to-end canary release features to help us control risks and impacts. The API publishing process can be further automated through CI/CD integration. By default, Cloud-native API Gateway provides built-in monitoring and observability features such as CloudMonitor and Prometheus. This helps us monitor the performance and usage of APIs in the production process. We can use monitoring and alerting to ensure the stability of our APIs and continuously meet the SLA requirements of our business.

The API-first approach emphasizes APIs during software development to ensure system flexibility, maintainability, and scalability. This emphasis runs through the whole lifecycle from requirement design and collaborated development all the way to publishing and O&M.

Cloud-native API Gateway can help us better practice API-first.

In the requirement design phase, the API-first approach encourages teams to clarify the functions and interface contract of the API at an early stage, and uses documentation to ensure that all parties have a consistent understanding of the API. This top-down design approach can effectively reduce communications and misunderstandings in subsequent development, thereby improving development efficiency.
Cloud-native API Gateway plays a key role in collaborated development. It not only provides unified API management, but also automatically generates SDKs and documentation according so that developers can quickly get started. Through Cloud-native API Gateway, teams can rapidly build, release, and iterate to ensure the flexibility of microservices architectures and enable developers to focus on the implementation of business logic rather than the underlying details.
In the publishing phase, API Gateway can help us with traffic management and security control to ensure the stability of APIs in the production environment. For example, traffic monitoring and throttling policies are used to detect and handle traffic anomalies in a timely manner to ensure user experience. At the same time, API Gateway can integrate authentication and authorization mechanisms to further enhance the security of the APIs.
In the O&M phase, Cloud-native API Gateway provides a variety of monitoring and analysis features to help O&M teams evaluate the performance and health status of APIs in real time. By analyzing API access logs and usage data, O&M personnel can quickly locate problems, optimize system performance, and ensure high service availability.

API Design Recommendations

This section provides some suggestions for API design based on production practices with API Gateway.

Ability to Evolve

The concept of version is used in API design. For APIs of the same version, each API change must consider the forward compatibility of the API, that is, whether earlier clients can normally access the new version.

• New optional fields are added.
• Existing methods, fields, and enumeration values are not deleted or modified.
• On the premise that the API operation is not changed, do not change the API semantics unless authorized.

For any API upgrade, we need to consider whether existing users will experience incompatibility due to any change.

If there are changes that cause incompatibility, the major version of the API needs to be upgraded, which requires clients to be upgraded at the meantime. The process will be very long. Frequent incompatible upgrades are unacceptable for API users, and that's why proper API design is important. In the process of API design, you can verify your own API design thoroughly to discover potential risks before publishing.

Regarding software upgrade and change, in the OpenAPI specification, API versions can be specified as descriptions in basic information. We can ensure the stability of the API during the introduction of new features or major changes through standard version control methods, thus ensuring that existing users and client applications are not affected by changes. This backward compatibility is critical to maintaining long-term user trust and application continuity. In addition, API version control helps us provide a smooth upgrade path for API users, so that developers and users can gradually migrate to new versions to avoid risks caused by one-time full upgrade.

Good API Documentation

The microservices architecture is becoming increasingly popular nowadays. In such an architecture, services rely on a large number of external APIs, and the importance of good API documentation cannot be overstated. Accurate and up-to-date API documentation can provide developers with clear interface descriptions and code examples, which can significantly reduce misuses and mistakes. At the same time, it greatly improves development efficiency and supports agile development by reducing communications and accelerating problem solving.

JDK ArrayList documentation

This is one of the reasons why a good API can endure. We can see from the above examples that good API documentation describe the functionality and implementation of APIs, provides suitable examples, and lists risks and precautions. All these can help users identify the applicable scenarios of the API in advance.

Limit the Number of APIs

A service with too many APIs increases costs for users and maintainers. Therefore, each API is rigorously reviewed before addition. Instead of creating new APIs, we try to meet new demands with well-designed APIs and API combinations. Once an API is opened, users and clients begin to integrate it, which means that its convergence, unpublishing, and changes will cause a lot of subsequent work.

Pros and Cons of API-first

API-first requires developers to put more efforts in API design. Comprehensive planning and design can incapacitate exploration attempts in later development. In addition, necessary later adjustments may also slow down development and cause design, management, and maintenance difficulties especially when there are a large number of APIs.

Parallel development reduces dependencies and bottlenecks, resulting in faster feature delivery. A comprehensive API management tool can also effectively reduce API management costs. In later development stages, such as post-publishing product experience, API-first provides more benefits. For example, your system can be integrated with external systems and third-party applications more easily and efficiently; your system can be securer and usable because security and scalability are considered at an early stage; and engineers and users are inspired to innovate based on data exposed by APIs. Therefore, although early development may seem a little slow, the overall development process is shorter and the product quality is higher.

With the rapid development of software technologies such as AI, mini programs, and serverless applications, the API-first development approach enables applications to integrate with these technologies more flexibly. Through API, both interaction with AI services and communication with mini programs or function computing services can be realized quickly. This not only accelerates product development, but also reduces the cost of trial and error, and provides strong support for enterprises to stay competitive in an ever-changing market.

Community

An API-first Development Practice Based on an API Gateway

What is API-first?

API-first Practices on Alibaba Cloud

Design APIs

Build an API in API Gateway

API Mocking

Implement API Logic

API Testing and Verification

API Policy Configuration

API publishing, Deployment, and Monitoring

API Design Recommendations

Ability to Evolve

Good API Documentation

Limit the Number of APIs

Pros and Cons of API-first

Read previous post:

Read next post:

Alibaba Cloud Native Community

You may also like

Comments

Alibaba Cloud Native Community

Related Products

API Gateway

Microservices Engine (MSE)

Cloud-Native Applications Management Solution

Database Gateway