:Best practices: Custom task release freeze control

Subscription configuration

When configuring a subscription, specify the event type as file commit or file deployment using an event rule.

• In standard mode workspaces: dataworks:FileChange:CommitFile.

• In basic mode workspaces: dataworks:FileChange:DeployFile.

Step 1: Configure a custom event bus

For detailed steps to enable and configure message subscriptions, see Enable message subscriptions. The following describes the core configuration steps and key considerations for this practice.

1. Log on to the EventBridge console. In the navigation pane on the left, click Event Bus to go to the event bus creation page.

   If you have not activated EventBridge, activate EventBridge and grant permissions.

2. Click the button and then click Create custom event bus.

   1. In the Bus section, enter a Custom event bus name, and then click Next step to configure the event source.

   2. Click Skip to skip configuring the Event source, Rule, and Target sections. Enter a Description. You can skip Steps 2 (Event source), 3 (Rule), and 4 (Target).

3. In the navigation pane on the left, click Event Bus to return to the event bus list. Find your custom event bus, and click its name to go to the overview page.

   • Click Event Rules in the navigation pane on the left to go to the event rules page, then click Create Rule to create a new event rule.

   • This practice configures the custom EventBridge bus to receive DataWorks file commit and file deployment events. The demo configuration and key parameters are as follows.

     1. Configure basic information: Enter a custom rule name.

     2. Configure event pattern:

        • Event source type: Select Custom event source.

        • Event source: Leave this field blank.

        • Pattern content: Write the pattern in JSON format as shown below. This example uses a standard mode workspace.

          Note

          If you use a basic mode workspace, set the value to dataworks:FileChange:DeployFile.

          {
              "source": [
                  "acs.dataworks"
              ],
              "type": [
                  "dataworks:FileChange:CommitFile"
              ]
          }

          • source: Specifies the product identifier as acs.dataworks.

          • type: Specifies the event type under the product as dataworks:FileChange:CommitFile.

        • Test event pattern: Update the values of source and type, and then test the event. After a successful test, click Next. Click Test. If the page displays "Match succeeded. The event can be triggered normally," the event pattern is configured correctly.

     3. Configure event target.

        • Service Type: Select HTTPS or HTTP. For more service types, see Manage event rules.

        • URL: Enter the URL that receives messages pushed from the custom event bus, for example: https://server address:port/extensions/consumer.

          Note

          Enter the server-side address where your extension is deployed.

        • Body: Select Complete event.

        • Network type: Select Internet.

Step 2: Configure an event distribution channel

1. Log on to the DataWorks console. In the target region, click More > Open Platform in the left-side navigation pane. Click Go to Open Platform to open the Developer Backend page.

2. On the Developer Backend page, click OpenEvent in the left-side navigation pane. On the page that appears, click Add Event Distribution Channel and configure the parameters in the dialog box.

   • Workspace for Distribution of Event Messages: Select the workspace that you created.

   • Specify Custom Event Bus in EventBridge for Distribution of Event Messages: Select the event bus that you created in Step 1.

3. After you save the event distribution channel, find the channel and click Enable in the Operation column to enable it.

Step 3: Register and configure an extension

1. On the Developer Backend page, click Extension in the left-side navigation pane. Then, click Register Extension and configure the parameters in the dialog box.

   1. Deployment method: Select Self-managed service deployment.

   2. Register the extension.

      • Extension Name: Enter a custom name.

      • Extension Point Event: Select Pre-file commit event and Pre-file deployment event.

      • Workspace for Testing: After configuration, you can test the extension in this workspace before submitting it.

      Configure other parameters as prompted.

2. After you complete the configurations, click Determine to save the registered extension.

3. In the Operation column for the extension you created, click Submit to start the review process.

   Note

   • Extensions are reviewed by the DataWorks platform. The review is typically completed within three business days.

   • To test the extension you created, you must configure a Workspace for Testing.

4. After the extension is approved, click Publish in the Actions column.

5. Click Manage Extensions to go to the Management Center > Extension page. Select your extension and enable it in the Enable column.

Sample code

This code receives messages pushed by EventBridge, compares the current date against holidays and network freeze dates, and returns the result to DataWorks using the UpdateIDEEventResult OpenAPI operation.

Environment: Java 8 with Maven.

package com.aliyun.dataworks.demo;
import com.alibaba.fastjson.JSON;
import com.alibaba.fastjson.JSONObject;
import com.aliyun.dataworks.config.Constants;
import com.aliyun.dataworks.config.EventCheckEnum;
import com.aliyun.dataworks.config.ExtensionParamProperties;
import com.aliyun.dataworks.services.DataWorksOpenApiClient;
import com.aliyun.dataworks_public20200518.Client;
import com.aliyun.dataworks_public20200518.models.UpdateIDEEventResultRequest;
import com.aliyun.dataworks_public20200518.models.UpdateIDEEventResultResponse;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import java.text.SimpleDateFormat;
import java.util.Arrays;
import java.util.List;
/**
 * @author dataworks demo
 */
@RestController
@RequestMapping("/extensions")
public class ExtensionsController {
  @Autowired(required = false)
  private DataWorksOpenApiClient dataWorksOpenApiClient;
  @Autowired
  private ExtensionParamProperties extensionParamProperties;
  /**
   * Receives messages pushed by EventBridge.
   *
   * @param jsonParam
   */
  @PostMapping("/consumer")
  public void consumerEventBridge(@RequestBody String jsonParam) {
    JSONObject jsonObj = JSON.parseObject(jsonParam);
    String eventCode = jsonObj.getString(Constants.EVENT_CODE_FILED);
    if (Constants.COMMIT_FILE_EVENT_CODE.equals(eventCode)) {
      // Initialize client
      Client client = dataWorksOpenApiClient.createClient();
      // Get current date
      SimpleDateFormat sdf = new SimpleDateFormat(Constants.YYYY_MM_DD);
      String now = sdf.format(System.currentTimeMillis());
      // Get 2022 holidays and network freeze dates
      List<String> holidayList = Arrays.asList(extensionParamProperties.getHolidayList().split(","));
      // Check if current date is a freeze date
      boolean isExists = holidayList.stream().anyMatch(day -> day.equals(now));
      // Prepare callback request
      UpdateIDEEventResultRequest updateIDEEventResultRequest = new UpdateIDEEventResultRequest();
      updateIDEEventResultRequest.setMessageId(jsonObj.getString("id"));
      updateIDEEventResultRequest.setExtensionCode(extensionParamProperties.getExtensionCode());
      // If freeze date, check fails
      if (isExists) {
        updateIDEEventResultRequest.setCheckResult(EventCheckEnum.FAIL.getCode());
        updateIDEEventResultRequest.setCheckResultTip("File submission is not allowed on freeze dates");
      } else {
        // If not a freeze date, check passes
        updateIDEEventResultRequest.setCheckResult(EventCheckEnum.OK.getCode());
        updateIDEEventResultRequest.setCheckResultTip(EventCheckEnum.OK.getName());
      }
      try {
        // Callback to DataWorks
        UpdateIDEEventResultResponse response = client.updateIDEEventResult(updateIDEEventResultRequest);
              // The unique ID of the request, used for troubleshooting
              System.out.println("response:" + response.getBody().getRequestId());
          } catch (Exception e) {
              System.out.println("consumerEventBridge error:" + e.getMessage());
              throw new RuntimeException(e);
          }
      } else {
          System.out.println("Other events were not filtered. Check your configuration steps.");
      }
  }
}

Deploy the sample project

1. Prepare the environment and project

   • Dependencies: Java 8 or later, Maven.

   • Project download link: extension-demo-deploycontroller.zip (30 KB).

2. Deployment options

   • Local deployment: Package the project into a JAR file and run it on a local server or Windows machine with Java 8 and Maven installed using the command java -jar yourapp.jar.

   • Cloud deployment: Package the project into a JAR file and deploy it to a runtime environment such as a Docker container or Elastic Compute Service instance.

3. After downloading the project, go to the root directory and run the following command to package it into a JAR file.

   mvn clean package -Dmaven.test.skip=true spring-boot:repackage

4. Run the JAR file:

   java -jar target/extension-demo-deploycontroller-1.0.jar

The application starts successfully, as shown in the following output:

.   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::                (v2.7.2)
2022-08-02 20:07:59.521  INFO 48144 --- [           main] com.aliyun.dataworks.DemoStarter          : Starting DemoStarter v1.0 using Java 1.8.0_151 on xxxdeMacBook-Pro.local with PID 48144 (/xxx/xxx/n-demo-deploycontroller/target/extension/demo-deploycontroller-1.0.jar started by xxx in xxx/xxx/extension-xxx-demo-deploycontroller)
2022-08-02 20:07:59.523  INFO 48144 --- [           main] com.aliyun.dataworks.DemoStarter          : No active profile set, falling back to 1 default profile: "default"
2022-08-02 20:08:00.440  INFO 48144 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 8080 (http)
2022-08-02 20:08:00.452  INFO 48144 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
2022-08-02 20:08:00.452  INFO 48144 --- [           main] org.apache.catalina.core.StandardEngine  : Starting Servlet engine: [Apache Tomcat/9.0.65]
2022-08-02 20:08:00.452  INFO 48144 --- [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
2022-08-02 20:08:00.576  INFO 48144 --- [           main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 1002 ms
2022-08-02 20:08:00.576  INFO 48144 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8080 (http) with context path ''
2022-08-02 20:08:00.945  INFO 48144 --- [           main] com.aliyun.dataworks.DemoStarter          : Started DemoStarter in 1.771 seconds (JVM running for 2.199)
2022-08-02 20:08:07.853  INFO 48144 --- [nio-8080-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring DispatcherServlet 'dispatcherServlet'
2022-08-02 20:08:07.854  INFO 48144 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet        : Initializing Servlet 'dispatcherServlet'

Open http://localhost:8080/index in your browser. If you see "hello world!", the application is deployed successfully. After connecting to the network, it can receive messages from EventBridge.