MaxCompute - Dynamic Filter Feature Released
Apr 07 2021MaxCompute
Target customers: developers, analysts, and users who want to optimize SQL operations. Features released: JOIN is a common operation in distributed systems. If you perform a JOIN operation on a large amount of data, the operation is both time-consuming and resource-consuming. MaxCompute now releases the Dynamic Filter feature to accelerate JOIN operations. This feature uses the equi-join attribute of JOIN operations and the filter that is dynamically generated during execution to filter data before the data is shuffled or joined. This accelerates queries. This feature is ideal for the scenarios in which JOIN operations are performed on dimension tables and fact tables. You can use flags to forcibly or intelligently enable the Dynamic Filter feature for a session. You can also specify hints in SQL statements to enable this feature for a session. In addition, you can use the dynamic partition pruning method to filter partitions before you use dynamic filters.