OpenSearch:FAQ about A/B testing

Q1: What value can I set the scene_tag parameter to?

Answer: Set this parameter to the name of the test group.

Q2: Search requests are received on the current day, but the A/B testing stops at the night of the current day. Does the report of the next day contain data?

Answer: Yes, the report of the next day contains data.

Q3: What is the flow_divider parameter used for?

Answer: You can add a value of the flow_divider parameter to the whitelist of a specific test. In this case, if you want to distribute traffic to the specified test for a query, set the flow_divider parameter to the value that you have added to the whitelist.

Q4: How can I improve the quality of the metrics for the A/B testing?

Answer: We recommend that you start the test based on previous behavioral data.

Q5: How do I encode URLs based on the values of the scene_tag and flow_divider parameters?

Answer: For example, the value of the scene_tag parameter is "test_1", and that of the flow_divider parameter is "Traffic Bucket".

The following example shows how to encode URLs in Python3.x based on the values of the scene_tag and flow_divider parameters:

import urllib
from urllib import parse
print(parse.quote("scene_tag:" + parse.quote("test_1") + ",flow_divider:" + parse.quote("Traffic Bucket")))
# Result: scene_tag%3Atest_1%2Cflow_divider%3A%25E6%25B5%2581%25E9%2587%258F%2520%25E5%2588%2586%25E6%25A1%25B6

You can set the scene_tag and flow_divider parameters to the following values in the OpenSearch console to perform a search test:

scene_tag:test_1,flow_divider:%E6%B5%81%E9%87%8F%20%E5%88%86%E6%A1%B6

Note: You can use online URL encoding tools to obtain the encoded value.