Overview
OpenSearch provides four categories of statistical metrics for hints: core metrics, traffic metrics, click metrics, and guided search metrics.
Core metrics
Metric | Definition | Calculation method | Condition | Description |
Shading PV | The number of successful hint requests. Failed requests due to reasons such as throttling are not counted. | Counts each hint request for which a success response is received. | Calculates the number of successful hint requests on the current day. | |
Shading UV | The number of users who sent hint requests and received results. Failed requests due to reasons such as throttling are not counted. | Counts each user only once regardless of how many hint requests the user sent. | Depends on the user_id field that is specified in hint requests when users send requests by calling an API operation or using an OpenSearch SDK. | Calculates the number of users who sent hint requests on the current day. |
Shading request error times | The number of times when a hint request is sent but an error response is received due to reasons such as throttling. | Counts each failed request. | Indicates whether you need to adjust the hint. For example, you may need to add more hints to the existing ones. | |
Shading PV-CTR | The click-through rate (CTR) of hints. | Divides Shading IPV by Shading PV. | Depends on the raw_query and from_request_id fields that are specified in search requests when users send requests by calling an API operation or using an OpenSearch SDK. | Calculates the CTR of hints. |
Shading UV-CTR | The CTR of users who sent hint requests and received results. | Divides Shading IPV_UV by Shading UV. | Depends on the raw_query and from_request_id fields that are specified in search requests when users send requests by calling an API operation or using an OpenSearch SDK, and the user_id field that is specified in hint requests. | Calculates the CTR of users who sent hint requests and received results. |
Guided search PV ratio | The ratio of hint-guided searches to all searches. | Divides Guided search PV by Search PV. | Depends on the from_request_id field that is specified in search requests when users send requests by calling an API operation or using an OpenSearch SDK. | Calculates the ratio of hint-guided searches to all searches. |
Guided search PV-CTR | The CTR of hint-guided searches. | Divides the item page view (IPV) of hint-guided searches by Guided search PV. | Depends on the raw_query and from_request_id fields that are specified in search requests when users send requests by calling an API operation or using an OpenSearch SDK. This metric also depends on the data collection feature and the uploaded behavioral data about user clicks. | Assesses the retrieval and sorting effects of hint-guided searches. |
Traffic metrics
Metric | Definition | Calculation method | Condition | Description |
Shading PV | The number of successful hint requests. Failed requests due to reasons such as throttling are not counted. | Counts each hint request for which a success response is received. | Calculates the number of successful hint requests on the current day. | |
Shading UV | The number of users who sent hint requests and received results. Failed requests due to reasons such as throttling are not counted. | Counts each user only once regardless of how many hint requests the user sent. | Depends on the user_id field that is specified in hint requests when users send requests by calling an API operation or using an OpenSearch SDK. | Calculates the number of users who sent hint requests on the current day. |
Shading request error times | The number of times when a hint request is sent but an error response is received due to reasons such as throttling. | Counts each failed request. | Indicates whether you need to adjust the hint. For example, you may need to add more hints to the existing ones. | |
Shading request error rate | The ratio of failed hint requests to all hint requests. The requests failed due to reasons such as throttling. | Divides Shading request error times by the sum of Shading PV and Shading request error times. | Indicates whether you need to adjust the hint. For example, you may need to add more hints to the existing ones. |
Click metrics
Metric | Definition | Calculation method | Condition | Description |
Shading IPV | The number of clicks on hints. | Counts each click on a hint. | Depends on the from_request_id field that is specified in search requests when users send requests by calling an API operation or using an OpenSearch SDK. | Calculates the number of clicks on hints. |
Shading IPV_UV | The number of users who clicked hints. | Counts each user only once regardless of how many times the user clicked hints. | Depends on the raw_query and from_request_id fields that are specified in search requests when users send requests by calling an API operation or using an OpenSearch SDK, and the user_id field that is specified in hint requests. | Calculates the number of users who clicked hints. |
Shading PV-CTR | The CTR of hints. | Divides Shading IPV by Shading PV. | Depends on the raw_query and from_request_id fields that are specified in search requests when users send requests by calling an API operation or using an OpenSearch SDK. | Calculates the CTR of hints. |
Shading UV-CTR | The CTR of users who sent hint requests and received results. | Divides Shading IPV_UV by Shading UV. | Depends on the raw_query and from_request_id fields that are specified in search requests when users send requests by calling an API operation or using an OpenSearch SDK, and the user_id field that is specified in hint requests. | Calculates the CTR of users who sent hint requests and received results. |
Shading has a click on the percentage of PV | The ratio of hint requests whose results are clicked by users to all hint requests. | Divides the number of hint requests whose results are clicked by users by Shading PV. | Depends on the from_request_id field that is specified in search requests when users send requests by calling an API operation or using an OpenSearch SDK. | Calculates the ratio of hint requests whose results are clicked by users to all hint requests. |
Guided search metrics
Metric | Definition | Calculation method | Condition | Description |
Guided search PV | The number of hint-guided searches. Search requests include paging requests. | Counts each time when a user clicks a hint. | Depends on the from_request_id field that is specified in search requests when users send requests by calling an API operation or using an OpenSearch SDK. | Calculates the number of hint-guided searches. |
Guided search UV | The number of users who clicked hints. | Counts each user only once regardless of how many times the user clicked hints. | Depends on the user_id and from_request_id fields that are specified in search requests when users send requests by calling an API operation or using an OpenSearch SDK. | Calculates the number of users who clicked hints. |
Guided search PV ratio | The ratio of hint-guided searches to all searches. | Divides Guided search PV by Search PV. | Depends on the from_request_id field that is specified in search requests when users send requests by calling an API operation or using an OpenSearch SDK. | Calculates the ratio of hint-guided searches to all searches. |
Guided search UV ratio | The ratio of users who clicked hints to all users who sent hint requests and received results. | Divides Guided search UV by Search UV. | Depends on the user_id and from_request_id fields that are specified in search requests when users send requests by calling an API operation or using an OpenSearch SDK. | Calculates the ratio of users who clicked hints to all users who sent hint requests and received results. |
Guided search no result rate | The ratio of hint-guided searches with empty results to all hint-guided searches. | Divides the number of hint-guided searches with empty results by Guided search PV. | Depends on the from_request_id field that is specified in search requests when users send requests by calling an API operation or using an OpenSearch SDK. | Calculates the ratio of hint-guided searches with empty results to all hint-guided searches. |
Guided search PV-CTR | The CTR of hint-guided searches. | Divides the IPV of hint-guided searches by Guided search PV. | Depends on the raw_query and from_request_id fields that are specified in search requests when users send requests by calling an API operation or using an OpenSearch SDK. This metric also depends on the data collection feature and the uploaded behavioral data about user clicks. | Assesses the retrieval and sorting effects of hint-guided searches. |
Guided search has clicks on the percentage of PV | The ratio of hint-guided searches whose results are clicked by users to all hint-guided searches. | Divides the number of hint-guided searches whose results are clicked by users by Guided search PV. | Depends on the from_request_id field that is specified in search requests when users send requests by calling an API operation or using an OpenSearch SDK. This metric also depends on the data collection feature and the synchronization of events about user clicking search results. | Calculates the ratio of hint-guided searches whose results are clicked by users to all hint-guided searches to assess the effects of search retrieval and sorting. |
Guided search for GMV | The gross merchandise volume (GMV) of purchases that users make based on hint-guided searches. | Aggregates the transaction amounts guided by hints. | Depends on the data collection feature and the uploaded behavioral data about user purchases. The value of the bhv_type field must be buy in the data that is uploaded. | Assesses the effect of hints on guiding users to make purchases. |
Guided search favorite conversion rate | The ratio of times when users add items from the results of hint-guided searches to favorites to all hint-guided searches. | Divides the number of times when users add items from the results of hint-guided searches to favorites by Guided search PV. | Depends on the data collection feature and the uploaded behavioral data about adding items to favorites. The value of the bhv_type field must be collect in the data that is uploaded. | Assesses the effect of hints on guiding users to add items to favorites. |
Guided search to the shopping cart conversion rate | The ratio of times when users add items from the results of hint-guided searches to the shopping cart to all hint-guided searches. | Divides the number of times when users add items from the results of hint-guided searches to the shopping cart by Guided search PV. | Depends on the data collection feature and the uploaded behavioral data about adding items to the shopping cart, bookshelf, or playlist. The value of the bhv_type field must be cart in the data that is uploaded. | Assesses the effect of hints on guiding users to add items to the shopping cart. |
Guided search like conversion rate | The ratio of times when users like items in the results of hint-guided searches to all hint-guided searches. | Divides the number of times when users like items in the results of hint-guided searches by Guided search PV. | Depends on the data collection feature and the uploaded behavioral data about liking items. The value of the bhv_type field must be like in the data that is uploaded. | Assesses the effect of hints on guiding users to like items. |
Guided search review conversion rate | The ratio of times when users post comments on items in the results of hint-guided searches to all hint-guided searches. | Divides the number of times when users post comments on items in the results of hint-guided searches by Guided search PV. | Depends on the data collection feature and the uploaded behavioral data about posting comments. The value of the bhv_type field must be comment in the data that is uploaded. | Assesses the effect of hints on guiding users to post comments on items. |
Usage notes
When the hint report displays metrics for a day, the report displays the metrics for the day before the current day by default. You can set the date picker control to another day as needed. After you upload behavioral data, the hint report based on the uploaded data is available the next day.
The hint report provides line charts to display the trends of metrics in the last seven days. You can set the date picker control to another time period as needed. You can view the line chart of a metric by clicking the corresponding icon.
To view a line chart, perform the following steps: In the left-side navigation pane, choose Report Statistics > Hint Report. Select an application from the Application Name drop-down list and a model from the Model Name drop-down list. Click a metric category tab. Then, set the date picker control to a time period as needed.