EAS allows you to configure dynamic parameters for a service at runtime. These changes take effect immediately without a service restart, making it ideal for adjusting rate-limiting configuration or tuning model parameters.
Overview
Dynamic parameters enable runtime configuration in EAS, allowing you to modify parameter settings while a service is running. These changes take effect immediately without requiring a service restart.
Difference from environment variables:
Environment variable: Injected when the service starts. Changes require a service restart to take effect.
Dynamic parameter: Can be modified at runtime. Changes take effect immediately without a service restart.
Key advantages:
Configuration changes take effect in real time and are synchronized across all container instances.
Service availability is not affected, preventing service interruptions caused by restarts.
Ideal for frequently adjusted settings, such as rate-limiting thresholds and model parameters.
Use cases
Dynamic parameters are suitable for the following scenarios:
Dynamically tune rate-limiting configuration: Adjust rate-limiting parameters such as
rate_limitin real time based on business traffic.Modify model inference parameters in real time: Adjust model parameters such as
temperature,top_p, andmax_tokens.Adjust log levels: Temporarily adjust the log output level for troubleshooting.
Control feature flags for canary releases: Use dynamic parameters to enable or disable new features.
Configure A/B testing: Quickly switch between different configurations to compare their performance.
Prerequisites
To use dynamic parameters, you must enable dynamic configuration in your service configuration. Add the features field at the top level of the service configuration JSON:
{
"name": "your-service-name",
...other configurations,
"features": {
"eas.aliyun.com/enable-properties": "true"
}
}Exception: If your service uses LLM-based intelligent routing, dynamic configuration is enabled by default. No manual configuration is required.
Procedure
Step 1: Configure dynamic parameters
You can update dynamic parameters in two ways:
Update in the Dynamic Parameters section (Recommended): Modify parameters in the Dynamic Parameters section on the service Overview page. Changes take effect in real time without triggering a service restart.
Update the service configuration (Not recommended): Directly modify the
propertiesfield in the service configuration JSON. This method triggers a service update.
The following steps describe the recommended method:
On the Inference Service tab, click the name of the target service to open its Overview page.
In the Environment Information section, add or modify parameters in Dynamic Parameters. You can add parameters in either of the following ways:
Table: Add parameter names and values row by row.
JSON: Add parameters in bulk in JSON format.
Click Submit to save the configuration.
Step 2: Verify the configuration
After you save the configuration, the system quickly synchronizes the parameters to the configuration file on each container instance.
The configuration file path is /eas/workspace/properties/service.json.
In your application code, read this configuration file to use the dynamic parameters.
Usage example
The following example shows how to configure rate-limiting parameters and then read them in your application code.
Configure rate-limiting parameters:
{
"properties": {
"rate_limit": 100,
"max_concurrent_requests": 50,
"timeout_seconds": 30
}
}Read dynamic parameters in an application (Python):
import json
# Load dynamic parameters.
def load_properties():
try:
with open('/eas/workspace/properties/service.json', 'r') as f:
properties = json.load(f)
return properties
except Exception as e:
print(f"Failed to load properties: {e}")
return {}
# Use the dynamic parameters.
properties = load_properties()
rate_limit = properties.get('rate_limit', 100)
max_concurrent = properties.get('max_concurrent_requests', 50)
print(f"Rate limit: {rate_limit}")
print(f"Max concurrent requests: {max_concurrent}")FAQ
Q: When do parameter changes take effect?
After you modify a dynamic parameter, the change is quickly synchronized to the configuration file on each container instance. However, your application must implement a configuration hot-reloading mechanism, such as file watching or scheduled polling, to apply the latest configuration.
Q: How do I implement configuration hot-reloading?
We recommend the following methods for implementing configuration hot-reloading:
File watching: Use a file system watcher, such as the Python watchdog library, to monitor the configuration file for changes and automatically reload the configuration when it is updated.
Scheduled polling: Periodically read the configuration file, for example, every 5 seconds. Check whether the content has changed and update the in-memory configuration accordingly.
Q: Can I use dynamic parameters for secrets?
We strongly recommend that you do not use dynamic parameters to store sensitive information. Secrets such as API keys and database passwords should be managed using environment variables or a dedicated secrets management service, such as Alibaba Cloud KMS, for enhanced security.