By default, MaxCompute Spark cannot connect to public endpoints. This topic describes two methods to enable access, using google.com:443 as an example.
Smartnat method
Configure the public access whitelist for your project. For more information, see Accessing the public network.
Add the following configuration items to the
spark-defaults.conffile or your DataWorks configuration:# Set smartnat to true. spark.hadoop.odps.cupid.smartnat.enable=true # Add the public access whitelist. spark.hadoop.odps.cupid.internet.access.list=google.com:443
Leased line connection method
Set up a leased line connection and verify that your VPC can access the public network. For more information, see VPC access solution (leased line connection).
Add the following configuration items to the
spark-defaults.conffile or your DataWorks configuration:# Add parameters for the leased line connection. spark.hadoop.odps.cupid.eni.enable=true spark.hadoop.odps.cupid.eni.info=regionid:vpc-********** # Add the public access whitelist. spark.hadoop.odps.cupid.internet.access.list=google.com:443