Description
The default port of Spark Master Web UI is 8080. The default port may allow external users to access data on the master node, imposing a data leakage risk.
Fix
Enable network access control
We recommend that you use a strict firewall policy and restrict the port to intranet access only. You can use Security group policies to limit access.
The Spark port information is as follows:
Enable UI access control
Configure spark.ui.filters to enable javax servlet filters, and configure SSL to enable HTTPS/SSL.
Enable authentication
After specifying the Java servlet filter, you can implement authentication.
Once a user logs on, Spark can query whether the user has permission to view the UI in the ACL list. The configuration parameters spark.acls.enable and spark.ui.view.acls control the ACL actions. Users who start the application always have the permission to access the UI. On YARN, the Spark UI uses the standard YARN web proxy and implements authentication through the installed Hadoop filters.
Spark also supports restricting the users permitted to modify a running Spark application by modifying the ACL table, such as to terminate an application or a task. Such operations are controlled by configuring spark.acls.enable and spark.modify.acls. If you are certifying the web UI, you must add users to the view acls table to use the terminate button on the web UI. On YARN, the modified ACLs table are passed in and control which users are allowed to access the web UI through the YARN interface.
Spark allows you to specify multiple administrators in the ACLs table so that different administrators can always view all applications and modify the permissions of the applications. This feature is controlled by spark.admin.acls and is applicable to a shared cluster.
For more information, see Spark official security solutions.