The default port of Spark Master Web UI is 8080. The default port may allow external users to access data on the master node, imposing a data leakage risk.
We recommend that you use a strict firewall policy and restrict the port to intranet access only. You can use Security group policies to limit access.
The Spark port information is as follows:
spark.ui.filters to enable javax servlet filters, and configure SSL to enable HTTPS/SSL.
After specifying the Java servlet filter, you can implement authentication.
Once a user logs on, Spark can query whether the user has permission to view the UI in the ACL list. The configuration parameters
spark.ui.view.acls control the ACL actions. Users who start the application always have the permission to access the UI. On YARN, the Spark UI uses the standard YARN web proxy and implements authentication through the installed Hadoop filters.
Spark also supports restricting the users permitted to modify a running Spark application by modifying the ACL table, such as to terminate an application or a task. Such operations are controlled by configuring
spark.modify.acls. If you are certifying the web UI, you must add users to the
view acls table to use the terminate button on the web UI. On YARN, the modified ACLs table are passed in and control which users are allowed to access the web UI through the YARN interface.
Spark allows you to specify multiple administrators in the ACLs table so that different administrators can always view all applications and modify the permissions of the applications. This feature is controlled by
spark.admin.acls and is applicable to a shared cluster.
For more information, see Spark official security solutions.