E-MapReduce supports MetaService, which allows you to access OSS data in the E-MapReduce environment without an AccessKey. The previous way of using an AccessKey and endpoint is also supported. Make sure that you use an internal IP address for the OSS endpoint. For more information about a complete list of endpoints, see OSS endpoints.
Allow Spark to access OSS
val conf = new SparkConf().setAppName("Test OSS") val sc = new SparkContext(conf) val pathIn = "oss://bucket/path/to/read" val inputData = sc.textFile(pathIn) val cnt = inputData.count println(s"count: $cnt") val outputPath = "oss://bucket/path/to/write" val outpuData = inputData.map(e => s"$e has been processed.") outpuData.saveAsTextFile(outputPath)
For the complete sample code, see: