Raja_KT
Architect
Architect
  • UID6384
  • Fans5
  • Follows3
  • Posts555
Reads:48530Replies:2

[Share]DataX-On-Hadoop supports uploading HDFS data to MaxCompute on the public cloud

Created#
More Posted time:Dec 15, 2018 16:00 PM
Interesting read.


Run the following commands to submit a MapReduce task on the Hadoop client. You only need to focus on the content in the configuration file of the job. Here, the ./bvt_case/speed.json file is used as an example.


Of course lots of questions , like supporting for bucketing, and why RC , when ORC is already there and many more














Street children suffer not because of their fault. We can help them if we want.Contact me.

GhulamQadir
Senior Engineer
Senior Engineer
  • UID5968
  • Fans4
  • Follows0
  • Posts98
1st Reply#
Posted time:Dec 15, 2018 18:23 PM
Nice to know about commands to submit a MapReduce task on the Hadoop client

Raja_KT
Architect
Architect
  • UID6384
  • Fans5
  • Follows3
  • Posts555
2nd Reply#
Posted time:Dec 15, 2018 20:04 PM
It is similar to the avro tool in Hadoop when we want to convert json to avro ...vice versa.

example below..JFI
$/home/raja/java -jar ~/avro-tools-x.y.z.jar fromjson --schema-file abcdef.avsc abcdef.json > abcdef.avro

It helps a lot when troubleshooting schema changes,,, when dealing with components like sqoop from rdbms to hdfs etc.
Street children suffer not because of their fault. We can help them if we want.Contact me.
Guest