This topic describes the packages that MaxCompute SDK for Java provides and the core classes of the SDK, such as instances, resources, tables, and functions.
For more information, see Storage pricing (pay-as-you-go), Computing pricing, and Download pricing (pay-as-you-go).
Background information
This topic describes the core classes of MaxCompute SDK for Java. For more information about the SDK, see the MaxCompute SDK for Java documentation.
<dependency>
<groupId>com.aliyun.odps</groupId>
<artifactId>odps-sdk-core</artifactId>
<version>X.X.X-public</version>
</dependency>
- Only the SDK of the 0.27.2-public version or later supports the new data types that are introduced in the MaxCompute V2.0 data type edition.
- You can obtain the latest version of the SDK by searching for odps-sdk-core at search.maven.org.
Package name | Description |
---|---|
odps-sdk-core | Provides the basic features of MaxCompute, such as the features that allow you to manage tables and projects. You can also use this package to access the Tunnel service. |
odps-sdk-commons | Provides common utilities. |
odps-sdk-udf | Provides user defined function (UDF) features. |
odps-sdk-mapred | Provides MapReduce features. |
odps-sdk-graph | Provides Graph SDK for Java. You can obtain the SDK by searching for odps-sdk-graph. |
AliyunAccount
You can call this API to obtain your Alibaba Cloud account. The constructor of the AliyunAccount class uses an AccessKey ID and an AccessKey secret as the input parameters. An AccessKey ID and an AccessKey secret constitute an AccessKey pair that is used to authenticate an Alibaba Cloud user. This API is also considered as a class that is used to initialize MaxCompute.
MaxCompute
Account account = new AliyunAccount("my_access_id", "my_access_key");
Odps odps = new Odps(account);
String odpsUrl = "<your odps endpoint>";
odps.setEndpoint(odpsUrl);
odps.setDefaultProject("my_project");
for (Table t : odps.tables()) {
....
}
MaxCompute Tunnel
MaxCompute Tunnel is developed based on Tunnel SDK. You can use MaxCompute Tunnel to upload data to and download data from MaxCompute. For more information, see MaxCompute Tunnel overview. MaxCompute Tunnel allows you to upload and download only tables, but not views.
MapReduce
For more information about MapReduce SDK, see Overview.
Projects
Account account = new AliyunAccount("my_access_id", "my_access_key");
Odps odps = new Odps(account);
String odpsUrl = "<your odps endpoint>";
odps.setEndpoint(odpsUrl);
Project p = odps.projects().get("my_exists");
p.reload();
...
Project
The Project class defines the details of a project. You can obtain a specified project by calling the get method of the Projects class.
SQLTask
The SQLTask class allows you to run and process SQL statements in MaxCompute. You can execute SQL statements by calling the run method of the SQLTask class.
import java.util.List;
import com.aliyun.odps.Instance;
import com.aliyun.odps.Odps;
import com.aliyun.odps.OdpsException;
import com.aliyun.odps.account.Account;
import com.aliyun.odps.account.AliyunAccount;
import com.aliyun.odps.data.Record;
import com.aliyun.odps.task.SQLTask;
public class testSql {
private static final String accessId = "";
private static final String accessKey = "";
private static final String endPoint = "http://service.odps.aliyun.com/api";
private static final String project = "";
private static final String sql = "select category from iris;";
public static void
main(String[] args) {
Account account = new AliyunAccount(accessId, accessKey);
Odps odps = new Odps(account);
odps.setEndpoint(endPoint);
odps.setDefaultProject(project);
Instance i;
try {
i = SQLTask.run(odps, sql);
i.waitForSuccess();
List<Record> records = SQLTask.getResult(i);
for(Record r:records){
System.out.println(r.get(0).toString());
}
} catch (OdpsException e) {
e.printStackTrace();
}
}
}
- You can submit and execute only one SQL statement at a time.
- To create a table by using MaxCompute SDK for Java, you must use the SQLTask class instead of the Table class. You need to pass the CREATE TABLE statement to SQLTask before you use an SQLTask object to execute the CREATE TABLE statement. For more information about the CREATE TABLE statement, see Table operations.
Instances
Account account = new AliyunAccount("my_access_id", "my_access_key");
Odps odps = new Odps(account);
String odpsUrl = "<your odps endpoint>";
odps.setEndpoint(odpsUrl);
odps.setDefaultProject("my_project");
for (Instance i : odps.instances()) {
....
}
Instance
Account account = new AliyunAccount("my_access_id", "my_access_key");
Odps odps = new Odps(account);
String odpsUrl = "<your odps endpoint>";
odps.setEndpoint(odpsUrl);
Instance instance= odps.instances().get("instance id");
Date startTime = instance.getStartTime();
Date endTime = instance.getEndTime();
...
Status instanceStatus = instance.getStatus();
String instanceStatusStr = null;
if (instanceStatus == Status.TERMINATED) {
instanceStatusStr = TaskStatus.Status.SUCCESS.toString();
Map<String, TaskStatus> taskStatus = instance.getTaskStatus();
for (Entry<String, TaskStatus> status : taskStatus.entrySet()) {
if (status.getValue().getStatus() != TaskStatus.Status.SUCCESS) {
instanceStatusStr = status.getValue().getStatus().toString();
break;
}
}
} else {
instanceStatusStr = instanceStatus.toString();
}
...
TaskSummary summary = instance.getTaskSummary("task name");
String s = summary.getSummaryText();
Tables
Account account = new AliyunAccount("my_access_id", "my_access_key");
Odps odps = new Odps(account);
String odpsUrl = "<your odps endpoint>";
odps.setEndpoint(odpsUrl);
odps.setDefaultProject("my_project");
for (Table t : odps.tables()) {
....
}
Table
Account account = new AliyunAccount("my_access_id", "my_access_key");
Odps odps = new Odps(account);
String odpsUrl = "<your odps endpoint>";
odps.setEndpoint(odpsUrl);
odps.setDefaultProject("my_project");
Table table = odps.tables().get("tablename");
for(Column c : table.getSchema().getColumns())
{
String name = c.getName();
TypeInfo type = c.getTypeInfo();
}
Account account = new AliyunAccount("my_access_id", "my_access_key");
Odps odps = new Odps(account);
String odpsUrl = "<your odps endpoint>";
odps.setEndpoint(odpsUrl);
Table t = odps.tables().get("table name");
t.reload();
Partition part = t.getPartition(new PartitionSpec("partition_col=partition_col_value"));
part.reload();
...
Resources
Account account = new AliyunAccount("my_access_id", "my_access_key");
Odps odps = new Odps(account);
String odpsUrl = "<your odps endpoint>";
odps.setEndpoint(odpsUrl);
odps.setDefaultProject("my_project");
for (Resource r : odps.resources()) {
....
}
Resource
Account account = new AliyunAccount("my_access_id", "my_access_key");
Odps odps = new Odps(account);
String odpsUrl = "<your odps endpoint>";
odps.setEndpoint(odpsUrl);
Resource r = odps.resources().get("resource name");
r.reload();
if (r.getType() == Resource.Type.TABLE) {
TableResource tr = new TableResource(r);
String tableSource = tr.getSourceTable().getProject() + "."
+ tr.getSourceTable().getName();
if (tr.getSourceTablePartition() != null) {
tableSource += " partition(" + tr.getSourceTablePartition().toString()
+ ")";
}
....
}
String projectName = "my_porject";
String source = "my_local_file.txt";
File file = new File(source);
InputStream is = new FileInputStream(file);
FileResource resource = new FileResource();
String name = file.getName();
resource.setName(name);
odps.resources().create(projectName, resource, is);
TableResource resource = new TableResource(tableName, tablePrj, partitionSpec);
//resource.setName(INVALID_USER_TABLE);
resource.setName("table_resource_name");
odps.resources().update(projectName, resource);
Functions
Account account = new AliyunAccount("my_access_id", "my_access_key");
Odps odps = new Odps(account);
String odpsUrl = "<your odps endpoint>";
odps.setEndpoint(odpsUrl);
odps.setDefaultProject("my_project");
for (Function f : odps.functions()) {
....
}
Function
Account account = new AliyunAccount("my_access_id", "my_access_key");
Odps odps = new Odps(account);
String odpsUrl = "<your odps endpoint>";
odps.setEndpoint(odpsUrl);
Function f = odps.functions().get("function name");
List<Resource> resources = f.getResources();
String resources = "xxx:xxx";
String classType = "com.aliyun.odps.mapred.open.example.WordCount";
ArrayList<String> resourceList = new ArrayList<String>();
for (String r : resources.split(":")) {
resourceList.add(r);
}
Function func = new Function();
func.setName(name);
func.setClassType(classType);
func.setResources(resourceList);
odps.functions().create(projectName, func);