O kit de desenvolvimento de software (SDK) Java fornece um conjunto de interfaces de programação Java para o MaxCompute. Use este SDK para gerencie projetos, tabelas, transferências de dados e funções com código Java. Este tópico apresenta uma visão geral do SDK Java, incluindo instâncias, recursos, tabelas e funções.
As taxas de computação e armazenamento para uso do SDK na chamada ao MaxCompute são idênticas às cobradas pelo uso direto do MaxCompute.
Informações básicas
Este tópico descreve as interfaces principais comuns do MaxCompute. Para mais detalhes, acesse SDK Java Doc.
Use o Maven para gerencie e configure a versão do SDK. O código a seguir mostra um exemplo de configuração do Maven.
<dependency>
<groupId>com.aliyun.odps</groupId>
<artifactId>odps-sdk-core</artifactId>
<version>X.X.X-public</version>
</dependency>Somente a versão 0.27.2-public e posteriores oferecem suporte aos new data types do MaxCompute V2.0.
Pesquise por odps-sdk-core em search.maven.org para obter a versão mais recente do SDK.
A tabela a seguir descreve os pacotes do SDK fornecidos pelo MaxCompute.
|
Nome do pacote |
Descrição |
|
odps-sdk-core |
Contém os recursos básicos do MaxCompute, como operações em tabelas e projetos, além do tunnel. |
|
odps-sdk-commons |
Inclui encapsulamentos Util. |
|
odps-sdk-udf |
Principais interfaces para o recurso UDF. |
|
odps-sdk-mapred |
Recurso MapReduce. |
|
odps-sdk-graph |
SDK Java para Graph. Pesquise pela palavra-chave odps-sdk-graph. |
AliyunAccount
Representa uma conta Alibaba Cloud. Os parâmetros de entrada são um AccessKey ID e um AccessKey secret, que funcionam como identidade e chave de autenticação da conta. Use esta classe para inicializar o MaxCompute.
ODPS
Ponto de entrada do SDK do MaxCompute. Esta classe permite obter todas as coleções de objetos em um projeto, como Projects, Tables, Resources, Functions e Instances.
Construa um objeto MaxCompute passando uma instância de AliyunAccount. O código a seguir mostra um exemplo.
// An AccessKey for an Alibaba Cloud account has all API access permissions, which poses a high security threat. Create and use a Resource Access Management (RAM) user for API calls or routine O&M. Log on to the RAM console to create a RAM user.
// This example shows how to store the AccessKey and AccessKey secret in environment variables. You can also store them in a configuration file as needed.
// Do not save the AccessKey and AccessKey secret in your code. This prevents key leaks.
Account account = new AliyunAccount(System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID"), System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET"));
Odps odps = new Odps(account);
String odpsUrl = "<your odps endpoint>";
odps.setEndpoint(odpsUrl);
odps.setDefaultProject("my_project");
for (Table t : odps.tables()) {
....
} Tunnel de dados em lote
O data tunnel do MaxCompute baseia-se no Tunnel SDK. Use o Tunnel SDK para fazer upload ou baixe de dados de tabelas do MaxCompute. Views não têm suporte.
MapReduce
Para mais informações sobre os SDKs compatíveis com o MapReduce, consulte Native SDK overview.
Projects
Representa uma coleção de todos os projetos no MaxCompute. Os elementos dessa coleção são objetos Project. O código a seguir mostra um exemplo.
// An AccessKey for an Alibaba Cloud account has all API access permissions, which poses a high security threat. Create and use a Resource Access Management (RAM) user for API calls or routine O&M. Log on to the RAM console to create a RAM user.
// This example shows how to store the AccessKey and AccessKey secret in environment variables. You can also store them in a configuration file as needed.
// Do not save the AccessKey and AccessKey secret in your code. This prevents key leaks.
Account account = new AliyunAccount(System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID"), System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET"));
Odps odps = new Odps(account);
String odpsUrl = "<your odps endpoint>";
odps.setEndpoint(odpsUrl);
Project p = odps.projects().get("my_exists");
p.reload();
...Project
Representa um projeto e suas informações. Obtenha o projeto correspondente na coleção Projects.
SQLTask
SQLTask é uma interface para execute e processar tarefas SQL. Execute instruções SQL diretamente usando a interface run.
A interface run retorna uma Instance, que permite recuperar o status de execução e o resultado da instrução SQL. O código a seguir mostra um exemplo.
import java.util.List;
import com.aliyun.odps.Instance;
import com.aliyun.odps.Odps;
import com.aliyun.odps.OdpsException;
import com.aliyun.odps.account.Account;
import com.aliyun.odps.account.AliyunAccount;
import com.aliyun.odps.data.Record;
import com.aliyun.odps.task.SQLTask;
public class TestSql {
// An AccessKey for an Alibaba Cloud account has all API access permissions, which poses a high security threat. Create and use a Resource Access Management (RAM) user for API calls or routine O&M. Log on to the RAM console to create a RAM user.
// This example shows how to store the AccessKey and AccessKey secret in environment variables. You can also store them in a configuration file as needed.
// Do not save the AccessKey and AccessKey secret in your code. This prevents key leaks.
private static String accessId = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID");
private static String accessKey = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET");
private static final String endPoint = "http://service.odps.aliyun.com/api";
private static final String project = "";
private static final String sql = "select category from iris;";
public static void
main(String[] args) {
Account account = new AliyunAccount(accessId, accessKey);
Odps odps = new Odps(account);
odps.setEndpoint(endPoint);
odps.setDefaultProject(project);
Instance i;
try {
i = SQLTask.run(odps, sql);
i.waitForSuccess();
List<Record> records = SQLTask.getResult(i);
for(Record r:records){
System.out.println(r.get(0).toString());
}
} catch (OdpsException e) {
e.printStackTrace();
}
}
}Envie e execute apenas uma instrução SQL por vez.
Para crie uma tabela, use a interface SQLTask em vez da interface Table. Passe a instrução de table operation para a interface SQLTask.
Instances
Representa uma coleção de todas as instâncias no MaxCompute. Os elementos dessa coleção são objetos Instance. O código a seguir mostra um exemplo.
// An AccessKey for an Alibaba Cloud account has all API access permissions, which poses a high security threat. Create and use a Resource Access Management (RAM) user for API calls or routine O&M. Log on to the RAM console to create a RAM user.
// This example shows how to store the AccessKey and AccessKey secret in environment variables. You can also store them in a configuration file as needed.
// Do not save the AccessKey and AccessKey secret in your code. This prevents key leaks.
Account account = new AliyunAccount(System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID"), System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET"));
Odps odps = new Odps(account);
String odpsUrl = "<your odps endpoint>";
odps.setEndpoint(odpsUrl);
odps.setDefaultProject("my_project");
for (Instance i : odps.instances()) {
....
}Instance
Representa uma instância e suas informações. Obtenha a instância correspondente na coleção Instances. O código a seguir mostra um exemplo.
// An AccessKey for an Alibaba Cloud account has all API access permissions, which poses a high security threat. Create and use a Resource Access Management (RAM) user for API calls or routine O&M. Log on to the RAM console to create a RAM user.
// This example shows how to store the AccessKey and AccessKey secret in environment variables. You can also store them in a configuration file as needed.
// Do not save the AccessKey and AccessKey secret in your code. This prevents key leaks.
Account account = new AliyunAccount(System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID"), System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET"));
Odps odps = new Odps(account);
String odpsUrl = "<your odps endpoint>";
odps.setEndpoint(odpsUrl);
Instance instance= odps.instances().get("instance id");
Date startTime = instance.getStartTime();
Date endTime = instance.getEndTime();
...
Status instanceStatus = instance.getStatus();
String instanceStatusStr = null;
if (instanceStatus == Status.TERMINATED) {
instanceStatusStr = TaskStatus.Status.SUCCESS.toString();
Map<String, TaskStatus> taskStatus = instance.getTaskStatus();
for (Entry<String, TaskStatus> status : taskStatus.entrySet()) {
if (status.getValue().getStatus() != TaskStatus.Status.SUCCESS) {
instanceStatusStr = status.getValue().getStatus().toString();
break;
}
}
} else {
instanceStatusStr = instanceStatus.toString();
}
...
TaskSummary summary = instance.getTaskSummary("task name");
String s = summary.getSummaryText();Tables
Representa uma coleção de todas as tabelas no MaxCompute. Os elementos dessa coleção são objetos Table. O código a seguir mostra um exemplo.
// An AccessKey for an Alibaba Cloud account has all API access permissions, which poses a high security threat. Create and use a Resource Access Management (RAM) user for API calls or routine O&M. Log on to the RAM console to create a RAM user.
// This example shows how to store the AccessKey and AccessKey secret in environment variables. You can also store them in a configuration file as needed.
// Do not save the AccessKey and AccessKey secret in your code. This prevents key leaks.
Account account = new AliyunAccount(System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID"), System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET"));
Odps odps = new Odps(account);
String odpsUrl = "<your odps endpoint>";
odps.setEndpoint(odpsUrl);
odps.setDefaultProject("my_project");
for (Table t : odps.tables()) {
....
}Table
Representa uma tabela e suas informações. Obtenha a tabela correspondente na coleção Tables. O código a seguir mostra um exemplo.
// An AccessKey for an Alibaba Cloud account has all API access permissions, which poses a high security threat. Create and use a Resource Access Management (RAM) user for API calls or routine O&M. Log on to the RAM console to create a RAM user.
// This example shows how to store the AccessKey and AccessKey secret in environment variables. You can also store them in a configuration file as needed.
// Do not save the AccessKey and AccessKey secret in your code. This prevents key leaks.
Account account = new AliyunAccount(System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID"), System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET"));
Odps odps = new Odps(account);
String odpsUrl = "<your odps endpoint>";
odps.setEndpoint(odpsUrl);
odps.setDefaultProject("my_project");
Table table = odps.tables().get("tablename");
for(Column c : table.getSchema().getColumns())
{
String name = c.getName();
TypeInfo type = c.getTypeInfo();
}Use o código a seguir para obter dados de partição da tabela.
// An AccessKey for an Alibaba Cloud account has all API access permissions, which poses a high security threat. Create and use a Resource Access Management (RAM) user for API calls or routine O&M. Log on to the RAM console to create a RAM user.
// This example shows how to store the AccessKey and AccessKey secret in environment variables. You can also store them in a configuration file as needed.
// Do not save the AccessKey and AccessKey secret in your code. This prevents key leaks.
Account account = new AliyunAccount(System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID"), System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET"));
Odps odps = new Odps(account);
String odpsUrl = "<your odps endpoint>";
odps.setEndpoint(odpsUrl);
Table t = odps.tables().get("table name");
t.reload();
Partition part = t.getPartition(new PartitionSpec("partition_col=partition_col_value"));
part.reload();
...Resources
Representa uma coleção de todos os recursos no MaxCompute. Os elementos dessa coleção são objetos Resource. O código a seguir mostra um exemplo.
// An AccessKey for an Alibaba Cloud account has all API access permissions, which poses a high security threat. Create and use a Resource Access Management (RAM) user for API calls or routine O&M. Log on to the RAM console to create a RAM user.
// This example shows how to store the AccessKey and AccessKey secret in environment variables. You can also store them in a configuration file as needed.
// Do not save the AccessKey and AccessKey secret in your code. This prevents key leaks.
Account account = new AliyunAccount(System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID"), System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET"));
Odps odps = new Odps(account);
String odpsUrl = "<your odps endpoint>";
odps.setEndpoint(odpsUrl);
odps.setDefaultProject("my_project");
for (Resource r : odps.resources()) {
....
}Resource
Representa um recurso e suas informações. Obtenha o recurso correspondente na coleção Resources. O código a seguir mostra um exemplo.
// An AccessKey for an Alibaba Cloud account has all API access permissions, which poses a high security threat. Create and use a Resource Access Management (RAM) user for API calls or routine O&M. Log on to the RAM console to create a RAM user.
// This example shows how to store the AccessKey and AccessKey secret in environment variables. You can also store them in a configuration file as needed.
// Do not save the AccessKey and AccessKey secret in your code. This prevents key leaks.
Account account = new AliyunAccount(System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID"), System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET"));
Odps odps = new Odps(account);
String odpsUrl = "<your odps endpoint>";
odps.setEndpoint(odpsUrl);
Resource r = odps.resources().get("resource name");
r.reload();
if (r.getType() == Resource.Type.TABLE) {
TableResource tr = new TableResource(r);
String tableSource = tr.getSourceTable().getProject() + "."
+ tr.getSourceTable().getName();
if (tr.getSourceTablePartition() != null) {
tableSource += " partition(" + tr.getSourceTablePartition().toString()
+ ")";
}
....
}O exemplo a seguir mostra como crie um recurso de arquivo.
String projectName = "my_porject";
String source = "my_local_file.txt";
File file = new File(source);
InputStream is = new FileInputStream(file);
FileResource resource = new FileResource();
String name = file.getName();
resource.setName(name);
odps.resources().create(projectName, resource, is);O exemplo a seguir mostra como crie um recurso de tabela.
TableResource resource = new TableResource(tableName, tablePrj, partitionSpec);
//resource.setName(INVALID_USER_TABLE);
resource.setName("table_resource_name");
odps.resources().update(projectName, resource);Functions
Representa uma coleção de todas as funções no MaxCompute. Os elementos dessa coleção são objetos Function. O código a seguir mostra um exemplo.
// An AccessKey for an Alibaba Cloud account has all API access permissions, which poses a high security threat. Create and use a Resource Access Management (RAM) user for API calls or routine O&M. Log on to the RAM console to create a RAM user.
// This example shows how to store the AccessKey and AccessKey secret in environment variables. You can also store them in a configuration file as needed.
// Do not save the AccessKey and AccessKey secret in your code. This prevents key leaks.
Account account = new AliyunAccount(System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID"), System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET"));
Odps odps = new Odps(account);
String odpsUrl = "<your odps endpoint>";
odps.setEndpoint(odpsUrl);
odps.setDefaultProject("my_project");
for (Function f : odps.functions()) {
....
} Function
Representa uma função e suas informações. Obtenha a função correspondente na coleção Functions. O código a seguir mostra um exemplo.
// An AccessKey for an Alibaba Cloud account has all API access permissions, which poses a high security threat. Create and use a Resource Access Management (RAM) user for API calls or routine O&M. Log on to the RAM console to create a RAM user.
// This example shows how to store the AccessKey and AccessKey secret in environment variables. You can also store them in a configuration file as needed.
// Do not save the AccessKey and AccessKey secret in your code. This prevents key leaks.
Account account = new AliyunAccount(System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID"), System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET"));
Odps odps = new Odps(account);
String odpsUrl = "<your odps endpoint>";
odps.setEndpoint(odpsUrl);
Function f = odps.functions().get("function name");
List<Resource> resources = f.getResources(); O exemplo a seguir mostra como crie uma função.
String resources = "xxx:xxx";
String classType = "com.aliyun.odps.mapred.open.example.WordCount";
ArrayList<String> resourceList = new ArrayList<String>();
for (String r : resources.split(":")) {
resourceList.add(r);
}
Function func = new Function();
func.setName(name);
func.setClassType(classType);
func.setResources(resourceList);
odps.functions().create(projectName, func); Referências
Para usar Python na interação com o MaxCompute e no processamento de dados, consulte Python SDK overview.