Maxcompute source connector

Description

Used to read data from Maxcompute.

Key features

Options

name type required default value
accessId string yes -
accesskey string yes -
endpoint string yes -
project string yes -
table_name string yes -
partition_spec string no -
split_row int no 10000
common-options string no
schema config no

accessId [string]

accessId Your Maxcompute accessId which cloud be access from Alibaba Cloud.

accesskey [string]

accesskey Your Maxcompute accessKey which cloud be access from Alibaba Cloud.

endpoint [string]

endpoint Your Maxcompute endpoint start with http.

project [string]

project Your Maxcompute project which is created in Alibaba Cloud.

table_name [string]

table_name Target Maxcompute table name eg: fake.

partition_spec [string]

partition_spec This spec of Maxcompute partition table eg:ds=’20220101’.

split_row [int]

split_row Number of rows per split, default: 10000.

common options

Source plugin common parameters, please refer to Source Common Options for details.

schema [config]

fields [Config]

The schema information of upstream data.

Examples

  1. source {
  2.   Maxcompute {
  3.     accessId="<your access id>"
  4.     accesskey="<your access Key>"
  5.     endpoint="<http://service.odps.aliyun.com/api>"
  6.     project="<your project>"
  7.     table_name="<your table name>"
  8.     #partition_spec="<your partition spec>"
  9.     #split_row = 10000
  10.     schema {
  11.         fields {
  12.             name = string
  13.             age = int
  14.             gender = string 
  15.         }
  16.     }
  17.   }
  18. }

Changelog

next version

  • [Feature] Add Maxcompute Source Connector(3640)
  • [Feature] Support Schema in MaxCompute Source(3640)