DataWorks:Memcache (OCS) - DataWorks

The Memcache (formerly OCS) data source lets you write data from other data sources to Memcache using DataWorks Data Integration. Configure sync tasks only in the code editor — wizard mode is not supported.

Limitations

Code editor only: Configure sync tasks using the code editor. Wizard mode is not available.

Supported write format

Memcache Writer supports one write format: text.

In text format, the first field in each source row becomes the Memcache key. All subsequent fields are serialized to string and joined with the fieldDelimiter you configure, forming the value.

Example: Given the following source data with fieldDelimiter set to \^:

ID	NAME	COUNT
23	CDP	100

The data is written to Memcache as:

KEY (OCS)	VALUE (OCS)
23	CDP\^100

Binary format is not currently supported.

Supported write modes

Only set is currently supported. The other write modes are not yet available in Memcache Writer.

Write mode	Behavior	Supported
`set`	Stores the data unconditionally	Yes
`add`	Stores the data only if the key does not exist	No
`replace`	Stores the data only if the key exists	No
`append`	Appends the data to an existing key's value (`expireTime` is ignored)	No
`prepend`	Prepends the data to an existing key's value (`expireTime` is ignored)	No

Add a data source

Before developing a synchronization task in DataWorks, add the Memcache data source by following the instructions in Data source management. View parameter descriptions in the DataWorks console to understand each parameter when adding a data source.

Configure a synchronization task

For the procedure, see Configure a task in the code editor.

For the script reference and all parameters, see Appendix: Script demo and parameter reference below.

Appendix: Script demo and parameter reference

Script demo

The following script shows a complete Memcache Writer configuration. All writer parameters go under steps[].parameter for the step where stepType is ocs.

{
    "type": "job",
    "version": "2.0",
    "steps": [
        {
            "stepType": "stream",
            "parameter": {},
            "name": "Reader",
            "category": "reader"
        },
        {
            "stepType": "ocs",
            "parameter": {
                "writeFormat": "text",
                "expireTime": 1000,
                "indexes": 0,
                "datasource": "",
                "writeMode": "set",
                "batchSize": "256"
            },
            "name": "Writer",
            "category": "writer"
        }
    ],
    "setting": {
        "errorLimit": {
            "record": "0"
        },
        "speed": {
            "throttle": true,
            "concurrent": 1,
            "mbps": "12"
        }
    },
    "order": {
        "hops": [
            {
                "from": "Reader",
                "to": "Writer"
            }
        ]
    }
}

Writer parameters

Parameter	Description	Required	Default
`datasource`	The name of the data source. The value must match the name of the data source added in DataWorks.	Yes	None
`writeMode`	The write mode. Only `set` is currently supported. See Supported write modes.	Yes	None
`writeFormat`	The write format. Only `text` is supported. In text format, the first field becomes the Memcache key and the remaining fields are concatenated with `fieldDelimiter` to form the value.	No	None
`expireTime`	The time-to-live (TTL) for cached values. Accepts either a Unix timestamp (a future point in time when the data expires) or a duration in seconds relative to the current time. If the value exceeds 2,592,000 seconds (30 days), the server treats it as a Unix timestamp. Set to `0` to keep data indefinitely.	No	`0`
`batchSize`	The number of records per batch write. A higher value reduces network round trips and improves throughput, but setting it too high risks an out-of-memory (OOM) error in the sync process.	No	`1,024`