All Products
Search

This Product

    Dataphin:Variable classification and assignment

    Document Center

    Dataphin:Variable classification and assignment

    Last Updated:Jan 21, 2025

    This topic outlines the variable classifications within Dataphin and details the process for assigning date and time values to local variables.

    Variable classification description

    Dataphin categorizes variables as follows:

    Variable Type

    Description

    Global Variable

    Global variables are predefined and established in Dataphin Planning- Common Business Logic-Global Variable, accessible across any node or code within the tenant. The system's default global variable is ${bizdate}.

    Note

    • Global variables are not compatible with sync tasks in current versions (supported in earlier versions).

    • Global variable settings. For specific operations, see Create a global variable.

    Space Name Variable

    These include ${project name} and ${module name}, as illustrated in the calculation task example below.

    image.png

    1. Retrieve the project or module name at ①.

    2. In the editor at ②, reference the project or module name using the ${project name or module name} syntax.

    3. The input parameter at ③ will be automatically identified by the system.

    Note

    Space name variables cannot be converted into other variable types.

    Local Variable

    Local variables do not require prior declaration. Use ${variable_name} within the code to define a local variable named variable_name. The example below demonstrates this in a calculation task.

    image.png

    Variable names should consist of numbers, letters, and underscores only. Avoid starting with dp_glb_, which is reserved for global variables, and limit the name to 32 characters.

    Important

    • If variable_name coincides with a global variable name, it will be treated as a global variable reference. You can change its type to a local variable in the parameter settings.

    • Should variable_name match a project or module name, it will be automatically considered a space name variable.

    • To prevent ${variable_name} from being treated as a variable within code content, configure it as non-variable in the parameter type settings.

    • Should ${variable_name} be a cross-node variable, configure it as such by selecting cross-node variable in the parameter type settings.

    • To set a local variable's parameter value to empty, Parameter Value and then Set To Empty with a single click.

    Cross-node Variable

    Cross-node variables are parameters passed from an ancestor node to the current object node, applicable in scenarios such as:

    • An ancestor node A sends a variable ${var} to a descendant node B, which then uses ${var} in its calculation logic. For instance, in finance, exchange rates stored in a physical table may be read by a Python node and passed to downstream nodes for calculations.

    • An ancestor node A transmits a variable ${var} to descendant node B, which bases its running policy on ${var}. For example, if a remote service's status is "stopped," node B might perform a dry-run, necessitating A to pass the status as a variable to B.

    Important

    • Cross-node variables are not supported in real-time tasks.

    • When backfilling data for a node that uses cross-node parameters, ensure to backfill data for both the ancestor node providing the parameters and the current node.

    Variable assignment description

    Date and time calculation expression

    For assigning date and time values to local variables, Dataphin supports the following quick date calculation operators:

    • Dataphin enables calculations between two date and time values:

      • For business date ${bizdate}, use the format: ${(yyyyMMdd|yyyy/MM/dd|yyyy-MM-dd) (+|-) n}.

      • For the expected running time (the node task's start time), use: $[(yyyyMMddHHmmss|yyyy/MM/dd HH:mm:ss|yyyy-MM-dd HH:mm:ss) (+|-) n].

    • Identifiers for date and time components:

      • Year: yyyy.

      • Month: mm for day-level granularity or MM for minute-level granularity (uppercase to differentiate from minute mm).

      • Day: dd.

      • Hour: HH.

      • Minute: mm.

      • Second: ss.

    • The interval unit for date and time calculations is one day:

      • Represent 1 hour as 1/24, 5 hours as 5/24.

      • Represent 1 minute as 1/1440 (1/24/60), 30 minutes as 30/1440.

    Example reference

    Current Date

    Configuration Based on Business Date (T-1)

    Actual Value

    Configuration Based on Expected Running Time (T)

    Actual Value

    2018-01-10

    ${yyyymmdd}

    20180109

    $[yyyymmdd]

    20180110

    2018-01-10

    ${yyyyMMdd}

    20180109

    $[yyyyMMdd]

    20180110

    2018-01-10

    ${yyyymmdd -1}

    20180108

    $[yyyymmdd -1]

    20180109

    2018-01-10

    ${yyyymmdd -7}

    20180102

    $[yyyymmdd -7]

    20180103

    2018-01-10

    ${yyyy-mm-dd}

    2018-01-09

    $[yyyy-mm-dd]

    2018-01-10

    2018-01-10

    ${yyyy-mm}

    2018-01

    $[yyyy-mm]

    2018-01

    2018-01-10

    ${yyyy-mm -1}

    2017-12

    $[yyyy-mm -1]

    2017-12

    2018-01-10

    ${yyyy}

    2018

    $[yyyy]

    2018

    2018-01-10

    ${yyyy+1}

    2019

    $[yyyy + 1]

    2019

    2018-01-10

    ${yyyy/MM/dd HH:mm:ss}

    2018/01/09 00:00:00

    $[yyyy/MM/dd HH:mm:ss]

    2018/01/10 00:00:00

    2018-01-10

    ${yyyy/MM/dd HH:mm:ss-1/24}

    2018/01/08 23:00:00

    $[HH:mm:ss-5/24/60]

    23:55:00