Basic modeling concepts - Cloud Monitor - Alibaba Cloud Documentation Center

UModel is a unified framework for modeling observable data that provides a comprehensive set of syntax and concepts. This topic explains the basic modeling concepts of UModel to help you understand data modeling.

The core modeling philosophy of UModel is based on the following basic concepts:

Field: The basic unit and property of data.
Node: An abstraction of data, entities, or storage.
Link: Links establish relationships between entities, and between entities and data.

Node

UModel uses the basic concepts of graph theory. It uses Nodes and Links (edges) to form a directed graph that describes an IT system. Nodes are classified into the following types:

EntitySet: A collection of entities that have the same properties.
TelemetryDataSet: A collection of observable data that has the same properties.
Storage: An abstraction of storage, such as SLS, Prometheus, or MySQL.

Definition of a Set

A Set is a special type of Node. It is an abstract collection of data or entities that share the same properties. A Set is similar to a table in a database or a class in a programming language. It provides a unified schema for data of the same type.

Core set types

UModel defines two core Set types:

1. EntitySet

An EntitySet is a collection of entities that have the same properties.

Key features:

Entity lifecycle management: Supports tracking of first observed time, last observed time, and includes a keepalive mechanism.
Primary key definition: Uniquely identifies an entity using primary_key_fields.
Dynamic entity support: Supports both static storage and dynamic generation modes.
Relationship modeling: Establishes relationships with other EntitySets and TelemetryDataSets.

Core configuration fields:

spec:
  primary_key_fields: ["field1", "field2"]  # A list of primary key fields
  first_observed_time_field: "first_seen"   # The field for the first observed time
  last_observed_time_field: "last_seen"     # The field for the last observed time
  keep_alive_seconds: 3600                  # The keepalive duration in seconds
  dynamic: false                            # Specifies whether the entity is dynamic

Scenarios:

IT infrastructure entities: Hosts, containers, and network devices.
Application layer entities: Services, interfaces, and database instances.
Business entities: Users, orders, and products.
Operations and Maintenance (O&M) entities: O&M engineers, code repositories, and deployment environments.

2. TelemetryDataSet

A TelemetryDataSet is a general representation of observable data. It is structurally similar to a database table but includes a time dimension and other features specific to observable data.

Core features:

Time dimension: All observable data includes a time field.
Field management: Supports the classification and management of displayed fields, hidden fields, and tag fields.
Sorting support: Defines the default sorting field and direction.
Inheritance mechanism: Other specific dataset types inherit from this base type.

Basic configuration:

spec:
  fields: []                    # A list of field definitions
  time_field: "timestamp"       # The time field
  name_fields: []               # The display name field
  tag_fields: []                # The tag field
  hidden_fields: []             # The hidden field
  ordered_fields: []            # The sorting field
  default_order: "asc"          # The default sorting direction

3. Specific dataset types

Based on TelemetryDataSet, UModel defines the following specific dataset types:

MetricSet

Defines a collection of metrics.
Supports tag management and metric aggregation.
Includes query syntax types (prom, spl, cms).

spec:
  labels:                       # The metric tag configuration
    keys: []                    # A list of tag fields
    dynamic: true               # Dynamic tag generation
  metrics: []                   # A list of detailed metrics
  query_type: "prom"            # The query syntax type

LogSet

Represents a collection of log data.
Inherits the basic capabilities of TelemetryDataSet.
Supports flexible field definitions and filtering.

TraceSet

Defines distributed tracing data.
Requires the following trace fields: trace_id, span_id, and parent_span_id.
Supports different tracing protocols. The default is opentelemetry.

spec:
  trace_id_field: "traceId"         # The TraceID field name
  span_id_field: "spanId"           # The SpanID field name  
  parent_span_id_field: "parentSpanId"  # The parent SpanID field name
  protocol: "opentelemetry"         # The tracing protocol

EventSet

Defines a collection of event data.
Inherits the basic capabilities of TelemetryDataSet.
Suitable for scenarios such as alert events and system events.

ProfileSet

Defines performance profiling data.
Supports multiple profiling protocols, with pprof as the default.
Suitable for performance analysis scenarios, such as CPU and memory analysis.

Definition of storage

Storage is an abstract concept in UModel that defines the physical storage of data. Storage decouples data modeling from specific storage implementations. It supports multiple storage types and provides a unified storage interface.

Core storage types

A Simple Log Service Logstore stores log data and provides powerful log search and analysis capabilities.
An SLS MetricStore stores time series metrics and provides high-performance metric query and aggregation capabilities.
Managed Service for Prometheus stores Prometheus metric data. It provides standard Prometheus metric storage and is compatible with the Prometheus Query Language (PromQL) syntax.
SLS EntityStore is the default storage class for EntitySet and EntitySetLink data. It is designed to store observable entities and relationship data, and provides graph query and relationship analysis capabilities.

Link

Definition of a link

A Link is a core concept in UModel used to define relationships between different Sets. Links establish relationships between entities, between entities and data, and between data and storage to build a complete relationship diagram of observable data.

General properties of a link

All Link types include the following general properties:

spec:
  src:                                # The source Set definition
    domain: "example.com"
    kind: "entity_set"
    name: "host"
  dest:                               # The destination Set definition  
    domain: "example.com"
    kind: "metric_set"
    name: "host_metrics"

Core link types

EntitySetLink defines the relationship between two EntitySets to build a topology between entities.
DataLink defines the relationship between an EntitySet/Link and a DataSet to associate entities with observable data.
StorageLink defines the relationship between an EntitySet/DataSet and Storage to decouple data modeling from physical storage.

Storage

Definition of storage

Core storage types

1. Simple Log Service Logstore

A Simple Log Service (SLS) Logstore stores log data and provides powerful log search and analysis capabilities.

Key features:

Region and project management: Supports storage configurations across multiple regions and projects.
Filter optimization: Supports multiple filtering methods, such as search_filter, spl_filter, and sql_filter.
View support: Supports Structured Process Language (SPL) and SQL views to dynamically generate data.

Core configuration:

spec:
  region: "cn-beijing"               # The storage region
  project: "my-project"              # The SLS project
  store: "app-logs"                  # The Logstore name
  search_filter: "*"                 # The search filter condition
  spl_filter: "status: 200"          # The SPL filter condition
  spl_view: "* | where status > 400" # The SPL view definition

2. SLS MetricStore

An SLS MetricStore stores time series metrics and provides high-performance metric query and aggregation capabilities.

Key features:

Time series optimization: Optimizes storage and queries for time series data.
Multi-metric support: Supports multiple metrics in the same store.
High-concurrency queries: Supports large-scale concurrent metric queries.

Core configuration:

spec:
  region: "cn-beijing"               # The storage region
  project: "my-project"              # The SLS project
  store: "app-metrics"               # The MetricStore name

3. SLS EntityStore

An SLS EntityStore is designed to store observable entities and relational data. It provides graph query and relationship analysis capabilities.

Key features:

Entity management: Optimized specifically for entity data.
Relationship storage: Supports the storage and query of relationships between entities.
Workspace isolation: Supports workspace isolation for multitenancy.

Core configuration:

spec:
  region: "cn-beijing"               # The storage region
  project: "my-project"              # The SLS project  
  workspace: "my-workspace"          # The workspace

4. Managed Service for Prometheus storage

Managed Service for Prometheus provides standard Prometheus metric storage and is compatible with PromQL syntax.

Key features:

Prometheus compatibility: Fully compatible with the Prometheus protocol and PromQL.
High availability architecture: Provides enterprise-grade high availability (HA) and disaster recovery capabilities.
SLS integration: Built on SLS storage to provide a unified observability experience.

Core configuration:

spec:
  region: "cn-beijing"               # The instance region
  instance_id: "prometheus-xxx"      # The Prometheus instance ID
  sls_project: "prometheus-project"  # The underlying SLS project
  sls_metricstore: "prometheus-data" # The underlying SLS MetricStore