Data service (OneService) represents the culmination of building a data mid-end with Dataphin, serving as a unified outlet for data services. It simplifies data market management and lowers the barriers to data sharing while maintaining data security.
Prerequisites
Ensure the data service value-added service is purchased and the current tenant has activated the data service module.
Function description
With the Dataphin data source feature now unpublished, the Dataphin JDBC Driver serves as an alternative. For more information, see Connect to Dataphin via JDBC.
Common data application issues
The standard process from requirement submission to delivery typically involves the following stages: requirement submission, analysis, product design and development documentation, development, functional testing, deployment, and finally, delivery. This process, which often spans about two weeks, requires iterations of development and testing. The interfaces developed are highly specific, catering mostly to individual business analysis needs. Once development is complete, monitoring the data interfaces' usage becomes challenging. Adjustments to business analysis necessitate the submission of new requirements and scheduling of iteration development, leading to several data application challenges:
Extended data development cycles and slow responsiveness: The process demands iteration scheduling, and the high threshold for data usage hinders immediate application.
Chimney-style development: Characterized by high development costs and low reusability, this approach provides specific data interfaces for each requirement.
Compromised data security and query performance: Development often prioritizes data accessibility over the security of data resources and the stability of data access.
Elevated maintenance costs: The absence of online platform management means that abnormal situations cannot be monitored or alerted, and online business issues are only addressed after code troubleshooting and fixes are deployed.
Data service value
To address these common data application issues, the data service system offers developers two modes-simple wizard and flexible script-to reduce the development threshold, enhance efficiency, standardize code quality, and streamline change management. Business users benefit from secure and stable data resource access, preventing sensitive data exposure at the application level. Moreover, data service includes monitoring and alerting features, enabling development and operations teams to quickly detect and resolve issues based on abnormal data patterns or alert notifications. The key benefits of data service products include:
Data service creation capabilities:
Support for associating single or multiple physical tables to create a logical table, enabling API development based on the logical table.
Wizard and script mode creation support: Simple queries can utilize wizard mode for API creation, while custom SQL can be crafted through scripts for complex joins, conditions, and data processing logic.
Data service management and operations capabilities:
Isolation of draft, development, and production environments: Development environment tests use development data, while production tests use production data.
Authentication, asset change ownership, and other service management features: Manage service call requests and data query authentication, and facilitate asset ownership transfers.
Configuration of service timeout, failure, and rate limit alerts: Implement alerts via DingTalk robots, email, text messages, and phone calls.
Monitoring of abnormal service call statistics: Track services and applications with unusual call patterns and trends.
Detailed service call log queries: Access comprehensive error information and specific SQL execution details.
Data service advantages
Unified interface standards: Data service provides a consistent data interface standard and service metadata, streamlining interface development, reducing the integration effort for downstream applications, and enhancing data access efficiency.
Data security and compliance assurance: Minimize the storage and exposure of detailed and sensitive data on the application side. Employ a unified platform data security control policy, including API request calls, authentication, flow control, and whitelisting, to reduce data security management costs.
Service monitoring: Support for data service call statistics and detailed call log queries, enabling operations and development teams to efficiently troubleshoot and address anomalies in data service calls.
Through the data service platform, you can create interfaces, call and manage services, and oversee the entire service lifecycle. The platform's simple configuration tools automatically produce and deploy data services, significantly boosting efficiency.
How it works
Data services enable the creation of service project groups from physical or logical tables generated by Dataphin, catering to various business scenarios. Applications can then invoke APIs as needed. Furthermore, data services offer Manage service monitoring API to assist in promptly identifying and resolving any issues that arise during API invocation.
Data services support data source types that read from physical tables. For more information, see data sources supported by Dataphin.
Scenarios
Data service is ideally suited for the following application scenarios:
Building an enterprise API ecosystem, such as providing open APIs to partners and developers, to facilitate the monetization and value realization of data assets.
Using Dataphin data sources for Quick BI to display, analyze, explore, and create business data reports.
Scenarios and procedures
Before utilizing data service, it is important to understand the service scenarios and the operational procedures for different roles within these scenarios.