Data center project management practice sharing

Introduction: This article summarizes the practical experience of enterprise-level data middle-end projects, hoping to provide experience for enterprises and individuals who are planning or have already implemented data middle-end projects.
Author: Wu Jianhong (Ji Yun)

【 Data Center Project Management 】 Introduction

Alibaba Cloud Data Center is an enterprise-level solution that includes implementation methodology, platform products and technical services. Alibaba Cloud Data Center uses big data computing platforms such as Maxcompute as the carrier, and uses three Ones as the theoretical basis to form a data center methodology, which realizes the management of the entire data life cycle in one platform.

This article summarizes the practical experience of enterprise-level data middle-end projects, hoping to provide experience for enterprises and individuals who are planning or have already implemented data middle-end projects.

The overall management picture and implementation process of Alibaba Cloud Data Middle-Taiwan projects can be summarized as the following big picture:

Data Center Project Management Practice Sharing (1) Project Launch

The data middle-end project is an enterprise-level project. At the beginning of the construction of each data middle-end project, a comprehensive and relatively comprehensive planning is required to avoid the 'single chimney' approach to building the middle-end.

The start-up phase is extremely important. Most of the planning and planning are produced in this phase. It is recommended that this phase should account for 15% of the entire project planning time. If the project plan is not adequately planned, the project implementation may be a process of filling holes. At the beginning of the project, you can follow 4 steps:

1. Set goals
2. Set up a team
3. Make a plan
4. Statutes

Before the start of the data middle platform project, the original intention and goals of the enterprise to build the middle platform need to be considered. Understand the company's current strategy, investigate the departments and department goals involved in each data middle-office scenario, and the connectivity between departments and scenarios. This will help to realize the integrated construction of the data center, clarify the goal of the data center construction, and avoid the rework of the follow-up work.

Based on corporate goals and strategies, disassemble the goals and KPIs of each department. When planning the data center, consider how to analyze, evaluate and evaluate through data, and display goals and progress through visualization. When investigating project goals, the project team needs to focus on:

1. What kind of data support are required for different roles in the enterprise, and where are these data distributed? Where is the data going? What is the original intention of the management to build a data center, and what data are they paying attention to?

For example, the original intention of some enterprises to build a data center is to conduct data governance and to unify the indicators that are currently inconsistent. If we can know which indicators are the biggest pain points of management, we can prioritize governance and meet some of the needs of management in advance. The construction of an enterprise-level data center must be supported by the enterprise-level management, and a data-based project is often a project with great long-term value but a boring process. Therefore, it is particularly important to continuously reflect the construction highlights of the project to the leadership.

2. How will the data of enterprise customers be used, and how to build the corresponding structure in terms of technical implementation?
For example, real-time and non-real-time scenarios, which also determine or affect the architecture of subsequent cloud migration.

3. What business processes are involved in these data?

In addition to clarifying the goals of the project, it is also necessary to consider the constraints of the contract during the implementation process, such as whether there are time constraints, the amount of work invested, whether to train employees, etc. Some detail factors can also have an impact on the project. For example, if the employee appraisal is on December 31 at the end of the year, it is best for the project to have a better output in early December in order to meet the performance appraisal of the project participants.

Through the above comprehensive consideration, the goal of the data center project and the sub-project goals of each scenario can be set.

set team
Large enterprise customers are particularly concerned about the project organization and division of labor. The data middle-end project is an enterprise-level project. A successful data middle-end project team must have the close participation of Party A's core management, business and technical parties. In many projects, due to the inability of the Party A team to participate deeply or the lack of roles, the coordination is insufficient, resulting in uncontrollable progress and quality. Especially for government and large enterprise projects, the most difficult thing to deal with is the relationship within the organization. The drawing of the organizational chart needs to think about how to achieve a horizontal level and meet the purpose of promoting the project.

It is recommended to set up a project management committee ( Project Control Borad , hereinafter referred to as PCB) for enterprise-level projects, which is participated by the core management of Party A and the core management of Party B. The role of the PCB is to define the goals of the project, resolve internal disagreements, and provide decision support when the project requires decisions. If the PCB is missing, when Party A participates in the project with multiple departments, it is easy to make the problem difficult to mediate due to the conflict of interest between the departments.

A common organizational structure in large enterprises is that the IT department is the contracting party for IT projects, but the leading department is the data department. The demands of the IT department and the data department for the project may even conflict. The structural design of the project team must fully consider the demands of each team, and under the general direction of seeking common ground while reserving differences, ensure that the big goals are consistent, so that each team is in a suitable position. To this end, on the basis of the traditional role, it is recommended to add the role of Product Owner. You can try to use the IT department as the PM. Data projects involve more internal processes of the IT department. It is more smooth for the PM of the IT department to coordinate the process, such as data permission activation, product permission activation, etc. The Product Owner can manage requirements and the priority of requirements.

Project role positioning

client side role

In the process of project delivery, the cooperation of the client is particularly important, so the role of the client is particularly important.

Customer demand decision maker Project Owner
1. Product Requirements Manager
2. Divergence between unified requirements
3. Iteratively define product and requirement priorities

Client project experience Project Manager
1. Solve the daily blocker of the team, focusing on solving all problems on the customer side.
2. Ensure that each iteration is completed to the maximum extent and be responsible for the overall progress.
3. Inform the customer of the required process needs to be quantifiable, testable and executable.
4. Organize daily standing meetings, weekly meetings and other regular meetings.

Customer business person in charge
1. Coordinate the business needs of customers in each scenario.
2. Define the Definition of Done for business requirements (such as indicator business logic).
3. Verify and accept the cloud migration results. ( Note: The quality results of the cloud data need to be verified by the business side from the beginning. During the process of project advancement, inaccurate indicators often occur due to lack of source data or substandard quality)
4. Validation and acceptance criteria.

Customer business partner
1. The maker of the customer's business needs.
2. Define the Definition of Done for business requirements (such as indicator business logic).
3. Verify and accept the cloud migration results.
4. Validation and acceptance criteria.

Customer Technical Lead (Customer TM)
1. Responsible for the overall delivery quality and the quality of each iteration .
2. Inform and assist customers in quality and management processes.
3. Coordinate data inventory and data migration to the cloud.

Customer technical implementer
1. Data inventory and data migration to the cloud.

Alibaba Cloud side role

To cooperate with it, Ali also needs to provide a five-in-one team to provide support:

Project Manager Project Manager
1. Solve the daily blocker of the team, focusing on solving all problems on the Alibaba side.
2. Ensure that each iteration is completed to the maximum extent and be responsible for the overall progress.
3. Organize daily standing meetings, weekly meetings and other regular meetings.

Architect Manager
1. Participate in business and data asset research, and organize data asset reports
2. Data model design
3. Facing the product development department, feedback product requirements and suggestions.

Technical Manager
1. Manage and carry out related development work, and be responsible for the overall delivery quality and the quality of each iteration.
2. Instruct technicians to use Ali products and comply with technical requirements such as development specifications.
3. Evaluate workload and allocate technical work reasonably.

Business AnalystBusiness Analyst
1. Be responsible for the overall consulting quality, and be responsible for refining the highlights of the project.
2. Summarize, empower and practice Data Alibaba's best practices and methodologies.

Product PD
1. Responsible for the design of visual display.
2. Ensure that the designed indicators can be implemented.
3. Responsible for internal self-test.

make a plan
Only after the project objectives and the project team are clear can the customization of the plan begin. The formulation of the project plan must be a rigorous, detailed, and collective process . The effect of a good plan is to allow everyone on the project team to go through everything that the project is about to experience in their minds . This is what Stephen Covey calls the first creation process in his book The Seven Habits of Highly Effective People.

In the process, many risks can often be foreseen. In many companies, many people are resistant to "creating a detailed plan" and prefer to start directly. In fact, this should not be done. When delivering ToB and ToG projects, if the preliminary planning is not done enough, it is likely to face challenges from customers. For example, customers may have the following problems:

1. Why is your plan different from the actual operation? How can I monitor your progress through the program?
2. A task in your plan lasted for two months. What does this task include?
3. From the original plan, we cannot see what Party A needs to cooperate with. Why do we often need urgent assistance from Party A?
4. Why does the project have the ability to anticipate risks?
5. What is the relationship between each item?

statutory law
There are rivers and lakes where there are people, especially the newly formed project team. Everyone comes from different teams and represents different interests. At the beginning of the project implementation, if the project team can be organized to jointly formulate the project charter, it will be of great help to the smooth implementation of the project. The purpose of creating a project charter is to agree on the rules of the game for multiple parties to work together to achieve the goal of jointly completing the project on the premise of satisfying their respective interests.

The project charter contains the project goals, team, and plan, as well as acceptance methods, prerequisites, and collaboration methods. At the same time, it is reminded that in order to make a charter with customers, it is necessary to have a good customer relationship as the basis, and only with a certain tacit understanding can we truly abide by it. Without human support, the project charter is worthless. Party A also needs to pay attention to the implementation of the project charter, which is also the protection of the cooperative relationship between Party A and Party B.

Data middle-end project management practice sharing (2) Demand research and design

The purpose of the requirements research and design stage is to undertake the products of the initial stage of the project, and to output detailed development and implementation requirements for the next stage of "technical implementation".

In order to speed up the implementation of the project, while doing the demand research, you can also simultaneously carry out the data uploading to the cloud and the design of the data architecture (public layer design) in the data center. The following 3 lines are available in parallel:

l The business line is responsible for business research
l The cloud line is responsible for uploading data to the cloud
l The architecture line is responsible for the design of the public layer data architecture

Business Line
Business research and industry best practices

A big difference in the implementation of Alibaba Cloud data middle-end projects is that Alibaba Cloud data middle-end projects are delivered based on business scenario-driven technologies. Each business scenario revolves around the establishment of an indicator/label system (hereinafter referred to as the indicator system) for the business scenario, and guides business operations through the indicator system to drive and realize the process of value creation.

The construction process of the indicator system is to sort out the existing indicators or indicator systems, and combine the industry or cross-industry (such as the Internet industry, new retail industry) with understanding and best practices to form a new set that can effectively guide business operations . index system. For the collection of the existing index system, Alibaba Cloud provides a series of templates, which allow Party A to collect and fill in based on daily experience.

For those who have not implemented the data middle-end project, they may not have a deep understanding of the relationship between the indicator/label system and operations, and do not understand how indicators/labels can play a role in operations. To give a related example, the AIPL marketing model commonly used in new retail is a model that quantitatively operates crowd assets. The details are as follows:

A (Awareness), brand awareness crowd. Including people who are reached by brand advertisements and searched by category terms;
I (Interest), the brand interest group. Including people who click on advertisements, browse brand/store homepages, participate in brand interaction, browse product detail pages, search for brand words, receive trial, subscribe/follow/join, and add to favorites;
P (Purchase), brand purchase crowd, refers to those who have purchased branded products;
L (Loyalty), brand loyalty crowd, including those who repurchase, comment, and share.

In the AIPL model, precise marketing can be carried out on the characteristics of each customer to effectively improve customer loyalty.

This is the process by which metrics and labels drive business value operations. There are 2 risks at this stage that are worth addressing in advance:

1. The leader of the mature standard industry has its own perfect operation method.
We have served a certain customer, and he is the largest industry leader in Asia. The industry in which he is located is highly process-oriented. As a delivery party, it is difficult for us to come up with any subversive indicator/labeling system.

2. The cycle of the new operation mode producing results is longer than the project construction cycle.
The construction cycle of a scene in the data center takes 6-12 months. Even if the client can be guided on how to operate, it is difficult to get the client to practice this mode of operation during the project cycle, because the change increases the client's incompatibility and uncertainty, often requiring suitable opportunities.

PRD design

In the research phase, the goal of the project is to output a large and comprehensive indicator/label system to help or inspire innovation on the customer's operational side. Therefore, the index system combed by the MRD link does not necessarily have to be fully developed and implemented. Some indicators/labels may not have a data foundation at present, but can be used as the direction of future enterprise data collection planning.

But in the PRD link, it is different. PRD considers the value of the indicators to determine the feasibility of the indicators, and design to display these indicators in a visual way.

After the PRD design link is completed, the scope of requirements of the project is theoretically clearer. At this time, it is recommended to produce a complete Product Backlog. What is expressed here is to agree with the customer, as the scope of requirements to be completed before the final acceptance, the priority that is full of requirements. The requirements summary table covers the MRD, PRD completed in the previous stage, the cloud migration list in this project, the public layer dimension and fact table construction list, the indicator/label list, etc. Only when the scope of requirements is clear and the priorities are clearly defined, can there be rules to follow in the subsequent development and avoid the spread of requirements.

data line
Data line, roughly divided into several steps

1. Determine the scope and priority of data inventory and cloud migration
2. Data inventory
3. Cloud architecture design and data migration to the cloud

Determine the scope and priority of data inventory and cloud migration

The goal of this stage is to explore the data required for each scenario, understand the systems in which these data are distributed, and produce a data inventory and cloud system list. It should be noted that this list not only needs to include the systems and tables that go to the cloud, but also the range of historical data refreshed to the cloud. The range of historical data refresh is based on how long the customer wants to see the data. For example, if a customer wants to see the comparison of sales in the past 2 years, the scope of the review must be more than 2 years.

Data inventory
According to the cloud system list, the data needed to be counted, the contents of the inventory include:

System process mapping table: Based on the business process, it lists the relationship between various business systems. Time limit requirements for mutual data access between systems.
Basic information of data source: Based on the system level, list the basic information of each business system, such as system type, database type, data volume, person in charge and other system-level information.
Data resource directory: Based on the table level, lists the content description, attribute information, cloud migration priority, etc. of each table.
Data dictionary: Based on the field level, the attributes and metadata information of each field are listed.

Note: The work of data inventory is not only for data cloud, but also some data governance work can be considered. For example, while conducting data inventory interviews, the scope of technical metadata and business metadata can also be investigated at the same time.

Cloud architecture design and data migration to the cloud
This stage is to design the cloud architecture based on the inventory data information and data usage requirements, and start cloud operations according to the architecture.

Architecture line
Architecture lines have two actions:

1. Sort out the business map of the company
2. Based on the business big picture, guide the construction of the public layer of the data center, that is, design the design of the fact table and dimension table.

The big picture of the data center business, focusing on the business actions based on business objects and the business objects involved in the process of business actions. The business action is reflected in the fact table in the middle platform, and the business object corresponds to the dimension table.

For example, a customer of an airline will buy a ticket, pay, and possibly refund the ticket. These are the business processes, and there is a record of the fact flow with relevant data, that is, the fact table. Regarding dimensions, it can be simply understood as which dimension /angle/object to analyze this fact table, such as the dimension of customers, the dimension of air tickets, the dimension of payment, etc.

When designing dimension tables and fact tables (public layer), data governance related issues need to be considered at the same time. In a previous project, the client questioned that the data of the public layer was somewhat biased. After review, it was found that there were two reasons for this:
Problem 1: Customer source data quality issues
Problem 2: Missing Links in Data Governance

The suggestion for problem 1 is that after the data is uploaded to the cloud, the business side starts to check the quality of the data, rather than troubleshooting after development. The quality of data on the cloud cannot be guaranteed, and no matter how accurate the calculation caliber can be, an accurate indicator/label cannot be obtained.

The process suggestion for problem 2 is to add the process of data governance during the implementation of the data center. The suggested process is as follows:

1. Design the data architecture of the public layer (dimension table and fact table) based on the business big picture.
2. Organize customers to review dimension tables and fact tables.
3. The customer information center completes data governance of technical metadata based on dimension tables and fact tables.
4. The customer business party completes the data governance of business metadata based on the dimension table and fact table.
5. The customer aggregates technical metadata and business metadata, and Alibaba Cloud develops it based on the content provided by the customer.

Data center project management practice sharing (3) Technology implementation

Traditional assembly line development

In the past, when doing data middle-end projects, the pipeline-type development method was used, and the next stage was only entered when there were clear and complete deliverables in the previous stage. For example, the design is only after the requirements are clear. Once the design is clear, development begins. After development is complete, acceptance begins. The advantages of this are: 1) It is convenient for demand management, and customers can determine the demand by setting milestones to reduce the spread of demand; 2) It is convenient to plan the investment of resources, and only one type of resource is required for a period of time. For example, only BA is invested in consulting, and only PD is invested in design.

But such questions are:

1. The upstream and downstream are often not connected, and the upstream requirements cannot be realized.
2. Repeated work, for example, BA investigates the indicator caliber from customers, but when PD/TM takes over the indicator list, PD/TM needs to sort out with customers again.
3. Since all indicators/tags are online at the same time, customers need to wait for a long time. Clients do not have good control over the priority of metrics.
4. It is also very unfavorable for Party B. After all the indicators have been developed, the customer will accept it. The risk of acceptance is great, the cycle is long, and the risk of rework is great.
5. The continuous cycle of the data center may be more than half a year. It is difficult to guarantee that the demand will remain unchanged in such a long cycle. Even if it is confirmed, it may be changed.

agile development

In order to solve the above problems, Alibaba Cloud introduced iterative development in project implementation. With bi-weekly as the iterative plan, each bi-week is a complete development unit.

For each iteration, an iterative planning meeting is required. From the Product Backlog, the customer selects the indicator with the highest value and the highest priority as the target of this iteration. This target is called the iteration list (Sprint Backlog). ).

For each iteration, we only work with the customer to complete the confirmation of the index of this iteration, and then carry out index development, index testing, and index acceptance. At the end of each biweekly, a general acceptance and review meeting will be held with customers.

This ensures that development is carried out according to the priority of customer value. Each iteration can have indicator acceptance and launch. For Party A, risks can be predicted in batches in advance, and customers can also use high-value indicators in advance.

To facilitate collaboration and project visualization, Teambition (TB) is recommended as a management tool. First, the project template is preset, so that members of the project team can easily find the required project content on the TB, and it is also very helpful for the management of the demand scope, such as the cloud list of data mentioned above, the list of dimension tables, the fact Table list, indicator/label list, iteration list, etc. Each type of list has development steps and processes, which are very suitable for visualization and process management through TB.

Finally, quality assurance must not wait until the last minute, which increases the risk of resumption of work. There should be a complete mechanism for quality assurance, which is ongoing.

Data Center - Project Closing
In the closing stage of the project, the deliverables are collected and archived and sent to the client, and a summary document is prepared for the completed project process and results for reporting. Design ceremonies to mark milestone moments. At the same time, review the highlights and shortcomings of the current project to help the next project.

Copyright statement: The content of this article is contributed by Alibaba Cloud's real-name registered users. The copyright belongs to the original author. The Alibaba Cloud developer community does not own the copyright and does not assume the corresponding legal responsibility. For specific rules, please refer to the " Alibaba Cloud Developer Community User Service Agreement " and " Alibaba Cloud Developer Community Intellectual Property Protection Guidelines ". If you find any content suspected of plagiarism in this community, fill out the infringement complaint form to report it. Once verified, this community will delete the allegedly infringing content immediately.

Related Articles

Explore More Special Offers

  1. Short Message Service(SMS) & Mail Service

    50,000 email package starts as low as USD 1.99, 120 short messages start at only USD 1.00

phone Contact Us