How to Design a Complex Business System

How to Design a Complex Business System Introduction:

The industry-level application architecture is constantly evolving and iterating, but I always feel that the formation process of enterprise application architecture is realized under a seemingly scientific methodology, but it is not completely scientific.

How to Solve Complex Business Designs


How to Design a Complex Business System.Software architecture design itself is a complex thing, but in fact, there is a consensus in the industry, that is, "separation of concerns is achieved through componentization to reduce local complexity". In fact, whether we are using containers, middleware, messages, databases, etc., are in a sense the product of componentization. The advantage of this is that it can be reused in different systems. Today, with the rise of cloud native, it is easier for us to use general and componentized services in the form of services. Therefore, if you do not enjoy the dividends of cloud native technology, you will be abandoned by the times.

How to Design a Complex Business System.Cloud native can technically solve many non-functional quality and technical requirements to the greatest extent (as shown in the figure above). As an enterprise-level application architecture, it will naturally shift the focus to the functional design of business applications. Now, for the design of a complex business architecture, if we want to do it quickly and well, there are only two situations: one is that the architect has a deep understanding of the business, strong ability, and consummate skill; the other is the original business The system itself has a clear model and enough "high cohesion and low coupling" to quickly analyze business changes and form a new business architecture design based on it. What we should pursue is the second situation, which means that the model design and business process must be carefully treated from the very beginning of enterprise-level model construction. Only with a solid foundation can there be subsequent "rapid iteration".

How to Design a Complex Business System.Let's go back to the essence of architectural design, which is why we design before code is implemented. Design is first and foremost about solving the complexity of the problem. So someone made an architecture and handed it over to a team to implement, and soon found that the architecture and design of the implementation are completely different things. Of course, the reason is very clear - lack of communication and communication; the second is to establish a consensus on team collaboration and communication. Even if we have done a good architectural design that a team has reached a consensus on, everyone has worked diligently to turn the design into reality. A problem that has plagued the software industry for a long time has arisen, and the requirements are always changing. No matter how "precise" the pre-design is, it is always It is found that the next pit is not far away, and the result is often that the situation is getting worse and worse, that is, the architecture we often say is "corrupted", and finally everyone has to accept the rewrite. These experiences have gradually made us clear that the essence of software architecture design is to reduce complexity through the separation of core problems, so that the system can respond to external business changes faster, and the system can continue to evolve. There is no need to start from scratch when encountering changes, ensuring that implementation costs are effectively controlled.

Therefore, I think from the perspective of architectural design, the following three points are the most critical:


•Let our model, components and business division be as close as possible to the essence of change. For example, for general e- commerce systems , it is users, commodities, transactions, payments, etc. This division allows us to "isolate" changes within a certain range ( business module), thereby helping us effectively reduce change points.

•In terms of design, the internal business model is highly cohesive, and the models are low-coupling, that is, the business completed by each is relatively independent, and the other party will not be implicated because one party is disconnected. For example, the product recommendation function fails, but the transaction and The payment business should continue to provide services normally, which may prompt users to temporarily be unable to provide recommended services, or simply downgrade to a bottom-line strategy.

•Models and components are reused as much as possible in business. It is this kind of reuse that makes today's Internet-level architecture. We do not start from scratch every time we build an e- commerce system . The business modules that are "reused" the most will obviously focus on design and operation and become the core business modules. Of course, such an e- commerce system will inevitably be more robust in terms of architecture.

How to Design a Complex Business System.The above three points undoubtedly point to the business. Starting from the business and facing the business change is the key to the success of our modern architecture design. Therefore, the core essence of the complex business architecture design is to ensure that we can be fast enough in the face of business changes. Responsiveness.

How to Design a Complex Business System.Domain Design


I mentioned the common diseases of business software development: from a small project to a large-scale business system, but with the continuous increase of new requirements, it eventually evolved into a nightmare for the development team. And most of these nightmares stem from the destruction of the conceptual integrity of software (the term "conceptual integrity" comes from the software engineering classic The Myth of the Moon and Man). These business codes may be stacked by generations of developers (we call it "shit mountain"), and no one consciously maintains the conceptual integrity of the software in the process. And DDD domain design, especially the concepts at the strategic modeling level provided by DDD, is a good medicine for maintaining the integrity of software concepts.

"Technology serves business, business drives technology" is the consensus of most people, especially for commercial companies. The DDD domain design advocates that the business domain itself should be the focus of attention in software design (in other words, software developers should understand the business), which is very consistent with this idea; and, DDD provides a practical solution for complex business software design. Solution, which is what I strongly advocate as an architect to learn and discuss the relevant knowledge of DDD domain design.

strategic modeling
At the strategic level, DDD places great emphasis on the analysis and decomposition of business problems, reducing the complexity of problems by identifying core problems. The two most important concepts for DDD to maintain the conceptual integrity of the model at the strategic level are the Bounded Context and the Anti-Corruption Layer.

well defined bounded context
Regarding the definition of bounded context, any book about DDD will explain it in detail. Here I just want to share some of my own understanding. At this time, some people will ask: How big is the bounded context to be appropriate, and are there any rules that can be followed for dividing the context?

How to Design a Complex Business System.rules for dividing the context are nothing more than "high cohesion, low coupling" that is universally applicable, which may be too false. In fact, what really makes everyone feel entangled is that the relationships between those things that you don't know how to segment, and some are even included in one context. In fact, I think instead of focusing on the "size" of the context, it is better to focus on the "quality" of the model, whether the integrity of the concept is easily broken. In my opinion, to judge whether the size is appropriate, we must combine the capabilities of the application development team to see how far the development team can control the conceptual integrity of the software. As long as there are no problems with the development team, this range is fine no matter how large it is.


If the level of the development team is upstream in the industry, the scope of the maintenance context is often very large; the level of the development team of some companies is uneven, so in the process of project implementation, it may be necessary to divide a relatively small context and reduce as much as possible. "Mountain of shit" keeps accumulating.

Do a good job of anti-corrosion layer

The bounded context needs to protect the boundaries it maintains and the integrity of the concepts within the boundaries at all times . At this time, the place where the concept of a context needs to be transformed into another context concept is called "anti-corrosion layer". There are many implementations of the anti-corrosion layer, typically implemented as an adapter, and in a broad sense, Gateway is also a typical anti-corrosion layer component. Of course, there must be obvious physical properties between the anti-corrosion layer code and other internal business models. Boundary (of course, it does not necessarily mean that the anti-corrosion layer should be used as an independent deployment process), at least we can consider building and maintaining the anti-corrosion layer as an independent class library, such as Star Ring, which is actually the way of thinking.

Tactical Modeling

The core concepts of DDD tactically are entities and aggregations. In order to better understand what aggregations, aggregation roots, and aggregation internal entities are, the following examples are given. Imagine an order-related model of an e- commerce system , we may get order Order, order header OrderHeader , order line item OrderItem three interrelated concepts:


•An aggregate called Order.
•The aggregate root of this order aggregation is an entity called OrderHeader , and the ID of the entity OrderHeader is called OrderId (order number).
•Through the OrderHeader entity, we can access an aggregate of OrderItem entities. The local ID of the OrderItem entity is called ProductId . Because business fickleness does not allow the same product to appear in different line items of the same order, we can choose the product ID as the partial ID of the line item.

"Aggregation is the unit of data modification", based on this principle, we can achieve "strong consistency within the aggregation and eventual consistency outside the aggregation", for example, whether we can accept that the sum of the sum of all order items in an order is not equal to the order header total amount, we have to divide the two entities Order Header and Order Line Item into the same aggregate.


•Principles of Designing Aggregations


Let's take a look at the description of the principles of aggregation design in the book "Implementing Domain-Driven Design". The original text is a bit difficult to understand. Let me explain it a little bit:


1.Model true invariant conditions within consistency boundaries. Aggregates are used to encapsulate true immutability rather than simply grouping objects together. There is a set of invariable business rules in the aggregation. Each entity and value object operates according to unified business rules to achieve the consistency of object data. Anything outside the boundary has nothing to do with the aggregation. This is the aggregation that can achieve high business cohesion. s reason.

2.Try to design small aggregates. If the aggregate design is too large, the aggregate will contain too many entities, resulting in too complicated management between entities, and concurrency conflicts or database locks will occur during high-frequency operations, which will eventually lead to poor system availability. The small aggregation design can reduce the possibility of aggregation reconstruction due to excessive business, and make the domain model more adaptable to business changes.

3.Reference other aggregates by unique identifiers. Aggregates are referenced by associating external aggregate root IDs rather than direct object references. Objects of external aggregation are managed within the aggregation boundary, which easily leads to unclear aggregation boundaries and increases the coupling between aggregations.

4.Use eventual consistency outside of bounds. Intra-aggregate data is strongly consistent, while inter-aggregate data is eventually consistent. In a transaction, at most one aggregate's state can be changed. If a business operation involves the change of multiple aggregation states, the related aggregations should be modified asynchronously in the form of domain events to achieve decoupling between aggregations (I will explain the relevant content in the domain events section).

5.Cross-aggregate service calls are implemented through the application layer. In order to achieve decoupling between aggregations within microservices, and to combine and split microservices in aggregate units in the future , cross-aggregate domain service calls and cross-aggregate database table associations should be avoided.

The above principles are some general design principles of DDD, or that sentence: "What suits you is the best." During the system design process, you must consider the specific situation of the project, if you face the convenience of use , high performance requirements, lack of technical capabilities and global transaction management and other influencing factors, these principles are not insurmountable, in short, everything starts from solving practical problems.


Steps to Design Aggregates

DDD domain modeling usually adopts similar event storms, generally through use case analysis, scenario analysis and user journey analysis, etc., through brainstorming to list all possible business behaviors and events, and then find out the domain objects that produce these behaviors, and sort out the domain The relationship between objects, find the aggregate root, find the entity and value object closely related to the aggregate root business , and then combine the aggregate root, entity and value object to build an aggregate.

Everyone is familiar with the e- commerce system , and there are many mature models for e- commerce business that can be directly used for reference; let's take another scenario - insurance business as an example, to see what the aggregation construction process mainly includes. The steps, of course, I have seen this example from other learning materials. It is typical and can be used as an example to illustrate:

Step 1: How to Design a Complex Business System.Using methods such as use case analysis or event storm, based on business behaviors, sort out all entities and value objects that occur in the process, such as application forms, subjects, customers, insureds, etc.

Step 2: How to Design a Complex Business System.Select the root entity that is suitable as the object manager from many entities, that is, the aggregate root. To determine whether an entity is an aggregate root, as mentioned in the previous chapter, you can analyze it in combination with the following scenarios: Is there an independent life cycle? Is there a globally unique ID? Can other objects be created or modified? Is there a dedicated module to manage this entity. The aggregate roots in the figure are the policy and customer entities, respectively.

Step 3: How to Design a Complex Business System.According to the principles of designing aggregates mentioned in the previous chapter, find all closely dependent entities and value objects associated with the aggregate root. Constructs a collection of objects containing an aggregate root, multiple entities, and value objects . This collection is an aggregate. In the diagram we construct two aggregates, Customer and Insurance.

Draw the reference and dependency model of the object according to the dependencies of the aggregation root, entity and value object . It needs to be explained here: the data of the insured and the insured are obtained from the customer aggregation by associating the customer ID. In the insurance aggregation, they are the value objects of the insurance application, and the data of these value objects is the redundant data of the customer. Even if the data aggregated by customers changes in the future, it will not affect the value object data of the application form. From the figure, we can also see the reference relationship between entities. For example, in the insurance application aggregation, the aggregate root of the insurance application refers to the quotation entity, and the quotation entity refers to the sub-entity of the quotation rule.

Step 5: How to Design a Complex Business System.Multiple aggregations are divided into the same bounded context according to business semantics and context.

The above is the complete process of the birth of an aggregation.

Domain Modeling Strategies in Different Scenarios
Due to the vast differences in the situation within the enterprise, the development process is also different. There are microservice transformation of legacy monolithic systems, business modeling and system design in new and unknown areas, and partial optimization of legacy systems. In different scenarios, the strategy of domain modeling will also be different. Let's take a look at how to perform domain modeling in several categories of scenarios.

New system

How to Design a Complex Business System.New system For complex business domains, the domain may need to be split at multiple levels before domain modeling can begin. Domains are split into subdomains, and even subdomains need to be split further. For example, insurance needs to be divided into sub-domains such as underwriting, claims settlement, collection and payment, and reinsurance, and the underwriting sub-domain is further divided into sub-sub-domains such as insurance application and policy management. If the complex domain is not further subdivided, because the problem domain is too large, the workload of domain modeling will be very large. It is not easy for you to complete a large domain modeling through event storms, and even if you do it barely, the effect is not necessarily good.


How to Design a Complex Business System.For complex domains, we can complete domain modeling and microservice design in three steps.

How to Design a Complex Business System.Split subdomains to establish a domain model, and refer to boundary factors such as process node boundaries or function aggregation modules according to the characteristics of the business domain. Combined with the discussions of domain experts and project teams, the domain is gradually decomposed into sub-domains of suitable size, and event storms are used for the sub-domains to divide aggregation and bounded contexts, and preliminarily determine the domain model in the sub-domain.

How to Design a Complex Business System.Domain model fine-tuning Sort out the domain models of all sub-domains in the domain, and fine-tune the domain models of each sub-domain. The fine-tuning process focuses on the reorganization of aggregates in different domain models. Synchronously consider the domain model and aggregate boundaries, services, and dependencies between events to determine the final domain model.

Microservices According to the domain model and the principle of splitting microservices , the splitting and design of microservices are completed.

How to Design a Complex Business System.Monolithic legacy system



If we are dealing with a monolithic legacy system, we only need to separate some functions into microservices, while the rest remain monolithic, and the whole remains unchanged. For example, the modules facing performance bottlenecks are split into microservices. We only need to understand this specific function as a simple sub-domain and refer to the way of modeling a simple domain. In the design of microservices , we also need to consider the compatibility of services and services between the old and new systems, and introduce an anti-corrosion layer if necessary.
Challenges in the Cloud Native Era
With the rise of cloud-native technologies, enterprise-level architecture is now more cloud-based, and the cloud-based architectural style has a new focus: elastic boundaries. Elastic boundary is a core concept of cloud-native enterprise-level application architecture. It refers to the system boundary delineated by taking elasticity as the top priority, which determines whether we can fully utilize the full capabilities of the cloud-native platform. So we need new ways to make up for the deficiencies of previous business models to meet the needs of the new cloud-native . It can now be said that microservices are basically based on cloud-native architecture, and using microservices architecture on a fixed and elastic platform has extremely high implementation costs. It can be said that cloud native should actually be a prerequisite for microservices .

In the cloud-native era, we need to model elasticity as a primary consideration . Then the elastic boundary is an important basis for us to divide the system. Moreover, we also need to consider the dependencies between elastic boundaries and try to avoid elastic coupling. For business modeling, in order to match the architecture of the cloud-native era, I think the following points should be done:

•Establishing a model structure can reflect the elastic boundary, and at this time, it is necessary to consider the principles of different elastic boundaries to divide the boundary context; if the two contexts obviously have different elastic demands, they should be split. And if there are consistent elastic demands, you can consider not dismantling first. How "micro" can microservices be split at this time ? Simply put, it is "micro" to be able to better utilize elasticity to control the size of the cost.

•From the perspective of the asynchronous model, to optimize the business logic; typical is the MQ message queue system, because of the broker, so the producer and consumer do not have to maintain availability and the same throughput at the same time, and the producer does not need to wait for a reply immediately .

•Loose coupling of locations: The typical service registry is the service registry. The consumer does not need to know the specific location of the provider directly, but all use the registry to find the service to access.

•When the elastic boundary divides the business context, the same elastic boundary maintains strong business consistency.

•When an asynchronous call generates an intermediate state exception, it is necessary to maintain the eventual consistency of the business.

Don't ignore the impact of organizational structure


"Conway's Law" tells us that organizational structure will determine the structure of team communication, and will also determine the structure of the product. The sorting out of the organizational structure will often be done during demand research. In terms of information collection, there is nothing special about business architecture design here. The difference is that the goal of business architecture is enterprise-level capability planning, hoping to break through barriers and form synergies. It is for this reason that the organizational structure has a great reaction force on the business architecture design. The enterprise-level digital transformation plan must match the organizational structure, otherwise it will be difficult to implement. It can be said that departmental interests are one of the biggest obstacles to doing enterprise-level architecture, and crossing this obstacle is also one of the requirements for the ability of architects . Of course, in some cases, immobility is an option when there is no better solution.


In my experience, there is no particularly good solution for this kind of problem. There are only two kinds of problems: one is that those with super-powers are leading, and with the support of the highest level, the decision-making is vigorously promoted, but the larger the enterprise, the more In a dominant enterprise, this structure is more difficult to form; the second is to strengthen the capacity and number of business architecture personnel within the enterprise (preferably, all departments have similar roles), and let these enterprise organization personnel work as partners. Participate in the whole process of the project, build a collaborative network during the implementation of the project, and improve decision-making efficiency, so that the organizational structure is no longer the bottleneck of the digital transformation of enterprises.


SOA - Microservices - Middle Platform: The Art of Compromise


Years ago, these traditional large-scale ERP business software actually maintained the integrity of business concepts on a large scale. After an ERP is installed, the database has seven or eight hundred tables (that is, seven or eight hundred entities) in the same bounded context. But it's admirable how well these ERPs maintain the integrity of the business concept within such a vastly bounded context.


How to Design a Complex Business System.It is very difficult to implement, but very easy to break. After a set of ERP customization project is implemented, there may be hundreds of more tables in the database, not to mention the non-standard naming looks strange. The ERP implementation consultants and developers of these manufacturers maintain this huge "shit mountain" day and night. We can't let these huge "monolithic applications" grow without limit, so we once again raised the banner of "divide and conquer". Software componentization technologies like SOA provide us with tools for splitting. We split a large bounded context into several relatively smaller bounded contexts according to the benefits; physically, we split a large monolithic application into several services.


How to Design a Complex Business System.Generally speaking, we basically stack the physical boundary of the service and the domain boundary of the bounded context. A bounded context is applied to one or more services that can be deployed independently . The service application includes the implementation of the core business logic of the bounded context. The physical boundaries of SOA's service components add some difficulties to calls between services, which makes developers simplify the relationship between objects and write more "highly cohesive, low-coupling" code. When there are not many service components, the workload of building the anti-corrosion layer will not be very large, and we only need to deal with the code between the components.


How to Design a Complex Business System.However, our architects and developers like "divide and conquer" too much. The widespread use of microservices is even abused. Let's see that many microservices are really "micro", almost a DDD aggregation can correspond to one Microservices that can be deployed independently . Such microservices can't do much business by themselves, which requires more and more microservices to "aggregate" together to provide business services to the outside world.


Of course, the development of microservice technology infrastructure has also provided more convenience for calls between services, and crossing the boundaries of microservices has become the norm; at this time, business developers distinguish between "service calls within the same context" and "service calls within the same context". The "anti-corrosion layer between contexts" needs to keep a clear head at all times. At this time, it is often difficult to align the physical boundaries of the bounded contexts and microservices, which inevitably increases the difficulty of maintaining the conceptual integrity of each bounded context .


Since it is getting more and more difficult to maintain the integrity of the "tiny" independent bounded context concepts, let's just aggregate them again? Combining them into a moderately sized bounded context is the so-called enterprise-level business architecture, which is what we now call the business middle-office. The ultimate goal can be said to want to achieve "enterprise-level" harmony.


So to a certain extent, software engineering is the art of compromise, the "middle of the mean". Whether we want a middle stage or a large middle stage, no matter the size of the enterprise, we should combine our own business goals and resources to "maintain a wider range of conceptual integrity" and "maintain more anti-corrosion layer code". A balance between them is one of the core things an enterprise architect has to do .


Our team has indeed accumulated and practiced some "business middle-stage methodology" over the years, and has practiced in some projects. Of course, one of the most soulful parts is the domain design mentioned above. In the past, many people said that the most difficult part of the DDD field design and even the business middle-stage methodology is that there is no suitable tool or platform to practice. Today, in fact, Alibaba's open source COLA and the internal use of Star Ring and BizWorks are all good tools and platforms.

Epilogue

Enterprise-level application architecture is constantly evolving and iterating, but I always feel that the formation process of enterprise application architecture is achieved under a seemingly scientific methodology, but it is not completely scientific. Think about it carefully, people who do software architecture actually envy those who do architectural architecture, because architectural architecture has a rigorous mechanical foundation as the base, and there are many things that can be accurately calculated, but software architecture does not have many components that can be accurately calculated, so , the "constant compromise" mentioned above is a feasible design idea and design art; in fact, this also fulfills the phrase "there is no silver bullet".




Related Articles

Explore More Special Offers

  1. Short Message Service(SMS) & Mail Service

    50,000 email package starts as low as USD 1.99, 120 short messages start at only USD 1.00