[Share]How can internet startups establish their own technical frameworks
Created#More Posted time:Oct 21, 2016 15:20 PM
This article targets small internet companies and is especially applicable to the background architecture of mobile Apps that can support 50,000 daily active users.
This article may introduce the technology selection for the techniques involved, as well as the procurement of related equipment.
Here I want to introduce the calculation process of some technical specifications to provide some reference.
• QPS: If the daily active users reach 50,000, the usage is concentrated to four hours every day and every user generates approximately 100 requests. So on average, our system may support requests: 50000 * 100 / (4 * 60 * 60) = 350 QPS/s
• Business data: Business volume. For us, it is news business, but there may be other businesses, such as games and shopping malls. Basically, the newly added business data every day is of the same magnitude, namely 10,000 every day. In addition, the information related to users is also a big part, such as the subscription by users. With 50,000 users, around 100 records of data are required to save related information.
• Cache size: Hotspot data about main businesses and users is stored in the cache. The data size is around 5G.
• Log size: User logs and request logs. The size is around 3G every day.
We establish the overall architecture on Alibaba Cloud as it is a relatively small company. Below are some instructions for the content and technology selection in the diagram.
Available schemes: SLB, Nginx.
- SLB charges fees and the price is cheap. Its service is stable and seldom fails. But there are few configuration options and it does not support mapping between the domain name and IP addresses.
- Nginx performs well all-aroundly, but it requires a certain amount of knowledge.
Suggestion: SLB + Nginx: SLB is bound to the domain name to serve as the uniform entry, and then Nginx is established on each server.
Used for caching static files. Qiniu and Alibaba are both good CDN service providers.
- Qiniu enjoys a longer history in the field, thus offering more improved image processing interfaces.
- The CDN performance of Alibaba is slightly better, but it does not offer the insecure access mode, compromising the access flexibility a little. Also, its image processing function is a little weaker.
Distributed call framework
At present, ZK + Dubbo, ZK + Motan, ZK + DubboX, and EDAS are available.
• Dubbo is a service governance framework of Alibaba. It is no longer maintained and is slow in responding to changes.
• DubboX is provided by Dangdang based on Dubbo. It is still in maintenance and is recommended.
• Motan is a service governance framework of micro blogs. It just went open-source and requires some learning to get started. It is recommended.
• EDAS is a service provided by Alibaba Cloud. It charges fees and is highly vulnerable to intrusions. Not recommended.
Available options: ActiveMQ, Alibaba Cloud Message and Notification Service, and rabbitMQ
They boast respective advantages, but Alibaba Cloud Message and Notification Service (MNS) is recommended considering the O&M difficulty.
Used for caching. Self-built Redis is costly as it requires Codis, sharding, clusters and master/slave nodes, which is very effort-consuming. It is recommended that you use the Redis service provided by Alibaba.
Considering the write/read splitting and master/slave replication, self-built databases and Alibaba DRDS are recommended.
- DRDS charges fees and is costly. It is not necessary.
- Self-built databases. Do not use middleware. Configure 1 for writes and 2 as read-only, and then configure the data sources for read/write splitting and the intranet SLB is used for reading clusters. The problem is solved.
ELK is recommended. You can synchronize databases automatically. Apart from the search engine functions, you can also perform log searches and monitor the system.
Some typical business scenarios
• Make the underlying services as the SOA module and provide external services through the distributed calling framework.
• Establish a small independent system to run timed tasks.
• Hotspot data is stored in the cache and cache is updated through MQ.
• Mongo can be considered for log data as necessary.