Chapter 378 [Solutions and Quotes]
afternoon.
Luo Sheng is still staying at Blue Star Technology Group. He was going to go to the Coast of Blue Lab to host a technical meeting this afternoon, but upon learning that the State Railway Group is coming to seek technical support and help, he temporarily revised his work arrangements and chose to give priority to meeting Vice President Liu.
Mainly, the end of the New Year is approaching, and people from all over the country, especially migrant workers who go out to work, are eager to go home for the New Year. This group is very difficult. Luo Sheng feels that he can do something for them. Not to mention anything else, it is also a very meaningful thing to help them buy a ticket to go home for the New Year and reunion more easily.
Besides, you can also gain a big customer.
...
At this moment, in a reception room of the company, Luo Sheng and Vice President Liu and seven or eight people from the National Railway Group were together.
After a few simple greetings, Vice President Liu found the entry point for key topics and said: "The development cost of the first phase of the 12306 ticketing system totaled 300 million yuan, including hardware. This system needs to be combined with the Ministry of Railways information database, and also needs to be combined with the related data such as telephone ticketing, window ticketing, etc. 12306 is not a simple online trading system, but the largest bulk material freight system in the country and even the world."
After a moment, a technical expert who accompanied him also said: "The technology of 12306 is indeed very difficult to solve. Using existing distributed databases, caching, and load balancing technologies can no longer meet the needs."
Speaking of this, the technical expert looked at Luo Sheng and added: "You know, during the Spring Festival travel rush, 12306 has nearly 30 billion visits per day. Such a huge number of train ticket inquiry is one of the important reasons affecting the performance of 12306, and it is likely to account for more than 90% of the visits. What's more difficult is that there is a huge difference in peak and valley query, and there is almost no way to make a good balance between cost and concurrency capabilities."
Luo Sheng nodded silently when he listened. After the deceased finished speaking, he asked: "How did you solve the cooperation with Alibaba?"
The technical expert admitted: "The current approach is to control traffic from several key entrances to ensure system availability, but it will affect the user experience. The website is still down and unable to open, and there has been a lot of criticism."
As he said that, the technical plane shook his head with a bitter smile.
Vice President Liu also said: "Ali provides technical support to 12306 for free, but the technology is still not strong enough. We are now putting our hope on the nebula."
Luo Sheng couldn't help but smile and said honestly: "Mr. Liu, first of all, I can give you a reply with certainty. The nebula system is completely fine. At the same time, I can guarantee that within one week, a complete set of solutions from software to hardware can be tailored for 12306, but this solution will not be supported as free as Alibaba."
Mr. Liu was stunned and smiled, saying, "I hope to hear the details."
Luo Sheng thought for a moment, quickly organized the language and said: "If Nebula supports the 12306 system, the 12306 website query access will be placed on the Nebula. Generally speaking, query is the most visited part of most ticket sales systems. I just heard from your experts that it is about 90%. During the transaction-related process, query requests will be submitted many times, not to mention the additional workload caused by the emergence of a large number of ticket swiping software. All of this makes the remaining ticket query system a concentrated place for the entire system."
"My solution is that after Nebula intervenes, it separates the remaining ticket query module from the existing 12306 system and has the ability to deploy independently; it can independently deploy a set of remaining ticket query system on the cloud, so that 12306 and the cloud have a set of remaining ticket query system, and the scheduling will be more flexible. Currently, the scale of Nebula cluster servers has reached more than 50,000 units."
Several technical experts from the railway group present also smacked their lips when they heard this. No wonder Alibaba's "Flying" system is so hard to fight. The former barely exceeds a thousand, while the latter has reached a scale of 50,000. No wonder Luo Sheng is so confident.
Next, Luo Sheng mainly discussed with several technical experts from the Railway Group.
"This morning, when I learned the news, I briefly studied the server architecture of '12306'." Luo Sheng said to everyone in an orderly manner: "The majority of visitors are criticizing it, but I know that as soon as the 12306 service was launched for trial operation, it was subject to qps that could not be surpassed by any flash sale system in the world. The concurrency of millions is normal."
The technical backbone personnel of the railway group present were moved in their hearts, long live to understand.
Leaders who do not understand technology have the most difficulty in communicating and feel that they have not tried their best.
Luo Sheng's words are still very important.
After a moment, Luo Sheng added: "The system architecture with high concurrency should adopt distributed cluster deployment, and the upper layer of the service has layers of load balancing and provides various disaster recovery methods. The so-called disaster recovery methods are dual-fire room, node fault tolerance, server disaster recovery, etc. To ensure the high availability of the system, traffic will also be balanced to different servers according to different load capacity and configuration strategies."
"Even so, the qps that a stand-alone machine in the cluster can bear is very high. So how to optimize the stand-alone machine performance to the extreme? To solve this problem, one thing must be understood first: Usually, the ticket booking system needs to handle the three basic stages of generating orders, deducting inventory, and user payment. What the system needs to do is to ensure that train tickets are not oversold. A lot of them are sold. Each ticket sold must be paid to be effective, and the system must also be guaranteed to bear extremely high concurrency."
Several technical experts from the railway group nodded repeatedly to express their agreement, and the technical pain points are here.
Luo Sheng continued: "Subscribe an order to reduce inventory. When the user's concurrent request arrives on the server, first create the order, then deduct the inventory, and wait for the user to pay. This order is the first solution that ordinary people will think of. In this case, it can also ensure that the order will not be oversold, but it will also cause some problems. The first is that in the case of extreme concurrency, any memory operation details will definitely affect performance, especially logic like creating an order, which basically needs to be stored in the disk database. The pressure on the database is conceivable. 12306 should be used as an Oracle database. Don't spend this money and put it on my nebula."
"The second is that if the user places a malicious order, only placing an order but not paying, the inventory will become smaller and there will be many orders sold. Although the server can limit the number of IP and user purchase orders, this is really not a good idea."
"Then it is to pay and reduce inventory. If you wait for the user to pay the order and then reduce the inventory, the first feeling is that you will not sell less. But this is a taboo in the concurrency architecture, because in the case of extreme concurrency, users may create many orders. When the inventory is reduced to zero, many users will find that the orders they have grabbed cannot be paid. This is the so-called "overselling", and it is impossible to avoid concurrent operation of database disk io."
"The last one is to withhold inventory. Considering the above two solutions, we can draw a conclusion: as long as the order is created, the database io must be operated frequently. So is there a solution that does not require direct operation of the database io? The answer is, to withhold inventory, first deduct inventory, ensure that it is not oversold, and then generate user orders asynchronously, so that the response to users will be much faster."
"So how to guarantee that a lot of sales is sold? What should I do if the user gets the order but does not pay? Orders should be valid for the period. For example, if the user does not pay within five minutes, the order will expire and new inventory will be added. The generation of orders is asynchronous and should be placed in the instant consumption queue for processing..."
Vice President Liu was confused when he heard this, but he found that Luo Sheng and the technical experts he brought were getting more and more enthusiastic, and seemed to have obtained a message.
Find the right person!
At this time, Luo Sheng opened the large screen on the wall in the room. He also brought a laptop and opened the work computer, signaling everyone to look at the main projector, and said while operating the computer:
"Go language is native to concurrent design. I will use Go language to demonstrate to you the specific process of grabbing tickets on stand-alone and my optimized solutions."
"The init function in the go package is executed before the main function, and it also mainly does some preparation work at this stage. The preparations that the system needs to do include: initializing the local inventory, initializing the hash key value of the remote redis storage unified inventory, and initializing the redis link pool."
"In addition, an int type with size 1 needs to be initialized, with the purpose of realizing the function of distributed locks. You can also directly use read and write locks or use redis and other methods to avoid resource competition, but using el is more efficient. This is the philosophy of Go language. It does not need to communicate through shared memory, but to share memory through communication. The redis library uses redigo, and the following is the code:
【
//localspike package structure definition
package localspike
type localspike struct {
lostot64
localsalesvolume int64
}
…
//Remotespike's definition of hash structure and redis connection pool
package remotespike
//Remote order storage of the skill value
type remotespikekeys struct {
...
】
...
Before I knew it, an hour and a half had passed. Anyway, Deputy Liu couldn't understand the code, and he was confused.
The technical issues have finally been discussed.
The technical expert of the railway group looked at Vice President Liu and nodded with a positive look. Vice President Liu immediately understood and turned to Luo Sheng decisively and asked: "Mr. Luo, if you provide such a complete set of solutions, what is the quotation?"
Luo Sheng said bluntly: "Pay 1 billion a year."
Vice President Liu almost didn't spit out a mouthful of blood when he heard this number. The budget for the first phase of the 12306 ticketing system was only 300 million yuan. Such a quotation was so expensive that it was unacceptable.
Chapter completed!