By Øystein Grøvlen
My name is Øystein Grøvlen, I was working as a Senior Staff Engineer in the MySQL Optimizer & SQL group of the PolarDB team at Alibaba for years. I am originally from Norway, but moved to China to work for Alibaba in 2018. After living one year in Hangzhou, I relocated to Alibaba's office in Sunnyvale, California.
My journey as a software developer started as an exchange student in the US where I was introduced to programming in BASIC on APPLE II computers. I found it very satisfying to be able to create my own programs in order to solve different kind of problems, and this inspired me to study computer science at the Norwegian Institute of Technology. During my studies, I found databases and information retrieval particularly interesting, and I decided to pursue a PhD in that area. I came in contact with the research department at the Norwegian Telecom, and I ended up working part-time on their research project, ClustRa, while working on my PhD on Compensation-Based Query Processing.
After my PhD, I started working at a start-up that was set up to commercialize the ClustRa database system. ClustRa was a highly available database system targeted at Telecom operations. The architecture of Clustra was pretty similar to MySQL Cluster, which many of you may have heard about. The focus of our work was on transactions, partitioning, replication, fail-over, and recovery. It was a fun time, solving a lot of interesting problems. Unfortunately, we ran out of funding when the dot-com bubble burst, but we were lucky that Sun Microsystems decided to pick up the pieces. At Sun, I continued to work on Clustra for while, before starting to work on Derby, the open-source version of the Cloudscape, a database implemented in Java. When Sun later bought MySQL, I switch to work on MySQL, and I got the opportunity to work in the Optimizer team. Sun was later bought by Oracle, and I worked on MySQL at Sun/Oracle for 10 years before joining Alibaba.
I find it really interesting to work on query optimization and query processing. My favorite part is investigating how we can get queries to run faster. POLARDB is based on MySQL code, so I can use my knowledge about the MySQL optimizer to make POLARDB a better product. What I find particularly rewarding about working for the POLARDB team, is that Alibaba is willing to put a lot of resources into POLARDB. We have a large group of people working on improving the product, and I think it is amazing what we have been able to do while I have been here. Most of my time I have spent on the project to support parallel query processing; while MySQL can only use one thread to execute a query, POLARDB can partition the query execution across multiple parallel threads.
I have been asked to give some advice to other developers on what it takes to make a career as a senior architect. I think there are many different paths to success, and some may claim that part of it may be the luck of being at the right place at the right time. However, based on my experience, I will discuss a few aspects that I feel has been essential to prepare myself for the role of software architect.
I think the only way to learn how a software system works, is to deep dive into the code. Do not just assume that you understand how the code is working, verify it by tracing the execution or stepping through the code with a debugger.
I have met quite a few developers that when faced with a bug, instead of getting a good understanding of how the code works, try to take short-cuts based on trial and error. They may succeed in fixing the bug, but they have learned very little about how the code works. Many times this approach also leads to bug fixes that seems to solve the problem, but the underlying problem may still exist. Do not just be satisfied when all your tests succeed, verify by trace/debugger that it runs as expected! It may take a bit longer to get the job done the first time, but you will get the reward later when you are assigned another task in the same area.
Optimizer Trace is a great tool to learn about the MySQL Query Optimizer
People are different. I find it easier to learn new code by following the execution of some specific examples, and then generalize from that. Other people may be successful in understanding the code just from reading it, but I guess my mental capacity is not at that level.
One of my first tasks at ClustRa was to make transaction rollback work. Much of the code was already prepared for the task, but there was still some missing pieces. By carefully tracing the execution of a transaction, it did not take me long to identify the missing pieces and make it work. I think it would have taken me much longer if I were to identify what was missing just by reading the code.
When you stumbled on a problem, do not be afraid to ask your co-workers. Share your understanding of the problem and ask for their advice. Most developers are more than happy to share what they know. However, make sure you have put some effort into understanding the problem before you ask. Otherwise, your co-workers may feel you are just trying to make them do your job.
Share your knowledge with your co-workers. Welcome their questions; you will often learn something yourself in the process.
Also, write about your work in articles and blog posts, and seek for opportunities to talk about it in public. This is a good way to make people outside your team or company aware of you and your work.
I feel many developers put too little effort into understanding the needs of their users. Often the developer prefer solutions that are either more straight-foward to implement, or more technically "interesting", over solving the real problem. Ask yourself: "How will this be used?", "Will the chosen interface be practical to the users?", and so on.
Before starting the design, it is a good idea to specify a few uses cases that describes how this new feature will be used, and what problem it will solve.
It takes time to become an expert on a technology, a product, or a code base. It took me several years before I could consider myself an expert on the MySQL query optimizer. While it is certainly good to have a wide experience, I think that if you change jobs or technologies too often, you will miss the opportunity to get the deep insights that will be necessary to establish yourself as an expert. In other words, be patient, and set long-term goals for yourself!
Learn more about PolarDB and the architecture:
https://www.youtube.com/watch?v=K87Ey4vqf5k&t=11s
[Best Practice] Instance Plan Management Released by AnalyticDB for PostgreSQL
[Infographic] Highlights | Database New Feature in January 2023
Alibaba Cloud Community - February 17, 2023
ApsaraDB - October 19, 2023
Alibaba Clouder - April 22, 2020
Alibaba Clouder - February 14, 2020
Key - February 20, 2020
yzq1989 - April 10, 2020
Alibaba Cloud PolarDB for MySQL is a cloud-native relational database service 100% compatible with MySQL.
Learn MoreAlibaba Cloud PolarDB for PostgreSQL is an in-house relational database service 100% compatible with PostgreSQL and highly compatible with the Oracle syntax.
Learn MoreAlibaba Cloud PolarDB for Xscale (PolarDB-X) is a cloud-native high-performance distributed database service independently developed by Alibaba Cloud.
Learn MoreAnalyticDB for MySQL is a real-time data warehousing service that can process petabytes of data with high concurrency and low latency.
Learn MoreMore Posts by ApsaraDB