What I Think Really Matters for Personal Growth in Software Development | From Junior Engineer to Expert

By Øystein Grøvlen

My name is Øystein Grøvlen, I was working as a Senior Staff Engineer in the MySQL Optimizer & SQL group of the PolarDB team at Alibaba for years. I am originally from Norway, but moved to China to work for Alibaba in 2018. After living one year in Hangzhou, I relocated to Alibaba's office in Sunnyvale, California.

My journey as a software developer started as an exchange student in the US where I was introduced to programming in BASIC on APPLE II computers. I found it very satisfying to be able to create my own programs in order to solve different kind of problems, and this inspired me to study computer science at the Norwegian Institute of Technology. During my studies, I found databases and information retrieval particularly interesting, and I decided to pursue a PhD in that area. I came in contact with the research department at the Norwegian Telecom, and I ended up working part-time on their research project, ClustRa, while working on my PhD on Compensation-Based Query Processing.

After my PhD, I started working at a start-up that was set up to commercialize the ClustRa database system. ClustRa was a highly available database system targeted at Telecom operations. The architecture of Clustra was pretty similar to MySQL Cluster, which many of you may have heard about. The focus of our work was on transactions, partitioning, replication, fail-over, and recovery. It was a fun time, solving a lot of interesting problems. Unfortunately, we ran out of funding when the dot-com bubble burst, but we were lucky that Sun Microsystems decided to pick up the pieces. At Sun, I continued to work on Clustra for while, before starting to work on Derby, the open-source version of the Cloudscape, a database implemented in Java. When Sun later bought MySQL, I switch to work on MySQL, and I got the opportunity to work in the Optimizer team. Sun was later bought by Oracle, and I worked on MySQL at Sun/Oracle for 10 years before joining Alibaba.

I find it really interesting to work on query optimization and query processing. My favorite part is investigating how we can get queries to run faster. POLARDB is based on MySQL code, so I can use my knowledge about the MySQL optimizer to make POLARDB a better product. What I find particularly rewarding about working for the POLARDB team, is that Alibaba is willing to put a lot of resources into POLARDB. We have a large group of people working on improving the product, and I think it is amazing what we have been able to do while I have been here. Most of my time I have spent on the project to support parallel query processing; while MySQL can only use one thread to execute a query, POLARDB can partition the query execution across multiple parallel threads.

I have been asked to give some advice to other developers on what it takes to make a career as a senior architect. I think there are many different paths to success, and some may claim that part of it may be the luck of being at the right place at the right time. However, based on my experience, I will discuss a few aspects that I feel has been essential to prepare myself for the role of software architect.

Make deep dives

I think the only way to learn how a software system works, is to deep dive into the code. Do not just assume that you understand how the code is working, verify it by tracing the execution or stepping through the code with a debugger.

I have met quite a few developers that when faced with a bug, instead of getting a good understanding of how the code works, try to take short-cuts based on trial and error. They may succeed in fixing the bug, but they have learned very little about how the code works. Many times this approach also leads to bug fixes that seems to solve the problem, but the underlying problem may still exist. Do not just be satisfied when all your tests succeed, verify by trace/debugger that it runs as expected! It may take a bit longer to get the job done the first time, but you will get the reward later when you are assigned another task in the same area.

Optimizer Trace is a great tool to learn about the MySQL Query Optimizer

Learn by example

People are different. I find it easier to learn new code by following the execution of some specific examples, and then generalize from that. Other people may be successful in understanding the code just from reading it, but I guess my mental capacity is not at that level.

One of my first tasks at ClustRa was to make transaction rollback work. Much of the code was already prepared for the task, but there was still some missing pieces. By carefully tracing the execution of a transaction, it did not take me long to identify the missing pieces and make it work. I think it would have taken me much longer if I were to identify what was missing just by reading the code.

Be not afraid to ask

When you stumbled on a problem, do not be afraid to ask your co-workers. Share your understanding of the problem and ask for their advice. Most developers are more than happy to share what they know. However, make sure you have put some effort into understanding the problem before you ask. Otherwise, your co-workers may feel you are just trying to make them do your job.

Share your wisdom

Share your knowledge with your co-workers. Welcome their questions; you will often learn something yourself in the process.

Also, write about your work in articles and blog posts, and seek for opportunities to talk about it in public. This is a good way to make people outside your team or company aware of you and your work.

Understand your users

I feel many developers put too little effort into understanding the needs of their users. Often the developer prefer solutions that are either more straight-foward to implement, or more technically "interesting", over solving the real problem. Ask yourself: "How will this be used?", "Will the chosen interface be practical to the users?", and so on.

Before starting the design, it is a good idea to specify a few uses cases that describes how this new feature will be used, and what problem it will solve.

Stay around

It takes time to become an expert on a technology, a product, or a code base. It took me several years before I could consider myself an expert on the MySQL query optimizer. While it is certainly good to have a wide experience, I think that if you change jobs or technologies too often, you will miss the opportunity to get the deep insights that will be necessary to establish yourself as an expert. In other words, be patient, and set long-term goals for yourself!

Learn more about PolarDB and the architecture:
https://www.youtube.com/watch?v=K87Ey4vqf5k&t=11s

Community

What I Think Really Matters for Personal Growth in Software Development | From Junior Engineer to Expert

Make deep dives

Learn by example

Be not afraid to ask

Share your wisdom

Understand your users

Stay around

Read previous post:

Read next post:

ApsaraDB

You may also like

Comments

ApsaraDB

Related Products

Database Overview

PolarDB for MySQL

PolarDB for PostgreSQL

PolarDB for Xscale