Predict IT trends from the deployment methods of Java applications – from DevOps to self-DevOps
Created#More Posted time:Oct 11, 2016 15:07 PM
One time, a coworker asked me a question: “Why are Spring Boot applications prone to be started directly by generating fat jar packages while the group's applications are typically started from application containers by generating war packages?” I explained the reason from the mainstream IT trend perspective is that Spring Boot applications are basically the products of the DevOps age, while most of the group's applications are products of the Dev-Ops-separation age.
Although abstract, this explanation works, but it fails to tell the whole story clearly enough. Then, you see this blog article after I re-arrange the explanation.
Deploying Java applications in application containers is actually influenced by J2EE. This is also a major differentiator between Java Web and other rapid Web development languages. For the delivery of a general Java Web application, the deliverable is normally a zip compressed package with the file name extension "war" and a definite internal directory structure. In this compression package, you can see (almost) the complete Web pages (templates), web site description, jar packages of different Java Web modules and dependent third-party jar files. O&M deploys this war file to an application container, and the corresponding site will be accessible.
Before the days when virtual machines dominate in data centers, Java applications were directly deployed on physical machines. However, a single Java Web application can barely take full advantage of all resources of physical machines. Because of this, multiple Java Web applications are deployed on the same physical machine to save costs, or to be more precise, multiple war files are deployed in the same application container for the same purpose.
Namely, by deploying multiple war files in the same application container, you can provide services for multiple sites through a single Java virtual machine process. This feature can effectively save memory resources and is especially significant during an age when memory was still valuable. Additionally, you can even deploy and uninstall war files dynamically without the need to restart Java virtual machines in the application container.
As mentioned above, a war file is a zip compression package. However, a more common practice is not to deploy this compression package but the directories extracted from the war file. At that Dev-Ops-separation age, developers are responsible for delivering war files while operators are for deploying those war files (extracted directories). As corporate business grows, Java Web applications host large traffic and some indiscoverable problems begin to emerge in application containers. Sometimes, these problems may even result in online failures. From that point on, dedicated teams gradually arise to maintain open-source application containers and to fix vulnerabilities for performance improvement.
Given that the R&D team delivers war files, the O&M team can complete redeploying Java Web applications without the interference of the R&D team when application containers need to be updated. Even when some common underlying frames are exposed to security vulnerabilities, the O&M team can also scan all machines for the jar files in the extracted war directories and fix those vulnerabilities by replacing problematic dependencies and restarting applications.
As hardware performance improves and the extensive deployment of virtual machine technologies in data centers, a dozen of Java Web applications under mixed-deployment on a single physical machine becomes a dozen of virtual machines, each of which has an application container and a Java Web application.
The extensive utilization of virtual machine technologies dramatically simplifies application operation and maintenance. With those technologies, you will no longer need to worry about disrupting the machine environments and conflicted dependences among applications. Instead, several lines of commands would produce a fully-configured out-of-the-box Linux Server. Also, you will no longer need to consider which applications need to be mixed-deployed or separately deployed. Instead, you can achieve oversold to reduce costs during low-traffic periods and migrate virtual machines to idle hosts to avoid resource competition before peak hours.
Later on, the development team enters the operation field and produces a new profession, DevOps. When both the development and O&M teams are the same team or even one person, deploying applications and application containers separately is very complicated. That is because there is no much difference for the DevOps to upgrade containers or applications, and a unified deployment process is preferred to complete them all. Probably, that is why embedded servlet containers arise. With such containers, application containers are no longer isolated from but part of applications, and they can be deployed and upgraded along with applications.
Especially, Spring Boot emerges during those years when the DevOps booms. By using embedded servlet containers, an application developed with Spring Boot embeds a Jetty or Tomcat container into its deliverable and packs the entire Java Web application and application containers into a single fat jar package. Then, you can start the application simply by running "java -jar webapp.jar" without even knowing the existence of application containers, as if you are running a common Java Application. Then, upgrading containers becomes as easy as upgrading a third-party class library, namely modifying the pom file, packaging and releasing.
Even later, we enter the Immutable Infrastructure age as a number of container technologies led by Docker emerge.
Immutable Infrastructure is a proactive concept put forward by Chad Fowler in 2013. In this mode, instances of any infrastructures (various software and hardware including servers and containers) will become read-only upon their creation, namely no change can be made to those instances. To modify or upgrade certain instances, you can only create new instances to replace them.
Before the container age comes to life, you can already pack Java applications into a fat jar package to distribute and deploy under the guidance of a framework like Spring Boot. However, in actual application scenarios, you are distributing far more content than this fat jar package to production environments. To ensure that application code can be deployed in different environments, inconsistent content among those environments will be extracted and stored in a configuration file so that Java Web applications can read it during startup. Such content includes database connection strings, passwords and IP address lists of ZooKeeper clusters. To dynamically adjust log levels and formats, independent configuration files will also be configured for different loggers including logback. In addition, changes of those configuration files will be monitored.
However, those configuration files outside of the fat jar package produce extra workload to application deployment and capacity expansion. Instead, containers want to take care of those all for you. By packing those configuration files into a container image, you will get an image that is ready for deployment at any time, simplifying application deployment and capacity expansion. However, this image is unable to adapt to different environments. For example, an image that works properly on cluster A may not be able to be deployed on cluster B. Because of this, environment variables are used to describe inconsistent content among different environments. Before a Java Web application starts, environment variables are used to generate a configuration file that is appropriate to the current environment. Then, you will need to use a configuration management database to manage those environment variables and replace application configurations from different clusters with a centralized CMDB.
To sum up, the deployment methods of Java Web applications go through the following combinations chronologically: application container + physical machine + mixed deployment of multiple applications, application container + virtual machine + independent deployment, fat jar + virtual machine and fat jar + Linux container. As O&M automation and intelligence evolved in the industry, we have witnessed the transition of engineers from specialized engineers to the DevOps or even full-stack engineers. By providing API-class computing resources that are accessible from anywhere at any time, cloud computing is accelerating this transition and making the jobs of the engineers involved in the transition easier and more interesting.
In this age, starting a business is not any easier than doing so in any previous ages but starting a cloud-computing-based business does become easier.