By Ilhan Adiyaman, Alibaba Cloud Tech Share Author. Tech Share is Alibaba Cloud's incentive program to encourage the sharing of technical knowledge and best practices within the cloud community.
This post features a recipe on how to write good quality Chef cookbooks and to keep your cookbooks up to date. You'll learn how to use Test Kitchen, Cookstyle, Foodcritic and ChefSpec to create a well documented Chef Cookbook which you can publish on Chef Supermarket.
In order to complete this walkthrough, you need to be set up for local development. It involves verifying your work on local test infrastructure before you deploy any changes out to production. Follow steps below, to setup your local development environment.
chef --version on terminal.Chef is a very popular, open sourced(very recently) and powerful infrastructure automation platform which transforms infrastructure to easily manageable and versioned code blocks. With Chef, you can automate how your infrastructure will be configured, deployed and managed across your network. It can work both in the cloud and on-premises no matter its size.
Chef has three fundamental components:
Apart from these three fundamental components, Chef's main feature is hidden behind Cookbooks which enable us to write our infrastructure in Ruby language.
Chef uses cookbooks to communicate with nodes in order to pass the configurations and policies. Inside of a cookbook, there are
recipes that specify the resources to use and the order in which they are to be applied,files which can be transferred to nodes,attributes which can be used to override the default settings on a node,metadata which provides information that helps client and server correctly deploy it,libraries to allow arbitrary Ruby code to be included in the cookbook,templates which are an Embedded Ruby(ERB) template that is used to dynamically generate static text files,tests to improve the quality of our cookbooks.Chef usually releases a backwards-compatible update of Chef client each month. However, each year they release a new major version which may include some deprecations for old functionalities. For example, Chef 13 was released in April 2017, Chef 14 was released in April 2018. You can see from here the upcoming update has some deprecations.
Moreover, we generally use chef cookbooks to install, configure and manage some specific software solutions such as Apache Server, PHP or MySQL on the our nodes. However, these softwares are also getting updated frequently. In order to get latest functionalities available on our recipes, we should regularly keep our cookbooks updated.
Depends on the size and structure of the organizations, some teams perform periodic maintenance on their notebooks every month or quarter whereas other teams updates less frequently. It all depends how big is your organization, how many cookbooks your team maintains and how often your team add features to your cookbooks. But it's crucial to remove legacy part from your cookbooks to ensure you have healthy infrastructure when you deploy on production.
Below, you can see the steps which should applied to keep your cookbooks up to date.
Now let's use these steps to update our sample Apache Cookbook. The example cookbook targets Chef 12. Here we'll use this cookbook to practice the update process.
Let's start by cloning the custom_apache cookbook from Github to your Chef workstation.
$ git clone https://github.com/learn-chef/custom_apache.gitThe cookbook contains default attributes and recipes, a resource to configure Apache, matchers for ChefSpec, ChefSpec tests, Test Kitchen configuration file, Vagrantfile, metadata and README for documentation.
We use Vagrantfile to run and verify our cookbook on a temporary VM that runs on Virtual Box. Before Test Kitchen, we had to use a separate Vagrantfile to achieve testing. However, Test Kitchen is creating and managing the Vagrantfile. Therefore, we can remove this file from the cookbook contents.
$ rm VagrantfileFoodcritic and Cookstyle are two popular linting tool for cookbooks. Linting tools help you to ensure that your code adheres to standart style guidelines that are followed for the language you write to avoid common problems. As both tools do not involve the creation of a test instance, they provide a faster way to validate the correctness of your cookbook.
Cookstyle is a linting tool based on the RuboCop Ruby linting tool. Cookstyle helps you to enforce style conventions and best practices, evaluates the code against metrics like "line length" and "function size", establishes uniformity of source code. It
On the other hand, Foodcritic checks cookbooks for common problems such as correctness, mistakes, deprecations of the cookbook.
Let's first run Foodcritic to see the violations.
$ foodcritic ~/cookbooks/custom_apache
Checking 5 files
x....
FC019: Access node attributes in a consistent manner: ./recipes/default.rb:13
FC019: Access node attributes in a consistent manner: ./recipes/default.rb:14
FC019: Access node attributes in a consistent manner: ./recipes/default.rb:15
FC064: Ensure issues_url is set in metadata: ./metadata.rb:1
FC065: Ensure source_url is set in metadata: ./metadata.rb:1
FC066: Ensure chef_version is set in metadata: ./metadata.rb:1
FC067: Ensure at least one platform supported in metadata: ./metadata.rb:1It reported 7 issues. There are issues for metadata and default recipe. Let's solve the issues of metadata first.
Add lines below to the end of the metadata.rb. We add these lines according to the issues numbered FC064, FC065, FC066, FC067. Fixing these issues are pretty straight forward after reading the description of the issues.
...
supports 'ubuntu'
issues_url 'https://github.com/learn-chef/custom_apache/issues'
source_url 'https://github.com/learn-chef/custom_apache'
chef_version '>= 12' if respond_to?(:chef_version)Issue number FC019 relates to how you accessing node attributes in your recipes. The standard Ruby bracket syntax are recommended over the older "method" or "dot" syntax. Let's change the recipes/default.rb accordingly.
...
# Load from node attributes some variables we'll need.
site_name = node['site']['name']
content_owner = node['site']['content']['owner']
content_group = node['site']['content']['group']
...Now, run Foodcritic again.
$ foodcritic ~/cookbooks/custom_apache
Checking 5 files
.....Perfect! No more violations. Now, it's time to check our coding style and automatically fix the violations with Cookstyle. Run Cookstyle as shown below.
$ cookstyle -a ~/cookbooks/custom_apache
Inspecting 8 files
C...C..C
Offenses:
...
8 files inspected, 12 offenses detected, 12 offenses correctedAmazing! It found 12 coding style violations and fixed them all without even modifying them by ourselves. If you run the Cookstyle again, you will see no offenses detected as an output.
Test Kitchen automatically test your cookbook across any combination of platforms and test suites defined in a kitchen.yml file. Here, we'll use its vagrant driver to test our cookbook in a test VM. To apply the cookbook to the Ubuntu virtual machine as defined in kitchen.yml file run kitchen converge as shown below.
$ kitchen converge
-----> Creating <default-ubuntu-1604>...
==> vagrant: A new version of Vagrant is available: 2.2.4!
==> vagrant: To upgrade visit: https://www.vagrantup.com/downloads.html
...
Deprecated features used!
Cloning resource attributes for apt_package[install libapache2 package] from prior resource
Previous apt_package[install libapache2 package]: /tmp/kitchen/cache/cookbooks/custom_apache/recipes/default.rb:32:in `block in from_file'
Current apt_package[install libapache2 package]: /tmp/kitchen/cache/cookbooks/custom_apache/recipes/default.rb:32:in `block in from_file' at 1 location:
- /tmp/kitchen/cache/cookbooks/custom_apache/recipes/default.rb:32:in `block in from_file'
See https://docs.chef.io/deprecations_resource_cloning.html for further details.
supports { manage_home: true } on the user resource is deprecated and will be removed in Chef 13, set manage_home true instead at 1 location:
- /tmp/kitchen/cache/cookbooks/custom_apache/recipes/default.rb:52:in `block in from_file'
See https://docs.chef.io/deprecations_supports_property.html for further details.
supports { non_unique: false } on the user resource is deprecated and will be removed in Chef 13, set non_unique false instead at 1 location:
- /tmp/kitchen/cache/cookbooks/custom_apache/recipes/default.rb:52:in `block in from_file'
See https://docs.chef.io/deprecations_supports_property.html for further details.
Chef Client finished, 11/16 resources updated in 30 seconds
Finished converging <default-ubuntu-1604> (0m53.43s).As you can see from the output, the kitchen converged successfully. However, when you look closer, you'll also notice about the using of deprecated features. To fix all these deprecations, it is easy to fix them one by one. To do that, let's enable deprecations_as_error setting to report deprecations as errors. Add line below to the provisioner section of kitchen.yml file and run kitchen converge again.
...
deprecations_as_errors: true
...Now you can see that it didn't converge successfully. When you read the description of the error, it points to deprecations_resource_cloning deprecations. On the page, Chef recommends that each resource have a unique name. Let's change that on our recipe/default.rb file as shown below.
...
package "install libapache2 package #{p}" do
...When you run kitchen converge again, this time it returns another error about the deprecations_supports_property. On the page, Chef explains how we should use supports metaproperty. Let's change that line in recipe/default.rb file accordingly.
...
manage_home true
non_unique false
...Now, when you run kitchen converge, you'll see there will be no error and it'll converge successfully.
ChefSpec is an extension of Ruby's RSpec a behaviour driven development framework for Ruby. ChefSpec is a framework that tests resources and recipes as part of a simulated chef-client run. In our example cookbook, there is already a few tests defined in spec/unit/recipes/default_spec.rb. Let's try to run them without error.
Run the ChefSpec test as shown below.
$ chef exec rspec ~/cookbooks/custom_apache
WARNING: you must specify a 'platform' and 'version' to your ChefSpec Runner and/or Fauxhai constructor, in the future omitting these will become a hard error. A list of available platforms is available at https://github.com/customink/fauxhai/blob/master/PLATFORMS.md
...Let's fix this warning in the file spec/unit/recipes/default_spec.rb by specifying our platform ubuntu and version 16.04 as shown below.
...
runner = ChefSpec::ServerRunner.new(platform: 'ubuntu', version: '16.04')
...When you run ChefSpec again, you'll see all 3 tests passed.
$ chef exec rspec ~/cookbooks/custom_apache
Finished in 0.53752 seconds (files took 2.17 seconds to load)
3 examples, 0 failuresFor brevity of the tutorial, we are not going to implement integration tests. However, you should take a look at InSpec to write integration test which helps you to verify multiple components function correctly together.
It is important to provide appropriate metadata and documentation that helps other team members understand what your cookbook does and how to use it. To do that, make sure you have at least update
You can add more files for documentation depending on your case. However, it's better to try to keep it tidy inside of the cookbook directory.
Perfect! In this walkthrough, you learn how to update a cookbook by using the tools that Chef provides to ensure following the best practices.
2,603 posts | 747 followers
FollowAlibaba Clouder - October 12, 2019
Alibaba Clouder - July 5, 2019
Alibaba Clouder - September 7, 2020
Alibaba Clouder - October 12, 2019
Alibaba Clouder - June 14, 2019
Alibaba Clouder - July 24, 2020
2,603 posts | 747 followers
Follow
ECS(Elastic Compute Service)
Elastic and secure virtual cloud servers to cater all your cloud hosting needs.
Learn More
Message Service
A message queuing and notification service that facilitates smooth transfer of messages between applications
Learn MoreLearn More
More Posts by Alibaba Clouder