Automate Terraform Documentation

Terraform Tips #2

Photo by Sam Dan Truong on Unsplash

I recently posted about restoring and recreating Terraform code and resources by delving into the state file.

I’d like to continue with another useful tip for Terraform code, this time from the coding aspect.

Here at BigPanda we manage and define our Cloud-based infrastructure as code using Terraform. In a complex SAAS Cloud-based product, our Terraform infrastructure is constantly growing and changing. Our Terraform code includes different workspaces, modules, and sub-modules. For a newcomer to the codebase, it can be hard to understand how to use each module, interact with them and what dependencies they require.

Like in all coding done by teams, it is useful to include documentation to assist new developers and ease their way into the codebase. I love documentation in the form of README.md files. One advantage of using README.md files is their simplicity and readability.
I especially like the fact that the documentation is part of the codebase and version controlled together with the code.

When creating documentation for Terraform, most tools generate Terraform documentation by inspecting the Terraform code directly. One of these tools is terraform-docs utility. You can install it as a standalone binary and then can be run in the directory in which the Terraform code resides. We can run the command below:

terraform-docs markdown . > README.md

This will output Markdown (my favorite documentation form!) documenting the Terraform module into a README.md file.
The information we get about the module is as follows:

• Terraform and provider version requirements.
• Module dependencies and their versions.
• Resources and sub-modules created by the module.
• Input variables the module expects.
• Output variables from the module.

For example, let's take a look at a module we have for managing AWS ACM (AWS Certificate Manager) resources.

This is the directory contents:

❯ ls -l
-rw-r--r--  1 jtavin  staff  1264 Dec 29 10:23 main.tf
-rw-r--r--  1 jtavin  staff   124 Dec 29 10:23 outputs.tf
-rw-r--r--  1 jtavin  staff   463 Dec 29 11:32 variables.tf
-rw-r--r--  1 jtavin  staff   220 Jan 16 11:27 versions.tf

And when running the command terraform-docs markdown . > README.md in that directory, we get the README.md file you can see below:

README.md example

So Far So Good!

We showed an easy method to generate Terraform documentation in our repository using a simple utility command. We can run this command in each Terraform module directory. Each module in the repository will then get its README.md file documenting the module properties. The only thing that needs to be done when changing the module code, is to rerun the terraform-docs command to update and show the new version of the documentation in the README.md file.

Is that Scalable?

Many developers across the company contribute to our Terraform codebase. Due to the fast pace and changes being done to the infrastructure constantly, developers tend to forget to run the terraform-docs command and re-generate the documentation for their new code. As a result, most of the time, we had an up-to-date codebase but mostly outdated documentation 🤕.

Automation to the Rescue — Git Hooks!

The software industry understands the high price we pay when issues are discovered after code is shipped to production.

Therefore, we try to catch code issues as soon as possible, even before the code has left the developer’s laptop. Unit Tests, Linters, and Code Style Guideline Enforcers to name a few, are methods and tools being used for this.

Fortunately for those using Git source control, there is a powerful mechanism for automation during the Git workflow process.

The feature is called Git Hooks and it enables us to “hook” into various stages of our Git workflow. These hooks can be on the client or server side, and they can run any executable we choose to place in the .git/hooks directory. Additionally, these hooks can run at many different stages of the Git workflow, before or after rebase, pre-commit, post-commit, and at many more stages.

It is of course possible to manage Git hooks manually by adding executable scripts and files directly to the .git/hooks directory. But for hooks that we want to run right before the Git commit is created, AKA pre-commit hooks, there is a better way!

Pre-Commit

One of the most popular frameworks for managing pre-commit Git hooks is called (Surprise Surprise…) pre-commit. It is a python based tool that enables a simple and easy framework for configuring and running pre-commit Git hooks. You can install it using the python pip package manager:

pip install pre-commit

Adding pre-commit checks to the Git repository is done as follows:

1. Add a file called .pre-commit-config.yaml to the repository root.
   The file must have some basic pre-commit configuration.
   You can even use pre-commit itself to auto-generate a sample configuration file for you by using:

pre-commit sample-config > .pre-commit-config.yaml

2. Install the pre-commit hooks specified in the config file to the .git/hooks directory.

pre-commit install

This step ensures that all the hooks specified in the pre-commit configuration file, exist as a Git hook under the .git/hooks directory.

Since we wanted our Terraform README.md files to stay up-to-date automatically. The following .pre-commit-config.yaml configuration file will do just that!

repos:
  - repo: https://github.com/antonbabenko/pre-commit-terraform
    rev: v1.77.0
    hooks:
      - id: terraform_docs

The terraform_docs hook will now run right before the commit is created and automatically change and update the markdown files with the up-to-date code documentation according to the current version of the code.

You can also add many other useful pre-commit checks to your .pre-commit-config.yaml file to run linting, validation, and code formatting for Terraform.

For example, add - id: terraform_fmt to the list of hooks under the hooks: keyword.
This hook will run the terraform fmt command for auto-formatting the code according to the default Terraform code styling.

Important!

When we ran the terraform-docs command manually, we controlled exactly where the output generated was being sent to.

However, when using the pre-commit hook, you must first add the following brackets into your README.md to have it automatically insert the documentation into the README.md files.

<!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->

When these brackets are added to the README.md and the terraform_docs Git hook is triggered, documentation will be automatically generated into all README.md files in the repository between those two brackets.

An additional advantage of using brackets is that you can still use the rest of the README.md file as usual for any other type of instructions or documentation outside these brackets without worrying that terraform-docs will override your regular README.md content.

Conclusion

We showed how to easily automate the documentation of your Terraform code using the terraform_docs pre-commit Git hook.

Using this pre-commit hook gives us the following advantages:

1. Documentation is generated automatically.
2. Documentation will stay up to date with the code.
3. We minimize the cognitive and technical effort of developers for documentation.

I hope this helps you streamline your Terraform workflow as it did ours!

Let me know your thoughts and feel free to connect at https://www.linkedin.com/in/jtavin/