printf("Writing a Terraform provider")
There is already a bunch of articles out there to help you create a Terraform provider. However after having done it myself I wanted to write about it. Mostly to keep track of how I did it but also to try to give you a few hints to write your own. The idea here is to go through the entire process.
If you are reading this post it’s likely that you already know which provider you want to implement. You also probably know that the API you will be using to write your plugin will need to implement the default CRUD functions. If not, you should take a look at the external data source resource.
Before starting, some assumptions:
- You use Terraform and are familiar with all the core concepts (in particular providers and resources).
- You know how to write in Go. No need to be an expert but you should be able to use any kind of SDK or API by reading the documentation.
Writing a Terraform provider
Basically a provider is composed of two parts : the “provider” itself and some resources. If you are using Terraform to manage your AWS infrastructure, you have something like :
provider "aws" {
access_key = "${var.aws_access_key}"
secret_key = "${var.aws_secret_key}"
region = "${var.aws_region}"
}
resource "aws_vpc" "main" {
cidr_block = "10.0.0.0/16"
}
Here, the “aws” provider part is taking care of setting up your AWS Client and authenticate you to the AWS API. Then the “aws_vpc” resource will create an AWS VPC with the correct CIDR block for you, using this AWS Client.
If you take a look at the existing providers, you will notice that the structure is almost always something like :
provider.go
: Implement the “core” of the Provider.config.go
: Configure the API client with the credentials from the Provider.resource_<resource_name>.go
: Implement a specific resource handler with the CRUD functions.import_<resource_name>.go
: Make possible to import existing resources. We won’t expand on resources in this post.data_source_<resource_name>.go
: Used to fetch data from outside of Terraform to be used in other resources. For example, you will be able to fetch the latest AWS AMI ID and use it for an AWS instance. Same as “import”, we won’t go further on this in this post.
Writing the “provider” is the easiest part so let’s start here.
The provider
To write your provider, you need to implement a terraform.ResourceProvider. It might seem complicated at first but it’s actually pretty easy.
This is how you start :
// File : provider.go
package main
import (
"github.com/hashicorp/terraform/helper/schema"
"github.com/hashicorp/terraform/terraform"
)
func Provider() terraform.ResourceProvider {
return &schema.Provider{
Schema: map[string]*schema.Schema{ },
ResourcesMap: map[string]*schema.Resource{ },
ConfigureFunc: configureProvider,
}
}
func configureProvider(d *schema.ResourceData) (interface{}, error) {
return nil, nil
}
Basically we have a function Provider()
which is returning a terraform.ResourceProvider
with all the required configuration to do the job :
Schema
is where you list the parameters of your provider. For instance, with AWS we have the access_key and the secret_key.ResourceMap
is the list of the resources managed by your provider.ConfigureFunc
is the function which, among other things, instantiates and configures the client you use to interact with the targeted API (AWS SDK for example).
Let’s go ahead and write a unit test for this provider. Even is the Provider is not operational right now, I always try to write the test as soon as possible.
// File : provider_test.go
import (
"os"
"testing"
"github.com/hashicorp/terraform/helper/schema"
"github.com/hashicorp/terraform/terraform"
)
var testAccProvider *schema.Provider
func init() {
testAccProvider = Provider().(*schema.Provider)
}
func TestProvider(t *testing.T) {
if err := Provider().(*schema.Provider).InternalValidate(); err != nil {
t.Fatalf("err: %s", err)
}
}
func TestProvider_impl(t *testing.T) {
var _ terraform.ResourceProvider = Provider()
}
func testAccPreCheck(t *testing.T) {
// We will use this function later on to make sure our test environment is valid.
// For example, you can make sure here that some environment variables are set.
}
The init()
function set the testAccProvider
variable with our provider and we’ll just make sure here that Terraform is happy with our implementation.
$ go test -v
=== RUN TestProvider
--- PASS: TestProvider (0.00s)
=== RUN TestProvider_impl
--- PASS: TestProvider_impl (0.00s)
PASS
ok github.com/Pryz/terraform-provider-fake 0.005s
Next you need to configure the client for the API you want to interact with. Let’s say we are writing a provider for a really simple API. This API requires a user and a token for authentication.
This will be in the Schema of your provider :
// File : provider.go
Schema: map[string]*schema.Schema{
"user": &schema.Schema{
Type: schema.TypeString,
Required: true,
DefaultFunc: schema.EnvDefaultFunc("API_USER", nil),
Description: "API User",
},
"token": &schema.Schema{
Type: schema.TypeString,
Required: true,
DefaultFunc: schema.EnvDefaultFunc("API_TOKEN", nil),
Description: "API Token",
},
}
Those two variables are strings so we use the type schema.TypeString
. Also we make sure that they are set by using Required: true
.
Get yourself familiar with schema.Schema
, you will also use it for the Resources. For instance, check the different types supported here :
schema#Schema.
You can now use those parameters to configure the client within configureProvider
. This function should return a configured API client. Hence the usage of interface{}
.
func configureProvider(d *schema.ResourceData) (interface{}, error) {
user := d.Get("user").(string)
token := d.Get("token").(string)
return SimpleFakeApi.New(user, token)
}
At this point you have a working provider ! Well, you don’t have any resource so even if you use it in a Terraform manifest, you will see nothing happening :)
The first resource
This is where the fun begins. As we did above with the Provider
, let’s define the skeleton of your first resource :
// File : resource_fake_object.go
package main
import (
"github.com/hashicorp/terraform/helper/schema"
)
func resourceFakeObject() *schema.Resource {
return &schema.Resource{
Create: resourceFakeObjectCreate,
Read: resourceFakeObjectRead,
Update: resourceFakeObjectUpdate,
Delete: resourceFakeObjectDelete,
Exists: resourceFakeObjectExists,
Schema: map[string]*schema.Schema{ },
}
}
func resourceFakeObjectExists(d *schema.ResourceData, meta interface{}) (b bool, e error) {
return true, nil
}
func resourceFakeObjectCreate(d *schema.ResourceData, meta interface{}) error {
return nil
}
func resourceFakeObjectRead(d *schema.ResourceData, meta interface{}) error {
return nil
}
func resourceFakeObjectDelete(d *schema.ResourceData, meta interface{}) error {
return nil
}
First things we have here is the definition of the CRUD functions :
Create
will simply create a new instance of your resource. The is also where you will have to set the ID (has to be an Int) of your resource. If the API you are using doesn’t provide an ID, you can always use a random Int.Read
will fetch the data of a resource.Update
is optional if your Resource doesn’t support update. For example, I’m not using update in the Terraform LDAP Provider. I just destroy and recreate the resource everytime there is a change.Exists
is called beforeRead
and obviously makes sure the resource exists.
Then, as you can see, we have the Schema
again ! Nothing really new here compare to the provider except if you have to use more complex attributes than string.
For instance, if you have something like a list of tags :
resource "fake_object" {
...
tags = ["spaceship", "beer"]
}
You will want to use the schema.TypeList
type. The declaration will be :
Schema: map[string]*schema.Schema{
...
"tags": &schema.Schema{
Type: schema.TypeList,
Required: true,
ForceNew: true,
Elem: &schema.Schema{Type: schema.TypeString},
},
}
To retrieve those tags
within the CRUD function, you can do something like :
objectTags := []string{}
for _, tag := range d.Get("tags").([]interface{}) {
objectTags = append(objectTags, tag.(string))
}
Things can quickly get tricky here. For example, if you need to implement attributes which can be specified multiple times like in the aws_security_group
resource :
resource "aws_security_group" "web" {
name = "web-http-https"
ingress {
from_port = 80
to_port = 80
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
ingress {
from_port = 443
to_port = 443
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
}
You will use the type schema.TypeSet
or schema.TypeMap
and use another level of schema.Schema
to define all the parameters of the ingress
and egress
rules. See resource_aws_security_group.go#L83.
In the case of the schema.TypeSet
, you will have the use the function Set()
in order to retrieve all the values. For instance :
ingressRules := d.Get("ingress").(*schema.Set).List()
Build it and try it
Throughout this post we have been using the package main
. We did this mostly because our plugin is standalone
. If you want to Pull Request your plugin within the Terraform Github, you will
have to change main
by the name of your plugin.
Since our plugin is standalone, we need a main :
package main
import (
"github.com/hashicorp/terraform/plugin"
)
func main() {
plugin.Serve(&plugin.ServeOpts{
ProviderFunc: Provider,
})
}
Build it with a name that Terraform can understand :
go build -o terraform-provider-fake
Tell Terraform where to find your plugin :
cat >> ~/.terraformrc <<EOF
providers {
ldap = "${GOPATH}/bin/terraform-provider-fake"
}
EOF
Write a quick manifest : main.tf
provider "blah" {
login = "foo"
password = "bar"
}
resource "blah_service" {
name = "foo"
content = "bar"
}
Plan and apply !
terraform plan
terraform apply
Final words
This post is a quick walkthrough to give you a starting point to write Terraform Providers. There is much more details to talk about like Imports
and Data Sources
, or also Partial States
.
Also I didn’t talk about how to test the Resources. There are really damn good examples out there. Take for instance this one : resource_datadog_monitor_test.go.
It’s always a good idea to look at the existing providers. Some are pretty complex like the AWS provider. Others are easier to understand, for instance the Datadog one.