Terraform Associate


1. Understanding IaC Concepts

Infrastructure as Code (IaC)

Types of IaC Tools:

  1. Configuration Management Tools

    • Examples: Ansible, Puppet, SaltStack, Chef
    • Purpose: Install/manage software on existing infrastructure
    • Features:
      • Maintains consistent code structure
      • Can run on multiple remote resources
      • Version control compatible
      • Idempotent (only makes necessary changes)
  2. Server Templating Tools

    • Examples: Docker, Vagrant, Packer
    • Purpose: Create custom VM/container images
    • Features:
      • Pre-installed software/dependencies
      • Promotes immutable infrastructure
      • Examples: VM images, custom AMIs, Docker images
  3. Infrastructure Provisioning/Orchestration Tools

    • Examples: Terraform, CloudFormation
    • Purpose: Provision infrastructure components
    • Features:
      • Uses declarative code
      • Supports various infrastructure components
      • CloudFormation: AWS-specific
      • Terraform: Vendor-agnostic, multi-cloud support

Key Differences:

  • Procedural vs Declarative Approach
    • Configuration tools (like Ansible): Procedural
      • Requires specific steps
      • Needs explicit instructions for each action
    • Orchestration tools (like Terraform): Declarative
      • Focuses on end state
      • Maintains state file
      • Easier resource management

Best Practices:

  1. Choose tools based on specific needs:

    • AWS-only: Consider CloudFormation
    • Multi-cloud/hybrid: Consider Terraform
  2. Combine tools effectively:

    • Use Terraform for infrastructure provisioning
    • Use configuration management tools for post-provisioning tasks
  3. Consider organizational requirements:

    • No one-size-fits-all solution
    • Evaluate based on specific use cases

Installing Terraform and HCL Basics

  1. Installation
  • Download binary for desired OS
  • Copy to system path
  • Supported on Windows, Mac, Linux, Solaris, OpenBSD
  1. HashiCorp Configuration Language (HCL)
  • Files use .tf extension
  • Composed of blocks and arguments
  • Used to define infrastructure resources
  1. Configuration File Structure a) Blocks
  • Identified by curly braces {}
  • Resource block starts with “resource” keyword
  • Contains: resource type and resource name

b) Resource Type Format

  • Provider_ResourceType (e.g., aws_instance)
  • Provider: word before underscore
  • Resource type: word after underscore

c) Arguments

  • Key-value pairs inside blocks
  • Specific to resource type
  • Can be mandatory or optional
  1. Resource Examples a) Local File
resource "local_file" "pet" {
    filename = "..."
    content = "..."
}

b) AWS EC2 Instance

resource "aws_instance" "web" {
    ami = "..."
    instance_type = "..."
}
  1. Terraform Workflow a) Core Steps
  2. Write configuration file
  3. terraform init (downloads required plugins)
  4. terraform plan (reviews execution plan)
  5. terraform apply (implements changes)

b) Additional Commands

  • terraform show: displays resource details
  • terraform apply -auto-approve: applies without confirmation
  1. Documentation
  • Available for all providers
  • Lists all supported resources
  • Details required and optional arguments
  • Provides usage examples
  1. Best Practices
  • Always review plan before applying
  • Use auto-approve with caution
  • Refer to documentation for argument details
  1. Resources Types
  • Can be local system resources
  • Cloud resources (AWS, GCP, Azure)
  • Services (S3, DynamoDB, Lambda)
  • Hundreds of resources across different providers

Updating and Destroying Infrastructure with Terraform

Resource Updates:

  • Update configuration file with new arguments/values
  • Run terraform plan (optional) to preview changes
  • -/+ symbol indicates resource will be deleted and recreated
  • “forces replacement” shows reason for recreation
  • terraform apply implements changes
  • Follows immutability concept

Destroying Resources:

  • Use terraform destroy command
  • Shows execution plan with minus (-) symbols
  • Requires confirmation unless using -auto-approve flag
  • Caution: -auto-approve destroys without confirmation

Configuration Directory Structure:

  • Terraform reads all .tf files in working directory
  • Multiple .tf files allowed in same directory
  • Common practices:
    • Single main.tf with all configurations
    • Split into multiple files (e.g., variables.tf, outputs.tf)
    • Can organize related resources in separate files

File Organization:

  • Can have multiple resource blocks in one file
  • Splitting into smaller files helps with management
  • Common files:
    • main.tf
    • variables.tf
    • outputs.tf
    • Other specific resource files

Best Practices:

  • Single file possible but may become difficult to manage
  • Splitting configurations recommended for better organization
  • Use consistent naming conventions
  • Consider maintainability when structuring files

2. Terraform Provider Basics

Terraform Providers

  1. Initial Command
  • terraform init: First command run with valid configuration
  • Installs required plugins for providers specified in configuration
  1. Provider Types a) Official Providers
  • Owned/maintained by HashiCorp
  • Includes major cloud providers (AWS, GCP, Azure)
  • Marked with official badge in Registry

b) Partner Providers

  • Owned/maintained by third-party companies
  • Reviewed and tested by HashiCorp
  • Identified by check-mark badge

c) Community Providers

  • Published/maintained by individual contributors
  1. Terraform Init Details
  • Safe command: Can be run multiple times without affecting infrastructure
  • Plugins downloaded to .terraform/plugins in working directory
  • Example plugin: hashicorp/local version 2.0.0
  1. Plugin Name Structure
  • Format: [hostname]/[organizational namespace]/[type]
  • Organizational namespace: Usually ‘hashicorp’ for Public Registry
  • Type: Name of provider (e.g., local, AWS, AzureRM, Google, Random)
  • Hostname: Optional, defaults to registry.terraform.io
  1. Provider Source Address Examples Full format: registry.terraform.io/hashicorp/local Shortened format: hashicorp/local

  2. Registry Information

  • Public Terraform Registry: registry.terraform.io
  • Supports hundreds of platforms
  • Uses plugin-based architecture

Multiple Providers in Terraform

  1. Basic Concept:
  • Multiple providers can be used within a single Terraform configuration
  • Allows provisioning resources across different platforms in same configuration
  1. Implementation:
  • When adding a new provider:
    • Must run ’terraform init’ command again
    • Previously installed providers are reused
    • New provider plugins are downloaded and installed
  1. Workflow Steps:
  1. terraform init
  2. terraform plan
  3. terraform apply
  1. Example Providers:

a) Local Provider:

  • Used for local file resources

b) Random Provider:

  • Logical provider (doesn’t create actual infrastructure)
  • Generates and stores random values
  • Types include:
    • random_pet: generates random pet names
      • Can specify number of words
      • Can add specific prefix
    • random_string: creates random strings
      • Can specify length
      • Can control character types (uppercase, special chars)
  1. Practical Example:
  • Combined configuration using:
    • Random provider (server-suffix resource)
      • Creates 6-character string
      • No uppercase/special characters
    • AWS provider
      • Creates AWS instance
      • Uses specific AMI ID and instance type
      • Tags include randomly generated string
  1. Resource Attributes:
  • Can reference attributes between resources
  • Helps link different resources together
  • Example: Using random string as part of AWS instance name tag

Provider Version Constraints in Terraform

  1. Default Behavior
  • terraform init downloads latest provider plugin version by default
  • This may cause compatibility issues with existing configurations
  1. Version Specification
  • Provider versions can be specified in terraform block
  • Uses required_providers block inside terraform block
  • Example structure:
    terraform {
      required_providers {
        local = {
          source = "[source_address]"
          version = "[version_number]"
        }
      }
    }
    
  1. Version Constraint Operators a) Exact Version
    • Specify exact version number (e.g., version = “1.4.0”)

b) Not Equal (!=)

  • Excludes specific version
  • Example: version != “2.0.0”

c) Comparison Operators

  • Greater than (>)
  • Less than (<)
  • Can be combined for range specification
  • Example: version > “1.2.0” and < “2.0.0”

d) Pessimistic Constraint (~>)

  • Allows incremental versions
  • Two formats:
    • ~> X.Y: Allows X.Y through X.9
    • ~> X.Y.Z: Allows X.Y.Z through X.Y.9
  1. Version Selection Process
  • Terraform selects highest available version within constraints
  • Downloads specified version during terraform init
  • Version must be available in registry
  1. Example Scenarios
  • ~> 1.2 allows versions 1.2 through 1.9
  • ~> 1.2.0 allows versions 1.2.0 through 1.2.9
  • Version selection limited by available versions in registry

Provider Aliases in Terraform

Definition & Purpose:

  • Provider aliases allow multiple configurations for the same provider
  • Commonly used to create resources in different regions within the same cloud platform

Implementation Example:

  1. Basic Configuration:

    • Default provider block (without alias):
      provider "aws" {
        region = "us-east-1"
      }
      
  2. Adding Alias Configuration:

    • Additional provider block with alias:
      provider "aws" {
        region = "ca-central-1"
        alias  = "central"
      }
      
  3. Resource Usage:

    • Default provider (implicit):

      • Resources use default provider if no specific provider is mentioned
      • Example: Resources created in us-east-1 region
    • Aliased provider (explicit):

      • Syntax: provider = <provider_name>.<alias_name>
      • Example: provider = aws.central
      • Resources specifically mention which provider configuration to use

Example Scenario:

  • Creating two EC2 key pairs:
    1. Alpha key pair: Created in us-east-1 (using default provider)
    2. Beta key pair: Created in ca-central-1 (using aliased provider)

Verification:

  • Use ’terraform show’ command to verify resources are created in intended regions
  • Resources without provider specification use default provider configuration
  • Resources with provider specification use the specified aliased configuration

3. Variables, Resource Attributes, and Dependencies

VARIABLES IN TERRAFORM

  1. Basic Structure:
  • Variables replace hard-coded values
  • Defined using ‘variable’ block
  • Commonly placed in variables.tf file (not mandatory)
  • Can be defined alongside resource blocks in main.tf
  1. Variable Declaration:
  • Syntax: variable “name”
  • Default values are optional
  • Usage in configuration: var.variable_name
  • No quotes needed when using variables in configuration
  1. Ways to Assign Variables:

a) Default Values:

  • Defined in variable block
  • Can be overridden during execution

b) Command Line:

  • Using -var option
  • Format: -var “variable_name=value”
  • Multiple variables can be passed

c) Environment Variables:

  • Format: TF_VAR_variable_name
  • Must be in uppercase

d) Variable Definition Files:

  • Extensions: .tfvars or .tfvars.json
  • Auto-loaded files:
    • terraform.tfvars
    • terraform.tfvars.json
    • *.auto.tfvars
    • *.auto.tfvars.json
  • Custom-named files require -var-file flag
  1. Variable Precedence (Highest to Lowest):

  2. Command-line flags (-var and -var-file)

  3. *.auto.tfvars files (alphabetical order)

  4. terraform.tfvars file

  5. Environment variables

  6. Important Notes:

  • Default values are optional
  • Multiple assignment methods can be used simultaneously
  • Higher precedence values override lower ones
  • Variables make configuration more flexible and reusable

VARIABLE BLOCK BASICS

  • Can be empty (requires value during execution)
  • Optional arguments:
    • default: Sets default value
    • description: Documents variable purpose
    • type: Defines data type
    • sensitive: Boolean (true/false) to suppress value display

VALIDATION RULES

  • Can add validation blocks inside variable blocks
  • Use conditions to enforce rules
  • Custom error messages for validation failures
  • Example: AMI ID validation using substr function

BASIC VARIABLE TYPES

  1. String: Alphanumeric values
  2. Number: Positive/negative numbers
  3. Boolean: true/false
  4. Any: Default type if not specified

TYPE CONVERSION

  • Possible between:
    • String ↔ Number
    • Boolean ↔ String
  • Must match type and default value
  • Terraform attempts automatic conversion when possible

COMPLEX DATA TYPES

  1. List

    • Numbered collection of values
    • Zero-based indexing
    • Access: var.variable_name[index]
  2. Map

    • Key-value pairs
    • Access values using keys
    • Example: var.instance_type[key]
  3. Set

    • Similar to list
    • No duplicate elements allowed
  4. Object

    • Complex data structure
    • Combines multiple variable types
    • Example: Can include strings, numbers, lists, booleans
  5. Tuple

    • Similar to list but allows different variable types
    • Must match defined type sequence exactly
    • Fixed length

TYPE CONSTRAINTS

  • Can combine type constraints (e.g., list(string), map(number))
  • Must match specified constraints
  • Invalid type combinations will cause errors

Output Variables in Terraform

Definition:

  • Output variables store values of expressions in Terraform
  • Complement to input variables

Syntax:

  1. Basic Structure:
    • Keyword: ‘output’
    • Variable name
    • Mandatory argument: value (reference expression)
    • Optional: description

Display/Usage:

  • Shown after running ’terraform apply’
  • Displays even without changes
  • Commands:
    • ’terraform output’ - shows all output variables
    • ’terraform output <variable_name>’ - shows specific variable

Example Use Case:

  • Storing EC2 instance’s public IP address from Web resource block

Primary Applications:

  1. Quick display of provisioned resource details
  2. Integration with other IaC tools:
    • Ad-hoc scripts
    • Ansible playbooks
    • Configuration management
    • Testing

Note: Not required for dependent resources using reference expressions between resource blocks.

Resource Attributes and Dependencies in Terraform

  1. Resource Attributes
  • Terraform records multiple values for each provisioned resource
  • Attributes can be viewed using ’terraform show’ command
  • Example: aws_key_pair resource exports attributes like:
    • arn
    • fingerprint
    • id
    • key_name
    • public_key
    • tags_all
  1. Using Resource Attributes
  • Exported attributes can be used as input for other resources
  • Reference Expression Syntax: <RESOURCE_TYPE>.<RESOURCE_NAME>.
  • Example: aws_key_pair.alpha.key_name
  1. Dependencies a) Implicit Dependencies
  • Occur when one resource uses attributes from another resource
  • Terraform automatically determines creation order
  • Example: EC2 instance depending on key_pair resource
  • Deletion occurs in reverse order

b) Explicit Dependencies

  • Used when there’s no attribute-based dependency
  • Implemented using ‘depends_on’ meta argument
  • Example: Making web instance depend on db instance
  • Syntax:
    depends_on = [resource_name]
    
  1. Resource Creation Behavior
  • By default, Terraform creates resources in parallel
  • Dependencies (implicit or explicit) affect creation order
  • Resources with dependencies wait for dependent resources to be created first

Data Sources in Terraform

Definition & Purpose:

  • Data sources allow access to resources that exist outside current Terraform configuration
  • Used to read information from specific resources not managed by current Terraform setup

When to Use Data Sources:

  • Accessing existing resources created manually
  • Resources created by other tools (CloudFormation, Ansible)
  • Resources from different Terraform configurations

Syntax & Structure:

  1. Data Block Format:

    • Keyword: ‘data’
    • Instance type (e.g., aws_key_pair)
    • Logical name (user-defined)
    • Arguments to identify resource
  2. Example Usage:

data "aws_key_pair" "cerberus-key" {
    key_name = "alpha"
}

Resource Identification Methods:

  • Direct identification (e.g., key_name)
  • Using key_id
  • Using filters (e.g., tags)
  • Multiple identification options available as per documentation

Comparison: Resources vs Data Sources

  1. Resources:

    • Created with ‘resource’ block
    • Used for creating, updating, destroying infrastructure
    • Called “managed resources”
  2. Data Sources:

    • Created with ‘data’ block
    • Used only for reading information
    • Called “data resources”
    • Cannot modify infrastructure

Reference Expression Usage:

  • Can reference data source attributes similar to regular resources
  • Used in resource configurations that depend on existing infrastructure

4. Terraform State

Terraform State

  1. Basic Concepts:
  • State file (terraform.tfstate) created after first terraform apply
  • Backup file (terraform.tfstate.backup) created by default
  • Contains complete JSON record of infrastructure
  • Acts as single source of truth for terraform operations
  1. State File Contents:
  • Resource IDs
  • Provider information
  • Resource attributes
  • Dependencies between resources
  • Sensitive information (passwords, SSH keys)
  1. State Operations:
  • Automatically refreshes during plan/apply commands
  • Checks current state against real-world infrastructure
  • Can disable refresh using -refresh=false option
    • Not recommended
    • Useful in large environments to save time
  • State file itself cannot be disabled (mandatory)
  1. State Management:
  • Tracks resource dependencies
  • Determines provisioning/destruction order
  • Example: DB instance created before web instance
  • During removal: web instance destroyed before DB instance
  1. Best Practices:
  • Never store state files in version control (GitHub, GitLab, Bitbucket)
  • Use secure remote backends (Amazon S3, Terraform Cloud)
  • Store only configuration files in version control
  • Never manually edit state files
  • Use terraform commands for state manipulation
  1. Security Considerations:
  • State files contain sensitive information
  • Store in secure remote backends
  • Default local storage as plain text is risky
  • Implement proper access controls
  1. State Refresh:
  • Occurs automatically during plan/apply
  • Ensures sync between configuration and real infrastructure
  • Can be disabled but not recommended
  • Important for maintaining consistency

Remote State in Terraform

Remote State Overview:

  • Alternative to local state files for better team collaboration
  • Stores state files in secured shared storage instead of local directory
  • Not recommended to store state files in version control systems due to:
    • Security concerns (sensitive information)
    • Lack of state locking support
    • Risk of conflicts and data loss

Benefits of Remote State:

  1. Automatic state file management
    • Loads from shared storage when needed
    • Updates after every apply
  2. State locking capability
    • Prevents concurrent operations
    • Maintains file integrity
  3. Enhanced security
    • Encryption at rest and in transit
    • Secure storage of sensitive information

Remote Backend Options:

  • AWS S3 bucket
  • HashiCorp Console
  • Google Cloud Storage
  • Terraform Cloud

Configuring Remote State (AWS S3 Example): Prerequisites:

  • Existing S3 bucket
  • Optional DynamoDB table (for state locking)

Configuration Steps:

  1. Create Terraform block
  2. Add backend block with:
    backend "s3" {
      bucket = "bucket-name"
      key    = "path/to/state-file"
      region = "aws-region"
      dynamodb_table = "table-name" #optional
    }
    
  3. Run terraform init
    • Initializes new backend
    • Option to copy existing local state to remote
  4. Remove local state file after successful migration

Operation:

  • State file managed automatically in remote storage
  • Locked during operations
  • Released after completion
  • No local state file stored in configuration directory

5. Read, Generate, and Modify Configuration

Count and For_each meta-arguments in Terraform

Count Meta-Argument:

  • Used to create multiple instances using the same resource block
  • Example: Setting count = 3 creates three EC2 instances
  • Resources are created as a list (aws_instance.web[0], web[1], web[2])
  • Can use list-type variables with length function instead of hard-coding count
  • Allows looping through list variables to assign tags using count.index

Drawback of Count:

  • Resources are managed as an ordered list
  • Removing elements from the list causes index shifting
  • Example: Removing first element causes:
    • Existing resources to be updated due to index reorganization
    • Last resource to be destroyed
    • Unexpected changes to infrastructure

For_each Meta-Argument:

  • Alternative method to create multiple instances
  • Works with set or map type variables (not list type)
  • Creates resources as a map instead of list
  • Syntax: for_each = var.webservers
  • References values using each.value expression

Advantages of For_each:

  • More predictable behavior when removing elements
  • Only affects the specific resource being removed
  • Other resources remain unaffected
  • Better for managing multiple similar resources

Key Differences:

  • Count creates ordered list of resources
  • For_each creates map of resources
  • For_each provides better stability when modifying resources
  • For_each requires set or map type variables

Provisioners in Terraform

PROVISIONERS OVERVIEW:

  • Used to execute tasks/commands on remote resources or locally
  • Runs after resource creation by default

TYPES OF PROVISIONERS:

  1. Remote-exec Provisioner:
  • Executes commands on remote resources
  • Requirements:
    • Network connectivity (SSH for Linux, WINRM for Windows)
    • Appropriate security groups
    • Authentication mechanism (e.g., SSH key pair)
  • Connection Block needed:
    • Specifies connection type (SSH/WINRM)
    • Host (public IP)
    • User credentials
    • Authentication keys
  1. Local-exec Provisioner:
  • Executes commands on local machine running Terraform
  • Useful for local data collection and file operations
  • Runs within resource block

PROVISIONER BEHAVIORS:

Timing:

  • Create-time provisioner (default): Runs after resource creation
  • Destroy-time provisioner: Runs before resource destruction
    • Uses ‘when = destroy’ argument

Failure Handling:

  • Default: Fails terraform apply if provisioner fails
  • Resource marked as “tainted” on provisioner failure
  • Can set ‘on_failure = continue’ to prevent operation failure

BEST PRACTICES:

  • Use provisioners as last resort
  • Prefer native resource options:
    • AWS: user_data
    • Azure: custom_data
    • GCP: meta_data

Example Use Cases:

  • Running scripts post-deployment
  • Installing/configuring software on instances
  • Collecting and storing resource information locally

Terraform Built-in Functions

  1. Interactive Console
  • Access using “terraform console” command
  • Loads state from configuration directory
  • Allows testing functions and interpolations
  • Can access stored variables
  1. Common Functions Previously Used
  • file: reads data from files
  • length: counts elements in lists/maps
  • toset: converts list to set, removes duplicates
  1. Function Categories a) Numeric Functions
  • max: returns greatest number
  • min: returns smallest number
  • ceil: rounds up to nearest whole number
  • floor: rounds down to nearest whole number

b) String Functions

  • split: transforms string to list using separator
  • lower: converts string to lowercase
  • upper: converts string to uppercase
  • title: capitalizes first letter of each word
  • substring: extracts part of string using offset and length
  • join: converts list to string

c) Collection Functions

  • length: counts elements
  • index: finds position of element in list
  • element: retrieves element at specific index
  • contains: checks if element exists (returns true/false)

d) Map Functions

  • keys: converts map to list of keys
  • values: converts map to list of values
  • lookup: finds value for specific key
    • Optional default value parameter
    • Returns error if key not found and no default provided
  1. Variable Usage
  • Use three dots (…) for expansion symbol
  • Variables can be tested in console
  • Can combine functions with variables
  1. Important Notes
  • Set data type cannot have duplicates
  • List indexing starts at 0
  • Function arguments syntax varies based on function type
  • Case sensitivity matters in contains function
  • Map lookups require exact key matches

These functions provide essential tools for data manipulation and transformation in Terraform configurations.

Operators and Conditional Expressions in Terraform

  1. Testing Operations using Terraform Console
  • Can test arithmetic and logical operations
  • Supports arithmetic, equality, logical, and comparison operators
  1. Types of Operators

a) Arithmetic Operators

  • Basic mathematical operations (+, -, *, /)

b) Equality Operators

  • Returns Boolean (true/false)
  • Double equals (==) for equality
  • Exclamation equals (!=) for inequality
  • Type-sensitive comparisons

c) Comparison Operators

  • Greater than (>)
  • Less than (<)
  • Greater than or equal to (>=)
  • Less than or equal to (<=)
  • Works with number type values

d) Logical Operators

  • AND (&&): Returns true only if both conditions are true
  • OR (||): Returns true if either condition is true
  • NOT (!): Inverts the Boolean result
  1. Conditional Expressions
  • Syntax: condition ? value_if_true : value_if_false
  • Used for conditional value assignment
  • Similar to if-else statements in other languages
  1. Practical Example: Random Password Generation
  • Uses random_password resource type
  • Implementation of minimum length requirement:
    length = var.length < 8 ? 8 : var.length
    
  • If length < 8, uses 8 as default
  • If length ≥ 8, uses provided length value
  1. Variable Usage
  • Can apply operators to variables
  • Variables can be used in conditional expressions
  • Values can be passed during terraform apply

This implementation allows for dynamic value assignment based on conditions while maintaining desired constraints in Terraform configurations.

Local Values in Terraform

Local Values Overview:

  • Local values allow reuse of common values within Terraform configurations
  • Help reduce repetition and simplify configuration management

Syntax & Usage:

  1. Basic Structure:

    • Defined using ’locals’ block
    • Format: locals { name = value }
    • Referenced using: local.NAME
  2. Example Implementation:

    • Common tags scenario:
      locals {
        common_tags = {
          department = "finance"
          project = "cerberus"
        }
      }
      
    • Applied to resources using: local.common_tags
  3. Advanced Usage:

    • Can reference:
      • Variables
      • Resource attributes
      • Other local values
  4. Practical Example - S3 Bucket:

    • Combines:
      • Random string resource
      • Variable values
      • Local values for bucket naming
    • Creates globally unique bucket names using pattern: project-randomstring-bucket

Benefits:

  • Reduces code duplication
  • Maintains consistency across resources
  • Simplifies configuration management
  • Makes code more maintainable

Note: Local values can use both literal constants and dynamic values through interpolation.

Dynamic Blocks and Splat Expressions

Dynamic Blocks:

  1. Purpose:
  • Used to create multiple nested blocks within a resource
  • Simplifies configuration when multiple similar nested blocks are needed
  • Alternative to writing repetitive nested blocks manually
  1. Basic Structure:
dynamic "block_name" {
  for_each = var.list_variable
  content {
    // block configuration
  }
}
  1. Example Implementation:
  • Used for creating multiple ingress rules in AWS security group
  • Components:
    • Variable definition (list of ports)
    • Dynamic block with for_each loop
    • Content block with configuration details
  1. Iterator Usage:
  • Optional feature to customize loop variable name
  • Syntax: iterator = “custom_name”
  • Replaces default dynamic block name in content references

Splat Expressions:

  1. Purpose:
  • Used to retrieve attributes from all elements in a list
  • Simplifies accessing specific attributes across multiple resources
  1. Syntax:
  • Uses special symbol [*]
  • Example: aws_security_group.backend-sg.ingress[*].to_port
  1. Implementation Example:
  • Used in output variables to display attributes
  • Returns list of values from specified attribute

Practical Example Context:

  • Creates AWS VPC with private subnet
  • Implements security group with multiple ingress rules
  • Uses dynamic blocks for port configuration
  • CIDR range: 10.0.0.0/16 for VPC
  • Allows inbound traffic on ports 22 and 8080

6. Use Terraform CLI

Common Terraform Commands

  1. terraform validate
  • Checks configuration correctness
  • Shows errors with line numbers and fixing hints
  • Displays successful validation message if correct
  1. terraform fmt
  • Formats code into canonical format
  • Improves configuration file readability
  • Shows changed files in directory
  1. terraform show
  • Displays current infrastructure state
  • Can use -json flag for JSON format output
  1. terraform providers
  • Lists all providers used in configuration
  1. terraform output
  • Prints all output variables
  • Can show specific variable value using variable name
  1. terraform refresh
  • Syncs Terraform with real infrastructure
  • Updates state file for changes made outside Terraform
  • Doesn’t modify infrastructure, only state file
  1. terraform graph
  • Creates visual representation of dependencies
  • Generates output in dot format
  • Compatible with visualization tools like GraphViz
  1. terraform state commands a) list
  • Shows all resources in state file
  • Can filter by resource address

b) show

  • Displays attributes of specific resource

c) move/mv

  • Moves items within or between state files
  • Requires manual configuration file update
  • Syntax: terraform state move

d) pull

  • Downloads remote state file locally
  • Output can be filtered using jq

e) rm

  • Removes items from state file
  • Doesn’t destroy actual resources
  • Resource blocks need manual removal from configuration

f) push

  • Overrides remote state with local state
  • Use with extreme caution
  • Has safety checks to prevent accidental pushes
  • Can use -force option (not recommended)

Note: State commands should be used instead of manual state file editing for safety and consistency.

Terraform Lifecycle Rules

LIFECYCLE RULES OVERVIEW:

  • Control how Terraform creates and destroys resources
  • Implemented within lifecycle blocks inside resource blocks

TYPES OF LIFECYCLE RULES:

  1. Create Before Destroy
  • Default behavior: Destroys existing resource first, then creates new one
  • With create_before_destroy: Creates new resource first, then destroys old one
  • Useful when updating resources like EC2 instances with new AMIs
  1. Prevent Destroy
  • Set prevent_destroy = true
  • Prevents resource destruction from configuration changes
  • Terraform rejects changes that would destroy the resource
  • Shows error message if destruction attempted
  • Note: Doesn’t prevent terraform destroy command
  • Useful for protecting critical resources like databases
  1. Ignore Changes
  • Prevents updates to specified attributes
  • Can list specific attributes to ignore within brackets
  • Example: ignore_changes = [tags]
  • Can use ‘all’ keyword to ignore changes to all attributes
  • Useful when external changes to resources should not trigger Terraform updates

IMPLEMENTATION:

lifecycle {
    create_before_destroy = true
    prevent_destroy = true
    ignore_changes = [tags]
}

These rules help manage resource lifecycle behavior and protect against unwanted changes or destruction of resources.

Terraform Debugging and Logging

  1. Enabling Debugging

    • Use environment variable: TF_LOG
    • Can be set to different log levels
  2. Log Levels (5 types)

    • Info
    • Warning
    • Error
    • Debug
    • Trace (most verbose)
  3. Logging Behavior

    • Additional logs appear based on selected level
    • Trace level shows extensive details (thousands of lines)
    • Includes internal Terraform plugin operations
    • Useful for troubleshooting internal Terraform issues
    • Can be used for bug reporting
  4. Log Storage

    • Use TF_LOG_PATH environment variable
    • Points to specific file for log storage
    • Ensures persistent log recording
  5. Disabling Logs

    • Unset the environment variables to disable logging completely

Note: Terraform apply output usually provides basic error information, but detailed logging helps when deeper investigation is needed.

Terraform Import

Purpose:

  • Used to bring existing resources under Terraform management
  • Helps manage resources created through other means (console, other IaC tools)

Key Points:

  1. Prerequisites:

    • Requires unique attribute of resource (e.g., instance ID for EC2)
    • Empty resource block in configuration file
  2. Import Command Characteristics:

    • Only updates state file, not configuration files
    • Configuration files must be manually updated
    • Syntax: terraform import <resource_type.resource_name> <unique_resource_attribute>
  3. Import Process: a. Create empty resource block with unique name b. Run terraform import command c. Complete resource block with required arguments

  4. Methods to Identify Required Arguments:

    • Inspect management console for attribute values
    • Review state file after import (preferred method)
  5. Post-Import:

    • Resource comes under Terraform control
    • Changes managed through normal Terraform workflow (plan/apply)
    • Terraform plan will show no changes if configuration matches existing resource

Important Note:

  • Resource names in configuration must be unique within current Terraform configuration and state
  • Import command doesn’t automatically generate configuration code

Terraform Workspaces

TERRAFORM WORKSPACES OVERVIEW:

  • Allows using same configuration directory for multiple infrastructure environments
  • Creates separate state files for different environments
  • Default workspace exists automatically in every Terraform configuration

WORKSPACE COMMANDS:

  • List workspaces: terraform workspace list
  • Create new workspace: terraform workspace new
  • Switch workspace: terraform workspace select
  • Current workspace indicated by * in workspace list

IMPLEMENTING WORKSPACES:

  1. Variable Configuration:
  • Use maps for environment-specific values
  • Example:
variable "instance_type" {
  type = map
  default = {
    development = "t2.micro"
    production = "m5.large"
  }
}
  1. Resource Configuration:
  • Use terraform.workspace expression to reference current workspace
  • Use lookup function to get workspace-specific values
  • Example:
lookup(var.instance_type, terraform.workspace)

STATE MANAGEMENT:

  • Local state storage: Uses terraform.tfstate.d directory
  • Separate subdirectories for each workspace
  • Each workspace directory contains its own terraform.tfstate file

PRACTICAL USAGE:

  • Useful for managing multiple environments (dev, prod, etc.)
  • Same configuration files, different resource specifications
  • Environment-specific tags and configurations possible
  • Reduces configuration duplication

WORKSPACE SWITCHING:

  • Can switch between workspaces to manage different environments
  • terraform apply executes with workspace-specific values
  • Resources created based on current workspace configuration

7. Terraform Modules

Terraform Modules

TERRAFORM MODULES - BASIC CONCEPTS

  • Any directory with Terraform configuration files is a module
  • Root module: Directory where Terraform commands are executed
  • Modules can call other modules (enables code reuse)

MODULE STRUCTURE

  1. Local Modules
  • Can specify relative or absolute paths
  • Example structure:
    root/terraform-projects/
      ├── AWS instance/ (child module)
      └── development/ (root module)
    
  • Module block syntax:
    module "dev-webserver" {
      source = "../AWS instance"  # or absolute path
    }
    
  1. Registry Modules
  • Available in Terraform Registry
  • Contains:
    • Publisher information
    • Version details
    • Usage instructions
    • Sub-modules for different use cases
  • Module block example:
    module "security_group" {
      source  = "terraform-aws-modules/security-group/aws"
      version = "4.0.0"  # Version specification recommended
    }
    

BENEFITS OF MODULES

  1. Simplified Configuration

    • Shorter, manageable root modules
    • Reduced complexity
  2. Reduced Risk

    • Pre-tested configurations
    • Validated code
    • Fewer human errors
  3. Code Reusability

    • Same module can be used multiple times
    • Standardized configurations
  4. Configuration Control

    • Can lock certain aspects
    • Maintains standard provisioning process

WORKFLOW WITH MODULES

  1. terraform init - Downloads provider plugins
  2. terraform plan - Reviews changes
  3. terraform apply - Implements changes

BEST PRACTICES

  • Always specify module versions
  • Use version constraints
  • Follow module documentation for required arguments
  • Consider using registry modules for common configurations

Creating and Using Terraform Modules

Project Context:

  • Client: Flex IT Consulting
  • Purpose: Deploy payroll software across multiple countries
  • Architecture Components:
    • EC2 instance (with custom AMI)
    • DynamoDB database (employee/payroll data)
    • S3 bucket (documents storage)

Module Structure:

  1. Base Directory: /root/terraform-projects/modules
  2. Configuration Elements:
    • EC2 instance configuration
      • Hardcoded instance type
      • Variable AMI based on region
    • S3 bucket configuration
      • Region-prefixed bucket names
      • Globally unique naming
    • DynamoDB configuration
      • Fixed table name and primary key
    • Explicit dependencies defined between resources

Variables Configuration:

  • app_region: Region specification
  • ami: AMI ID variable
  • bucket: Includes default value
  • Some values intentionally hardcoded for consistency

Deployment Process:

  1. US Region Deployment:

    • Directory: us-payroll-app
    • Main.tf configuration
    • Uses module block with source path
    • Specifies us-east-1 region and AMI
  2. UK Region Deployment:

    • Directory: uk-payroll-app
    • Similar configuration
    • Uses eu-west-2 region
    • Different AMI ID for UK region

Resource Addressing Syntax: Format: module.[module_name].[resource_type].[resource_name] Example: module.us_payroll.aws_dynamodb_table.payroll_db

Benefits of Modules:

  1. Simplified configuration management
  2. Reduced risk of human error
  3. Code reusability across regions
  4. Standardized configurations
  5. Easier testing and validation
  6. Comparable to programming libraries/packages

Workflow:

  1. terraform init - Downloads module and plugins
  2. terraform plan - Reviews changes
  3. terraform apply - Implements infrastructure

8. Terraform Cloud

Terraform Cloud Introduction

  1. Production Usage & Team Collaboration
  • Traditional Terraform usage limited to single-user perspective
  • Local storage of configuration files and state files not suitable for team environments
  • Security concerns with manual sharing of state files
  1. Current Challenges
  • State files contain sensitive infrastructure information
  • Version control systems (GitHub, GitLab, Bitbucket) suitable only for configuration files
  • Remote backends (S3, Google Cloud Storage, Azure RM) needed for secure state management
  • Version compatibility issues between team members
  • Potential disruption of existing projects due to version updates
  1. Terraform Cloud Solution
  • SaaS platform designed for team collaboration
  • Built-in shared state management
  • No dependency on external remote backends
  • Provides consistent working environment
  1. Key Features
  • Core workflows (init, plan, apply) run on Terraform Cloud
  • Eliminates version compatibility issues
  • Web-based user interface
  • Access control mechanisms
  • Secret management capabilities
  • Private registry
  • Policy controls
  • Centralized state management
  1. Benefits
  • Enhanced security for state file management
  • Consistent environment across team members
  • Streamlined collaboration
  • Reduced configuration conflicts
  • Professional team-oriented features

Terraform Cloud Plans

  1. Free Plan
  • Available at app.terraform.io
  • Includes basic Terraform binary features
  • Supports remote state management and operations
  • Private module registry available
  • Limited to 5 active users
  • No team management features
  1. Team Plan
  • Includes all Free Plan features
  • Added team management capabilities
  • Fine-grain permission control for projects
  • Suitable for team collaboration
  1. Team and Governance Plan
  • Includes all Team Plan features
  • Policy-as-code framework support
  • Sentinel policy implementation
  • Enables infrastructure standardization enforcement
  • Example: EC2 instance type restrictions
  1. Business Tier Advanced features include:
  • Enhanced security, compliance, and governance
  • Single sign-on integration (Currently Okta)
  • Future support for Azure AD and SAML 2.0
  • Parallel job processing
  • Self-hosted agents
    • x86 64-bit Linux OS support
    • Docker container compatibility
  • Premium support
  • Control over Terraform run environments

Additional Notes:

  • Standard plans execute Terraform runs on Terraform’s infrastructure
  • Business tier allows runs on organization’s infrastructure

Terraform Cloud demo

Initial Setup Process:

  1. Account Creation
  • Created at app.terraform.io
  • Required email verification
  • Basic user settings configuration
  1. Organization Setup
  • Created organization with unique name: “KodeKloud-terraform-cloud-demo-01”
  • Initial organization settings and configuration
  1. Workspace Creation Options:
  • Version Control Workflow (chosen for demo)
  • CLI Driven Workflow
  • API Driven Workflow

VCS Integration:

  1. GitHub Connection
  • Connected to existing GitHub repository
  • Repository contained:
    • main.tf (EC2 instance configuration)
    • provider.tf (AWS provider config)
    • Sentinel policies
  1. Variable Configuration:
  • Added required variables:
    • AMI ID
    • Instance type
    • Subnet ID
    • Region
    • AWS access key (sensitive)
    • AWS secret key (sensitive)

Workspace Features:

  1. Overview Tab:
  • Shows workspace status
  • Recent runs
  • Resource details
  • VCS integration status
  • Execution mode settings
  1. Runs Tab:
  • Execution plans history
  • Apply/destroy operations
  • Plan details and logs
  1. State Tab:
  • Remote state storage
  • Resource tracking
  • State file management
  1. Variables Tab:
  • Variable management
  • Sensitive variable handling
  • Variable editing capabilities

Operations Demonstrated:

  1. Resource Creation:
  • Generated execution plan
  • Cost estimation
  • Policy checks
  • Manual approval process
  1. Resource Modification:
  • VCS-triggered changes
  • Automatic plan generation
  • State updates
  1. Resource Destruction:
  • Destroy plan generation
  • Cost savings estimation
  • Confirmation process

Settings & Configuration:

  1. Workspace Settings:
  • Execution mode (remote/local)
  • Terraform version selection
  • Auto-apply configuration
  • Workspace locking
  • VCS settings
  1. Organization Settings:
  • Team management
  • User access control
  • API token management
  • Plan and billing options

Plan Tiers:

  1. Free Plan:
  • Up to 5 users
  • Basic features
  • Limited functionality
  1. Trial Plan (30 days):
  • Team features
  • Governance capabilities
  • No credit card required
  1. Team/Business Plans:
  • Full feature access
  • Advanced team management
  • Additional integrations

Advanced Features:

  1. Cost Estimation:
  • Monthly cost projections
  • Cost change tracking
  • Resource-specific costs
  1. Registry:
  • Custom provider publishing
  • Module management
  • Organization-specific registry
  1. Sentinel Policy Implementation:
  • Policy set creation
  • VCS-based policy management
  • Enforcement levels:
    • Soft mandatory
    • Hard mandatory
    • Advisory
  • Custom policy paths
  • Policy checking during runs
  1. Security Features:
  • Two-factor authentication
  • API token management
  • Workspace locking
  • Sensitive variable handling

Integration Capabilities:

  • GitHub/GitLab/Bitbucket support
  • API integration
  • CLI integration
  • Custom provider support

The demo concluded with a practical example of policy enforcement, showing how Sentinel policies can restrict EC2 instance types and the override process for soft mandatory policies.