Eagle's blog

Environment Variables Update Automation using Ansible and Jenkins

Ugochukwu Charles Eugene's photo
Ugochukwu Charles Eugene
·

9 min read

Environment Variables Update Automation using Ansible and Jenkins

Table of contents

Introduction

Ansible: Ansible is an open-source automation tool that simplifies the management and configuration of systems, applications, and infrastructure. It allows you to describe your desired state of systems and execute tasks across multiple machines simultaneously. With Ansible, you can automate repetitive tasks, manage configurations, deploy applications, and orchestrate complex workflows, making it easier to maintain and scale your infrastructure efficiently.

Jenkins: Jenkins is an open-source automation server that facilitates continuous integration and delivery (CI/CD) processes. It allows you to automate various stages of software development, including building, testing, and deploying applications. Jenkins provides a web-based interface and a powerful plugin ecosystem, enabling teams to easily configure, schedule, and monitor automated workflows. By automating these processes, Jenkins helps streamline software development, increase productivity, and ensure the quality and reliability of software releases.

How it Works

The idea is this, Ansible is responsible for retrieving and updating the environment variables, while Jenkins automates the process by scheduling the execution of the Ansible playbook at regular intervals.

The steps involved:

  1. Writing an Ansible playbook: Create a playbook that retrieves the environment variables from their storage location in AWS. Then, update the variables as needed and return the new values to the server.

  2. Configuring Jenkins: Set up Jenkins to schedule and automate the execution of the Ansible playbook. You can specify the interval at which you want the playbook to run.

  3. Continuous updating: Once Jenkins is configured, it will execute the Ansible playbook at the specified interval. This allows the environment variables to be continuously updated with the latest values.

Ansible Playbook for Env Variable Update

Below is an example ansible-playbook to achieve this:

---
- name: Update Environment Variables
  hosts: localhost
  gather_facts: false

  vars:
    aws_access_key: "YOUR_AWS_ACCESS_KEY"     # Replace with your AWS access key
    aws_secret_key: "YOUR_AWS_SECRET_KEY"     # Replace with your AWS secret key
    aws_region: "us-west-2"                   # Replace with your desired AWS region
    secret_name: "my-secret"                  # Replace with your secret name

  tasks:
    - name: Install AWS CLI
      apt:
        name: awscli
        state: present
      become: true

    - name: Configure AWS credentials
      shell: |
        aws configure set aws_access_key_id {{ aws_access_key }}
        aws configure set aws_secret_access_key {{ aws_secret_key }}
        aws configure set default.region {{ aws_region }}
      become: true

    - name: Retrieve current secret value
      shell: |
        aws secretsmanager get-secret-value --secret-id {{ secret_name }} --region {{ aws_region }}
      become: true
      register: current_secret

    - name: Parse secret variables
      set_fact:
        secret_variables: "{{ current_secret.stdout | from_json | json_query('SecretString') | from_json }}"

    - name: Generate new secret values
      vars:
        updated_variables: {}
      loop: "{{ secret_variables | dict2items }}"
      block:
        - name: Generate random value
          community.general.random_string:
            length: 16                              # Specify the desired length of the random string
            chars: 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
          register: new_value

        - name: Add updated variable
          set_fact:
            updated_variables: "{{ updated_variables | combine({item.key: new_value.result}) }}"

    - name: Update secret values
      shell: |
        echo "{{ updated_variables | to_json }}" | aws secretsmanager put-secret-value --secret-id {{ secret_name }} --secret-string file:///dev/stdin --region {{ aws_region }}
      become: true

    - name: Return updated secret value
      debug:
        msg: "{{ updated_variables }}"

Breaking down each step in the ansible-playbook

tasks:
    - name: Install AWS CLI
      apt:
        name: awscli
        state: present
      become: true
  • This line Installs the AWS Command Line Interface (CLI) on the localhost, enabling interaction with AWS services
 - name: Configure AWS credentials
      shell: |
        aws configure set aws_access_key_id {{ aws_access_key }}
        aws configure set aws_secret_access_key {{ aws_secret_key }}
        aws configure set default.region {{ aws_region }}
      become: true
  • Sets the AWS access key, secret key, and region for the AWS CLI using the provided variables.
- name: Retrieve current secret value
      shell: |
        aws secretsmanager get-secret-value --secret-id {{ secret_name }} --region {{ aws_region }}
      become: true
      register: current_secret
  • Executes the AWS CLI command to retrieve the current secret value from AWS Secrets Manager and stores it in the current_secret variable.
- name: Parse secret variables
      set_fact:
        secret_variables: "{{ current_secret.stdout | from_json | json_query('SecretString') | from_json }}"
  • Parses the current_secret value to extract the individual environment variable key-value pairs and stores them in the secret_variables variable.
- name: Generate new secret values
      vars:
        updated_variables: {}
      loop: "{{ secret_variables | dict2items }}"
      block:
        - name: Generate random value
          community.general.random_string:
            length: 16                              # Specify the desired length of the random string
            chars: 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
          register: new_value

        - name: Add updated variable
          set_fact:
            updated_variables: "{{ updated_variables | combine({item.key: new_value.result}) }}"
  • Loops over each key-value pair in secret_variables and generates a random value using the community.general.random_string module. The generated values are stored in the updated_variables dictionary.

Note: To generate random values for different keys in a file containing multiple environment variables, you can modify the playbook to iterate over each key and generate a random value individually. That is basically what we did in the example-ansible-playbook above.

- name: Update secret values
      shell: |
        echo "{{ updated_variables | to_json }}" | aws secretsmanager put-secret-value --secret-id {{ secret_name }} --secret-string file:///dev/stdin --region {{ aws_region }}
      become: true
  • Uses the updated_variables dictionary to update the secret values in AWS Secrets Manager using the AWS CLI put-secret-value command.
- name: Return updated secret value
      debug:
        msg: "{{ updated_variables }}"
  • Prints the updated_variables dictionary as a debug message, showing the updated key-value pairs.

Important details to note:

  • Replace the placeholder values (YOUR_AWS_ACCESS_KEY, YOUR_AWS_SECRET_KEY, us-west-2, my-secret) with your own AWS access key, secret key, desired AWS region, and secret name.

  • The 'length' parameter in the community.general.random_string module specifies the desired length of the randomly generated string.

  • The 'chars' parameter in the community.general.random_string module defines the characters that can appear in the randomly generated string.

  • The 'updated_variables' dictionary stores the updated key-value pairs, where each key represents an environment variable name, and the corresponding value is the randomly generated string.

Configuring Jenkins

To upload and run the Ansible playbook using Jenkins, you follow these simple steps:

  1. Configure Jenkins:

    • Install Jenkins on a server or use an existing Jenkins installation.

    • Set up any necessary plugins required for Ansible integration. For example, you may need to install the "Ansible" and "Pipeline" plugins.

    • Configure Jenkins to connect to your version control system (e.g., Git) and any required credentials for accessing the code repository.

  2. Create a Jenkins Pipeline job:

    • Open Jenkins and navigate to the Jenkins dashboard.

    • Click on "New Item" to create a new job.

    • Enter a name for your job and select "Pipeline" as the job type.

    • Scroll down and configure the pipeline settings, such as defining the pipeline script from SCM.

    • Specify the SCM (Source Code Management) details, including the repository URL, credentials, and branch name.

    • Set the Jenkinsfile path to point to the location of the Jenkinsfile in your code repository.

  3. Configure Jenkinsfile:

    • Create a Jenkinsfile in your code repository with the necessary pipeline steps to run the Ansible playbook.

    • Add the required stages and steps to the Jenkinsfile to perform the following actions:

      • Check out the code repository.

      • Set up any necessary environment variables specific to your Jenkins server and the playbook.

      • Execute the Ansible playbook using the ansible-playbook command.

      • Ensure the Jenkins server has the necessary permissions and prerequisites to execute Ansible.

  4. Configure Playbook Execution:

    • Within the Jenkinsfile, you'll need to define a stage that executes the Ansible playbook using the ansible-playbook command.

    • Specify the path to the playbook file within the Jenkins workspace.

    • Define any additional arguments or options required by the Ansible playbook command, such as specifying the inventory file or additional variables.

  5. Save and Run the Jenkins Job:

    • Save the Jenkins job configuration and trigger a build manually or set up a schedule for automatic builds.

    • Jenkins will fetch the code from the specified repository, execute the pipeline, and run the Ansible playbook as defined in the Jenkinsfile.

    • The output of the Ansible playbook execution will be displayed in the Jenkins job console output.

Here's an example of a Jenkinsfile that demonstrates the integration of Ansible playbook execution within a Jenkins Pipeline job:

pipeline {
  agent any

  stages {
    stage('Checkout') {
      steps {
        // Check out the code repository
        git url: 'https://github.com/your-repo-url.git', branch: 'main'
      }
    }

    stage('Install Dependencies') {
      steps {
        // Install required dependencies (e.g., Ansible)
        sh 'pip install ansible'
      }
    }

    stage('Execute Ansible Playbook') {
      environment {
        AWS_ACCESS_KEY_ID = credentials('aws-access-key') // Jenkins credential ID for AWS access key
        AWS_SECRET_ACCESS_KEY = credentials('aws-secret-key') // Jenkins credential ID for AWS secret key
        AWS_REGION = 'us-west-2'
      }

      steps {
        // Execute the Ansible playbook
        sh 'ansible-playbook -i inventory.ini playbook.yml'
      }
    }
  }
}

In this example Jenkinsfile:

  • The pipeline is defined using the pipeline directive with the agent any statement, indicating that it can run on any available Jenkins agent.

  • The stages section defines the different stages of the pipeline.

  • The first stage, "Checkout," checks out the code repository using the git step.

  • The second stage, "Install Dependencies," installs any required dependencies using the sh step. In this case, Ansible is installed.

  • The third stage, "Execute Ansible Playbook," defines the environment variables required for the Ansible execution, such as AWS credentials and region.

  • Inside the "Execute Ansible Playbook" stage, the sh step executes the Ansible playbook using the ansible-playbook command. Adjust the inventory file and playbook file names as per your project's structure.

Continuous updating

To schedule Jenkins to automatically run the Ansible playbook at set intervals, thereby continuously updating AWS with the new values of the env variables at that set intervals do these:

  1. Open the Jenkins web interface and navigate to your Jenkins project.

  2. Click on "Configure" or "Configure Project" to modify the project's configuration.

  3. In the project configuration, find the section related to "Build Triggers" or "Build Periodically" (the exact terminology may vary depending on Jenkins version).

  4. Enable the build trigger option for periodic builds.

  5. Specify the schedule using the cron syntax. For example, to run the playbook every day at 8:00 AM, you would use 0 8 * * *.

  6. Save the project configuration.

Jenkins will now automatically trigger the project at the specified interval, and the Ansible playbook will be executed accordingly.

Summary

  • To automate environment variables update in AWS, a typical approach involves using Ansible and Jenkins.

  • Ansible playbooks can be created to retrieve environment variables from AWS Secrets Manager, update their values, and return the new values to the secret store.

  • Jenkins can be configured to schedule and automate the execution of the Ansible playbook at specified intervals.

  • The Ansible playbook can be executed within a Jenkins Pipeline job, where the necessary dependencies and environment variables are set up.

  • By combining these tools and setting up the appropriate configurations, it becomes possible to continually update and manage environment variables securely in AWS using automation.

Jenkinsansibleansible-playbookAWSEnvironment variables