How to create Jenkins jobs using Jenkins job builder

Jenkins Job builder (abbreviated JJB) is a python script to maintain and simplify configuration of Jenkins jobs. Jenkins internally stores configuration of jobs in an XML format. JJB instead maintains jobs as simple descriptions in YAML or JSON format, which are then expanded to XML and uploaded to Jenkins through its HTTP API.

You can keep your job descriptions in human readable text format in a version control system to make changes and auditing easier. It also has a flexible template system, so creating many similarly configured jobs is easy.

1. Ensure pip is installed

Jenkins jobs builder is available as a python package. Pip is a python package manager. You have to ensure that pip is installed before proceeding.

Please consult your OS package manager to learn how to install python pip in your system.

2. Install jenkins job builder

Once Python pip is installed, we can use it to install jenkins-job-builder. Use this command to install:

pip install --user jenkins-job-builder

Confirm the version installed

➜ pip freeze | grep jenkins-job-builder
jenkins-job-builder==3.12.0

3. Configure Jenkins Job Builder

JJB requires a basic authentication file with Jenkins credentials. Authentication is only needed if you intent to deploy changes to Jenkins. For local testing and validation, no authentication credentials are needed (just create a dummy file with empty user and password fields).

To obtain an API token, log into Jenkins and open the “Configure” tab of your account (JENKINS_URL/user/USERNAME/configure;

Click the “Add API Token”, give it a name (eg. “jjb”), and then copy the your new API token.

Create a jenkins_jobs.ini file within your integration/config.git checkout:

[job_builder]
allow_duplicates=False
update=all

[jenkins]
user=admin
password=110bd6d23d89139451c8c54c811c21766f
url=http://34.221.63.133:8080/
query_plugins_info=False

4. Creating a job with Jenkins job builder

The next part of the configuration for using the Jenkins Job Builder is writing the YAML file. YAML files are the ones that are used for providing the configurations for creating the Jenkins jobs. There are two types of YAML that we will be looking at:

singlejob.yaml — Creates the job as per the configurations in the yaml
multiplejobtemplate.yaml — Creates multiple jobs all having the same configuration as per the job template. All the jobs will be created at one go.

Create a job for singlejob.yaml with this content

- job:
    scm:
      - Simple-Test-Job
    builders:
      - shell: |
          TIME=`date`
          echo "It is ${TIME}"
    parameters:
      - string:
          name: BRANCH
          default: '{branch|master}'
    name: simple-test-job
    project-type: freestyle
    defaults: global
    description: 'Do not edit this job through the web!'
    disabled: false
    display-name: 'Fancy Test Job name'
    concurrent: true
    quiet-period: 5
    block-downstream: false
    block-upstream: false
    retry-count: 3
    build-discarder:
      daysToKeep: 3
      numToKeep: 20
      artifactDaysToKeep: -1
      artifactNumToKeep: -1

- scm:
    name: Simple-Test-Job
    scm:
      - git:
          url: https://github.com/etowett/apisim.git
          branches:
            - main

This YAML consists of these blocks.

The first is** scm. **This corresponds to the version control and its details we will be using in the Jenkins job. In the above example, it corresponds to Git version control, the URL, and the branch.

The second block is **builders.**This corresponds to the build management tool, which is going to be used by the Jenkins job. In the above example, it corresponds to shell multi line script.

The third block is the generic job details. This corresponds to the generic configuration details in the Jenkins job General tab.

To create the job:

jenkins-jobs --conf ./jenkins_jobs.ini update jobs/simple-job.yaml

Where:

./jenkins_jobs.ini is the ini configuration file in the current path
jobs/simple-job.yaml is the path to the job yaml definition
update is the argument to execute the job

Create a job for multiple.yaml with this content

- project:
    name: Test-Job-Template-Temp
    jobs:
      - 'Test-Job-Template-{num}':
          num: 1
      - 'Test-Job-Template-{num}':
          num: 2
      - 'Test-Job-Template-{num}':
          num: 3

- job-template:
    disabled_var:
    command: df
    type: freestyle
    scm:
      - Test-Job-Template
    builders:
      - shell: |
          TIME=`date`
          echo "{num} It is $TIME"
          echo "Job-Name Test-Job-Template-{num}"
    parameters:
      - string:
          name: BRANCH
          default: '{branch|master}'
    name: 'Test-Job-Template-{num}'
    id: 'Test-Job-Template-{num}'
    project-type: freestyle
    defaults: global
    description: 'Do not edit this job through the web!'
    disabled: false
    display-name: 'Fancy Test Job name {num}'
    concurrent: true
    quiet-period: 5
    block-downstream: false
    block-upstream: false
    retry-count: 3
    build-discarder:
      daysToKeep: 3
      numToKeep: 20
      artifactDaysToKeep: -1
      artifactNumToKeep: -1

- scm:
    name: Test-Job-Template
    scm:
      - git:
          url: https://github.com/etowett/apisim.git
          branches:
            - main

We introduced job-template. This corresponds to the name of the job template using which the further set of jobs will be created. Using this YAML, multiple jobs having the same configuration but different names can be created by just running the command line once.

Checkout more information about job definition here.

5. Testing Job definition

JJB creates Jenkins XML configuration file from a YAML/JSON definition file and just uploads it to Jenkins. JJB provides a convenient test command to allow you to validate the XML before you attempt to upload it to Jenkins.

Test a YAML job definition:

jenkins-jobs test jobs/simple-job.yaml

The above command prints the generated Jenkins XML to the console. If you prefer to send it to a directory:

jenkins-jobs test -o output jobs/simple-job.yaml

The output directory will contain files with the XML configurations.

6. Updating Jenkins job definition

Once you’ve tested your job definition and are happy with it then you can use the update command to deploy the job to Jenkins. The update command requires a configuration file. An example file is supplied in the etc folder, you should update it to match your Jenkins master:

jenkins-jobs --conf ./jenkins_jobs.ini update jobs/simple-job.yaml

The above command will update your Jenkins master with the generated jobs.

Please note that JJB caches Jenkins job information locally. Changes made using the Jenkins UI will not update that cache, which may lead to confusion.

7. Working with JSON job definitions

You can also define your jobs in json instead of yaml:

jenkins-jobs --conf etc/jenkins_jobs.ini-sample update tests/jsonparser/fixtures/simple.json

The above command just uses a simple job definition. You can also convert any of the YAML examples to JSON and feed that to JJB.

8. Deleting a job

To delete a job:

jenkins-jobs --conf ./jenkins_jobs.ini delete jobs/simple-job.yaml

The above command deletes the job simple from the Jenkins master.

9. Providing plugins info

To generate a plugins info, using an account with Administrator rights:

jenkins-jobs --conf ./jenkins_jobs.ini get-plugins-info -o plugins_info.yaml

To run JJB update using the plugins_info.yaml:

jenkins-jobs update -p plugins_info.yaml ./myjobs

Conclusion

In this guide we learnt how to use jenkins job builder to create jenkins jobs using either yaml or json job deefinitions. This will enable you to keep track of jenkins jobs as code, where you can commit to git.