Jenkins Pipeline Job Types¶

Jenkins Pipelines are written as a Jenkinsfile and can have either a Declarative¹ or Scripted² format. Whenever possible for as much as possible, always write your pipelines in Declarative format.

How they are configured and run is a classic case of it depends.... This page goes into the job types available and when to use which and why.

Job Types¶

We classify the job types as follows:

• Something that isn't a pipeline job and you should never ever use (I'm looking at you Freestyle)
• Pipeline
• Multibranch Pipeline
• Organization Job (named after the most commonly used GitHub Organization Job)

Pipeline¶

A Pipeline job⁹ is the most fundamental building block available. You can create a New Item in Jenkins and choose the type Pipeline.

Here you can choose two different configurations, either you write the Pipeline inline or from SCM. Inline means that you write the Pipeline directly in the Job configuration itself. Where from SCM let's check out a Jenkinsfile from an SCM (such as GitHub).

Aside from testing Snippets, you should never use this Job type. It is very limited in what it offers beyond running a Pipeline.

Multibranch Pipeline¶

Now, this is starting to look like something we can use!

The Multibranch Pipeline³ allows you to map a Pipeline to an SCM Repository. Where a Pipeline Job maps to a specific Branch (or Branch scheme) of a Repository, a Multibranch Pipeline Job targets the entire repository.

This means that it scans the repository for its Branches. For each branch that meets the criteria you set, it creates a Pipeline Job targeting this specific branch.

By default, the only criteria configured is the existence of a file called Jenkinsfile. However, you can add other criteria such as a Branch Naming Scheme - either based on Regex or Wildcards.

Benefits Beyond Pipeline Job¶

Aside from not having to manage Job configurations for branches, as Multibranch handles both creation and deletion, this job type has additional benefits. It automatically populates environment variables such as BRANCH_NAME and many other related to the Git Commit. This allows you to take full leverage of the When¹⁰ Directive.

pipeline {
    stages {
        stage('Only Run If Master Branch') {
            when { branch 'master' }
            steps {
                echo 'This only runs iff we are in the master branch'
            }
        }
    }
}

To leverage the Multibranch Pipeline well, I recommend going through this excellent end-to-end tutorial⁴.

Branch Sources¶

The Multibranch Pipeline Job lets you configure to wich SCM Repository it should map to. This can be merely pointing to a Git or SVN (please don't) repository, but more importantly, you can map it to specific Branch Sources.

What are Branch Sources? These are specific Git providers that 1) give Jenkins more information, and 2) have their own unique handler plugins providing better integration with the Git provider.

In most cases, this means you can also build Pull Requests and deal with Tags in a way that leverages Jenkins Pipeline Directives such as the already mentioned When¹⁰ directive.

As of this writing - August 2019 - there are four such Branch Source plugins available; GitHub⁵ (most mature), GitLab⁸ (latest addition), Bitbucket⁶, and Gitea⁷.

Organization Job¶

I name these types of jobs after the first one that worked very well: GitHub Organization Job. This plugin is replaced by GitHub Branch Source, now the common name for the plugins providing this kind of integration.

The goal of Organization Pipeline Job is to scan an entire namespace of Git repositories for branches containing Jenkinsfile. In the case of GitHub, this is called an Organization, in the case of Bitbucket cloud, this is a Team and so forth. Creating several benefits, the essential being, you create a single job within Jenkins and manage the Pipeline for every application in that Organization/Team/Whatever.

Moreover, don't worry, there are ample settings to configure if you want to limit the job to specific branches or repositories.

Benefits¶

• reduced maintenance: a single job maintained in Jenkins takes care of all applications in an Organization/Team/... including creation and deletion of the repositories and branches!
• reduced complexity: due to the configuration working as a standardization, there are no special snowflakes, there's just one way a job gets configured, and that includes the naming
• visible coupling: a Pipeline is coupled to a repository, the job clearly shows this connection with links and icons
• manages status reporting: although this does depend on the specific integration, in general, every Pipeline automatically reports the build status of each commit to your SCM without requiring configuration

Models¶

What people often fail to realize, is that the mentioned Pipeline Job Types are not on their own. They are part of a single model, where they build on top of each other. The Pipeline job being the smallest building block, then Multibranch, and last but not least, Organization job extends Multibranch.

I've created some models to try and visualize this.

Abstract¶

At the very abstract level, we see this babushka doll type of layering.

Basic¶

Starting from the outer layer, we have a GitHub Organization job. In essence, it is a folder that contains Multbranch Pipeline jobs. A Multibranch Pipeline job is also a folder, and it contains Pipeline jobs.

Mapping¶

This layering is created by mapping the different types of jobs to corresponding resources in the SCM you are talking to. In the example of a GitHub Organization job, we're talking to a GitHub Organization. Jenkins scans each repository in that Organization, and per repository, scan each branch. For each branch that meets the criteria - by default the presence of a Jenkinsfile - it creates the Multibranch Pipeline job for that repository.

The Multibranch Pipeline job creates a Pipeline job for each branch that met the criteria.

So as summary:

• GitHub Organization Job -> GitHub Organization / Bitbucket Team / ...
• Multibranch Pipeline Job -> Repository
• Pipeline Job -> Branch

References¶