Saturday 1 October 2016

Performing nightly build steps with a Jenkinsfile

Note: 2018-12-13: I have a new post with an updated version that works with declarative pipelines. 


Using a Jenkinsfile to control your jenkins builds is an important part of the jenkins 2 workflow for pipeline-as-code. A Jenkinsfile allows you to control what you build, were you build it and all other aspects of your CI flow.

Typically when using pipeline-as-code your build would be triggered by a commit or push from your source control repository. However, there can still be times when you want your build to run on a schedule to perform a long running task e.g. static analysis or a full rebuild of your repository.

Running a nightly build

Jenkins supports running jobs using a trigger which can be controlled with a cron like format. From a Jenkinsfile this can be setup using triggers
  
def triggers = []
triggers << cron('H H(0-2) * * *')
properties (
    [
        pipelineTriggers(triggers)

    ]
)
This will cause your build to trigger sometime between midnight and 2am every day. The above works correctly, however it will cause a build to trigger for every branch in your repository. To limit it to a specific branch you can change it to


def triggers = []
if (env.BRANCH_NAME == "master) {
    triggers << cron('H H(0-2) * * *')
}
properties (
    [
        pipelineTriggers(triggers)

    ]
)
This will limit your scheduled build to only run on the master branch.

Limiting parts of the build to only run at night

Now that you have your build running every night, how do you limit the long running tasks to only trigger from the nightly build?

To do this you must examine the cause of the build. This involves getting the rawBuild data and searching all causes for a particular line in the description. Below is a handy function I've written which can be used to get that information.

// check if the job was started by a timer
// check if the job was started by a timer
@NonCPS
def isJobStartedByTimer() {
    def startedByTimer = false
    try {
        def buildCauses = currentBuild.rawBuild.getCauses()
        for ( buildCause in buildCauses ) {
            if (buildCause != null) {
                def causeDescription = buildCause.getShortDescription()
                echo "shortDescription: ${causeDescription}"
                if (causeDescription.contains("Started by timer")) {
                    startedByTimer = true
                }
            }
        }
    } catch(theError) {
        echo "Error getting build cause"
    }

    return startedByTimer
}

Note: As this is a NonCPS function it must be run outside of a node block.
Note: To get this to work correctly you may have to go to Manage Jenkins > In Process Script Approval, and approve the following signatures

method groovy.lang.Binding getVariables
method hudson.model.Cause getShortDescription
method hudson.model.Run getCause java.lang.Class
method hudson.model.Run getCauses
method org.jenkinsci.plugins.workflow.support.steps.build.RunWrapper getRawBuild


When I run my build I change my trigger section to

def triggers = []
def startedByTimer = false
if (env.BRANCH_NAME == "master) {
    triggers << cron('H H(0-2) * * *')
    startedByTimer = isJobStartedByTimer()
}
properties (
    [
        pipelineTriggers(triggers)

    ]
)

Then later in my build I can check if the build is a timed build and run the additional analysis checks. For example

if ( startedByTimer ) {
    node("analysis_server") {
        sh script: "make analysis"
    }
}

7 comments:

  1. Interesting post! Recently I've been reading about pipelines in Jenkins, and was wondering about exactly this (how to perform a nightly build using a pipeline script)

    ReplyDelete
  2. ummm jenkins has an environment variable BUILD_CAUSE which is equal to TIMEDTRIGGER if job was run by timer or MANUALTRIGGER if a user manually runs it..
    just use that for build cause

    ReplyDelete
    Replies
    1. Do you have any documentation on that? I don't see a BUILD_CAUSE in my environment variables on Jenkins 2.73

      Delete
  3. Random bypasser comment: I'm guessing Imran has https://wiki.jenkins.io/display/JENKINS/EnvInject+Plugin which defines the BUILD_CAUSE variable.

    ReplyDelete
  4. how to specify parameter in crons..

    if (env.BRANCH_NAME == "master) {
    triggers << cron('H H(0-2) * * *')
    }

    ReplyDelete
    Replies
    1. i have tried cron('H H(0-2) * * * %param1=true')

      but it's not working for me.. what's the problem for this...???

      Delete
    2. I haven't tried it but think you might need the parameterized scheduler plugin and the format of the trigger would be

      parameterizedCron('H H(0-2) * * * %param1=true')

      Delete