A tool to trigger all Jenkins jobs in a Jenkins folder – Part 2

Introduction

I decided to build this in.net6, a cross-platform command line app. C# is still probably my strongest computer language, and I already have code I can borrow from another project. Let us not get too deeply into the workings of global tools, but you should be able download this tool via nuget and install it on Mac, Linux, and, yes, Windows. Let’s briefly examine the core of the program:

public static int Execute(RunOptions options)
{
    Options = options;
    Console.WriteLine($"Starting at {DateTime.Now}");
    GetInitializedRestClientWithCrumb();
    if (!GetJenkinsProjectList())
    {
        return 1;
    }

    foreach (var url in JenkinsProjectUrls)
    {
        RestTriggerJenkinsBuild(url);
    }
    
    Console.WriteLine($"Ending at {DateTime.Now}");
    return 0;
}

We call the Execute() method with a RunOptions object (shown a bit further down). Then, using our login information, we call Jenkins to obtain a Crumb (a CSRF token).  We then send this token with each request to the server. After this, we obtain a list of Jenkins projects in the desired Jenkins folder.

using CommandLine;
namespace jenkins_trigger_all_builds.Options;

[Verb("run", HelpText = "Runs the jenkins trigger command.")]
public class RunOptions
{
    [Value(0)]
    [Option('u', "url", Required = false,
        HelpText = "A valid Jenkins base URL.",
        Default = true)]
    public string JenkinsServerUrl { get; set; } 

    [Value(1)]
    [Option('p', "path", Required = true,
        HelpText = "Unix-like path to a Jenkins jobs folder, which will be enumerated for contained projects.",
        Default = "")]
    public string Path { get; set; }
    
    [Value(2)]
    [Option('w', "wait", Required = false,
        HelpText = "When set, the program will wait for the build to complete before continuing.",
        Default = true)]
    public bool Wait { get; set; }
    
    [Value(3)]
    [Option('t', "token", Required = false,
        HelpText = "AuthToken for triggered builds.",
        Default = "")]
    public string ProjectTriggerToken { get; set; }
    
    [Value(4)]
    [Option('m', "max-wait-delay", Required = false,
        HelpText = "Max wait delay in minutes, before the build times out.",
        Default = 30)]
    public int MaximumBuildTimeInMinutes { get; set; }
    
    [Value(5)]
    [Option('u', "username", Required = false,
        HelpText = "Jenkins username",
        Default = "")]
    public string JenkinsUsername { get; set; }
    
    [Value(6)]
    [Option('w', "password", Required = false,
        HelpText = "Jenkins password.  Use JENKINS_PASSWORD env variable if you want to be secure.",
        Default = "")]
    public string JenkinsPassword { get; set; }
}

The parameters above should be fairly self-explanatory. It is important to note that the token on line 29 must match the settings we set up in the Jenkins job. Although you can specify this in the program arguments, this software can use utilize the environment variables JENKINS_USER and JENKINS_PASSWORD. It’s generally not a good idea to use your password in the command line. This is because the computer stores it in your shell history and possibly in the operating system’s process table.

The path is really the part of the URL that takes you to the Jenkins folder of interest.

Getting the crumb was interesting and yielded a bunch of learning. I’m using RestSharp, a library that makes doing REST much easier. Newtonsoft.Json helps decode the Json.

private static void GetInitializedRestClientWithCrumb()
{
    if (!string.IsNullOrEmpty(Options.JenkinsUsername))
        JenkinsUser = Options.JenkinsUsername;
    else
        JenkinsUser = Environment.GetEnvironmentVariable("JENKINS_USER") ?? "";
    if (!string.IsNullOrEmpty(Options.JenkinsPassword))
        JenkinsPassword = Options.JenkinsPassword;
    else
        JenkinsPassword = Environment.GetEnvironmentVariable("JENKINS_PASSWORD") ?? "";

    if (string.IsNullOrEmpty(JenkinsUser) || string.IsNullOrEmpty(JenkinsPassword))
        throw new Exception("Missing JENKINS_USER or JENKINS_PASSWORD env variable or option");

    var cookieJar = new CookieContainer();
    RestClient = new RestClient(Options.JenkinsServerUrl)
    {
        Authenticator =
            new HttpBasicAuthenticator(JenkinsUser, JenkinsPassword),
        CookieContainer = cookieJar
    };
    var request = new RestRequest("/crumbIssuer/api/json", Method.GET);
    var response = RestClient.Execute(request);
    if (!response.IsSuccessful)
    {
        Console.WriteLine("Request failed for crumb, StatusCode: " +
                          response.StatusCode);
        throw new Exception("Failed to get a crumb from Jenkins");
    }
    
    var responseCookie = response.Cookies.SingleOrDefault
        (x => x.Name.StartsWith("JSESSION", StringComparison.Ordinal));
    Debug.Assert(responseCookie != null, nameof(responseCookie) + " != null");
    Console.WriteLine($"Cookie: Name: {responseCookie.Name}, " +
                      $" Value: {responseCookie.Value}, " +
                      $"Path: {responseCookie.Path}, Domain: {responseCookie.Domain}");
    cookieJar.Add(new Cookie(responseCookie.Name,
        responseCookie.Value, responseCookie.Path, responseCookie.Domain));

    var jObj = JObject.Parse(response.Content);
    SavedCrumb = (jObj["crumb"] ?? string.Empty).Value<string>();
    Debug.Assert(SavedCrumb != string.Empty);
}

Keep in mind that we send the request to the /crumbIssues/api/json with basic authentication using our Jenkins login information. The header we require begins with the letters JSESSION and is present in the response headers from the crumb request. We build the cookieJar object and fill it with a freshly baked cookie. Additionally, we also save the crumb in a static class variable. Keep in mind that the RestClient is a static private object. All subsequent calls are made using this RestClient, which is the one holding the CookieJar. 

Armed with RestClient object with the right cookieJar and cookie, as well as a crumb, we can now call Jenkins to trigger and monitor projects.

To trigger builds note that if you only have the ?token= parameter, call the [server][project]/build?token={your guid}, but if you have more parameters, you need [server][project]//buildWithParameters?token={yourGuid}&key1=value1&key2=value2.

After starting the build, you must wait for Jenkins to start the new job for about 15 seconds. The build’s ID may then be obtained from the /lastbuild endpoint. Subsequent calls can utilize the /id/api/json parameter to obtain the current build status.

private static bool WaitForBuild(string url)
{
    IRestResponse response = null;
    var url2 = FixUrl(url) + "/lastBuild/api/json";
    var request = new RestRequest(url2, Method.GET);
    request.AddHeader("Jenkins-Crumb", SavedCrumb);

    var duration = new TimeSpan(0, 0, Options.MaximumBuildTimeInMinutes, 0);
    var endTime = DateTime.Now.Add(duration)
        .Ticks;

    var url3 = GetUrlOfBuildJustLaunched(url2);
    request = new RestRequest(url3, Method.GET);
    request.AddHeader("Jenkins-Crumb", SavedCrumb);

    // poll jenkins for build status with timeout
    while (DateTime.Now.Ticks < endTime)
    {
        Console.WriteLine($"Checking status of build at {url3}");
        response = RestClient.Execute(request);
        var jObj = JObject.Parse(response.Content);
        var result = (jObj["result"] ?? string.Empty).Value<string>();
        if (string.IsNullOrEmpty(result))
        {
            // wait five seconds before checking again
            Console.WriteLine($"Build at {url3} is still in progress, waiting...");
            Thread.Sleep(5000);  
            continue;
        }
        // we have a result, ABORTED, FAILED, SUCCESS..., so we break out of the wait loop
        Console.WriteLine($"Build {url3}, finished with result: {result}");
        break;
    }

    if (DateTime.Now.Ticks > endTime)
    {
        Console.WriteLine($"Timed out waiting for {url3}");
        return false;
    }

    return true;
} 

This is what the console output of the program looks like:

tb run --url [Reacted] --path job/{redacted}/job/veracode --wait --token [Redacted] --max-wait-delay 31 --username [Redacted] --password [Redacted]
Cookie: Name: JSESSIONID.9[recacted],  Value: [redacted], Path: /, Domain: [Redacted]
Triggering jenkins build with: /job/[redacted]/job/veracode/job/[redacted]/build?token=[redacted]
Triggered build OK for /job/[redacted]/job/veracode/job/[redacted]/build?token=[redacted]
Making status request to /job/[redacted]/job/veracode/job/[redacted]/lastBuild/api/json
Build id 12 is the latest build for /job/[redacted]/job/veracode/job/[redacted]/lastBuild/api/json
Checking status of build at /job/[redacted]/job/veracode/job/[redacted]/12/api/json
Build at /job/[redacted]/job/veracode/job/[redacted]/12/api/json is still in progress, waiting...
Checking status of build at /job/[redacted]/job/veracode/job/[redacted]/12/api/json
Build at /job/[redacted]/job/veracode/job/[redacted]/12/api/json is still in progress, waiting...
Checking status of build at /job/[redacted]/job/veracode/job/[redacted]/12/api/json
Build at /job/[redacted]/job/veracode/job/[redacted]/12/api/json is still in progress, waiting...
Checking status of build at /job/[redacted]/job/veracode/job/[redacted]/12/api/json
Build at /job/[redacted]/job/veracode/job/[redacted]/12/api/json is still in progress, waiting...
Checking status of build at /job/[redacted]/job/veracode/job/[redacted]/12/api/json
Build /job/[redacted]/job/veracode/job/[redacted]/12/api/json, finished with result: SUCCESS
Triggering jenkins build with: /job/[redacted]/job/veracode/job/[redacted]/build?token=[redacted]
Triggered build OK for /job/[redacted]/job/veracode/job/[redacted]/build?token=[redacted]
Making status request to /job/[redacted]/job/veracode/job/[redacted]/lastBuild/api/json
Build id 5 is the latest build for /job/[redacted]/job/veracode/job/[redacted]/lastBuild/api/json
Checking status of build at /job/[redacted]/job/veracode/job/[redacted]/5/api/json
Build at /job/[redacted]/job/veracode/job/[redacted]/5/api/json is still in progress, waiting...
Checking status of build at /job/[redacted]/job/veracode/job/[redacted]/5/api/json
Build at /job/[redacted]/job/veracode/job/[redacted]/5/api/json is still in progress, waiting...
Checking status of build at /job/[redacted]/job/veracode/job/[redacted]/5/api/json
Build at /job/[redacted]/job/veracode/job/[redacted]/5/api/json is still in progress, waiting...
Checking status of build at /job/[redacted]/job/veracode/job/[redacted]/5/api/json

Part 5 – A python tool to perform and analyze veracode scans