Jenkins (CJE)

Jenkins is an Open Source automation and orchestration engine. Jenkins offers a simple way to help automate the non-human parts of the software development process. Jenkins implements a continuous integration and continuous delivery environment for almost any combination of languages and source code repositories using pipelines, as well as automating other routine development tasks. Cloudbees Jenkins Enterprise (CJE) is a centrally managed Enterprise class version of Jenkins.

Features Include:

- - - Continuous Integration: Commit changes to the source code in a shared repository several times a day or more frequently. Every commit made in the repository is then built. This allows the teams to detect the problems early.
    - Add-On Plugins: Focus on typical Jenkins-related growth challenges faced by enterprises.
    - Agile Methodology: Deliver better products and optimize utilization of resources while getting rapid resolutions on issues.

QUICK START GUIDE

Requesting Access to a CloudBees Jenkins Master

Step 1: If you do not have a HARP account or an EIDM or EUA account, register for a HARP ID. For instructions on the HARP registration process, refer to the HARP page.

Step 2: Once the HARP account has been created, log into HARP and request a QualityNet CloudBees Jenkins entitlement via a HARP User Role.

NOTE: Due to the Jenkins RBAC security model, users requesting the Jenkins_Admin or Jenkins_Developer role must also request the Jenkins_Browse role.
Until this functionality is developed in HARP, Admins/Developers will need to submit a second role request for the Jenkins_Browse role.

Users requesting access to the HIDS Master will need to select the Jenkins_Browse role and then reach out to the SecDevOps team via Slack (#hids-clouddevops-support) or email (dl-hcqis_devops@ventechsolutions.com) to indicate which specific Jenkins job folder they need access to.

Select User Roles from the top of the page and select Request a Role.
On the Select a Program Page, select QualityNet-CloudBees Jenkins.
On the Select an Organization page, select the Jenkins Master you are requesting access to.

Don't see your organization listed?

On the Select Roles page, select QualityNet-CloudBees Jenkins user role (choose one)

Jenkins_Admin
Jenkins_Browse
Jenkins_Deployer (not available in all Masters)
Jenkins_Developer
Jenkins_SO

Select the Submit button
Enter your reason for requesting the selected role in the Request Reason text field.
Select the Submit button

Step 3: Users requesting the Jenkins_Admin or Jenkins_Developer role must also request the Jenkins_Browse role, repeat Step 2 and request the Jenkins_Browse role. See NOTE above.

Step 4: The organization's Security Official reviews and approves/denies the user role request. You will be notified via email that your request has been submitted, and again when your role has been approved or denied.

Accessing CloudBees Jenkins:

Step 1: If you do not have Zscaler access, please follow instructions on the Zscaler Getting Started page.

Step 2: Log into Zscaler

Step 3: Log into CloudBees Jenkins at https://jenkins.hcqis.org/ using your HARP credentials. Alternatively, you may also log in to the CMS.gov|IDM application portal at https://idm.cms.gov and select the CloudBees Jenkins tile

Note: you must be connected to Zscaler before logging into CloudBees Jenkins

Requesting a New CloudBees Jenkins Master

Step 1: Log into CCSQ ServiceNow using your HARP credentials. If you do not have access to CCSQ ServiceNow, please follow instructions on the
CCSQ ServiceNow Getting Started page.

Step 2: Select ServiceNow after logging in.

Step 3: Locate the SecDevOps Catalog item

Type “catalog” in the Filter Navigator
Select IT Services Catalog
Select SecDevOps
Select ADO Onboarding Request
Select CloudBees Jenkins Onboarding Request
Complete online form and select Submit.

Required Information for online form:

Name of the new CloudBees Jenkins Master
A brief justification for the new CloudBees Jenkins Master
- EX: This new Master will be used by the <Your LOB> team to subdivide major applications that are managed by different teams on our program.

FAQs

CJE CORE

The HCQIS CloudBees Jenkins CORE can be accessed using the following URL: https://jenkins.hcqis.org/

You can either login using the above url which will redirect you to HARP login. Alternatively, you can login to HARP and click on the Jenkins app icon.

Users who created a personal Jenkins API token before the Jenkins HARP cutover may need to create a new token. This is dependent on whether your AD ID is different from your HCQIS ID in your Okta profile. For users whose AD ID is the same as their HCQIS ID, your previously generated, the previously generated token should work.

For users whose AD ID is different from their HCQIS ID you will need to generate a new API token.

Steps to create a new personal Jenkins API Token:

Log in to Jenkins using your HARP credentials
Select your user name in the upper right and select Configure
In the API Token section, select Add new token, provide a name, and select Generate
Update any credentials that used a previous personal API token

100.65.16.0/24
100.65.17.0/24
100.65.18.0/24

Follow instructions here: How to request ec2 access for Service user for Jenkins Agent connections

The HCQIS CloudBees Jenkins can be accessed using the following URL: https://jenkins.hcqis.org To login, enter your Active Directory ID (i.e.gl1234) and password.

If you are seeing the below screenshot, it means that you have landed at https://jenkins.hcqis.org/cjoc/ which is the operations center managed by the DevOps team. It also implies that your master is probably not migrated yet. Please schedule a call with the DevOps team to discuss the plan for migration.

Requesting Access to a CloudBees Jenkins Master

Step 1: If you do not have a HARP account or an EIDM or EUA account, register for a HARP ID. For instructions on the HARP registration process, refer to the HARP page.

Step 2: Once the HARP account has been created, log into HARP and request a QualityNet CloudBees Jenkins entitlement via a HARP User Role.

NOTE: Due to the Jenkins RBAC security model, users requesting the Jenkins_Admin or Jenkins_Developer role must also request the Jenkins_Browse role.
Until this functionality is developed in HARP, Admins/Developers will need to submit a second role request for the Jenkins_Browse role.

Users requesting access to the HIDS Master will need to select the Jenkins_Browse role and then reach out to the SecDevOps team via Slack (#hids-clouddevops-support) or email (dl-hcqis_devops@ventechsolutions.com) to indicate which specific Jenkins job folder they need access to.

Select User Roles from the top of the page and select Request a Role.
On the Select a Program Page, select QualityNet-CloudBees Jenkins.
On the Select an Organization page, select the Jenkins Master you are requesting access to.

Don't see your organization listed?

On the Select Roles page, select QualityNet-CloudBees Jenkins user role (choose one)

Jenkins_Admin
Jenkins_Browse
Jenkins_Developer
Jenkins_SO

Select the Submit button
Enter your reason for requesting the selected role in the Request Reason text field.
Select the Submit button

Step 3: For users requesting the Jenkins_Admin or Jenkins_Developer role, repeat Step 2 and request the Jenkins_Browse role. See NOTE above.

Step 4: The organization's Security Official reviews and approves/denies the user role request. You will be notified via email that your request has been submitted, and again when your role has been approved or denied.

The new URL is the only change https://jenkins.hcqis.org . The previously used url "https://jenkins-cloud.hcqis.org" will cease to exist after the migration. Your access and roles to CJE Master should remain same. All the existing jobs, configurations and plugins will remain same.

Yes, there are some slight changes to building jobs in CJE CORE (2.x). Instead of using docker agents to build jobs, you will now have to use Kubernetes pod templates. If you are already using permanent slave, there would be no change

The process to create custom kubernetes pod templates is documented here: How to create Kubernetes Pod Templates in CJE-CORE

The process to build docker images is documented here: CJE-CORE Building docker images using Buildah

Yes. If you have the role of an ADO Admin, you may install plugins using the plugin manager.

Troubleshooting

On 12/20/2020 GitHub moved to a .gov URL. The hcqis.org URL redirect does not allow Jenkins multi-branch pipelines to function correctly. URLs for these jobs MUST be updated to https://qnetgit.cms.gov.

If you are the seeing the symptom below, please continue reading:

Symptom:

I am trying to run Nexus Upload job to copy a deployment package to Nexus, but it is not running. The page seems to be spinning and uploads to 25-30 % and starts from 0 and times out after a few attempts.

Resolution:

Please confirm if the size of the file you are uploading is greater than 250 MB.

We discovered that this is being caused due to the request being too large which is being rejected by Nginx. The Nginx has been configured to accept a max_body_size of 250 MB. This configuration cannot be changed on the Managed masters on the fly and even if we did, the change on the docker container would not be persistent. We have requested CloudBees to create a feature enhancement request to increase this limit in the upcoming versions.

Our recommendation is to use the Nexus Web-UI upload option to upload large files greater than 250 MB to your repository. Reach out to dl-hcqis_devops@ventechsolutions.com if you need help with this.

The nexus gem is available at RubyGems and provides features to interact with Nexus Repository Manager Pro including pushing gems to a hosted gem repository including the necessary authentication. The initial invocation will request the URL for the gem repository and the credentials needed for deployment. If you are getting the above error, our recommendation is to create a user token in nexus which can be created by accessing this url: http://nexus.hcqis.org:8081/#user/usertoken

Once it is created, replace the token in the following file on the target server :

~/.gem/nexus,

which looks like this:

---
:url: https://nexus.hcqis.org/repository/gems
:authorization: Basic aSBsb3ZlIHBvdGF0b2Vz=

Replace the token here using a "sed" command and re-run your job.

Need Help ?

CCSQ Support Central: Provides you with multi-program support to submit a new ticket, and track the status of an existing case, incident, or request. No login required. https://cmsqualitysupport.servicenowservices.com/ccsq_support_central

HIDS DevOps: If you need help or assistance please contact the HIDS DevOps team. They can be reached via the following methods:

Send an email to: dl-hcqis_devops@ventechsolutions.com

Contact them on QualityNet Slack #hids-clouddevops-support

Visit the HIDS SecDevOps Support page.