This document is specific to the London RedCentric lab, but should evolve to a more generic setup once we have more labs. For now, there are some hard-coded logic in the wiki as well as the scripts, to make sure we can reproduce at least the one lab we have. Once we have more labs, we'll work to automate that using configuration files, command line options, etc.

Install Debian as you normally would for a server, do care to install the ssh server and to plan for enough space for the Jenkins logs (a bare minimum of 500Go 500GB for the Jenkins VM is desirable).

...

Warning: This script will restart your network. It has been tested remotely (via SSH), but you may want to have a physical terminal nearby just in case.

Setting up the VMs

With Warning: This script will set the /etc/ansible/hosts file to reflect the HPC Lab's IP layout. Please edit file to reflect your own topology.

Setting up the VMs

With the network in place, you can create the VMs.

Code Block

root@hpc-admin # cd ~/labconf/kvm
root@hpc-admin # ./jenkins_virt_install.sh
root@hpc-admin # ./mrp_virt_install.sh
root@hpc-admin # ./fileserver_virt_install.sh
root@hpc-admin # ./login_virt_install.sh

For all, the preseed will setup statis IPs (10.40.0.11 and 10.40.0.12 and 10.40.0.13 respectively), and they should be visible from the wider network, including the host.

...

The network setup step above assumes the same IPs, so everything is fixed. In time we'll use configuration files so you don't have to change too many scripts.

Installing the MrP service

The Login node still doesn't use LDAP (TODO), but accounts can be created by hand, for now.

Installing the MrP service

You need to run both KEA and MrP roles to install a fully working provisioner. This can be done via the infra-server playbook:

Code Block

	bash
	bash

root@hpc-admin # cd ~/labconf/ans_setup_mrp/
root@hpc-admin # ./pre-setup.sh
root@hpc-admin # ansible-playbook mrp_setup.yml -vvvv -u root

Ansible will start MrProvisioner automatically, so you should be able to just open the URL on your browser (assuming you have a route to the machine's IP):

...

Code Block

	bash
	bash

root@hpc-admin # cd ~/labconf/ans_setup_jenkins
root@hpc-admin # ansible-playbook configure-jenkins.yml -vvvv -u root

Ansible will start Jenkins automatically, so you should be able to just open the URL on your browser (assuming you have a route to the machine's IP):

...

You may get two warnings when you log in to Jenkins, which can be corrected on the Global Security screen:

ERROR in config.xml: Jenkins may complain "version 1.1" is not supported, only 1.0. Editing /var/lib/jenkins/config.xml and changing that on the first line seems to work.
Agent to master security subsystem is currently off: Go to Security Settings and check the box saying "Enable Agent → Master Access Control"
Jenkins instance uses deprecated protocols: JNLP3-connect: Go to Security Settings > Agents and clear the box "Java Web Start Agent Protocol/3" in "Agent Protocols"

...

SSH HOST KEY VERIFIERS ARE NOT CONFIGURED FOR ALL SSH SLAVES: They are (host key verification), but Jenkins wants you to mark that manually, by entering all slaves' configuration and hitting "Save".

Themes: Install the Simple Theme Plugin and choose one from the list by updating the theme URL in the general settings.

Save the configuration and you should be all set.

Installing the Jenkins Jobs

Clone the repository

...

WARNING : The following playbook does not run on versions of ansible inferior to 2.4.0 since it makes use of the 'include_tasks' module.

Clone the repository

Code Block

	bash
	bash

root@hpc-admin # git clone https://github.com/Linaro/hpc_lab_jenkinssetup.git
root@hpc-admin # cd hpc_lab_jenkinssetup

Create the authorisation files

...

This will show your user ID and token.

hpc_lab_jenkinssetup/vars/jenkins_cred.yml.secret:

...

NOTE: The API TOKEN is the one from the admin user hpc-sig-admin users, not regular users.

Also, you need a token for Mr-Provisioner, to upload the preseeds. If you haven't got one yet, create it on the UI by clicking on your username's link (top right) > Tokens > "+". This will create a token.

Create a new file and copy the token hash into it.

hpc_lab_jenkinssetup/vars/mrp_creds.yml.secret:

No Format
mr_provisioner_auth_token: {TOKEN}

Run the Jenkins playbook

Since we're pushing changes to Mr-Provisioner, we've added it's Ansible as a submodule. So first, The first playbook you need to update itrun is Jenkins:

Code Block

	bash
	bash

root@hpc-admin # git submodule updateansible-playbook --init --recursive

Then run the playbook:

Code Block
bash	bash	root@hpc-admin # ansible-playbook -vvv -v -u root hpc_jobs_deployjenkins.yml

Create the users accounts

Mr-Provisioner

You need to create the Jenkins account by hand in MrP, add the ssh keys and generate the MrP token. This requirement will be dropped when bug 102 is fixed.

For now, the process is the following:

Log in as "admin"
Create a user "jenkins", set its password to some random string (use 'pwgen')
Logout as "admin" - Log in as "jenkins"
Add the SSH keys of all slaves to it
Generate a token, copy and paste somewhere
Log out as "jenkins"

Jenkins

Having the APITOKEN generated by the step above, add it to hpc_lab_job_deploy/vars/jslave_tokens.yml.secret in the following format (same token for all users):

...

jslave_tokens:
  - jslave: d05ohpc
    token: APITOKEN
  - jslave: qdcohpc
    token: APITOKEN
  - jslave: d03bench
    token: APITOKEN
  - jslave: d05bench
    token: APITOKEN
  - jslave: qdcbench
    token: APITOKEN
  - jslave: tx2bench
    token: APITOKEN

and then run the playbook:

...

root@hpc-admin # ansible-playbouok -vvv -u root put_mrp_tokens.yml

Now that the SSH Keys, Tokens and accounts are in place, all you have to do is assign the slaves to the right machines and ensure that you use the jinja templating in the preseeds in MrP.

File Server

The file server VM setup job handles the user creation + SSH keys copy, but this will move to the hpc_lab_jenkins repo.

Updating Jenkins Jobs

Once the jobs are installed and working, on every change pertaining the Jenkins configuration, you just need to update the repo and run the same playbook again:

...

This also works to update once there are changed. This playbook will create the nodes, jobs, users, ssh keys, etc.

The Jenkins playbook is a requirement for the other two: MrP and FS.

Warning: If you only want to install the services, and do not have (access, since it requires sudo) /etc/ansible/hosts configured, use this command :

To populate the hosts, please refer to the doc, and define a group named "jenkins".

Code Block

	bash
	bash

root@hpc-admin # ansible-playbook -i hosts -v -u root jenkins.yml

Create Mr-Provisioner users account

You need to create the Jenkins account by hand in Mr-Provisioner, add the SSH keys and generate the token. This requirement will be dropped when bug 102 is fixed.

The Jenkins playbook will create an SSH key in hpc-jenkins' /var/lib/jenkins/.ssh/id_rsa.pub. That's the one you should update in Mr-Provisioner's Jenkins account.

For now, the process is the following:

Log in as "admin"
Create a user "jenkins", set its password to some random string (use 'pwgen')
Logout as "admin" - Log in as "jenkins"
Add the SSH keys of all slaves to it
Generate a APITOKEN, copy and paste somewhere
Log out as "jenkins"

Add the APITOKEN generated by the step above, add it to vars/jslave_tokens.yml.secret in the following format (same token for all users):

Code Block

	yaml
	yaml

jslave_tokens:
  - jslave: d05ohpc
    token: APITOKEN
  - jslave: qdcohpc
    token: APITOKEN
  - jslave: d03bench
    token: APITOKEN
  - jslave: d05bench
    token: APITOKEN
  - jslave: qdcbench
    token: APITOKEN
  - jslave: tx2bench
    token: APITOKEN

Run Mr-Provisioner and File System playbooks

Code Block

	bash
	bash

root@hpc-admin # ansible-playbook -v -u root mrp.yml
root@hpc-admin # ansible-playbook -v -u root fs.yml

Warning: If you want to not depend on /etc/ansible/hosts, populate a hosts file with three groups : "jenkins", "provisioner", "fileserver" and use the command :

Code Block

	bash
	bash

root@hpc-admin # ansible-playbook -v -u root -i hosts mrp.yml
root@hpc-admin # ansible-playbook -v -u root -i hosts fs.yml

Updating Jenkins Jobs

Once the jobs are installed and working, on every change pertaining the Jenkins configuration, you just need to update the repo and run the same playbook again:

Code Block

	bash
	bash

root@hpc-admin # cd hpc_lab_setup
root@hpc-admin # git fetch -a & git pull
root@hpc-admin # ansible-playbook -v -u root jobs.yml

Warning: If you want to not depend on /etc/ansible/hosts, populate a hosts file with this groups : "jenkins" and use the command :

Code Block

	bash
	bash

root@hpc-admin # ansible-playbook -vvvv -u root -i hosts hpc_jobs_deploy.yml

Version	Old Version 64	New Version Current
Changes made by	Former user	Former user
Saved on	Jun 20, 2018	Jul 18, 2019

Page Comparison

Versions Compared

Key

Setting up the VMs

Setting up the VMs

Installing the MrP service

Installing the MrP service

Installing the Jenkins Jobs

Clone the repository

WARNING : The following playbook does not run on versions of ansible inferior to 2.4.0 since it makes use of the 'include_tasks' module.

Clone the repository

Create the authorisation files

Run the Jenkins playbook

Create the users accounts

Mr-Provisioner

Jenkins

File Server

Updating Jenkins Jobs

Create Mr-Provisioner users account

Run Mr-Provisioner and File System playbooks

Updating Jenkins Jobs