Foreman & Katello 2.x Full Setup Guide - Part 2

Contents

In this second part I’ll explain how to:

  • Creation of the Katello Client Product (or another Product)
  • Configuration of Ansible, Puppet and Remote Execution

in the third part I’ll explain how to:

  • Configuration of VMWare and virt-who
  • Configuration of Provisioning
  • Provisioning of new server and KickStart configuration
  • Configuration of OpenSCAP
  • Final Setup

Creation of Katello Product

As written in the first article, I added the Katello repository to my servers registered to Foreman, so I’m using an external repository and now I want to use an internal one, as the same of EPEL. With this procedure I can add how many different repositories I want (eg. docker, some sw, internal repos…), I’ve just to change the yum repository upstream URL and the name. The following steps are for CentOS7 and 8, I’ll do the same for RHEL, just change name and label.

First of all I’ll create a new credential key for Katello Client:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
wget https://yum.theforeman.org/releases/2.0/RPM-GPG-KEY-foreman
hammer content-credentials create --organization 'kraba.lan' \
       --key RPM-GPG-KEY-foreman --name 'RPM-GPG-KEY-foreman' \
       --content-type gpg_key
hammer content-credentials list --organization kraba.lan --name RPM-GPG-KEY-foreman

---|--------------------
ID | NAME               
---|--------------------
5  | RPM-GPG-KEY-foreman
---|--------------------

and a new product for Katello Client on CentOS 7 (I’ll run also sync):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
hammer product create --name 'Katello Client 7' --organization 'kraba.lan' \
       --description 'Katello Client 7 Repos' --gpg-key 'RPM-GPG-KEY-foreman' \
       --sync-plan 'Weekly_Sync'
hammer product list --organization 'kraba.lan' --name 'Katello Client 7'

---|------------------|------------------------|--------------|--------------|-----------
ID | NAME             | DESCRIPTION            | ORGANISATION | REPOSITORIES | SYNC STATE
---|------------------|------------------------|--------------|--------------|-----------
6  | Katello Client 7 | Katello Client 7 Repos | kraba.lan    | 0            |           
---|------------------|------------------------|--------------|--------------|-----------

hammer repository create --organization 'kraba.lan' --product 'Katello Client 7' \
  --name 'Client7' --label 'Client7' \
  --content-type 'yum' --download-policy 'on_demand' \
  --gpg-key 'RPM-GPG-KEY-foreman' \
  --url 'https://yum.theforeman.org/client/2.0/el7/x86_64/' \
  --mirror-on-sync 'no'

hammer repository list --organization 'kraba.lan' --product 'Katello Client 7'

---|---------|------------------|--------------|--------------------------------------------------
ID | NAME    | PRODUCT          | CONTENT TYPE | URL                                              
---|---------|------------------|--------------|--------------------------------------------------
68 | Client7 | Katello Client 7 | yum          | https://yum.theforeman.org/client/2.0/el7/x86_64/
---|---------|------------------|--------------|--------------------------------------------------

### The ID is 68, I'll run the sync for repos ID 68
hammer repository synchronize --async --organization 'kraba.lan' --product 'Katello Client 7' --id 68

hammer product list  --name 'Katello Client 7' --organization kraba.lan

ID | NAME             | DESCRIPTION            | ORGANISATION | REPOSITORIES | SYNC STATE       
---|------------------|------------------------|--------------|--------------|------------------
6  | Katello Client 7 | Katello Client 7 Repos | kraba.lan    | 1            | Syncing Complete.
---|------------------|------------------------|--------------|--------------|------------------

and the new product for Katello Client on CentOS 8 (I’ll run also sync):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
hammer product create --name 'Katello Client 8' --organization 'kraba.lan' \
       --description 'Katello Client 8 Repos' --gpg-key 'RPM-GPG-KEY-foreman' \
       --sync-plan 'Weekly_Sync'
hammer product list --organization 'kraba.lan' --name 'Katello Client 8'

---|------------------|------------------------|--------------|--------------|-----------
ID | NAME             | DESCRIPTION            | ORGANISATION | REPOSITORIES | SYNC STATE
---|------------------|------------------------|--------------|--------------|-----------
7  | Katello Client 8 | Katello Client 8 Repos | kraba.lan    | 0            |           
---|------------------|------------------------|--------------|--------------|-----------

hammer repository create --organization 'kraba.lan' --product 'Katello Client 8' \
  --name 'Client8' --label 'Client8' \
  --content-type 'yum' --download-policy 'on_demand' \
  --gpg-key 'RPM-GPG-KEY-foreman' \
  --url 'https://yum.theforeman.org/client/2.0/el8/x86_64/' \
  --mirror-on-sync 'no'

hammer repository list --organization 'kraba.lan' --product 'Katello Client 8'

---|---------|------------------|--------------|--------------------------------------------------
ID | NAME    | PRODUCT          | CONTENT TYPE | URL                                              
---|---------|------------------|--------------|--------------------------------------------------
69 | Client8 | Katello Client 8 | yum          | https://yum.theforeman.org/client/2.0/el8/x86_64/
---|---------|------------------|--------------|--------------------------------------------------

### The ID is 69, I'll run the sync for repos ID 69
hammer repository synchronize --async --organization 'kraba.lan' --product 'Katello Client 8' --id 69

hammer product list  --name 'Katello Client 8' --organization kraba.lan

---|------------------|------------------------|--------------|--------------|-----------
ID | NAME             | DESCRIPTION            | ORGANISATION | REPOSITORIES | SYNC STATE
---|------------------|------------------------|--------------|--------------|-----------
7  | Katello Client 8 | Katello Client 8 Repos | kraba.lan    | 1            | Running   
---|------------------|------------------------|--------------|--------------|-----------

Creating the Content Credential via web GUI:

  • Content -> Content Credentials
  • Create Content Credential
  • Add Name, Type, paste/import the downloaded key and Save

Creating the Product via web GUI:

  • Content -> Products
  • Create Product
  • Add Name, Label, GPG Key, Sync Plan and Save

Creating the repositories via web GUI:

  • Content -> Products -> Katello Client
  • New Repository (el7 and el8)
  • Add Name, Label, Type yum
  • Upstream URL: the URL (cut & paste the one from hammer command)
  • Verify SSL : ticked
  • Download Policy: On Demand (or immediate if you’ve enough space)
  • GPG Key: RPM-GPG-KEY-foreman
  • Save (and repeat for all the repos)

Now I’ll add the repository ID 40 (the client for CentOS7) to my CentOS7 Content View and the repository ID 41 to my CentOS8 Content View.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
hammer content-view list --name CentOS7_CV
----------------|------------|------------|-----------|---------------------|----------------
CONTENT VIEW ID | NAME       | LABEL      | COMPOSITE | LAST PUBLISHED      | REPOSITORY IDS
----------------|------------|------------|-----------|---------------------|----------------
4               | CentOS7_CV | CentOS7_CV | false     | 2020/04/26 04:15:00 | 7, 8, 9, 10, 11
----------------|------------|------------|-----------|---------------------|----------------

hammer content-view add-repository --organization kraba.lan --name CentOS7_CV \
  --product 'Katello Client' --repository-id 68

hammer content-view list --name CentOS7_CV
----------------|------------|------------|-----------|---------------------|--------------------
CONTENT VIEW ID | NAME       | LABEL      | COMPOSITE | LAST PUBLISHED      | REPOSITORY IDS     
----------------|------------|------------|-----------|---------------------|--------------------
4               | CentOS7_CV | CentOS7_CV | false     | 2020/05/05 10:45:06 | 7, 8, 9, 10, 11, 68
----------------|------------|------------|-----------|---------------------|--------------------

hammer content-view list --name CentOS8_CV
----------------|------------|------------|-----------|---------------------|------------------
CONTENT VIEW ID | NAME       | LABEL      | COMPOSITE | LAST PUBLISHED      | REPOSITORY IDS   
----------------|------------|------------|-----------|---------------------|------------------
3               | CentOS8_CV | CentOS8_CV | false     | 2020/04/26 01:23:23 | 2, 3, 4, 5, 6, 12
----------------|------------|------------|-----------|---------------------|------------------

hammer content-view add-repository --organization kraba.lan --name CentOS8_CV \
  --product 'Katello Client 8' --repository-id 69

hammer content-view list --name CentOS8_CV
----------------|------------|------------|-----------|---------------------|----------------------
CONTENT VIEW ID | NAME       | LABEL      | COMPOSITE | LAST PUBLISHED      | REPOSITORY IDS       
----------------|------------|------------|-----------|---------------------|----------------------
3               | CentOS8_CV | CentOS8_CV | false     | 2020/04/26 01:23:23 | 2, 3, 4, 5, 6, 12, 69
----------------|------------|------------|-----------|---------------------|----------------------

And the last step is publishing a new CV:

1
2
3
4
5
hammer content-view publish --organization kraba.lan --name CentOS7_CV \
  --description "Katello Client added"

hammer content-view publish --organization kraba.lan --name CentOS8_CV \
  --description "Katello Client added"

Updating the CV via web GUI:

  • Content -> Content Views -> Select Content View
  • Yum Content tab -> Repositories
  • Add tab -> Tick the repos
  • Click Publish New Version (up right)

Now I’m checking the Content Credentials and adding them to the Activation Key, I’ll add the Katello Client 7 and 8.

1
2
3
4
hammer subscription list --organization kraba.lan | grep Katello

6  | 402881e471e47fc90171e484bfbb0002 | Katello Client 7 | Physical |          |         |         | 2020/05/05 11:07:17 | 2049/12/01 00:00:00 | Unlimited | 0       
7  | 402881e471e47fc90171e488677d000c | Katello Client 8 | Physical |          |         |         | 2020/05/05 11:11:16 | 2049/12/01 00:00:00 | Unlimited | 0

The IDs I need are the 6 and the 7 and I’ll add them to my Activation Key:

1
2
3
4
5
hammer activation-key add-subscription --organization kraba.lan --name 'CentOS8_Key' \
  --quantity '1' --subscription-id 7

hammer activation-key add-subscription --organization kraba.lan --name 'CentOS7_Key' \
    --quantity '1' --subscription-id 6

Adding the subscription to the CV via web GUI:

  • Content -> Activation Keys -> Subscription tab -> Add
  • Select the proper Katello (7 or 8)
  • Details tab -> select the CentOS7_CV or CentOS8_CV as Content View

From this point all new server will registered with the new repository (Katello Client) attached. If I want to add it to a registered server:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
hammer host subscription product-content --host foreman.kraba.lan
---|------------------|-----------------------------------|------------------|---------
ID | NAME             | LABEL                             | DEFAULT ENABLED? | OVERRIDE
---|------------------|-----------------------------------|------------------|---------
7  | CentOS 7 OS      | kraba_lan_CentOS7_CentOS7_OS      | yes              |         
8  | CentOS 8 Extras  | kraba_lan_CentOS7_CentOS7_Extras  | yes              |         
10 | CentOS 8 SCL     | kraba_lan_CentOS7_CentOS7_SCL     | yes              |         
9  | CentOS 8 Updates | kraba_lan_CentOS7_CentOS7_Updates | yes              |         
11 | EPEL7            | kraba_lan_EPEL7_EPEL7             | yes              |         
---|------------------|-----------------------------------|------------------|---------

### Subscription ID 15 is for Katello Client 7, foreman.kraba.lan is a CentOS7 servers
hammer host subscription attach --subscription-id 15 --host foreman.kraba.lan

hammer host subscription product-content --host foreman.kraba.lan

---|------------------|------------------------------------|------------------|---------
ID | NAME             | LABEL                              | DEFAULT ENABLED? | OVERRIDE
---|------------------|------------------------------------|------------------|---------
7  | CentOS 7 OS      | kraba_lan_CentOS7_CentOS7_OS       | yes              |         
8  | CentOS 8 Extras  | kraba_lan_CentOS7_CentOS7_Extras   | yes              |         
10 | CentOS 8 SCL     | kraba_lan_CentOS7_CentOS7_SCL      | yes              |         
9  | CentOS 8 Updates | kraba_lan_CentOS7_CentOS7_Updates  | yes              |         
15 | Client7          | kraba_lan_Katello_Client_7_Client7 | yes              |         
11 | EPEL7            | kraba_lan_EPEL7_EPEL7              | yes              |         
---|------------------|------------------------------------|------------------|---------

Adding subscription via web GUI:

  • Content -> Content hosts
  • Select Hosts
  • Select Actions -> Manage Subscriptions
  • Add Subscription

Configuration of Ansible and Remote Execution

I want to configure Ansible, Puppet and Remote Execution because I need something who is able to talk with my server, the Katello remote agent will be deprecated in a bit. I’ll configure all the three solutions and choose (or not) the better for my environment.

First of all I’ll install & enable Ansible and Remote Execution on Foreman, the new version of Foreman needs that.

1
2
3
4
5
yum install tfm-rubygem-foreman_ansible rubygem-smart_proxy_ansible tfm-rubygem-foreman_ansible rubygem-smart_proxy_ansible ansible

foreman-installer --enable-foreman-plugin-ansible --enable-foreman-proxy-plugin-ansible --enable-foreman-plugin-remote-execution --enable-foreman-proxy-plugin-remote-execution-ssh

foreman-mantain service restart

and I configure Ansible on my Foreman server, adding those lines (adding/changing/etc):

1
2
3
4
5
6
7
8
9
vi /etc/ansible/ansible.cfg

[defaults]
callback_whitelist = foreman
[callback_foreman]
url = 'https://foreman.kraba.lan'
ssl_cert = /etc/foreman-proxy/foreman_ssl_cert.pem
ssl_key = /etc/foreman-proxy/foreman_ssl_key.pem
verify_certs = /etc/foreman-proxy/foreman_ssl_ca.pem

Now I’ve to create an ssh key for the user foreman-proxy, who is the user who run the Ansible jobs and the remote execution. I’ll save the key into the proper home directory:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
mkdir ~foreman-proxy/.ssh
chown foreman-proxy ~foreman-proxy/.ssh
sudo -u foreman-proxy ssh-keygen -f ~foreman-proxy/.ssh/id_rsa_foreman_proxy -N ''
### I need also this for SSH Remote Execution
ln -s /usr/share/foreman-proxy/.ssh /usr/share/foreman-proxy/ssh


ls -ltr /usr/share/foreman-proxy/.ssh/
total 12
-rw-r--r-- 1 foreman-proxy foreman-proxy  193 May  5 15:22 known_hosts
-rw-r--r-- 1 foreman-proxy foreman-proxy  738 May  5 16:42 id_rsa_foreman_proxy.pub
-rw------- 1 foreman-proxy foreman-proxy 3243 May  5 16:42 id_rsa_foreman_proxy

and after I’ll copy the key into all the servers, just for test I copy it into my foreman server, it will connect to itself and send/collect all the facts to itself:

1
ssh-copy-id -i /usr/share/foreman-proxy/.ssh/id_rsa_foreman_proxy.pub root@foreman.kraba.lan

Yes, I’m running/copying the key to root account, it’s just for test. I know it’s wrong but it’s just for test. I’ll explain later how to setup the proper user. Now I setup the Remote Execution, or better I’ll check if the setup is the default one (running with root), the fastest way is checking the web GUI/console:

  • Administer -> Settings -> Remote execution

and I have:

and it’s the default one, ssh user and effective user are root! I’ll move to a test host, always the client foreman.kraba.lan, and run an ansible playbook:

  • Hosts -> All Hosts -> Tick my server -> Select Action -> Schedule Remote Job

  • Select “Ansible Playbook” as Job Category -> Ansible Roles - Ansible Deault as Job Template -> Submit

and wait:

If I move now to :

  • Hosts -> All Hosts -> Click my server
  • Facts, order by Reported At

I’ll have the facts reported by Ansible:

Good, the VM/client can send all facts to my Foreman, it’s useful and it’s a good check if all my server estate is working. I’ll create now a Recurring Job for obtaining an Ansible Callback (I’ll collect all the facts) every 30 minutes. To create a new Recurring Job I’ll use the web GUI:

  • Monitor -> Jobs -> Run Job
  • Select “Ansible Playbook” as Job Category -> Ansible Roles - Ansible Deault as Job Template
  • Write organization = kraba.lan without into Search Query
  • Type Of Query dynamic
  • Schedule -> Set Up Recurring Logic
  • Repeats -> Cronline and add */30 * * * * as cronline
  • Save

If I move to:

  • Monitor -> Recurring Logics

I can see the new job and all the information about. We can check it also via hammer cli:

1
2
3
4
5
6
7
8
hammer recurring-logic list

---|--------------|----------|-----------|-------
ID | CRON LINE    | END TIME | ITERATION | STATE
---|--------------|----------|-----------|-------
1  | 0 2 * * 6    |          | 2         | active
2  | */30 * * * * |          | 1         | active
---|--------------|----------|-----------|-------

the ID 1 is my weekly sync, the ID 2 is the Ansible Callback job.
Now I’m ready to run ansible jobs/playbook from my Foreman to every server registered to it. But…but…I’m using the root account into the server, what I need and prefer is creating a service account (eg. svcforeman) with sudo privileges in every server and copy the ssh key of foreman-proxy to them. After that I’ve to change some settings on Foreman:

  • Administer -> Settings -> Remote execution
  • SSH User -> the new user eg. svcforeman
  • Effective User -> root
  • Effective User Method -> sudo

if the ssh key exchange is working I don’t need the password/other setting.

Configuration of Puppet

Puppet is part of Foreman, I’ll configure it on my Foreman client/VMs for my purpose and just for few thinks, later for the OpenSCAP plugin. I can use it instead of Ansible for obtaining the facts every 30 minutes or running some jobs. By default I’ve only one Puppet Environment, the Production one, but if I want to create a new one and associate my servers to a different environment (eg. dev, test…) I can create a new Puppet Environment:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
hammer puppet-environment list
---|-----------
ID | NAME      
---|-----------
2  | Production
---|-----------

hammer puppet-environment create --name test --locations krabaDC  --organizations 'kraba.lan'
hammer puppet-environment list
---|-----------
ID | NAME      
---|-----------
2  | Production
8  | test      
---|-----------

hammer puppet-environment info --id 8
Id:            8
Name:          test
Puppetclasses:

Locations:     
    krabaDC
Organisations:
    kraba.lan
Created at:    2020/05/07 08:23:57
Updated at:    2020/05/07 08:23:57

Or use the web GUI:

  • Configure -> Puppet -> Environments
  • Create Puppet Environment

To configure Puppet properly I’ve to install the repo in every server or, following the previous chapter, add it to Foreman as a local repo. I’ll run it via remote executions and check if it works, I’ll add it later maybe. What I run in all server is:

Tip

I don’t have to install puppet repo and puppet-agent into my Foreman server, just on the other servers Change the /opt/puppetlabs/puppet/bin/puppet config set –section agent environment setting to your new one (or leave in Production) if you want to change Puppet Environment
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
### CentOS7/RHEL7
yum -y localinstall  https://yum.puppet.com/puppet6-release-el-7.noarch.rpm
yum install -y puppet-agent
content=`hostname -a`
/opt/puppetlabs/puppet/bin/puppet config set --section agent server foreman.kraba.lan
/opt/puppetlabs/puppet/bin/puppet config set --section agent certname ${content}.kraba.lan
/opt/puppetlabs/puppet/bin/puppet config set --section agent runinterval 1800
/opt/puppetlabs/puppet/bin/puppet config set --section agent environment Production
/opt/puppetlabs/puppet/bin/puppet config set --section agent listen false
/opt/puppetlabs/puppet/bin/puppet config set --section agent report true
/opt/puppetlabs/puppet/bin/puppet config set --section agent usecacheonfailure true
/opt/puppetlabs/puppet/bin/puppet config set --section agent pluginsync true
systemctl enable puppet
systemctl restart puppet

or

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
### CentOS8/RHEL8
yum -y localinstall  https://yum.puppet.com/puppet6-release-el-8.noarch.rpm
yum install -y puppet-agent
content=`hostname -a`
/opt/puppetlabs/puppet/bin/puppet config set --section agent server foreman.kraba.lan
/opt/puppetlabs/puppet/bin/puppet config set --section agent certname ${content}.kraba.lan
/opt/puppetlabs/puppet/bin/puppet config set --section agent runinterval 1800
/opt/puppetlabs/puppet/bin/puppet config set --section agent environment Production
/opt/puppetlabs/puppet/bin/puppet config set --section agent listen false
/opt/puppetlabs/puppet/bin/puppet config set --section agent report true
/opt/puppetlabs/puppet/bin/puppet config set --section agent usecacheonfailure true
/opt/puppetlabs/puppet/bin/puppet config set --section agent pluginsync true
systemctl enable puppet
systemctl restart puppet

Tip

I don’t have to install puppet repo and puppet-agent into my Foreman server, just on the other servers

I choose to run it via Remote Execution, just for testing it via web GUI:

  • Hosts -> All Hosts -> Tick my server -> Select Action -> Schedule Remote Job
  • Commands as Job Category -> Run Command - SSH Default as Job Template
  • I’ll cut & paste all the commands into command
  • Submit

When the job is finished I can check - just to be sure if my puppet agent is configured properly:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
cat /etc/puppetlabs/puppet/puppet.conf
...
[agent]
    classfile = $statedir/classes.txt
    default_schedules = false
    environment = Production
    localconfig = $vardir/localconfig
    masterport = 8140
    noop = false
    report = true
    runinterval = 1800
    splay = false
    splaylimit = 1800
    usecacheonfailure = true
    ### i'm running it on foreman...it will be different if i'm running it on server1.kraba.kraba_lan_CentOS7_CentOS7_Extras
    certname = foreman.kraba.lan
    listen = false
    pluginsync = true
    server = foreman.kraba.lan

and I can check if I’m receveing some puppet facts:

  • Hosts -> All Hosts -> Click my server
  • Facts, order by Reported At

and if it’s working, Puppet is done.