An Ansible role - Tomcat

In my previous post we set out with Ansible. This time I'm going to build on that and take a look at the concept of roles. I will create a role for installing a Tomcat 7 application server.

http://www.deviantart.com/art/The-Black-Tomcat-159041026

Now, to keep the complexity limited (this also gives me topics for future posts) I am going to make a couple of assumptions :

• Tomcat depends on Java ... I'm going to assume Java is present and in a known location (the same for all the target hosts).
• The target hosts have access to the internet (this allows them to download the Tomcat archive).
• The target hosts use upstart.

I understand that because of these assumptions my example may not reflect a real-life situation. I want to keep my posts limited in size though. Rest assured that these difficulties will be addressed in later posts.

A first new concept is that of groups. Those are defined in the Ansible hosts file.
[ansible@ansibleworkstation ~]$ vi /etc/ansible/hosts
ansibleworkstation

[tomcat-servers]
athena

Quite simple, a group ... groups together hosts of the same kind. So here we have a group tomcat-servers. Note that a host can be in multiple groups.

Similar to variables for a host, there are variables for a group.
[ansible@ansibleworkstation ~]$ mkdir /etc/ansible/group_vars
[ansible@ansibleworkstation ~]$ mkdir /etc/ansible/group_vars/tomcat-servers
[ansible@ansibleworkstation ~]$ vi /etc/ansible/group_vars/tomcat-servers/vars
tomcat7_http_port: 8080
tomcat7_version: 7.0.73
tomcat7_admin_username: "{{ vault_tomcat7_admin_username }}"
tomcat7_admin_password: "{{ vault_tomcat7_admin_password }}"
[ansible@ansibleworkstation ~]$ vi /etc/ansible/group_vars/tomcat-servers/vault
vault_tomcat7_admin_username: admin
vault_tomcat7_admin_password: adminsecret
[ansible@ansibleworkstation ~]$ ansible-vault encrypt /etc/ansible/group_vars/tomcat-servers/vault

Remember that we don't want multiple vault passwords, so use the same one you used for the host variables. I think it's quite clear what the variables mean, so lets move straight on to the playbook.
[ansible@ansibleworkstation ~]$ vi /etc/ansible/playbooks/tomcat7.yml
---
- hosts: tomcat-servers
  remote_user: root
  become: yes
  become_method: sudo
  roles:
    - tomcat7

No rocket science there either, this playbook will execute the role tomcat7 on the hosts in the tomcat-servers group. Before we can run the playbook however we have to define the role. 
[ansible@ansibleworkstation ~]$ mkdir /etc/ansible/roles
[ansible@ansibleworkstation ~]$ mkdir /etc/ansible/roles/tomcat7
[ansible@ansibleworkstation ~]$ mkdir /etc/ansible/roles/tomcat7/tasks
[ansible@ansibleworkstation ~]$ vi /etc/ansible/roles/tomcat7/tasks/main.yml
---
# http://docs.ansible.com/ansible/group_module.html
- name: add group "tomcat"
  group:
    name: tomcat

# http://docs.ansible.com/ansible/user_module.html
- name: add user "tomcat"
  user:
    name: tomcat
    group: tomcat
    home: /usr/share/tomcat
    createhome: no

# http://docs.ansible.com/ansible/unarchive_module.html
- name: extract archive
  unarchive:
    remote_src: yes
    src: "http://archive.apache.org/dist/tomcat/tomcat-7/v{{ tomcat7_version }}/bin/apache-tomcat-{{ tomcat7_version }}.tar.gz"
    dest: /opt
    creates: "/opt/apache-tomcat-{{ tomcat7_version }}"

# http://docs.ansible.com/ansible/file_module.html
- name: symlink installation directory
  file:
    src: "/opt/apache-tomcat-{{ tomcat7_version }}"
    path: /usr/share/tomcat
    owner: tomcat
    group: tomcat
    state: link

# http://docs.ansible.com/ansible/file_module.html
- name: change ownership of the installation
  file:
    path: "/opt/apache-tomcat-{{ tomcat7_version }}"
    owner: tomcat
    group: tomcat
    recurse: yes
    state: directory

# http://docs.ansible.com/ansible/template_module.html
- name: configure server
  template:
    src: server.xml
    dest: "/opt/apache-tomcat-{{ tomcat7_version }}/conf"
    mode: 0600
    owner: tomcat
    group: tomcat

# http://docs.ansible.com/ansible/template_module.html
- name: configure users
  template:
    src: tomcat-users.xml
    dest: "/opt/apache-tomcat-{{ tomcat7_version }}/conf"
    mode: 0600
    owner: tomcat
    group: tomcat

# http://docs.ansible.com/ansible/template_module.html
- name: upstart script
  template:
    src: tomcat7.conf
    dest: "/etc/init"
    mode: 0644
    owner: root
    group: root
  when: ansible_service_mgr == "upstart"

# http://docs.ansible.com/ansible/service_module.html
- name: enable and start service
  service:
    name: tomcat7
    state: started
    enabled: yes

A role has tasks. Note that added a reference to the documentation for each type of task mentioned. What are the tasks ?

• Create a group tomcat
• Create a user tomcat
• Download and extract the Tomcat archive (which one depends on the version variable) into /opt
• Create a symbolic link /usr/share/tomcat to the installation.
• Set the ownership of the installation.
• Update server.xml based on the variables and a template.
• Update tomcat-users.xml based on the variables and a template.
• Install the upstart script.
• Start Tomcat. 

If you study the tasks in detail you'll see that only existing Ansible modules are used (you can of course write your own) and that none of the tasks is complex in/by itself. There is actually only one fancy thing in there and that's the use of the when-clause to determine whether a target host has upstart or not.

Three templates are used in the tasks (server.xml, tomcat-users.xml, tomcat7.conf). Templates are files that are modified based on available variables and facts. Note that I only show the relevant bits of the files below.
[ansible@ansibleworkstation ~]$ mkdir /etc/ansible/roles/tomcat7/templates
[ansible@ansibleworkstation ~]$ vi /etc/ansible/roles/tomcat7/templates/tomcat-users.xml
<?xml version='1.0' encoding='utf-8'?>
...
<!-- {{ ansible_managed }} -->
...
<tomcat-users>
  <role rolename="manager-gui"/>
  <user username="{{ tomcat7_admin_username }}" password="{{ tomcat7_admin_password }}" roles="manager-gui" />
</tomcat-users>

[ansible@ansibleworkstation ~]$ vi /etc/ansible/roles/tomcat7/templates/server.xml
<?xml version='1.0' encoding='utf-8'?>

<!-- {{ ansible_managed }} -->
...
  <Service name="Catalina">
...
    <Connector port="{{ tomcat7_http_port }}" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="8443" />
...

[ansible@ansibleworkstation ~]$ vi /etc/ansible/roles/tomcat7/templates/tomcat7.conf
# {{ ansible_managed }}
description "Tomcat {{ tomcat7_version }} Server"

start on runlevel [2345]
stop on runlevel [!2345]
respawn
respawn limit 10 5

setuid tomcat
setgid tomcat

env JAVA_HOME=/usr/lib/jvm/java-8-oracle/jre
env CATALINA_HOME=/opt/apache-tomcat-{{ tomcat7_version }}
env JAVA_OPTS="-Djava.awt.headless=true -Djava.security.egd=file:/dev/./urandom"
env CATALINA_OPTS="-Xms512M -Xmx1024M -server -XX:+UseParallelGC"

exec $CATALINA_HOME/bin/catalina.sh run

# cleanup temp directory after stop
post-stop script
  rm -rf $CATALINA_HOME/temp/*
end script

And that concludes our role definition. Now we can run the playbook.
[ansible@ansibleworkstation ~]$ ansible-playbook /etc/ansible/playbooks/tomcat7.yml --ask-vault-pass

Cleanup up is not automated at the moment, so if you want to experiment a bit, here are the instructions to clean up on a target host.
ansible@athena:~$ sudo service tomcat7 stop
tomcat7 stop/waiting 
ansible@athena:~$ sudo rm -rf /etc/init/tomcat7.conf
ansible@athena:~$ sudo initctl reload-configuration
ansible@athena:~$ sudo rm -rf /usr/share/tomcat 
ansible@athena:~$ sudo rm -rf /opt/apache-tomcat-7.0.72/
ansible@athena:~$ sudo userdel tomcat
ansible@athena:~$ sudo groupdel tomcat
groupdel: group 'tomcat' does not exist

As you can see, roles are very powerful and they can be customized with your own variables or with system facts. Enjoy !

Setting out with Ansible

In 1998 Steve Traugott and Joel Huddleston wrote a paper titled Bootstrapping an Infrastructure. They were not the first to struggle with Configuration Management (as it is called today) but they set the stage, defined what was needed and gave concepts a name. SysAdmins from all over the world (myself included) gobbled it up.

Nowadays you can have your pick of tools that help you manage your infrastructure. The one mentioned in the paper, cfengine, is still around (albeit very different from the 1998 tool). And there's Puppet, Chef, Saltstack, ... to name but a few of the more popular ones.

Which one you use depends on a number of factors. Your manager may have fallen for a steak-and-strippers proposal. Or you absolutely want to use a specific programming language to write the tools (Ruby and Python are common). You want a server-agent setup or you want to work agentless. Today we're going to take a look at how to get up-and-running with Ansible.

http://www.deviantart.com/art/setting-out-113310530

Ansible is an agentless tool. You manage everything from a workstation which uses ssh to connect to your servers and pushes whatever you want done. That sounds simple and ... it is. Lets take a look.

For my workstation I set up an Oracle Linux 7.3. (Windows is not an option for the workstation). There are several ways to install Ansible, I picked the Python pip method (because it works on most Linux distributions) :

sudo pip install ansible

That didn't work the first time round, the following packages were needed on my system : libffi-devel, python-devel, openssl-devel. Well, that's the downside of working with pip. I installed them and all was well.

sudo mkdir /etc/ansible
sudo chown <youruser>:<yourgroup> /etc/ansible

Note that I'm not using root. While a couple of sudo's are required to get started, ansible does not require root. Substitute <youruser> and <yourgroup> with whatever your user-with-sudo-permissions is. For the remainder of this post I'll assume it is ansible.

Ansible is agentless but it doesn't work by magic. It uses ssh as its primary means to connect to servers. So ... we're going to need a key.

[ansible@ansibleworkstation ~]$ ssh-keygen -t rsa
[ansible@ansibleworkstation ~]$ cd .ssh
[ansible@ansibleworkstation .ssh]$ cp id_rsa.pub authorized_keys

The last statement is the clue, we are going to need the public key we just generated put into the .ssh/authorized_keys file of the target user on every server that we want to manage.

Hold on. What is the target user ? Well, ansible is going to connect to the servers with ssh. Preferably not to the root user but to a user that can become another user (through sudo or other means, there are multiple options). The target user. For this post I'll assume it is also ansible, uses sudo to become another user and requires a password to execute sudo, but it can be any user you wish (we'll see further on how to define it).

The next step is to define the servers that we want to manage. As a reminder, each host mentioned below requires our public key in the .ssh/authorized_keys file of the target user !

[ansible@ansibleworkstation ~]$ vi /etc/ansible/hosts
ansibleworkstation ansible_ssh_user=ansible
athena ansible_ssh_user=ansible

As you can see we're also going to manage the workstation itself. To check if things are working we can do the following.

[ansible@ansibleworkstation ~]$ export ANSIBLE_HOST_KEY_CHECKING=False
[ansible@ansibleworkstation ~]$ ansible all -m ping
athena | SUCCESS => {
    "changed": false, 
    "ping": "pong"
}
ansibleworkstation | SUCCESS => {
    "changed": false, 
    "ping": "pong"
}
[ansible@ansibleworkstation ~]$ export ANSIBLE_HOST_KEY_CHECKING=True

That looks good. I set the ANSIBLE_HOST_KEY_CHECKING to False because this is the first time Ansible connects to the servers and they are not in our .ssh/known_hosts yet. Once the command runs they are added so that's covered now (unless you add a server or reinstall a server of course ... keeping the .ssh/known_hosts correct is needed to be able to run things without manual intervention).

Ok, Ansible has executed the ping on the servers as the target user. That's great, but most installations will require root-permissions.

[ansible@ansibleworkstation ~]$ vi /etc/ansible/hosts
ansibleworkstation ansible_ssh_user=ansible ansible_become_pass=XXX
athena ansible_ssh_user=ansible ansible_become_pass=YYY

[ansible@ansibleworkstation ~]$ ansible all -a "touch /var/tmp/touched" --sudo
ansibleworkstation | SUCCESS | rc=0 >>
athena | SUCCESS | rc=0 >>

[ansible@ansibleworkstation ~]$ ll /var/tmp/touched 
-rw-r--r--. 1 root root 0 Dec 11 11:20 /var/tmp/touched

ansible@athena:~$ ll /var/tmp/touched 
-rw-r--r-- 1 root root 0 Dez 11 11:20 /var/tmp/touched

That worked, but this way the /etc/ansible/hosts file will quickly become unwieldy. Lets fix that.

[ansible@ansibleworkstation ~]$ mkdir /etc/ansible/host_vars
[ansible@ansibleworkstation ~]$ vi /etc/ansible/host_vars/ansibleworkstation
ansible_ssh_user: ansible
ansible_become_pass: XXX
[ansible@ansibleworkstation ~]$ vi /etc/ansible/host_vars/athena
ansible_ssh_user: ansible
ansible_become_pass: YYY
[ansible@ansibleworkstation ~]$ vi /etc/ansible/hosts
ansibleworkstation
athena

Excellent. However, we do have those passwords in plaintext, don't we ? No way we can put /etc/ansible under version control unless we change that.

In a first step we are going to create a file that contains all the variables (those we want in plaintext and those we don't want in plaintext) for a specific host.

[ansible@ansibleworkstation ~]$ mv /etc/ansible/host_vars/athena /etc/ansible/host_vars/athena_vars
[ansible@ansibleworkstation ~]$ mv /etc/ansible/host_vars/ansibleworkstation /etc/ansible/host_vars/ansibleworkstation_vars
[ansible@ansibleworkstation ~]$ mkdir /etc/ansible/host_vars/athena
[ansible@ansibleworkstation ~]$ mkdir /etc/ansible/host_vars/ansibleworkstation
[ansible@ansibleworkstation ~]$ mv /etc/ansible/host_vars/ansibleworkstation_vars /etc/ansible/host_vars/ansibleworkstation/vars

[ansible@ansibleworkstation ~]$ mv /etc/ansible/host_vars/athena_vars /etc/ansible/host_vars/athena/vars

In a second step we are going to create a second file that contains only the variables that we don't want in plaintext and we are going to redirect to those variables from the vars file.

[ansible@ansibleworkstation ~]$ vi /etc/ansible/host_vars/athena/vars
ansible_ssh_user: "{{ vault_ansible_ssh_user }}"
ansible_become_pass: "{{ vault_ansible_become_pass }}"
[ansible@ansibleworkstation ~]$ vi /etc/ansible/host_vars/athena/vault
vault_ansible_ssh_user: ansible
vault_ansible_become_pass: YYY

[ansible@ansibleworkstation ~]$ vi /etc/ansible/host_vars/ansibleworkstation/vars
ansible_ssh_user: "{{ vault_ansible_ssh_user }}"
ansible_become_pass: "{{ vault_ansible_become_pass }}"
[ansible@ansibleworkstation ~]$ vi /etc/ansible/host_vars/ansibleworkstation/vault
vault_ansible_ssh_user: ansible
vault_ansible_become_pass: XXX

And finally we encrypt the vault files (use the same password, having multiple vault-passwords is a pain).

[ansible@ansibleworkstation ~]$ ansible-vault encrypt /etc/ansible/host_vars/athena/vault
New Vault password: 
Confirm New Vault password: 
Encryption successful

[ansible@ansibleworkstation ~]$ ansible-vault encrypt /etc/ansible/host_vars/ansibleworkstation/vault
New Vault password: 
Confirm New Vault password:
Encryption successful

Verify the vault files, no more plaintext and /etc/ansible is ready for version control ! This does require a change in how we execute the commands though.

[ansible@ansibleworkstation ~]$ ansible all -a "echo /var/tmp/touched" --sudo --ask-vault-pass
Vault password: 
athena | SUCCESS | rc=0 >>
ansibleworkstation | SUCCESS | rc=0 >>

There, you are now all set up to securely execute Ansible commands. To conclude this post we're going to create an example Playbook (Ansible's name for a recipe/manifest/<whatever you call it>).

[ansible@ansibleworkstation ~]$ mkdir /etc/ansible/playbooks
[ansible@ansibleworkstation ~]$ vi /etc/ansible/playbooks/example.yml
---
- hosts: all
  tasks:
    - name: create /var/tmp/touched file
      shell: echo "{{ ansible_bios_version }}" > /var/tmp/touched
[ansible@ansibleworkstation ~]$ ansible-playbook /etc/ansible/playbooks/example.yml --sudo --ask-vault-pass
Vault password: 

PLAY [all] *********************************************************************

TASK [setup] *******************************************************************
ok: [ansibleworkstation]
ok: [athena]

TASK [create /var/tmp/touched file] ********************************************
changed: [athena]
changed: [ansibleworkstation]

PLAY RECAP *********************************************************************
ansibleworkstation         : ok=2    changed=1    unreachable=0    failed=0   
athena                     : ok=2    changed=1    unreachable=0    failed=0   

[ansible@ansibleworkstation ~]$ ll /var/tmp/touched 
-rw-r--r--. 1 root root 11 Dec 11 14:25 /var/tmp/touched
[ansible@ansibleworkstation ~]$ cat /var/tmp/touched
VirtualBox
ansible@athena:~$ ll /var/tmp/touched 
-rw-r--r-- 1 root root 11 Dez 11 14:25 /var/tmp/touched
ansible@athena:~$ cat /var/tmp/touched
VirtualBox

The ansible_bios_version is a fact that Ansible gathers from the servers. There are many such facts, check them as follows

[ansible@ansibleworkstation ~]$ ansible all -m setup

And that concludes this blogpost. This is of course just the start of the journey with Ansible. Every journey starts with a first step though and I hope I helped with that.