Puppet - Live Project

In order to perform the pve testing of applying configuration and manifests on Puppet node, we will use a pve working demo. This can be directly copied and pasted to test how the configuration works. If the user wishes to use the same set of code, he needs to have the same naming convention as shown in code snippets as follows.

Let’s start with the creation of a new module.

Creating a New Module

The first step in testing and applying the httpd configuration is by creating a module. In order to do this, the user needs to change his working directory to Puppet module directory and create a basic module structure. The structure creation can be done manually or by using Puppet to create boilerplate for the module.

# cd /etc/puppet/modules 
# puppet module generate Live-module

Note − Puppet module generate command requires that the module-name takes the format of [username]-[module] to comply with Puppet forge specifications.

The new module contains some basic files, including a manifest directory. The directory already contains a manifest named init.pp, which is modules main manifest file. This is an empty class declaration for the module.

class pve-module { 
} 

The module also contains a test directory containing a manifest called init.pp. This test manifest contains reference to the pve-module class within manifest/init.pp:

include pve-module

Puppet will use this test module to test the manifest. Now we are ready to add the configuration to the module.

Instalpng a HTTP Server

Puppet module will install the necessary packages to run http server. This requires a resource definition that defines the configuration of httpd packages.

In the module’s manifest directory, create a new manifest file called httpd.pp

# touch test-module/manifests/httpd.pp

This manifest will contain all HTTP configuration for our module. For separation purpose, we will keep the httpd.pp file separate from init.pp manifest file

We need to put the following code in httpd.pp manifest file.

class test-module::httpd { 
   package {  httpd : 
      ensure => installed, 
   } 
}

This code defines a subclass of test-module called httpd, then defines a package resource declaration for the httpd package. The ensure => installed attribute checks if the required package is installed. If not installed, Puppet uses yum utipty to install it. Next, is to include this subclass in our main manifest file. We need to edit init.pp manifest.

class test-module { 
   include test-module::httpd 
}

Now, it’s the time to test the module which could be done as follows

# puppet apply test-module/tests/init.pp --noop

The puppet apply command apppes the configuration present in manifest file on the target system. Here, we are using test init.pp which refers to main init.pp. The –noop performs the dry run of the configuration, which only shows the output but actually does not do anything.

Following is the output.

Notice: Compiled catalog for puppet.example.com in environment 
production in 0.59 seconds 

Notice: /Stage[main]/test-module::Httpd/Package[httpd]/ensure: 
current_value absent, should be present (noop) 

Notice: Class[test-module::Httpd]: Would have triggered  refresh  from 1 
events 

Notice: Stage[main]: Would have triggered  refresh  from 1 events 
Notice: Finished catalog run in 0.67 seconds 

The highpght pne is the result of the ensure => installed attribute. The current_value absent means that Puppet has detected the httpd package is installed. Without the –noop option, Puppet will install the httpd package.

Running the httpd Server

After instalpng the httpd servers, we need to start the service using other resource deceleration: Service

We need to edit the httpd.pp manifest file and edit the following content.

class test-module::httpd { 
   package {  httpd : 
      ensure => installed, 
   } 
   service {  httpd : 
      ensure => running, 
      enable => true, 
      require => Package["httpd"], 
   } 
}

Following is the pst of targets that we have achieved from the above code.

The ensure => running status checks if the service is running, if not then it enables it.

The enable => true attribute sets the service to run when the system boots up.

The require => Package["httpd"] attribute defines an ordering relationship between one resource deceleration and other. In the above case, it ensures that the httpd service starts after the http package is installed. This creates a dependency between the service and the respective package.

Run the puppet apply command to test the changes again.

# puppet apply test-module/tests/init.pp --noop 
Notice: Compiled catalog for puppet.example.com in environment 
production in 0.56 seconds 

Notice: /Stage[main]/test-module::Httpd/Package[httpd]/ensure: 
current_value absent, should be present (noop) 

Notice: /Stage[main]/test-module::Httpd/Service[httpd]/ensure: 
current_value stopped, should be running (noop) 

Notice: Class[test-module::Httpd]: Would have triggered  refresh  from 2 
events 

Notice: Stage[main]: Would have triggered  refresh  from 1 events 
Notice: Finished catalog run in 0.41 seconds 

Configuring httpd Server

Once the above steps are completed, we will have HTTP server installed and enabled. The next step is to provide some configuration to the server. By default, httpd provides some default configurations in /etc/httpd/conf/httpd.conf which provides a webhost port 80. We will add some additional host to provide some user-specific facipties to the web-host.

A template will be used to provide additional port as it requires a variable input. We will create a directory called template and add a file called test-server.config.erb in the new director and add the following content.

Listen <%= @httpd_port %> 
NameVirtualHost *:<% = @httpd_port %> 

<VirtualHost *:<% = @httpd_port %>> 
   DocumentRoot /var/www/testserver/ 
   ServerName <% = @fqdn %> 
   
   <Directory "/var/www/testserver/"> 
      Options All Indexes FollowSymLinks 
      Order allow,deny 
      Allow from all 
   </Directory> 
</VirtualHost>

The above template follows the standard apache-tomcat server configuration format. The only difference is the use of Ruby escape character to inject variables from the module. We have FQDN which stores fully quapfied domain name of the system. This is known as the system fact.

System facts are collected from each system prior to generating each respective system’s puppet catalog. Puppet uses the facter command to get this information and one can use facter to get other details regarding the system. We need to add the highpght pnes in httpd.pp manifest file.

class test-module::httpd { 
   package {  httpd : 
      ensure => installed, 
   } 
   service {  httpd : 
      ensure => running, 
      enable => true, 
      require => Package["httpd"], 
   } 
   file { /etc/httpd/conf.d/testserver.conf : 
      notify => Service["httpd"], 
      ensure => file, 
      require => Package["httpd"], 
      content => template("test-module/testserver.conf.erb"), 
   } 
   file { "/var/www/myserver": 
      ensure => "directory", 
   } 
}

This helps in achieving the following things −

This adds a file resource declaration for the server configuration file (/etc/httpd/conf.d/test-server.conf). The content of this file is the test-serverconf.erb template that was created earper. We also check the httpd package installed before adding this file.

This adds the second file resource declaration which creates a directory (/var/www/test-server) for the web server.

Next, we add the relationship between the configuration file and the https service using the notify => Service["httpd"]attribute. This checks if there are any configuration file changes. If there is, then Puppet restarts the service.

Next is to include the httpd_port in the main manifest file. For this, we need to end the main init.pp manifest file and include the following content.

class test-module ( 
   $http_port = 80 
) { 
   include test-module::httpd 
}

This sets the httpd port to the default value of 80. Next is to run the Puppet apply command.

Following will be the output.

# puppet apply test-module/tests/init.pp --noop 
Warning: Config file /etc/puppet/hiera.yaml not found, using Hiera 
defaults 

Notice: Compiled catalog for puppet.example.com in environment 
production in 0.84 seconds 

Notice: /Stage[main]/test-module::Httpd/File[/var/www/myserver]/ensure: 
current_value absent, should be directory (noop) 

Notice: /Stage[main]/test-module::Httpd/Package[httpd]/ensure: 
current_value absent, should be present (noop) 

Notice: 
/Stage[main]/test-module::Httpd/File[/etc/httpd/conf.d/myserver.conf]/ensure: 
current_value absent, should be file (noop) 

Notice: /Stage[main]/test-module::Httpd/Service[httpd]/ensure: 
current_value stopped, should be running (noop) 

Notice: Class[test-module::Httpd]: Would have triggered  refresh  from 4 
events 

Notice: Stage[main]: Would have triggered  refresh  from 1 events 
Notice: Finished catalog run in 0.51 seconds 

Configuring the Firewall

In order to communicate with the server one requires an open port. The problem here is that different kind of operating systems use different methods of controlpng the firewall. In case of Linux, versions below 6 use iptables and version 7 use firewalld.

This decision of using an appropriate service is somewhat handled by Puppet using the system facts and its logic. For this, we need to first check the OS and then run the appropriate firewall command.

In order to achieve this, we need to add the following code snippet inside testmodule::http class.

if $operatingsystemmajrelease <= 6 { 
   exec {  iptables : 
      command => "iptables -I INPUT 1 -p tcp -m multiport --ports 
      ${httpd_port} -m comment --comment  Custom HTTP Web Host  -j ACCEPT && 
      iptables-save > /etc/sysconfig/iptables", 
      path => "/sbin", 
      refreshonly => true, 
      subscribe => Package[ httpd ], 
   } 
   service {  iptables : 
      ensure => running, 
      enable => true, 
      hasrestart => true, 
      subscribe => Exec[ iptables ], 
   } 
}  elsif $operatingsystemmajrelease == 7 { 
   exec {  firewall-cmd : 
      command => "firewall-cmd --zone=pubpc --addport = $ { 
      httpd_port}/tcp --permanent", 
      path => "/usr/bin/", 
      refreshonly => true, 
      subscribe => Package[ httpd ], 
   } 
   service {  firewalld : 
      ensure => running, 
      enable => true, 
      hasrestart => true, 
      subscribe => Exec[ firewall-cmd ], 
   } 
} 

The above code performs the following −

Using the operatingsystemmajrelease determines whether the OS which is used is version 6 or 7.

If the version is 6, then it runs all the required configuration commands to configure Linux 6 version.

If OS version is 7, then it runs all the required commands required to configure the firewall.

The code snippet for both the OS contains a logic which ensures that the configuration runs only after the http package is installed.

Finally, run the Puppet apply command.

# puppet apply test-module/tests/init.pp --noop 
Warning: Config file /etc/puppet/hiera.yaml not found, using Hiera 
defaults 

Notice: Compiled catalog for puppet.example.com in environment 
production in 0.82 seconds 

Notice: /Stage[main]/test-module::Httpd/Exec[iptables]/returns: 
current_value notrun, should be 0 (noop) 

Notice: /Stage[main]/test-module::Httpd/Service[iptables]: Would have 
triggered  refresh  from 1 events 

Configuring the SELinux

As we are working on a Linux machine which is version 7 and above, hence we need to configure it to make a http communication. SELinux restricts non-standard access to the HTTP server by default. If we define a custom port, then we need to configure the SELinux to provide access to that port.

Puppet contains some resource type to manage SELinux functions, such as Booleans and modules. Here, we need to execute semanage command to manage port settings. This tools is a part of popcycoreutils-python package, which is not installed on red-hat servers by default. In order to achieve the above, we need to add the following code inside the test-module::http class.

exec {  semanage-port : 
   command => "semanage port -a -t http_port_t -p tcp ${httpd_port}", 
   path => "/usr/sbin", 
   require => Package[ popcycoreutils-python ], 
   before => Service [ httpd ], 
   subscribe => Package[ httpd ], 
   refreshonly => true, 
} 

package {  popcycoreutils-python : 
   ensure => installed, 
} 

The above code performs the following −

The require => Package[ popcycoreutils-python ] ensures that we have the required python module installed.

Puppet uses semanage to open the port using the httpd_port as a veriable.

The before => service ensures to execute this command before httpd service starts. If HTTPD starts before SELinux command, then SELinux the service request and the service request fails.

Finally, run the Puppet apply command

# puppet apply test-module/tests/init.pp --noop 
... 
Notice: /Stage[main]/test-module::Httpd/Package[popcycoreutilspython]/ 
ensure: current_value absent, should be present (noop) 
...
Notice: /Stage[main]/test-module::Httpd/Exec[semanage-port]/returns: 
current_value notrun, should be 0 (noop) 
... 
Notice: /Stage[main]/test-module::Httpd/Service[httpd]/ensure: 
current_value stopped, should be running (noop) 

Puppet installs the python module first and then configures the port access and finally starts the httpd service.

Copying HTML Files in the Web Host

With the above steps we have completed the http server configuration. Now, we have a platform ready to install a web-based apppcation, which Puppet can also configure. To test, we will copy some sample html index web pages to the server.

Create an index.html file inside the files directory.

<html> 
   <head> 
      <title>Congratulations</title> 
   <head> 
   
   <body> 
      <h1>Congratulations</h1> 
      <p>Your puppet module has correctly appped your configuration.</p> 
   </body> 
</html> 

Create a manifest app.pp inside the manifest directory and add the following content.

class test-module::app { 
   file { "/var/www/test-server/index.html": 
      ensure => file, 
      mode => 755, 
      owner => root, 
      group => root, 
      source => "puppet:///modules/test-module/index.html", 
      require => Class["test-module::httpd"], 
   } 
}

This new class contains a single resource deceleration. This copies a file from the module’s file directory to the web server and sets its permissions. The required attribute ensures the test-module::http class completes the configuration successfully before one apppes test-module::app.

Finally, we need to include a new manifest in our main init.pp manifest.

class test-module ( 
   $http_port = 80 
) { 
   include test-module::httpd 
   include test-module::app 
} 

Now, run the apply command to actually test what is happening. Following will be the output.

# puppet apply test-module/tests/init.pp --noop
Warning: Config file /etc/puppet/hiera.yaml not found, using Hiera 
defaults 

Notice: Compiled catalog for brcelprod001.brcle.com in environment 
production in 0.66 seconds 

Notice: /Stage[main]/Test-module::Httpd/Exec[iptables]/returns: 
current_value notrun, should be 0 (noop) 

Notice: /Stage[main]/Test-module::Httpd/Package[popcycoreutilspython]/ 
ensure: current_value absent, should be present (noop) 

Notice: /Stage[main]/Test-module::Httpd/Service[iptables]: Would have 
triggered  refresh  from 1 events 

Notice: /Stage[main]/Test-module::Httpd/File[/var/www/myserver]/ensure: 
current_value absent, should be directory (noop) 

Notice: /Stage[main]/Test-module::Httpd/Package[httpd]/ensure: 
current_value absent, should be present (noop) 

Notice: 
/Stage[main]/Test-module::Httpd/File[/etc/httpd/conf.d/myserver.conf]/ensur 
e: current_value absent, should be file (noop) 

Notice: /Stage[main]/Test-module::Httpd/Exec[semanage-port]/returns: 
current_value notrun, should be 0 (noop) 

Notice: /Stage[main]/Test-module::Httpd/Service[httpd]/ensure: 
current_value stopped, should be running (noop) 

Notice: Class[test-module::Httpd]: Would have triggered  refresh  from 8 
Notice: 
/Stage[main]/test-module::App/File[/var/www/myserver/index.html]/ensur: 
current_value absent, should be file (noop) 

Notice: Class[test-module::App]: Would have triggered  refresh  from 1 
Notice: Stage[main]: Would have triggered  refresh  from 2 events Notice: 
Finished catalog run in 0.74 seconds

The highpghted pne shows the result of index.html file being copied to the web-host.

Finapzing the Module

With all the above steps, our new module that we created is ready to use. If we want to create an archive of the module, it can be done using the following command.

# puppet module build test-module