Friday, 8 March 2019

Getting Started with Puppet

For premium DevOps and AWS Courses: https://imojo.in/2cox7em
  1. AWS with the project 
  2. DevOps Real-Time 
  3. DevOps with the project 
  4. AWS Real-Time
  5. Linux Admin

1.1 How Puppet Works



- Ravi is a DevOps person in charge of managing company infrastructure
- He manually configure or do changes on all the virtual machines
- It takes weeks or may be months depending on the size of infrastructure.






- Is there a way to manage this better?
- What if he can deploy change concurrently to all the systems


Puppet helps to automate repetitive and error prone tasks
- Provision servers
- Deploy application
- Proactively manage infrastructure




It helps to automate full infrastructure- Like OS, network, middleware, application.





Ravi can manage resources by writing in Puppet’s Simple Language. It will work on Linux, Solaris, AIX, windows,etc.

It also possible to deploy in different env, like public, private and hybrid clouds.




1.2 What is Puppet?


·         Ruby based
·         Can run in Client-Server or standalone model
·         Developed by Luke Kanies
·         used to manage a host throughout its lifecycle: from initial build and installation, to upgrades, maintenance, and finally to end-of-life
·         designed to continuously interact with your hosts, unlike provisioning tools which build your hosts and leave them unmanaged.
·         Uses puppet declarative language( focus is what instead of how)
Four Major challenges in keeping up the configuration of systems and application:
1>    ensuring that servers we deploy have consistent base configurations
2>    remediating inconsistent or incorrect configurations
3>    deploying applications in a consistent and reliable way
4>    automating all of these things

Three Components of puppet operating model:
1>    Deployment
2>    Configuration Language and resource Abstraction Layer
3>    Transactional Layer





1.3 Why can’t we achieve above things by script


·         Take lot of time – Write, Test and Debug
·         Procedural – goes in multiple of line to achieve a result, focus on how.
·         Mulitple ways of doing same thing, difficult for other person to understand
·         Need different code for different OS
·         For Same OS, different version have diff commands



1.4 Why Puppet?

·         Overcomes shortcomings of other configuration management tools.
e.g: Chef- Ruby is req, CEconfig- Complex
·         Puppet programs are Declarative
·         Focus is “What” instead of “How”, less complex .



1.5 Advantages





1.10 Architecture


·         Usually deployed in a simple client-server model
·         Server = Puppet Master  (Linux Server), Puppet Master application is written in Ruby on Rails
·         Client = Nodes
·         Master and Nodes communicate via SSL over TCP/8140
·         Nodes pull config from Master
·         Each individual configuration item in a Puppet program is called a resource declaration.



Installing and Configuring the Puppet Master


2.1 Lab Setup


Master – Centos 7
Client1- CentOS 7





2.4 Installing the PuppetMaster


Add the puppet repository: to download puppet binaries.
#yum –y install https://yum.puppetlabs.com/puppetlabs-release-el-7.noarch.rpm
#yum –y install puppet-server

To check the version of Puppet Master
#puppet master –version


2.5 Setting Up Directory Environments


Directory Env allow to maintain various env like test, dev and prod on same puppet master.
Puppet Loads the site manifest by Default (pp = Puppet Program)
/etc/puppet/manifests/site.pp
Tasks:
1>    Create a prod env
2>    Lower the env timeout
3>    Set DNS alternative names
Environment Configuration file: /etc/puppet/environments/production/environment.conf

Three Section /etc/puppet/puppet.conf ( file exists on both master and agent node)
1.      Master
2.      Agent
3.      Main
[main]
  logdir = /var/log/puppet
  rundir = /var/run/puppet
  ssldir = $vardir/ssl

[master]
  environmentpath = $confdir/environments
  basemodulepath = $confdir/modules:/opt/puppet/share/puppet/modules

[agent]
  classfile = $vardir/classes.txt
  localconfig = $vardir/localconfig


Creating directory structure
#mkdir –p /etc/puppet/environments/production/{modules,manifests}
#cd /etc/puppet/environments/production
#vi environment.conf
  modulepath = /etc/puppet/environments/production/modules
  environment_timeout = 5s

#cd /etc/puppet/
#vi puppet.conf    ( there is no master section : create it)
[main]
……
    dns_alt_names = puppet, puppetmaster, puppetmaster.company.com

[master]
    environmentpath = $confdir/environments
    basemodulepath = $confdir/modules:/opt/puppet/share/modules/

[agent]
….






2.6 Security: Dealing with SELinux


Put it into permissive mode.
#setenforce permissive
File: /etc/sysconfig/selinux


2.7 Puppet Master: Generating Certificates


To generate Certificates: need to perform only one time
#puppet master –verbose –no-daemonize
Break it ^c
#ls –la  /var/lib/puppet/ssl

Suppose we want to re-generate the keys if the current ones are compromised. Delete ssl (rm –r /var/lib/puppet/ssl)

Chapter 3

Installing and Configuring the Puppet Agent


3.1 Boot the client1 server






3.2 Adding the PuppetMaster to the hosts file


On both client:
#echo 192.168.56.201 puppetmaster >> /etc/hosts

3.3 Installing the Puppet Agent on CentOS


#yum –y install puppet

Now open the /etc/puppet/puppet.conf file, without server entry puppet agent will not function.
#vi /etc/puppet/puppet.conf
[agent]
    server =  puppetmaster






3.5 Generating Certificate from Client


On both client nodes run below command:
#puppet agent --verbose --no-daemonize –onetime


3.6 Signing Agent Certificates


PuppetMaster by default don’t accept the certificates
#puppet cert list
# puppet cert sign client1
# puppet cert sign client2
#puppet cert list --all

Chapter 4

Puppet Concept







4.1 Resource Declaration


Each individual configuration item in a Puppet program is called a resource declaration.
To manage resources on a target system, you should declare them in Puppet manifests.

Resource Declaration answer two basic questions:
·         What aspect – user, service , package, file
·         What state – stop, start

******************************
type { title:
attribute => value,
}
******************************




4.2 Package Resource


To browse the resource package, run below command:

*************************************
# puppet resource package
package { 'GeoIP':
  ensure => '1.5.0-11.el7',
}
package { 'NetworkManager':
  ensure => '1.4.0-17.el7_3',
}
package { 'NetworkManager-libnm':
  ensure => '1.4.0-17.el7_3',
}
package { 'NetworkManager-team':
  ensure => '1.4.0-17.el7_3',
}

************************************************************

Package resource Type:
package { ‘ntp’:
  ensure => ‘installed’,
 }





4.3 File Resource


Description:
Manages files, including their content, ownership, and permissions.
File contents can be managed directly with the content attribute, or downloaded from a remote source using the source attribute

file { ‘/readme.txt’:
  ensure => ‘present’,
  content => “Comments are here”,
}


Run below command to check the various attributes of file /etc.
******************************************************
# puppet resource file /etc
file { '/etc':
  ensure   => 'directory',
  ctime    => '2017-03-17 09:11:06 -0400',
  group    => '0',
  mode     => '755',
  mtime    => '2017-03-17 09:10:43 -0400',
  owner    => '0',
  selrange => 's0',
  selrole  => 'object_r',
  seltype  => 'etc_t',
  seluser  => 'system_u',
  type     => 'directory',
}

*****************************************************





4.4 Service Resource


Manage running services

e.g:
service { ‘ntpd’:
  ensure => ‘running’ ,
  enable => true ,   #(note: Boolean attributes shouldn’t be in quotes, in quotes it takes all true)
}

Run below command to check all the services on the server
******************************************************
# puppet resource service
service { 'NetworkManager-dispatcher.service':
  ensure => 'stopped',
  enable => 'true',
}
service { 'NetworkManager-wait-online.service':
  ensure => 'running',
  enable => 'false',
}
service { 'NetworkManager.service':
  ensure => 'running',
  enable => 'true',
}

*****************************************************************





4.5 Facter


Facter is a standalone tool that house’s environment-level variables.
Each key-value in facter is referred to as “facts”
Command:
# facter
architecture => x86_64
augeasversion => 1.4.0
bios_release_date => 12/01/2006
bios_vendor => innotek GmbH
bios_version => VirtualBox
blockdevice_sda_model => VBOX HARDDISK
blockdevice_sda_size => 8589934592
blockdevice_sda_vendor => ATA
blockdevice_sr0_model => CD-ROM
blockdevice_sr0_size => 1073741312
blockdevice_sr0_vendor => VBOX
blockdevices => sda,sr0
boardmanufacturer => Oracle Corporation
boardproductname => VirtualBox
boardserialnumber => 0
facterversion => 2.4.6
……….










Chapter 5

Creating Manifests


·         Puppet programs are stored in files called manifests.
·         A manifest is just a regular text file with a .pp extension that contains Puppet code.
·         A Puppet manifest consists of various elements such as node definitions, resource declarations, variables, and classes that control what configuration get applied to what nodes.
·         full path of directory will be /etc/puppet/environments/production/manifests
·         Puppet will automatically load all of the .pp files in this directory


5.1 Node Definitions


#vi /etc/puppet/environments/production/manifests/nodes.pp
node ‘client1’ {
}

node ‘client2’{
}


5.2 Managing Files


·         Earlier we looked at three common resource types for managing: packages, files, and services.
·         to incorporate these resource types into our new manifest
Create a simple text file on each server that says, "Created by Puppet", and then appends the date and time that the file was created
Make  the entry in nodes.pp
  file { '/info.txt':
    ensure  => 'present',
    content => inline_template("Created by Puppet at <%= Time.now %>\n"),
  }



5.4 Managing Packages


On puppetmaster: nodes.pp
  package { 'ntp':
    ensure => 'installed',
  }
On Client: CentOS
# puppet agent --verbose --no-daemonize –onetime
Once the ntp is installed, verify:
CentOS:
# yum info ntp
Installed Packages
Name        : ntp
Arch        : x86_64






5.5 Managing Services


On Puppetmaster:

  service { 'ntpd':
    ensure  => 'running',
    enable  => true,
  }


On client run below command:
# puppet agent --verbose --no-daemonize –onetime


Now we delete the ntp package and run again the puppet agent command:


Here, puppet determined that first the package need to be installed and after that only services can be brought up. However, there are chances that it can get it wrong. Will see how to take care of this later.







5.6 Selectors


·         On our client1 server which runs CentOS, the ntp service is called ntpd.
·         On the client2 Ubuntu server it's called ntp.
·         We had to specify separate resource declarations with different titles under each node definition in our manifest
·         Not an efficient way
·         If we are having many different nodes, it becomes unmanageable.
·         To make it scalable and readable by using a Puppet feature called a selector
·         Uses variables
e.g:
$ntpservice = $osfamily ? {
  ‘redhat’ =>  ‘ntpd’,
  ‘debian’ =>  ‘ntp’,
   default  =>  ‘ntp’,
}

How to use it:
Replace:
  service { 'ntpd':
    ensure  => 'running',
    enable  => true,
  }


With:
  service { $ntpservice:
    ensure  => 'running',
    enable  => true,
  }


5.7 Classes

·         At this point, we have a lot of repetition in our code.
·         Issues: inefficient, doesn’t scale very well,  difficult to read and understand.
·         Should not repeat the code.
·         Class can be used to clean up the code and arrange it better.
·         Puppet Class is very different from Object Oriented Class.
·         A Puppet class is simply a named collection of resources.
·         Adv:  class is that you can write it once and then reuse it throughout the Puppet programs
e.g:
class linux {
  package { ‘ntp’:
    ensure  =>  ‘installed’,
  }
}

node  ‘wiki’ {
  { class ‘linux’:}
}

nodes.pp

node 'client1' {
  class { 'linux': }
}

node 'client2' {
  class { 'linux': }
}

class linux {

  $ntpservice = $osfamily ? {
    'redhat' =>  'ntpd',
    'debian' =>  'ntp',
    default  =>  'ntp',
  }

  file { '/info.txt':
    ensure  => 'present',
    content => inline_template("Created by Puppet at <%= Time.now %>\n"),
  }

  package { 'ntp':
    ensure => 'installed',
  }

  service { $ntpservice :
    ensure  => 'running',
    enable  => true,
  }
}



5.8 Using Variables


We can use variables in another way to avoid repetition.
Suppose we have three packages we want to install, Nano, Git, and Screen.
Instead of writing resource for each, we can use variables to write it once.

class linux {
  $packlist = ['nano', 'screen']

  package { $packlist:
    ensure => 'installed',
  }  
…….
Old Entries
……..
}
Package might not installed in the order in which they are specified.


Chapter 6

Creating and Using Modules



6.1 Definition

Modules are directories with a predictable structure.
Puppet can automatically load manifests, files, and plugins from modules in its modulepath.





6.2 Finding Modules


Use puppet master --configprint modulepath to see where Puppet expects to find modules on your system.
# puppet master --configprint modulepath
/etc/puppet/environments/production/modules


6.3 Creating Modules


#cd /etc/puppet/environments/production/modules/
#puppet module generate yourname-mediawiki --environment production
#mv yourname-mediawiki mediawiki
#cd mediawiki/manifests

Check the file init.pp. It is having one class called mediawiki.





6.11 Customising Module


Create below entry inside class mediawiki
# cat /etc/puppet/environments/production/modules/mediawiki/manifests/init.pp
class mediawiki {

  $phpmysql  = $osfamily ? {
    'redhat' => 'php-mysql',
    'debian' => 'php5-mysql',
    default  => 'php-mysql',
  }

  package { $phpmysql:
    ensure => 'present',
  }

}








6.12 How to use class from a module inside a node definition


Now include above class inside each node definition in nodes.pp

# cat /etc/puppet/environments/production/manifests/nodes.pp              
node 'client1' {

  class { 'linux': }
  class { 'mediawiki': }

}

node 'client2' {

  class { 'linux': }
  class { 'mediawiki': }

}

class linux {

  $packlist = ['git', 'nano', 'screen']

  package { $packlist:
    ensure => 'installed',
  }

  $ntpservice = $osfamily ? {
    'redhat' =>  'ntpd',
    'debian' =>  'ntp',
    default  =>  'ntp',
  }

  file { '/info.txt':
    ensure  => 'present',
    content => inline_template("Created by Puppet at <%= Time.now %>\n"),
  }

  package { 'ntp':
    ensure => 'installed',
  }

  service { $ntpservice :
    ensure  => 'running',
    enable  => true,
  }

}

Run the agent command on both server client1 and client2.

# puppet agent --verbose --no-daemonize –onetime









6.13 Conditionals


If we want to install a package only on CentOS but not on Ubuntu then we use conditionals.

Add this in init.pp inside mediawiki class.
  if $osfamily == 'redhat' {
    package { 'php-xml':
      ensure => 'present',
    }
  }

Run the agent command on both server client1 and client2. This package php-xml got  installed  only on client1.

# puppet agent --verbose --no-daemonize –onetime

CentOS:
# systemctl list-unit-files | grep -i mysql

When you run:
#yum install mysql
command by default it installs MariaDB not MySQL.
So try the following command:
#yum list installed | grep mariadb
if mariadb-server is missing try this following command:
#yum install mariadb-server
it installs the server package then start the service:
#systemctl start mariadb






Or create below entries in nodes.pp
***************************************************
$packlist = ['nano', 'screen', 'mariadb', 'mariadb-server']

  package { $packlist:
    ensure => 'installed',
  }
  if $osfamily == 'redhat' {
  service { 'mariadb':
      ensure => 'running',
    }
  }
*********************************************


 For Complete DevOps Course : Link
 For Complete AWSCourse : Link