tsu - a new social networking platform which pays users to post!
https://www.tsu.co/ashoklucky
https://nsrc.org/workshops/ws-files/2011/sanog17/exercises/
Source from: nsrc.org
Exercise 1: Set up a kerberos client
A Kerberos client is easy to set up to work with existing Kerberos infrastructure.Setup
Install the Kerberos client packages:# apt-get install krb5-user
To show how little configuration is really needed, we will move the auto-generated config file out of the way (which contains a load of junk anyway) and create a new minimal one with just 4 lines:# mv /etc/krb5.conf /etc/krb5.conf.example
# editor /etc/krb5.conf
[libdefaults]
default_realm = WS.NSRC.ORG
dns_lookup_realm = true
dns_lookup_kdc = true
Now check that the ssh client program will try Kerberos authentication. It's enabled by default in Ubuntu, but in other operating systems it might not be.# editor /etc/ssh/ssh_config (on MacOSX it's /etc/ssh_config)
...
GSSAPIAuthentication yes # check this line present (near end)
GSSAPIKeyExchange yes # add this line too
...
Testing ssh client
The class Kerberos setup has an account "testuser" which you can use.$ kinit testuser
... enter password when prompted
(password is "nsrc2020" unless you've been told otherwise)
Now you should now be able to login to servers in your Kerberos realm without re-entering your password:$ ssh testuser@noc.ws.nsrg.org
-- logout, then login somewhere else
$ ssh testuser@noc2.ws.nsrc.org
-- logout
Have a look at the tickets you've picked up:$ klist
You should see your own TGT plus tickets for the servers you've connected to.To get rid of them, use
kdestroy
; check that you can no longer login (the server will give you a password prompt instead)Reference material [not part of the exercise]
We've actually enabled two different authentication mechanisms.GSSAPIAuthentication
is the standard one (gssapi-with-mic), and just authenticates the user. GSSAPIKeyExchange
is a new one (gssapi-keyex) which also validates the authenticity of the host. It avoids the need to accept host keys into your known_hosts
file, because Kerberos already provides mutual authentication.GSSAPIKeyExchange
is available as a patch to ssh, and is included by default in recent Debian/Ubuntu, RHEL6, and Fedora 13+. But it probably won't work when connecting to older Unix boxes or to BSD boxes, in which case you'll fall back to gssapi-with-mic.More info at http://www.sxw.org.uk/computing/patches/openssh.html
Enabling Kerberos in HTTP clients
For curl, you must supply an empty username and password in option-u
$ curl --negotiate -u: http://noc.ws.nsrc.org/secure/
For Firefox:- Go to
about:config
- Filter on "negotiate"
network.negotiate-auth.trusted-uris ws.nsrc.org
/opt/google/chrome/google-chrome \
--auth-server-whitelist=*.ws.nsrc.org
Enabling Kerberos in LDAP client
Under Ubuntu you have to have the appropriate SASL-GSSAPI module installed.# apt-get install ldap-utils libsasl2-modules-gssapi-mit
$ ldapsearch -Y GSSAPI -h ldap.ws.nsrc.org \
-b "dc=ws,dc=nsrc,dc=org" "(cn=*candler*)"
---------------------------------------------------------------------------------------
Exercise2 : enable Kerberos logins to a host
This is to connect a server to an existing Kerberos/LDAP infrastructure. In part 1 we'll just enable Kerberos authentication, and in part 2 we'll enable LDAP authorization for uid/gid mapping.Part 1: Kerberos authentication
Your machine must already be working as a Kerberos client (i.e./etc/krb5.conf
is setup correctly)Sync your clock
Kerberos only works if your clocks are in close sync (certainly within 5 minutes). You can do a one-off clock sync like this:# ntpdate ntp.ubuntu.com
But it's better to install the ntp daemon, which will continuously keep your clock in accurate sync.# apt-get install ntp
Enable kerberos for sshd
In thesshd_config
file, uncomment the GSSAPIAuthentication
line and set it to yes
; also add GSSAPIKeyExchange yes
. Then restart sshd.# editor /etc/ssh/sshd_config
...
# GSSAPI options
GSSAPIAuthentication yes
GSSAPIKeyExchange yes
....
# service ssh restart
You're also going to add some extra lines to the end of krb5.conf, because of a current limitation in the kadmin utility.# editor /etc/krb5.conf
...
[realms]
WS.NSRC.ORG = {
admin_server = kdc1.ws.nsrc.org
}
Now you need to set up a shared key between the KDC and the host's keytab. Call over an instructor to do this for you (because we're not letting you have admin access to the workshop Kerberos server :-) Here's what they will do:# kadmin -p nsrc/admin
... enter secret instructor password
addprinc -randkey host/pcN.ws.nsrc.org
ktadd host/pcN.ws.nsrc.org
^D
This has (1) created the principal on the KDC, and (2) has copied the key into a local keytab file.You can examine your own keytab using either
klist -k
for a summary, or ktutil
for more detailled information. Inside ktutil
use '?' for help and ^D (ctrl-D) to exit.# ktutil
rkt /etc/krb5.keytab
list -e
^D
(The key will be listed multiple times, for use with different encryption algorithms)Now you're going to allow ssh logins for 'testuser'. Because we don't have LDAP configured yet, we'll have to create a local account - but there's no need to set a password.
# useradd -m -s /bin/bash testuser
Now you're ready to test it. You can try logging in from your own machine to yourself, and you can get someone else to login to your machine.$ kinit testuser # use password from exercise 1
$ ssh testuser@pcN.ws.nsrc.org
If instead of logging straight in you see a password prompt from ssh, then there's a problem which needs investigating.Debugging
The first things to check are:- Is your system clock at the correct time?
- Does your host have working forward and reverse DNS?
- Did you get a ticket? (klist)
- Did you restart sshd after setting
GSSAPIAuthentication yes
?
# /usr/sbin/sshd -d -p99
Then try to connect to it:$ ssh -v -p99 testuser@pcN.ws.nsrc.org
You will see logs at both the client and server side.Part 2: LDAP authorization
This centralises your user/uid/gid/homedir management, and removes the need to add local users.We're going to configure it manually. In practice you'd probably just make a tarball containing all the preconfigured files and drop them onto each server which needed it.
Install LDAP utils
# apt-get install ldap-utils libsasl2-modules-gssapi-mit
# editor /etc/ldap/ldap.conf
BASE dc=ws,dc=nsrc,dc=org
URI ldap://ldap.ws.nsrc.org
Now this is configured, you should be able to query the LDAP directory - but the server is set up only to allow read access if you have a Kerberos ticket.$ kinit testuser
$ ldapsearch
... should list everything in ldap
$ ldapsearch "(cn=*test*)"
... should list only entries matching the search query
Automated Kerberos ticket
For security, the LDAP server only permits connections which are Kerberos authenticated and encrypted. This means that the host itself needs a Kerberos ticket when doing LDAP lookups. (We could instead have used TLS security, but that would involve setting up a Certificate Authority, and we may as well leverage our Kerberos setup)We will run a cronjob every hour, using our existing keytab entry to obtain a ticket for
host/pcN.ws.nsrc.org
# editor /etc/cron.hourly/kerberos
#!/bin/sh
/usr/bin/kinit -k host/`hostname` -c /tmp/krb5cc_host
Make it executable, and then run it once manually:# chmod +x /etc/cron.hourly/kerberos
# /etc/cron.hourly/kerberos
You can use this keytab to be able to do 'ldapsearch' when logged in as root:# KRB5CCNAME=/tmp/krb5cc_host ldapsearch
Note: our LDAP queries will come from the caching daemon (nscd) which runs as root, and so this is all we need. If you wanted to run nscd as a different user, you would have to extend the cron script to make the ticket readable to that user. e.g.#!/bin/sh
/usr/bin/kinit -k host/`hostname` -c /tmp/krb5cc_host
umask 077
cp /tmp/krb5cc_host /tmp/krb5cc_nscd
chown nscd /tmp/krb5cc_nscd
Install nss_ldap
This is the library which intercepts password/group lookups and redirects them to LDAP.# apt-get install libnss-ldap nscd
(Beware: there is an alternative package libnss-ldapd
with a trailing "d". These exercises have been tested using libnss-ldap
)Answer prompted questions as:
LDAP server [ldap://ldap.ws.nsrc.org]
Distinguished name of search base [dc=ws,dc=nsrc,dc=org]
LDAP version to use [3]
Make local root database admin [No]
Does the LDAP database require login? [No]
NOTE: The LDAP server starts with ldap:// not ldapi:// !The configuration file is
/etc/ldap.conf
. We're going to replace it with a simple one which uses Kerberos authentication:# mv /etc/ldap.conf /etc/ldap.conf.example
# editor /etc/ldap.conf
krb5_ccname /tmp/krb5cc_host
use_sasl on
rootuse_sasl on
base dc=ws,dc=nsrc,dc=org
uri ldap://ldap.ws.nsrc.org
ldap_version 3
sasl_secprops minssf=56
nss_initgroups_ignoreusers backup,bin,bind,daemon,games,gnats,irc,libuuid,list,lp,mail,man,news,nslcd,ntp,openldap,proxy,root,sshd,sync,sys,syslog,uucp,www-data
We are also going to tweak nscd.conf
to reduce caching of group information from 3600 to 600 seconds.# editor /etc/nscd.conf
...
positive-time-to-live group 600
...
Configure nsswitch
This is the point where we actually start to use LDAP for lookups.# editor /etc/nsswitch.conf
...
passwd: compat ldap
group: compat ldap
shadow: compat ldap
...
# service nscd restart
At this point you're ready to test: you should be able to map a username to a uid over LDAP, even though it doesn't exist in your local /etc/passwd
file# id ldapuser
uid=10004(ldapuser) gid=100(users) groups=100(users)
If it fails, places to look are:- /var/log/auth.log
- has root got a kerberos ticket? Use the following commands as root:
# KRB5CCNAME=/tmp/krb5cc_host klist # KRB5CCNAME=/tmp/krb5cc_host ldapsearch
- after any changes which might fix the problem, remember to restart nscd because it has both positive and negative caching
- try stopping nscd, then try
id ldapuser
as root - there will be no caching, so it will make a separate LDAP query for each request. You can look at tcpdump, and logs on the LDAP server side. - if you control the LDAP server, run it in debug mode:
# /etc/init.d/slapd stop # KRB5_KTNAME=/etc/ldap/krb5/krb5.keytab slapd -d 255 -u openldap -g openldap
Home directory creation
When a user first logs into a machine, you can create their home directory automatically via PAM (Pluggable Authentication Modules).# editor /etc/pam.d/common-session
...
session required pam_mkhomedir.so umask=0022 skel=/etc/skel/
...
Miscellaneous Notes
Using kadmin to extract the key on the target host itself is almost always the best way to set up the shared host/service key.If joining a Microsoft Active Directory domain: you will need a third-party tool such as css_adkadmin (or it may be possible to use bits from samba). This may be a starting point:
kinit Administrator
css_adkadmin -r AD.NSRC.ORG -s dc1.ad.nsrc.org -q "ank -k -attr description=$HOSTNAME +randkey host/$HOSTNAME"
css_adkadmin -r AD.NSRC.ORG -s dc1.ad.nsrc.org -q "xst +randkey host/$HOSTNAME"
The alternative is to set a manual password on a host principal and then use ktutil to create a keytab with the same password. This is awkward, because (1) you need to use the right kvno, and (2) you need to repeat it for each encryption type which might be used on your network.[KDC side]
kadmin
addprinc host/pcX.ws.nsrc.org
... you are prompted for password
getprinc host/pcX.ws.nsrc.org
... make a note of the kvno
[host side]
# ktutil
ktutil: addent -password -p host/pcX.ws.nsrc.org -k 4 -e aes256-cts
Password for host/pcX.ws.nsrc.org@WS.NSRC.ORG:
ktutil: addent -password -p host/pcX.ws.nsrc.org -k 4 -e arcfour-hmac
Password for host/pcX.ws.nsrc.org@WS.NSRC.ORG:
ktutil: addent -password -p host/pcX.ws.nsrc.org -k 4 -e des3-hmac-sha1
Password for host/pcX.ws.nsrc.org@WS.NSRC.ORG:
ktutil: list
slot KVNO Principal
---- ---- ---------------------------------------------------------------------
1 4 host/pcX.ws.nsrc.org@WS.NSRC.ORG
2 4 host/pcX.ws.nsrc.org@WS.NSRC.ORG
3 4 host/pcX.ws.nsrc.org@WS.NSRC.ORG
ktutil: wkt /etc/krb5.keytab
****************************************************************************
Exercise 3: Build a KDC
These exercises assume your host is pc1.ws.nsrc.org and you will be building a realm REALM1.WS.NSRC.ORG - change1
to the appropriate value for your machine.1. Install ntp
Because tickets are timestamped, it's important that your machine always has an accurately synchronized clock. Double-check it's installed:# apt-get install ntp
This will run a daemon (ntpd) which keeps your clock synchronized.Note: if you have a large network then it makes sense to have two local timeservers and all other machines sync to those. You would then change /etc/ntpd.conf to point to your time server(s).
2. Install kdc packages
# apt-get install krb5-kdc krb5-admin-server
If you get a dialog box about setting up a kerberos realm, just accept [Ok]3. Configure krb5 library
Now we are going to configure the krb5 client library, manually associating your own machine with your own realm. (We could instead change the DNS, but doing it here lets you control your own settings)# editor /etc/krb5.conf
[libdefaults]
default_realm = REALM1.WS.NSRC.ORG
dns_lookup_realm = true
dns_lookup_kdc = true
[realms]
REALM1.WS.NSRC.ORG = {
kdc = pc1.ws.nsrc.org
admin_server = pc1.ws.nsrc.org
}
[domain_realm]
pc1.ws.nsrc.org = REALM1.WS.NSRC.ORG
4. Create and initialize your database
Create Kerberos database:kdb5_util create -r REALM1.WS.NSRC.ORG -s
# This can pause for several minutes. Eventually you will be asked to
# choose a database master password. Use "abcd" for this exercise,
# but normally you'd choose something much stronger.
Create the ACL file and grant admin rights to all */admin principals:# editor /etc/krb5kdc/kadm5.acl
*/admin@REALM1.WS.NSRC.ORG *
Now we use kadmin.local (running as root) to create the first few principals: a host principal for the host itself (putting the randomly-chosen key into its own keytab file); a regular principal "student" and a KDC admin principal "student/admin"# kadmin.local
addprinc -randkey host/pc1.ws.nsrc.org
ktadd host/pc1.ws.nsrc.org
addprinc student
-- you'll be prompted to choose a password
addprinc student/admin
-- you'll be prompted to choose a password
^D
Now start the daemons:# /etc/init.d/krb5-kdc start
# /etc/init.d/krb5-admin-server start
At this point, you should be able to get a ticket from your own realm:$ kinit student
Password for student@REALM1.WS.NSRC.ORG:
$ klist
(Note that you've obtained a ticket from your own realm, using the username and password you created in your own KDC)If this doesn't work, then debug as follows:
- Check your clock is synchronized (type
date
) - Check your krb5.conf
- Check kdc is running:
ps auxwww | grep krb5kdc
- Look at KDC logs in /var/log/auth.log
5. Managing principals
From now on, you don't needkadmin.local
as root to administer the Kerberos database; you can use kadmin
instead, as a normal user. This runs over a TCP socket (so it can be run from a remote workstation, not necessarily the KDC). The traffic is authenticated and encrypted using Kerberos.$ kadmin -p student/admin
... enter password for 'student/admin' which you chose earlier
Type '?' for a list of commands. The important ones are:listprincs -- list principals
addprinc -- add principal
cpw -- change password
delprinc -- delete principal
Use ^D (ctrl-D) to exit.Try the following:
- Change the password for the
student
principal. In another screen, check that you cankinit student
with the new password. - Create a new user principal of your choice, and check you can get a ticket for it with kinit.
6. Kerberos ssh
Since we have modified krb5.conf earlier, restart sshd:# service ssh restart
Create a local student
user:# useradd -m -s /bin/bash student
Note that we have not set any password. However you should be able to login using a kerberos ticket:$ kinit student
$ ssh student@localhost
7. Extra exercises
If you have spare time, you and your neighbour can add [realms] and [domain_realms] sections for each other, and you can try to get a ticket directly from the other realm and use it to login to the other machine:kinit student@REALM2.WS.NSRC.ORG
ssh student@pc2.ws.nsrc.org
You could also try setting up cross-realm authentication. Each KDC has to share a secret with its neighbour. On both KDCs create the following principals; for each principal enter the same (preferably long and complex) passphrase on both KDCs.# kadmin -p student/admin
addprinc krbtgt/REALM2.WS.NSRC.ORG@REALM1.WS.NSRC.ORG
addprinc krbtgt/REALM1.WS.NSRC.ORG@REALM2.WS.NSRC.ORG
(The first sets up realm2 to trust realm1; the second sets up realm1 to trust realm2)At this point, Kerberos authentication will work, but ssh on pc1 will reject a user who has authenticated as someone@REALM2 because pc1 is part of REALM1.
-- on pc1, logging in to pc2
$ kinit student
$ ssh -v student@pc2.ws.nsrc.org
-- should be rejected
To permit it, create file ~/.k5login
containing the authorized principal(s), one per line.-- on pc2
# editor ~student/.k5login
student@REALM1.WS.NSRC.ORG
Notes
In a real environment, you would add at least one slave KDC for resilience. There is information how to do this at http://web.mit.edu/kerberos/krb5-1.8/krb5-1.8.3/doc/krb5-install.html#Install%20the%20Slave%20KDCs***********************************************************************
Exercise 4: build an LDAP server
This is by far the hardest part of the operation, due to the cryptic way OpenLDAP 2.4 now stores its configs within LDAP instead of in a config file. Seeman slapd-config
We are going to build an LDAP server with Base DN "dc=realm1,dc=ws,dc=nsrc,dc=org". Change this to match your own realm.
Install the slapd server and ensure everything else needed is present:
# apt-get install slapd ldap-utils libsasl2-modules-gssapi-mit
Set up the service principal with keytab readable by slapd:# mkdir /etc/ldap/krb5
# kadmin -p student/admin
addprinc -randkey ldap/pc1.ws.nsrc.org
ktadd -k /etc/ldap/krb5/krb5.keytab ldap/pc1.ws.nsrc.org
^D
# chown -R openldap:openldap /etc/ldap/krb5
# chmod 550 /etc/ldap/krb5
# chmod 440 /etc/ldap/krb5/krb5.keytab
# editor /etc/default/slapd
...
export KRB5_KTNAME=/etc/ldap/krb5/krb5.keytab
# service slapd restart
Install the schemas we need:# ldapadd -Y EXTERNAL -H ldapi:/// -f /etc/ldap/schema/cosine.ldif
# ldapadd -Y EXTERNAL -H ldapi:/// -f /etc/ldap/schema/nis.ldif
# ldapadd -Y EXTERNAL -H ldapi:/// -f /etc/ldap/schema/inetorgperson.ldif
# ldapadd -Y EXTERNAL -H ldapi:/// -f /etc/ldap/schema/misc.ldif
NOTE: For a real production LDAP server, read the files in /usr/share/doc/slapd
carefully, especially README.Debian.gz and README.DB_CONFIG.gzNow we need to run some scripts - either paste directly to a root shell, or create in temporary files and then run them.
BEWARE: in LDIF files, superfluous spaces at the end of lines may cause attributes to be rejected as invalid!
First is
create_database.sh
ldapadd -Y EXTERNAL -H ldapi:/// <
(Note: we have no olcRootDN or olcRootPW. This is a pure Kerberos config)
Next is init_database.sh
ldapadd -Y EXTERNAL -H ldapi:/// <
And finally config.sh
# Because these are 'replace' operations it's safe to modify this script
# and re-run it
ldapmodify -Y EXTERNAL -H ldapi:/// <
The simple ACL says that full access is permitted to */admin
principals in our realm, and otherwise read access is granted to any valid user (i.e. anyone with a kerberos ticket that we recognize). See OpenLDAP ITS#6757 for a note on the format of the auth DN.
We also require SASL, which disables anonymous and simple binds.
Note: minssf enforces encryption. GSSAPI always returns 56 for ssf, regardless of the strength of the actual crypto mechanism in use. See http://lists.andrew.cmu.edu/pipermail/cyrus-sasl/2006-September/000628.html
The following tweak disables unused sasl mechanisms, and means that -Y GSSAPI
can be omitted form the ldapsearch command line.
# editor /etc/ldap/sasl2/slapd.conf
mech_list: gssapi external
Test LDAP
Update /etc/ldap/ldap.conf
to point to your own LDAP server and base DN. Use ldapsearch to check that you can query it.
$ kinit testuser
$ ldapsearch [-Y GSSAPI] [-b "dc=realm1,dc=realm1,dc=ws,dc=nsrc,dc=org"]
This should dump back the (mostly empty) LDAP database, as long as you have a Kerberos ticket
Create user
We are going to create a user called "newuser"
Firstly in Kerberos:
$ kadmin -p student/admin
addprinc newuser
... choose a password
^D
And now in LDAP:
$ kinit student/admin
$ ldapadd <
Test NSS
Now update your LDAP nscd/nsswitch configuration (part 2 of exercise 2). You will need to edit /etc/ldap.conf
with your own PC as LDAP server and your own Base DN. Restart nscd after doing this.
To verify all is working:
$ id newuser
You should see a result with the uid and gid. Repeated operations should be quick as nscd will be caching them.
Troubleshooting:
- Make sure clock is synced
- Make sure you have a kerberos ticket for root. Re-run
/etc/cron.hourly/kerberos
if necessary.
- Restart nscd
- If you stop nscd, then 'id ' will make an LDAP query as the current user. Make sure that the current user has a kerberos ticket.
ldapscripts
There is a package called ldapscripts
with simple tools for adding and managing LDAP users. Unfortunately it doesn't know about GSSAPI authentication, so you need to patch it. See ldapscripts-sasl.diff
Then configure it:
# editor /etc/ldapscripts/ldapscripts.conf
SASLAUTH="GSSAPI"
...
SERVER="ldap://pc1.ws.nsrc.org"
...
SUFFIX="dc=realm1,dc=ws,dc=nsrc,dc=org"
Then adding a new user in LDAP is as simple as:
$ kinit student/admin
$ ldapadduser someuser users
$ kdestroy
Adding groups and users into groups:
$ ldapaddgroup noc
$ ldapaddusertogroup someuser noc
You may need to restart nscd before 'id' shows this information.
Configure backup LDAP
OpenLDAP replication is left as an exercise to the reader.
*******************************************************************
Preparation for the Kerberos lab
To get the "wow" factor for the initial exercises, showing how easy it is to add clients and servers to a Kerberised network, we need to have a Kerberos/LDAP environment already up and running.
Prepare the DNS
Build the NOC machine and ensure forward and reverse DNS is fully working, if not already done.
Create kdc1 as an alias to whatever machine will be running the primary KDC (probably also the NOC), and add the Kerberos service records.
# apt-get install bind9 bind9-utils
# editor /etc/bind/named.conf.local
zone "ws.nsrc.org" { type master; file "/etc/bind/ws.nsrc.org"; };
zone "10.in-addr.arpa" { type master; file "/etc/bind/10.in-addr.arpa"; };
# editor /etc/bind/ws.nsrc.org
$TTL 600
@ IN SOA noc.ws.nsrc.org. root.noc.ws.nsrc.org. (
1 ; Serial
604800 ; Refresh
600 ; Retry
2419200 ; Expire
600 ) ; Negative Cache TTL
@ IN NS noc.ws.nsrc.org.
_kerberos IN TXT "WS.NSRC.ORG"
_kerberos-master._udp IN SRV 0 0 88 kdc1
_kerberos-adm._tcp IN SRV 0 0 749 kdc1
_kpasswd._udp IN SRV 0 0 464 kdc1
; List all active Kerberos servers here
_kerberos._udp IN SRV 0 0 88 kdc1
ldap IN A 10.10.254.250
noc IN A 10.10.254.250
kdc1 IN A 10.10.254.250
$GENERATE 1-20 $ IN PTR pc$.ws.nsrc.org.
# editor /etc/bind/10.in-addr.arpa
$TTL 600
@ IN SOA noc.ws.nsrc.org. root.noc.ws.nsrc.org. (
1 ; Serial
604800 ; Refresh
600 ; Retry
2419200 ; Expire
600 ) ; Negative Cache TTL
@ IN NS noc.ws.nsrc.org.
$GENERATE 1-20 $.1.10 IN PTR pc$.ws.nsrc.org.
250.254.10 IN PTR noc.ws.nsrc.org.
# editor /etc/resolv.conf
search ws.nsrc.org
nameserver 127.0.0.1
Test it:
# dig pc1
# dig -x 10.10.1.1
# dig _kerberos._udp.ws.nsrc.org. srv
If the NOC machine also has an entry for itself in /etc/hosts, make sure its FQDN is the first one listed.
# editor /etc/hosts
...
10.10.254.250 noc.ws.nsrc.org noc
Build slave DNS if desired
You like resilience don't you? :-) And it sets a good example!
Build primary KDC
See also the MIT Kerberos docs (esp. install guide) at http://web.mit.edu/kerberos/krb5-1.8/#documentation
Install ntp:
# apt-get install ntp
Install kdc packages:
# apt-get install krb5-kdc krb5-admin-server
Answer the annoying debconf questions:
Default Kerberos 5 realm: [WS.NSRC.ORG]
Kerberos servers for your realm: [kdc1.ws.nsrc.org]
Administrative server: [kdc1.ws.nsrc.org]
Create the client library config:
# mv /etc/krb5.conf /etc/krb5.conf.example
# editor /etc/krb5.conf
[libdefaults]
default_realm = WS.NSRC.ORG
dns_lookup_realm = true
dns_lookup_kdc = true
[realms]
WS.NSRC.ORG = {
admin_server = kdc1.ws.nsrc.org
}
(note that kadmin doesn't yet support looking up the admin server via DNS; hence we have to list it in krb5.conf)
Create Kerberos database:
kdb5_util create -r WS.NSRC.ORG -s
# You will be asked to choose a database master password. We use "abcd"
Create the ACL file and grant admin rights to all */admin principals:
# editor /etc/krb5kdc/kadm5.acl
*/admin@WS.NSRC.ORG *
Now create some principals: a host principal for the host itself, putting the random key into its own keytab file; an instructor principal ("nsrc"); and an instructor KDC admin principal ("nsrc/admin")
# kadmin.local
addprinc -randkey host/noc.ws.nsrc.org
ktadd host/noc.ws.nsrc.org
addprinc nsrc
-- you'll be prompted to choose a password
addprinc nsrc/admin
-- you'll be prompted to choose a password
^D
Now start the daemons:
# /etc/init.d/krb5-kdc start
# /etc/init.d/krb5-admin-server start
At this point, you should be able to kinit nsrc
and get a ticket. If you get "Cannot resolve network address for KDC" then check the DNS.
Note: new principals can now be added using kadmin
instead of kadmin.local
, and you can use it from any remote machine where you can kinit. (So you never need to do a local login to the KDC)
Build slave KDC if desired
If you want to build a slave KDC, see the instructions at http://web.mit.edu/kerberos/krb5-1.8/krb5-1.8.3/doc/krb5-install.html#Install%20the%20Slave%20KDCs
Add another _kerberos._udp
srv entry in the DNS pointing to kdc2 so that clients know about both of them.
Enable Kerberized ssh
The first exercise requires students to make a single-signon ssh login.
Create a 'testuser' kerberos principal in the KDC, which the students will login as.
$ kadmin -p nsrc/admin
addprinc testuser
... choose a password which will be given to the students, e.g. nsrc2020
^D
On the machine(s) where the students will be allowed to login via kerberos, create a 'testuser' account (but don't set any local password) and enable Kerberos authentication for sshd. Note we're not doing LDAP yet.
# useradd -m -s /bin/bash testuser
# editor /etc/ssh/sshd_config
...
GSSAPIAuthentication yes
GSSAPIKeyExchange yes
...
# service ssh restart
Now we'll test it. You can either do this from the noc to itself, or you can use a separate throw-away client VM, or your own laptop. If using a separate machine you'll first have to make it a Kerberos client like this:
# apt-get install krb5-user
# mv /etc/krb5.conf /etc/krb5.conf.example
# editor /etc/krb5.conf
[libdefaults]
default_realm = WS.NSRC.ORG
dns_lookup_realm = true
dns_lookup_kdc = true
Check that /etc/ssh/ssh_config
has GSSAPIAuthentication yes
and if supported GSSAPIKeyExchange yes
(On Mac OSX: /etc/ssh_config
)
Now it should work:
$ kinit testuser
$ ssh testuser@noc
If it still doesn't work after this, Kerberos problems can be tricky to debug. First check both machines have accurately synced clocks, and that forward and reverse DNS is working. Then you can try starting another sshd in debug mode on another port:
# /usr/sbin/sshd -d -p99
Then on the client:
$ ssh -v -p99 nsrc@noc
Test with Kerberized Apache
If Apache isn't already installed on the noc, install it, and also curl for testing.
# apt-get install apache2 curl
Create a test directory and test page:
# mkdir /var/www/secure
# editor /var/www/secure/index.html
... whatever you like
-- check it works and is not yet protected
# curl http://noc.ws.nsrc.org/secure/
Now we're going to kerberize it.
# apt-get install libapache2-mod-auth-kerb
# editor /etc/apache2/conf.d/topsecret
AuthName "Hello Kerberos World"
AuthType Kerberos
# Allow fallback to Basic Auth?
KrbMethodK5Passwd Off
KrbAuthRealms WS.NSRC.ORG
Krb5Keytab /etc/apache2/krb5/krb5.keytab
# require user nsrc@WS.NSRC.ORG
require valid-user
# service apache2 restart
Check that the page is no longer visible (curl gives a 403 error)
We now have to create a service principal, and extract its key into the keytab specified in the config above, readable to the apache daemon.
# mkdir /etc/apache2/krb5
# kadmin -p nsrc/admin
addprinc -randkey HTTP/noc.ws.nsrc.org
ktadd -k /etc/apache2/krb5/krb5.keytab HTTP/noc.ws.nsrc.org
^D
# chown -R www-data:www-data /etc/apache2/krb5
# chmod 550 /etc/apache2/krb5
# chmod 440 /etc/apache2/krb5/krb5.keytab
Check that the page is visible while you have a ticket with a client which supports HTTP Negotiate (and also check it isn't after kdestroy
)
# curl --negotiate -u: http://noc.ws.nsrc.org/secure/
If you want to test from Firefox or Chrome, see the presentation.
NOTE: testing suggests that modauthnzldap only does anonymous binds or fixed simple binds (i.e. I couldn't get it to use SASL, even if the www-data user has a ticket). We can gloss over this for now.
Configure master LDAP server
This is by far the hardest part of the operation, due to the cryptic way OpenLDAP 2.4 now stores its configs within LDAP instead of in a config file. See man slapd-config
There is a good series of articles here: http://www.opinsys.fi/en/setting-up-openldap-on-ubuntu-10-04-alpha2 But there are many others which user openldap <2.4 and slapd.conf
Install the slapd server and Kerberos bits:
# apt-get install slapd ldap-utils libsasl2-modules-gssapi-mit
Set up the service principal with keytab readable by slapd:
# mkdir /etc/ldap/krb5
# kadmin -p nsrc/admin
addprinc -randkey ldap/noc.ws.nsrc.org
ktadd -k /etc/ldap/krb5/krb5.keytab ldap/noc.ws.nsrc.org
^D
# chown -R openldap:openldap /etc/ldap/krb5
# chmod 550 /etc/ldap/krb5
# chmod 440 /etc/ldap/krb5/krb5.keytab
# editor /etc/default/slapd
...
export KRB5_KTNAME=/etc/ldap/krb5/krb5.keytab
# service slapd restart
Install the schemas we need:
# ldapadd -Y EXTERNAL -H ldapi:/// -f /etc/ldap/schema/cosine.ldif
# ldapadd -Y EXTERNAL -H ldapi:/// -f /etc/ldap/schema/nis.ldif
# ldapadd -Y EXTERNAL -H ldapi:/// -f /etc/ldap/schema/inetorgperson.ldif
# ldapadd -Y EXTERNAL -H ldapi:/// -f /etc/ldap/schema/misc.ldif
NOTE: For a real production LDAP server, read the files in /usr/share/doc/slapd
carefully, especially README.Debian.gz and README.DB_CONFIG.gz
Now we need to run some scripts. First is create_database.sh
ldapadd -Y EXTERNAL -H ldapi:/// <
(Note: we have no olcRootDN or olcRootPW. This is a pure Kerberos config)
Next is init_database.sh
ldapadd -Y EXTERNAL -H ldapi:/// <
And finally config.sh
# Because these are 'replace' operations it's safe to modify this script
# and re-run it
ldapmodify -Y EXTERNAL -H ldapi:/// <
The following tweak disables unused sasl mechanisms, and means that -Y GSSAPI
can be omitted form the ldapsearch command line.
# editor /etc/ldap/sasl2/slapd.conf
mech_list: gssapi external
Test LDAP
(On the same machine, or on a separate client)
# apt-get install ldap-utils libsasl2-modules-gssapi-mit
# editor /etc/ldap/ldap.conf
BASE dc=ws,dc=nsrc,dc=org
URI ldap://ldap.ws.nsrc.org
$ kinit nsrc
$ ldapsearch [-Y GSSAPI] [-b "dc=ws,dc=nsrc,dc=org"]
This should dump back the (mostly empty) LDAP database, as long as you have a Kerberos ticket
Create user
We are going to create a central user called "ldapuser"
Firstly in Kerberos:
$ kadmin -p nsrc/admin
addprinc ldapuser
... password: foo123bar
^D
And now in LDAP:
$ kinit nsrc/admin
$ ldapadd <
Test NSS
Now configure LDAP nscd/nsswitch, as per part 2 of exercise 2.
To verify all is working:
$ id ldapuser
You should see a result with the uid and gid. Repeated operations should be quick as nscd will be caching them.
Troubleshooting:
- Make sure clock is synced
- Make sure you have a kerberos ticket for root. Re-run
/etc/cron.hourly/kerberos
if necessary.
- Restart nscd
- If you stop nscd, then 'id ' will make an LDAP query as the current user. Make sure that the current user has a kerberos ticket.
For easier management you can install the ldapscripts
package, but see the patches at the end of exercise 4.
Configure backup LDAP
OpenLDAP replication is left as an exercise to the reader. You can configure round-robin DNS for ldap.ws.nsrc.org and it should work.
***************************************************************************
Nagios Installation and Configuration
Notes:
------
* Commands preceded with "$" imply that you should execute the command as
a general user - not as root.
* Commands preceded with "#" imply that you should be working as root.
* Commands with more specific command lines (e.g. "RTR-GW>" or "mysql>")
imply that you are executing commands on remote equipment, or within
another program.
Exercises
---------
Exercises Part I
----------------
0. Log in to your PC or open a terminal window as the sysadm user.
1. You may need to install Nagios version 3. You would do this as root or as the sysadmin
user and use the "sudo" command. As sysadm:
$ sudo apt-get install nagios3
Unless you already have an MTA installed, nagios3 will install
postfix as a dependency. Select "Internet Site" option. (If you had wanted
to use a different MTA likely you'd install it before nagios3)
You will be prompted for nagiosadmin password. Give it the normal
workshop password.
To get the documentation in /usr/share/doc/nagios3-doc/html/ (which
can also be read via the nagios web interface), do:
$ sudo apt-get install nagios3-doc
Now lets install the Apaceh web server so that you'll be running some additional
services for us to monitor.
$ sudo apt-get install apache2
2. Look at the file which contains the password. It's hashed (encrypted)
$ cat /etc/nagios3/htpasswd.users
3. You should already have a working Nagios!
- Open a browser, and go to
http://pcN.ws.nsrc.org/nagios3/
Check with the instructor or your neighbor if you are in doubt.
- At the login prompt, login as:
user: nagiosadmin
pass:
Browse to the "Host Detail" page to see what's already configured.
4. Let's look at the configuration layout... But, first, let's become the root
user on your machine:
$ sudo bash
# cd /etc/nagios3
# ls -l
-rw-r--r-- 1 root root 1882 2008-12-18 13:42 apache2.conf
-rw-r--r-- 1 root root 10524 2008-12-18 13:44 cgi.cfg
-rw-r--r-- 1 root root 2429 2008-12-18 13:44 commands.cfg
drwxr-xr-x 2 root root 4096 2009-02-14 12:33 conf.d
-rw-r--r-- 1 root root 26 2009-02-14 12:36 htpasswd.users
-rw-r--r-- 1 root root 42539 2008-12-18 13:44 nagios.cfg
-rw-r----- 1 root nagios 1293 2008-12-18 13:42 resource.cfg
drwxr-xr-x 2 root root 4096 2009-02-14 12:32 stylesheets
# cd conf.d
# ls -l
-rw-r--r-- 1 root root 1695 2008-12-18 13:42 contacts_nagios2.cfg
-rw-r--r-- 1 root root 418 2008-12-18 13:42 extinfo_nagios2.cfg
-rw-r--r-- 1 root root 1152 2008-12-18 13:42 generic-host_nagios2.cfg
-rw-r--r-- 1 root root 1803 2008-12-18 13:42 generic-service_nagios2.cfg
-rw-r--r-- 1 root root 210 2009-02-14 12:33 host-gateway_nagios3.cfg
-rw-r--r-- 1 root root 976 2008-12-18 13:42 hostgroups_nagios2.cfg
-rw-r--r-- 1 root root 2167 2008-12-18 13:42 localhost_nagios2.cfg
-rw-r--r-- 1 root root 1005 2008-12-18 13:42 services_nagios2.cfg
-rw-r--r-- 1 root root 1609 2008-12-18 13:42 timeperiods_nagios2.cfg
Notice that the package installs files with "nagios2" in their name.
This is because they are the same files as were used for the Nagios
version 2 Debian package. However there was a change made to the
host-gateway configuration file, so this has a new name.
5. You have a config which is already monitoring your own system
(localhost_nagios2.cfg) and your upstream default gateway
(host-gateway_nagios3.cfg).
Have a look at the config file for the default gateway: it's very simple.
(Note: tab completion is useful here. Type cat host-g then hit tab; the
filename will be filled in for you)
# cat host-gateway_nagios3.cfg
# a host definition for the gateway of the default route
define host {
host_name gateway
alias Default Gateway
address 10.10.0.254
use generic-host
}
PART II
Configuring Equipment
-----------------------------------------------------------------------------
0. Order of configuration
Conceptually we will build our configuration files from the "nearest" device
then the further away ones.
By going in this order you will have defined the devices that act as parents
for other devices.
The classroom GW router is already defined (10.10.0.254).
1. First we need to tell Nagios to monitor the gateway for the router instances,
which is 10.10.254.254 or gw-254.ws.nsrc.org.
# cd /etc/nagios3/conf.d/
Create the routers gateway like this:
# editor routers.cfg
define host {
use generic-host
host_name gw-254
alias Routers Gateway
address 10.10.254.254
parents gateway
}
Exit and save this file.
*NOTE* - "gateway" is the same machine as gw.ws.nsrc.org. Nagios has simply given
this machine the name "gateway". While it's nice to have the host_name mactch the
name in DNS it is not strictly necessary. This will be our only exception.
2. the final parent we have in our network is our backbone switch. Create
a file called switches.cfg and add an entry for this item:
# editor switches.cfg
define host {
use generic-host
host_name sw
alias Backbone Switch
address 10.10.0.253
parents gateway
}
At this point Nagios is configured to monitor whether our core hosts (the parents)
are up on our classroom network. Your next steps are to add in the individual hosts
such as the classroom virtual PC images (pc1 to pc26), the Wireless Access Points
(ap1 and ap2), the virtual router images (r1 through r26) and the classroom noc
host.
Be sure you add in a proper "parents" entry for each host.
To understand the parent relationship in our network review the logical
network diagram located here:
http://nocws.nsrc.org/wiki/wiki/NetworkDiagram
Note the Nagios parent bullet points:
Nagios Parent Relationships
Parents are "gw", "sw" and "gw-254". The parent relations are:
* gw is the parent of sw and gw-254
* gw-254 is the parent of r1 through r26
* sw is the parent of s0, s1, s2, ap1, ap2, noc and pc1 through pc26
STEPS 2a - 2c SHOULD BE REPEATED WHENEVER YOU UPDATE THE CONFIGURATION!
2a. Verify that your configuration files are OK:
# nagios3 -v /etc/nagios3/nagios.cfg
... You should get :
Warning: Host 'bb-sw' has no services associated with it!
Warning: Host 'bb-gw' has no services associated with it!
...
Total Warnings: 2
Total Errors: 0
Things look okay - No serious problems were detected during the check.
Nagios is saying that it's unusual to monitor a device just for its
existence on the network, without also monitoring some service.
2b. Reload/Restart Nagios
# /etc/init.d/nagios3 restart
Not always 100% reliable to use the "restart" option due to a bug in the Nagios init script.
To be sure you may want to get used to doing:
# /etc/init.d/nagios3 stop
# /etc/init.d/nagios3 start
2c. Go to the web interface (http://pcN.ws.nsrc.org/nagios3) and check that the hosts
you just added are now visible in the interface. Click on the "Host Detail" item
on the left of the Nagios screen to see this. You may see it in "PENDING"
status until the check is carried out.
HINT: You will be doing this a lot. If you do it all on one line, like this,
then you can hit cursor-up and rerun all in one go:
nagios3 -v /etc/nagios3/nagios.cfg && /etc/init.d/nagios3 restart
The '&&' ensures that the restart only happens if the config is valid.
3. Create entries for ther routers and PCs in the classroom
Now that we have our routers and switches defined it is quite easy to create
entries for all our PCs. Think about the parent relationships:
Remember, if you do not understand the parent relationship refer back to the
classroom network diagram here:
http://noc.ws.nsrc.org/wiki/wiki/NetworkDiagram
Below are three sample entries. One for the NOC, one for pc1 and one for
pc26. You should be able to use this example to create entries for all
classroom PCs plus the NOC.
We could put these entries in to separate files, but as our network is small
we'll use a single file called pcs.cfg.
NOTE! You do not add in an entry for your own PC or router. This has already
been defined in the file /etc/nagios3/conf.d/localhost_nagios2.cfg. This
definition is what defines the Nagios network viewpoint. So, when you come to
the spot where you might add an entry for your PC you should skip this and go
on to the next PC in the list.
# editor pcs.cfg
# Our classroom NOC
define host {
use generic-host
host_name noc
alias Workshop NOC machine
address 10.10.0.250
parents sw
}
# PCs
define host {
use generic-host
host_name pc1
alias pc1
address 10.10.0.1
parents sw
}
define host {
use generic-host
host_name pc26
alias pc26
address 10.10.0.26
parents sw
}
Take the three entries above and now expand this to create the remaining
entries for all active PCs. That is, fill in for PCs 2 through 25 (rememember to
skip your PC).
Exit and save the file pcs.cfg
As before, repeat steps 2a-2c to verify your configuration, correct any
errors, and activate it.
4. Now configure Nagios to start monitoring the multiple router instances
we have available. These are from 10.10.254.1 through 10.10.254.26.
Let's create the first router in our file called routers.cfg. Add this line
to the bottom of the file:
define host {
use generic-host
host_name r1
alias router 1
address 10.10.254.1
parents gw-254
}
Now create the remaining routers 2-26. Or, just create a few if you don't
want to spend too long on this particular part of the exercise. But, remember
which router instances you have defined!
Second router:
define host {
use generic-host
host_name r2
alias router 2
address 10.10.254.2
parents gw-254
}
Repeat this until router number 26:
define host {
use generic-host
host_name r26
alias router 26
address 10.10.254.26
parents gw-254
}
Save the file.
5. Look at your Nagios instance on the web. Note that "Status Map" gives
you a graphical view of the parent-child relationships you have defined.
PART III
Configure Service check for the classroom NOC
-----------------------------------------------------------------------------
0. Configuring
Now that we have our hardware configured we can start telling Nagios what services to monitor
on the configured hardware, how to group the hardware in interesting ways, how to group
services, etc.
1. Associate a service check for our classroom NOC
# joe hostgroups_nagios2.cfg
- Find the hostgroup named "ssh-servers". In the members section of the defintion
change the line:
members localhost
to
members localhost,noc
Exit and save the file.
Verify that your changes are OK:
# nagios3 -v /etc/nagios3/nagios.cfg
Restart Nagios to see the new service assocation with your host:
# /etc/init.d/nagios3 restart
Click on the "Service Detail" link in the Nagios web interface to see your new entry.
PART IV
Defining Services for all PCs
-----------------------------------------------------------------------------
0. For services, the default normal_check_interval is 5 (minutes) in
generic-service_nagios2.cfg. You may wish to change this to 1 to speed up
how quickly service issues are detected, at least in the workshop.
1. Determine what services to define for what devices
- This is core to how you use Nagios and network monitoring tools in
general. So far we are simply using ping to verify that physical hosts
are up on our network and we have started monitoring a single service on
a single host (your PC). The next step is to decide what services you wish
to monitor for each host in the classroom.
- In this particular class we have:
routers: running ssh and snmp
switches: running telnet and possibly ssh as well as snmp
pcs: All PCs are running ssh and http and should be running snmp
The NOC is currently running an snmp daemon
So, let's configure Nagios to check for these services for these
devices.
2.) Verify that SSH is running on the routers and workshop PCs images
- In the file services_nagios2.cfg there is already an entry for the SSH
service check, so you do not need to create this step. Instead, you
simply need to re-define the "ssh-servers" entry in the file
/etc/nagios3/conf.d/hostgroups_nagios2.cfg. The initial entry in the file
looked like:
# A list of your ssh-accessible servers
define hostgroup {
hostgroup_name ssh-servers
alias SSH servers
members localhost,noc
}
What do you think you should change? Correct, the "members" line. You should
add in entries for all the classroom pcs, routers and the switches that run ssh.
With this information and the network diagram you should be able complete this entry.
The entry will look something like this:
define hostgroup {
hostgroup_name ssh-servers
alias SSH servers
members localhost,pc1,pc2,pc3,pc4....,noc,ap1,ap2,r1,r2,r3....
}
Note: leave in "localhost" - This is your PC and represents Nagios' network point of
view. So, for instance, if you are on "pc3" you would not include "pc3" in the list
of all the classroom pcs as it is represented by the "localhost" entry.
The "members" entry will be a long line and will likely wrap on the screen.
Remember to include all your PCs and all your routers that you have defined. Do no
include any entries if they are not already defined in pcs.cfg, switches.cfg or
routers.cfg.
- Once you are done, run the pre-flight check:
# nagios3 -v /etc/nagios3/nagios.cfg
If everything looks good, then restart Nagios
# /etc/init.d/nagios3 stop
# /etc/init.d/nagios3 start
and view your changes in the Nagios web interface.
3.) Check that http is running on all the classroom PCs.
- This is almost identical to the previous exercise. Just make the change to the
HTTP service adding in each PC (no routers or switches). Remember, you don't need
to add your machine as it is already defined as "localhost".
4.) OPTIONAL EXTRA: as opposed to just checking that a web server is
running on the classroom PCs, you could also check that the nagios3
service is available, by requesting the /nagios3/ path. This means
passing extra options to the check_http plugin.
For a description of the available options, type this:
# /usr/lib/nagios/plugins/check_http
# /usr/lib/nagios/plugins/check_http --help
and of course you can browse the online nagios documentation or google
for information on check_http. You can even run the plugin by hand to
perform a one-shot service check:
# /usr/lib/nagios/plugins/check_http -H localhost -u /nagios3/
So the goal is to configure nagios to call check_http in this way.
define command{
command_name check_http_arg
command_line /usr/lib/nagios/plugins/check_http -H '$HOSTADDRESS$' $ARG1$
}
define service {
hostgroup_name nagios-servers
service_description NAGIOS
check_command check_http_arg!-u /nagios3/
use generic-service
}
and of course you'll need to create a hostgroup called nagios-servers to
link to this service check.
Once you have done this, check that Nagios warns you about failing
authentication (because it's trying to fetch the page without providing
the username/password). There's an extra parameter you can pass to
check_http_arg to provide that info, see if you can find it.
WARNING: in the tradition of "Debian Knows Best", their definition of the
check_http command in /etc/nagios-plugins/config/http.cfg
is *not* the same as that recommended in the nagios3 documentation.
It is missing $ARG1$, so any parameters to pass to check_http are
ignored. So you might think you are monitoring /nagios3/ but actually
you are monitoring root!
This is why we had to make a new command definition "check_http_arg".
You could make a more specific one like "check_nagios", or you could
modify the Ubuntu check_http definition to fit the standard usage.
PART V
Create More Host Groups
-----------------------------------------------------------------------------
0. In the web view, look at the pages "Hostgroup Overview", "Hostgroup
Summary", "Hostgroup Grid". This gives a convenient way to group together
hosts which are related (e.g. in the same site, serving the same purpose).
1. Update /etc/nagios3/conf.d/hostgroups_nagios2.cfg
- For the following exercises it will be very useful if we have created
or update the following hostgroups:
debian-servers
routers
switches
If you edit the file /etc/nagios3/conf.d/hostgroups_nagios2.cfg you
will see an entry for debian-servers that just contains localhost.
Update this entry to include all the classroom PCs, including the
noc (this assumes that you created a "noc" entry in your pcs.cfg
file). Remember to skip your PC entry as it is represented by the
localhost entry.
# editor /etc/nagios3/conf.d/hostgroups_nagios2.cfg
Update the entry that says:
# A list of your Debian GNU/Linux servers
define hostgroup {
hostgroup_name debian-servers
alias Debian GNU/Linux Servers
members localhost
}
So that the "members" parameter contains something like this. Use your
classroom network diagram to confirm the exact number of machines and names
in your workshop.
members localhost,pc1,pc2,pc3,pc4,pc5,pc6,pc7,pc8,pc9
pc10,pc11,pc12,pc13,pc14,pc15,pc16,pc17,pc18,
pc19,pc20,pc21,pc22,pc23,pc24,pc25,pc26
Be sure that the line wraps and is not on two separate lines. Otherwise
you will get an error when you go to restart Nagios. Remember that
your own PC is "localhost".
- Once you have done this, add in two more host groups, one for routers and
one for switches. Call these entries "routers" and "switches".
- When you are done be sure to verify your work and restart Nagios.
2. Go back to the web interface and look at your new hostgroups
PART VI
Extended Host Information ("making your graphs pretty")
-----------------------------------------------------------------------------
1. Update extinfo_nagios2.cfg
- If you would like to use appropriate icons for your defined hosts in
Nagios this is where you do this. We have the three types of devices:
Cisco routers
Cisco switches
Ubuntu servers
There is a fairly large repository of icon images available for you to
use located here:
/usr/share/nagios/htdocs/images/logos/
these were installed by default as dependent packages of the nagios3
package in Ubuntu. In some cases you can find model-specific icons for
your hardware, but to make things simpler we will use the following
icons for our hardware:
/usr/share/nagios/htodcs/images/logos/base/debian.*
/usr/share/nagios/htdocs/images/logos/cook/router.*
/usr/share/nagios/htdocs/images/logos/cook/switch.*
- The next step is to edit the file /etc/nagios3/conf.d/extinfo_nagios2.cfg
and tell nagios what image you would like to use to represent your devices.
# editor /etc/nagios3/conf.d/extinfo_nagios2.cfg
Here is what an entry for your routers looks like (there is already an entry
for debian-servers that will work as is). Note that the router model (3600)
is not all that important. The image used represents a router in general.
define hostextinfo {
hostgroup_name routers
icon_image cook/router.png
icon_image_alt Cisco Routers (3600)
vrml_image router.png
statusmap_image cook/router.gd2
}
Now add an entry for your switches. Once you are done check your
work and restart Nagios. Take a look at the Status Map in the web interface.
It should be much nicer, with real icons instead of question marks.
PART VII
Create Service Groups
-----------------------------------------------------------------------------
1. Create service groups for ssh and http for each set of pcs.
- The idea here is to create three service groups. Each service group will
be for a quarter of the classroom. We want to see these PCs grouped together
and include status of their ssh and http services. To do this edit
and create the file:
# editor /etc/nagios3/conf.d/servicegroups.cfg
Here is a sample of the service group for group 1:
define servicegroup {
servicegroup_name group1-servers
alias group 1 servers
members pc1,SSH,pc1,HTTP,pc2,SSH,pc2,HTTP,pc3,SSH,pc3,HTTP,pc4,SSH,pc4
}
- Note that the members line should wrap and not be on two lines.
- Note that "SSH" and "HTTP" need to be uppercase as this is how the service_description is
written in the file /etc/nagios3/conf.d/services_nagios2.cfg
- You should create an entry for other groups of servers too
- Save your changes, verify your work and restart Nagios. Now if you click on
the Servicegroup menu items in the Nagios web interface you should see
this information grouped together.
PART VIII
Configure Guest Access to the Nagios Web Interface
-----------------------------------------------------------------------------
1. Edit /etc/nagios3/cgi.cfg to give read-only guest user access to the Nagios
web interface.
- By default Nagios is configured to give full r/w access via the Nagios
web interface to the user nagiosadmin. You can change the name of this
user, add other users, change how you authenticate users, what users
have access to what resources and more via the cgi.cfg file.
- First, lets create a "guest" user and password in the htpasswd.users
file.
# htpasswd /etc/nagios3/htpasswd.users guest
You can use any password you want (or none). A password of "guest" is
not a bad choice.
- Next, edit the file /etc/nagios3/cgi.cfg and look for what type of access
has been given to the nagiosadmin user. By default you will see the following
directives (note, there are comments between each directive):
authorized_for_system_information=nagiosadmin
authorized_for_configuration_information=nagiosadmin
authorized_for_system_commands=nagiosadmin
authorized_for_all_services=nagiosadmin
authorized_for_all_hosts=nagiosadmin
authorized_for_all_service_commands=nagiosadmin
authorized_for_all_host_commands=nagiosadmin
Now let's tell Nagios to allow the "guest" user some access to
information via the web interface. You can choose whatever you would
like, but what is pretty typical is this:
authorized_for_system_information=nagiosadmin,guest
authorized_for_configuration_information=nagiosadmin,guest
authorized_for_system_commands=nagiosadmin
authorized_for_all_services=nagiosadmin,guest
authorized_for_all_hosts=nagiosadmin,guest
authorized_for_all_service_commands=nagiosadmin
authorized_for_all_host_commands=nagiosadmin
- Once you make the changes, save the file cgi.cfg, verify your
work and restart Nagios.
- To see if you can log in as the "guest" user you may need to clear
the cookies in your web browser. You will not notice any difference
in the web interface. The difference is that a number of items that
are available via the web interface (forcing a service/host check,
scheduling checks, comments, etc.) will not work for the guest
user.
OPTIONAL
--------
* Check that SNMP is running on the classroom NOC
- First you will need to add in the appropriate service check for SNMP in the file
/etc/nagios3/conf.d/services_nagios2.cfg. This is where Nagios is impressive. There
are hundreds, if not thousands, of service checks available via the various Nagios
sites on the web. You can see what plugins are installed by Ubuntu in the nagios3
package that we've installed by looking in the following directory:
# ls /usr/lib/nagios/plugins
As you'll see there is already a check_snmp plugin available to us. If you are
interested in the options the plugin takes you can execute the plugin from the
command line by typing:
# /usr/lib/nagios/plugins/check_snmp
# /usr/lib/nagios/plugins/check_snmp --help
to see what options are available, etc. You can use the check_snmp plugin and
Nagios to create very complex or specific system checks.
- Now to see all the various service/host checks that have been created using the
check_snmp plugin you can look in /etc/nagios-plugins/config/snmp.cfg. You will
see that there are a lot of preconfigured checks using snmp, including:
snmp_load
snmp_cpustats
snmp_procname
snmp_disk
snmp_mem
snmp_swap
snmp_procs
snmp_users
snmp_mem2
snmp_swap2
snmp_mem3
snmp_swap3
snmp_disk2
snmp_tcpopen
snmp_tcpstats
snmp_bgpstate
check_netapp_uptime
check_netapp_cupuload
check_netapp_numdisks
check_compaq_thermalCondition
And, even better, you can create additional service checks quite easily.
For the case of verifying that snmpd (the SNMP service on Linux) is running we
need to ask SNMP a question. If we don't get an answer, then Nagios can assume
that the SNMP service is down on that host. When you use service checks such as
check_http, check_ssh and check_telnet this is what they are doing as well.
- In our case, let's create a new service check and call it "check_system". This
service check will connect with the specified host, use the private community
string we have defined in class and ask a question of snmp on that ask - in this
case we'll ask about the System Description, or the OID "sysDescr.0" -
- To do this start by editing the file /etc/nagios-plugins/config/snmp.cfg:
# joe /etc/nagios-plugins/config/snmp.cfg
At the top (or the bottom, your choice) add the following entry to the file:
# 'check_system' command definition
define command{
command_name check_system
command_line /usr/lib/nagios/plugins/check_snmp -H '$HOSTADDRESS$' -C
'$ARG1$' -o sysDescr.0
}
You may wish to copy and paste this vs. trying to type this out.
Note that "command_line" is a single line. If you copy and paste in joe the line
may not wrap properly and you may have to manually add the part:
'$ARG1$' -o sysDescr.0
to the end of the line.
- Now you need to edit the file /etc/nagios3/conf.d/services_nagios2.cfg and add
in this service check. We'll run this check against all our servers in the
classroom, or the hostgroup "debian-servers"
- Edit the file /etc/nagios3/conf.d/services_nagios2.cfg
# joe /etc/nagios3/conf.d/services_nagios2.cfg
At the bottom of the file add the following definition:
# check that snmp is up on all servers
define service {
hostgroup_name snmp-servers
service_description SNMP
check_command check_system!xxxxxx
use generic-service
notification_interval 0 ; set > 0 if you want to be renotified
}
The "xxxxxx" is the community string previously (or to be) defined in class.
Note that we have included our private community string here vs. hard-coding
it in the snmp.cfg file earlier. You must change the "xxxxx" to be the snmp
community string given in class or this check will not work.
- Now we must create the "snmp-servers" group in our hostgroups_nagios2.cfg file.
Edit the file /etc/nagios3/conf.d/hostgroups_nagios2.cfg and go to the end of the
file. Add in the following hostgroup definition:
# A list of snmp-enabled devices on which we wish to run the snmp service check
define hostgroup {
hostgroup_name snmp-servers
alias snmp servers
members noc
}
- Note that for "members" you could, also, add in the switches and routers for
group 1 and 2. But, the particular item (MIB) we are checking for "sysDescr.0"
may not be available on the switches and/or routers, so the check would then fail.
- Now verify that your changes are correct and restart Nagios.
- If you click on the Service Detail menu choice in web interface you should see
the SNMP check appear for the noc host.
- After we do the SNMP presentation and exercises in class, then you could come
back to this exercise and add in all the classroom PCs to the members list in the
hostgroups_nagios2.cfg file, snmp-servers hostgroup definition. Remember to list
your PC as "localhost".