UNIX tricks and treats

Aller au contenu | Aller au menu | Aller à la recherche

mercredi 10 septembre 2008

Purging a sendmail mailqueue on AIX

Tested on: IBM AIX 5.2

Sendmail processes may run wild, due to huge process loads, or even badly configured applications sending automatized mails.

When sendmail processes are overloaded, they may clog up the mailqueue and spawn multiple sendmail processes to treat the mailqueue, ultimately consuming most of your server's swap area, degrading performance, or even prevent other applications from running.

Here are the steps needed to stop rogue sendmail processes, and cleanly purge the sendmail mailqueue on IBM AIX 5.2. The process is similar on other UNIXes, except for the sendmail stop and start commands, which vary, depending of your OS. On Solaris, for example, you would use your own stop and start scripts in /etc/rcX.d/ or in /etc/init.d/.

First, find and kill the multiple sendmail processes if they have gone havoc.

# ps -ef | grep sendmail
 
# kill -9 SENDMAIL_PIDS

Then, stop sendmail cleanly (the commands depend of your OS. This one works only on IBM AIX).

# stopsrc -s sendmail  

You may check the number of messages that are in the queue, which will give you an idea of the time it will take to process the queue:

# sendmail -bp 

Check that there are no longer any sendmail processes running:

# ps -ef | grep sendmail
 
# kill -9 SENDMAIL_PIDS

Rename the current mailqueue to another directory:

# mv /var/spool/mqueue /var/spool/omqueue 

Restart sendmail

# startsrc -s sendmail
0513-059 The sendmail Subsystem has been started. Subsystem PID is 62118
 

Now process the old queue (may take time, depending upon the number of messages to process):

# /usr/sbin/sendmail -oQ/var/spool/omqueue -q -v

Running /var/spool/omqueue/m7HKkOM60666 (sequence XXXX of XXXXX)
Running /var/spool/omqueue/m7HKkOM60666 (sequence XXXX+1 of XXXXX)...
etc... 

Now, you may safely delete all messages in the old queue:

# rm -rf /var/spool/omqueue

Create a new mailqueue directory.

# mkdir /var/spool/mqueue

Stop and start sendmail:

# stopsrc -s sendmail

# startsrc -s sendmail

You're done!

Happy computing.

Drop me a comment if this post has been useful to you, or if you see any reason for add-on or modification.

Nixman

mercredi 28 mai 2008

Using Pen load balancer as a port-forwarding proxy


Suppose you're a Paris-based firm that has several databases spread across different locations.

For example:
a) several Oracle databases listening on port 1521 on your own network, on addresses 10.75.75.1 - 10,
b) one Oracle database listening on port 1521 in Turku, Finland linked by a VPN  tunnel, on address 172.16.2.2,
c) one oracle database listening on port 1521 in Toulouse linked by a leased line, on address 172.31.31.31,
d) one Oracle database listening on port 21521 on a public internet WAN port in Tanger, with just source-address filtering as security, on address 212.66.66.66,
e) plus about a dozen other databases in different parts of the world at your clients' sites.

You have a partner providing an extra service to your clients, and he has to connect in real time on all of your databases. He doesn't want to spend money on a network connection to each and everyone of your clients. He proposes to pay a leased line to your Paris site, and you will do the dispatching.
Of course, you don't want him to know too much about your network, so you will restrict his access to only one address, which will be a firewall at your end of the line between your two sites .

Let's suppose the outside interface address of your firewall is 192.168.15.100, and the inside address (the one on your network) 10.75.75.254.

Providing  connectivity to the (a) databases on your local  network is pretty easy: you just have to give a port-forwarding rule and an access list to your router.

For example:
port 2001 on the outside interface ---> port 1521 on server1 at  10.75.75.1,
port 2002 on the outside interface ---> port 1521 on server2 at 10.75.75.2,
port 2003 on the outside interface ---> port 1521 on server3 at 10.75.75.3,
and so on...
Then, you just tell your partner to configure his tnsnames.ora to point at address 192.168.15.100 and port 2001 for server1, address 192.168.15.100 and port 2002 for server2, and so on...

However, forwarding the ports to the external (b), (c) and (d) databases is another affair.

Luckily, Pen is there for you. It was designed as load-balancing piece of software for server farms, but its features allow it to be used as a port-forwarding proxy, which is what we need in this case. It is available prepackaged for rpm- as well as deb-based Linux  distros, or as GPL'ed source code. You may learn more on its numerous features on its website: http://siag.nu/pen/

All you need is is a standard PC on your network, with a Linux Distro, let's say Debian, installed on it , as well as one (yes, only one) NIC.

Do an apt-get install pen (or an rpm-Uvh pen on an rpm-based distro).

Let's suppose you've given address 10.75.75.75 to this computer.
It has to know the routes to reach the Turku, Toulouse, and Tanger based servers, and of course the route to reach your partner who wants to connect to them. It has also to be allowed to reach them on the ports on which they are listening (i.e 1521, 1521 and 21521 respectively).

All you need now is to write a little snippet of shell code in a file that you would call for example port-fwrd.sh:

###############
#!/bin/bash

# This is for the Turku-based database
pen 10.75.75.75:2011 172.16.2.2:1521

# This is for the Toulouse-based database
pen 10.75.75.75:2012 172.31.31.31:1521

# This is for the Tanger-based database
pen 10.75.75.75:2013 212.66.66.66:21521

exit
###############

Make port-fwrd.sh executable by a chmod, and launch it: ./port-fwrd.sh
Have it start in your init scripts at rc3 level, so that it will get executed upon reboot of your machine.

All you have to do now on your firewall is to forward:
port 2011 on the outside interface ---> port 2011 on 10.75.75.75
port 2012 on the outside interface ---> port 2012 on 10.75.75.75
port 2013 on the outside interface ---> port 2013 on 10.75.75.75

and tell your partner to configure his tnsnames.ora to reach:
Turku on address 192.168.15.100 and port 2011,
Toulouse on address 192.168.15.100 and port 2012,
Tanger on address 192.168.15.100 and port 2013.

Beautiful and simple, ain't it?

Note: of course, if the server on one of the locations doesn't support shared sockets (as it is the case with for example a Windows 2000 Server failsafe cluster), you won't be able to use portforwarding, since the answering port on the target server will a dynamic one, and thus unpredictable.

Happy computing.

Drop me a comment if this post has been useful to you, or if you see any reason for add-on or modification.

Nixman