Troubleshooting using strace

Hi,

Today I faced some issues on bind9, as I was unable to start the bind service using service command. I just went through the logs /var/log/syslog. But didn't get any information about this issue. Finally I used strace which was very usefull to troubleshoot.


strace -f /usr/sbin/named -u bind

It will trace the whole process . Then I was able to find permission denied error for /usr/lib/ssl/openssl.cnf. I fixed the permission issue and then I was able to start the bind.

Cheers
Syamkumar.M

How to enable Java Plugin to firefox on centos

Hi,

Below are the steps to install and enable java plugin for firefox. First we need to install java on the machine and then do the following steps to enable the plugin on the firefox.



root@test:~# ln -sf /opt/java/jre1.7.0_05/lib/i386/libnpjp2.so /usr/lib/mozilla/plugins/

root@test:~# export JAVA_HOME="/opt/java/jre1.7.0_05"

Testing the plugin.

root@test:~# firefox http://java.com/en/download/testjava.jsp

Regards
Syamkumar.M

How to Detect and secure Linux Cpanel Server From Spamming

Hi,

We faced some severe spamming issue on a Cpanel server, upon  checking the email queue, I can see that there is more than 1 lakh of emails. On checking the headers I can see that almost all of them are spam. Below are the steps that I have  taken for fixing the spamming issues on the server.

Exim command to detect the email queue is

exim -bpc

exim -bp|exiqsumm 

command will give you summary of the queue. Once you get the Message IDs from the above command you can view the Header of an email using the command

exim -Mvh Message ID


This will list the headers including the php script which is sending emails, if any.

X-php-script

You can check the body of the message as well in exim to make sure that it was a body of spam email

exim -Mvb Message-ID


Mostly you will get the php script from the exim -Mvh command. But some times you won't get the php scripts from the headers. The hackers might made some custom changes so that it won't appear in the headers.

Then we can use the below command which will detect the directory under which exim is sending emails.

grep cwd /var/log/exim_mainlog | grep -v /var/spool | awk -F"cwd=" '{print $2}' | awk '{print $1}' | sort | uniq -c | sort -n

Also you will get the common subject of spam emails by searching for  duplicate subject lines from the exim logs.

awk -F"T=\"" '/<=/ {print $2}' /var/log/exim_mainlog | cut -d\" -f1 | sort | uniq -c | sort -n

The above commands will give you the directory in which exim is currently executing the emails.The above command will also list the count of emails. So you will know from which directory the emails are sending heavily. Just cd into the directory and check the php files inside or scan with available antivirus(clamav inside that directory)

clamscan -i -r Directory-Name


This will list the vulnerable php files. You just need to delete those files first.

After you find the files which caused the spamming you might need to run a full system scanning on the server using clamav antivirus to find out any other suspicious files are residing on the server.


clamscan -i -r /


Below are the steps to make the system more secure against spamming

1)First install a antivirus like clamav on the server and then scan whole the server and automate it to run daily or weekly using crontab.

2)Install CSF firewall on the server and enable lfd on it

3)Install chrootkit or RK hunter on the server and automate the scanning of those using cron jobs

4)Disable  shell access for users  on the Cpanel servers unless it is required.

5)Disable any unanonymous FTP uploads on the Cpanel server.

6)Disable variables like open_basedir and php exec_shell and register_globals of php on the server

Regards
Syamkumar.M

How to change Outgoing IP of exim in a Cpanel server

Hi All,

Some times the IP of a cpanel server may got black listed due to spamming and hence we shouldn't be able to send emails to gmail or yahoo etc. This would affect whole customers of a server as no email communication is not possible.  A work around or temperory solution for this issue was to change the outgoing email IP of the cpanel server. Below is the steps to change the ip of a outgoing exim email server in Cpanel.

1)First you need to stop the email service exim through command line.

root@server [~]# /etc/init.d/exim stop

2)Now Login to your WHM control panel and do the changes on the section Exim configuration Manager on WHM.

Reference /etc/mailips for outgoing SMTP connections [?]

Send outgoing mail from the IP address that matches the domain name in /etc/mailips (more information)

Make the above settings to on.

3)Now login to the server as root and then open the configuration file /etc/mailips

add the new outgoing ip adress for exim as 

*:X.X.X.X

Where x.x.x.x is the ip adress you need to change. Save the configuration file and exit.

4)Restart the exim service

/etc/init.d/exim restart

Now the email will be sending from new fresh ip which is not black listed.

Regards

Syamkumar.M

psql: FATAL: parameter "listen_addresses" cannot be changed without restarting the server

Hi,

We faced an issue on one of the machine, where the postgresql commands are not runnning as root. Always getting the below error while executing any psql command

psql: FATAL:  parameter "listen_addresses" cannot be changed without restarting the server.

Initially I thought it was some thing to do with the settings on the pg_hba.conf file and postgresql.conf file where the variable listen_address is specified .The variable listen_adresses has been changed on postgresql.conf from

listen_addresses = '*'

to

listen_addresses = '10.11.16.112'

Then restarted the postgresql service. But still got the same error. One of the strange thing is that I am able to connect to postgresql command line from another machine to the machine having issue, but not from with in the machine as root.

Finally On checking, I found that there was a variable named PGOPTIONS was the issue.  Below is the steps I had done to resolve it.

Find the value of variable PGOPTIONS on the command line using the command

echo $PGOPTIONS
-i

SO I found out that the variable was defined in the server some where . I found out the variable on the environment file /etc/environment.

I changed the value from -i to Null value on /etc/environment

vi /etc/environment
PGOPTIONS=""

 Now the command echo $PGOPTIONS showed the null value

And now I am able to execute any psql command operations like pg_dump.

Regards
Syamkumar.M

/etc/rc.local not running on the boot

Hi,

   /etc/rc.local is the file which runs once the system boots up. If we need to run few commands immediately once the system starts up, we can do it by adding those commands in /etc/rc.local.

Last week I faced an issue in which the commands inside the rc.local was not working, once the system boot.  I tried starting and stopping the systems so many times and tried to change the settings of /etc/rc.local, but still it was not working.  Then I read in one of the Centos Forums that it was some issue with the first boot.

In my machine firstboot got hanged and init was not able to execute the content inside /etc/rc.local. Below is the steps to resolve it

check to see if firstboot is still running and kill it:


ps ax |grep firstboot
killall firstboot

This will kill the first boot .

Then switch the firstboot permenantly using chkconfig

chkconfig firstboot off


The first boot issue was caused by some of the software installations which may confuse the firstboot.

Regards
Syamkumar.M
 

How to change ulimit values permanently for a user in Centos

Hi,

ulimit is the parameter which defines the limits a process can use on a linux system.  It will provide control over the resources a user or a process in a shell can use.  You can list the current setting of ulimit values by login as your user and type the following commands

# ulimit -Hn
# ulimit -Sn

Default values should be 4096 and 1024 respectively.
 

So it will also determine the number of open files a user can open or edit. For increasing the ulimit, you need to change those paramentes in a configuration file called

/etc/security/limits.conf

If I need to change the ulimit value for my user on  a linux system. Then you should login to machine as root.

Open the above file in editor and then add the following content to it.

syam        soft nofile 9000
syam        hard nofile 65000


Now you should see the changes to the ulimit value for a user when you switch user from the root. But still the limit won't get reflected when user login as ssh with the password. We need to add the following steps to get the changes reflected for the user upon ssh login as user .

1. In /etc/pam.d/sshd added the line

session required pam_limits.so

2. In /etc/pam.d/login added the line

session required pam_limits.so

3. In /etc/ssh/sshd_config added

UsePAM yes

Now restart the sshd service

4. /etc/init.d/sshd restart

Now the ulimit values will be changed permenantly and you can see the values once you ssh into the machine as user.


ulimit -Hn
65000

ulimit -Sn

9000

Regards
Syamkumar.M