Showing posts with label system performance. Show all posts
Showing posts with label system performance. Show all posts

Wednesday, November 24, 2010

How to clear the cache from memory

Linux has a supposedly good memory management feature that will use up any "extra" RAM you have to cache stuff. This section of the memory being used is SUPPOSED to be freely available to be taken over by any other process that actually needs it, but unfortunately Linux thinks that cache memory is too important to move over for anything else that actually needs it.

I noticed that whenever the server is booted, everything runs great. But as soon as it fills up with cache, performance degrades. It's terrible..

Up until just now, I have been forced to restart every time this happens because I simply cannot get any work done while in this state of retardation. I can close every single program I'm running - and even then, simply right clicking would require some extended thinking before loading the context menu. Ridiculous.

What consumes System Memory?

The kernel - The kernel will consume a couple of MB of memory. The memory that the kernel consumes can not be swapped out to disk. This memory is not reported by commands such as "free" or "ps".

Running programs - Programs that have been executed will consume memory while they run.

Memory Buffers - The amount of memory used is managed by the kernel. You can get the amount with "free".

Memory Cached - The amount of memory used is managed by the kernel. You can get the amount with "free".

Cached memory is freed on demand by the kernel.... it's done this way to make your system more responsive. Trust Linus Torvalds, he knows what he's doing.

There are few ways to check memory usage in Linux. By using the commands like free, vmstat and ps you will be able to check your linux memory usage:-

# free

The 'free -m' command displays the total amount of free and used physical and swap memory in the system, as well as the buffers used by the kernel.

Output:
total used free shared buffers cached
Mem: 1033612 354636 678976 0 240 201508
-/+ buffers/cache: 152888 880724
Swap: 1967952 0 1967952

You can also ask free to display results in every 5 seconds, in order to track the increases/decreases on memory usage.

# free -s 5

# vmstat

The vmstat command reports information about processes, memory, paging, block IO, traps, and cpu activity.

Output:

procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
1 0 0 131620 35432 341496 0 0 42 82 737 1364 15 3 81 1


# ps

The command "ps" is a c program that reads the "/proc" filesystem.

There are two elements that are useful when determining the per process memory usage.

They are:
a. RSS
b. VSIZE

Per Process Memory Usage

The inputs to this section were obtained with the command:

# ps -eo pid,ppid,rss,vsize,pcpu,pmem,cmd -ww --sort=pid


Luckily, I found a way to clear out the cache being used. Simply run the following command as root and the cache will be cleared out.

Linux Command :

# sync; echo 3 > /proc/sys/vm/drop_caches

The main point "Don't Panic! Your ram is fine!" is absolutely correct.

The Linux disk cache is very unobtrusive. It uses spare memory to greatly increase disk access speeds, and without taking any memory away from applications. A fully used store of ram on Linux is efficient hardware use, not a warning sign.

Wednesday, July 28, 2010

Zabbix 1.8.2 on CenOS 5.5 (x86_64)


Installing Zabbix 1.8.2 on CenOS 5.5 and generating fake load to check the installation
==================================================================================

Install the basic CentOS 5.5 on a machine.

==================================================================================

Update CentOS using the following command:

yum -y update

Note : Disable firewall and SELinux only for testing

==================================================================================

Install EPEL using the following command:

rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-3.noarch.rpm

==================================================================================

Install mysql and php using yum:

yum install mysql mysql-server php php-gd php-bcmath php-xml php-mbstring php-mysql

==================================================================================

Download Zabbix rpm file from http://fedora.danny.cz/danny-el/5Server/

wget http://fedora.danny.cz/danny-el/5Server/x86_64/zabbix-web-mysql-1.8.2-2.el5.x86_64.rpm
wget http://fedora.danny.cz/danny-el/5Server/x86_64/zabbix-server-1.8.2-2.el5.x86_64.rpm
wget http://fedora.danny.cz/danny-el/5Server/x86_64/zabbix-1.8.2-2.el5.x86_64.rpm
wget http://fedora.danny.cz/danny-el/5Server/x86_64/zabbix-docs-1.8.2-2.el5.x86_64.rpm
wget http://fedora.danny.cz/danny-el/5Server/x86_64/zabbix-agent-1.8.2-2.el5.x86_64.rpm
wget http://fedora.danny.cz/danny-el/5Server/x86_64/zabbix-web-1.8.2-2.el5.x86_64.rpm
wget http://fedora.danny.cz/danny-el/5Server/x86_64/zabbix-server-mysql-1.8.2-2.el5.x86_64.rpm

==================================================================================

Install the downloaded Zabbix rpm's using the following command :

yum --nogpgcheck localinstall zabbix-1.8.2-2.el5.x86_64.rpm zabbix-server-1.8.2-2.el5.x86_64.rpm zabbix-server-mysql-1.8.2-2.el5.x86_64.rpm zabbix-agent-1.8.2-2.el5.x86_64.rpm zabbix-docs-1.8.2-2.el5.x86_64.rpm zabbix-web-1.8.2-2.el5.x86_64.rpm zabbix-web-mysql-1.8.2-2.el5.x86_64.rpm

==================================================================================

service httpd restart

chkconfig httpd on

==================================================================================
Configuring the Database part (MySQL)

service mysqld restart

chkconfig mysqld on

mysqladmin -u root password passw0rd!

mysql --user=root --password=passw0rd!

Welcome to the MySQL monitor. Commands end with ; or \g.

mysql> create database zabbix character set utf8;

mysql> create user 'zabbix'@'localhost' identified by 'password';

mysql> grant all on zabbix.* to zabbix;

mysql> flush privileges;

mysql> quit
Bye

Insert schema, data and images

cd /usr/share/doc/zabbix-server-mysql-1.8.2/create/schema

mysql --user=zabbix --password=password zabbix <>

cd ../data/

mysql --user=zabbix --password=password zabbix <>

mysql --user=zabbix --password=password zabbix <>

==================================================================================

Configuing PHP

edit /etc/php.ini
;...
;max_execution_time = 600
;max_input_time = 600
;memory_limit = 256M
...
;upload_max_filesize = 16M
...
;post_max_size = 32M
...
;date.timezone = Europe/Brussels
...
;mbstring.func_overload = 2
...

==================================================================================

Config Zabbix-server

edit /etc/zabbix/zabbix_server.conf
...
# Database user
DBUser=zabbix
....
#DBPassword=
DBPassword=
...

==================================================================================

Config Zabbix-webinterface

create /usr/share/zabbix/conf/zabbix.conf.php (remove the ;)

;
;global $DB_TYPE,
;$DB_SERVER,
;$DB_PORT,
;$DB_DATABASE,
;$DB_USER,
;$DB_PASSWORD,
;$IMAGE_FORMAT_DEFAULT;
;$DB_TYPE = "MYSQL";
;$DB_SERVER = "localhost";
;$DB_PORT = "0";
;$DB_DATABASE = "zabbix";
;$DB_USER = "zabbix";
;$DB_PASSWORD = "";
;$IMAGE_FORMAT_DEFAULT = IMAGE_FORMAT_PNG;
;?>

==================================================================================



service zabbix-server start

chkconfig zabbix-server on

service zabbix-agent start

chkconfig zabbix-agent on

lsof -i:10050

more /var/log/zabbix/zabbix_agentd.log

==================================================================================

http://server/zabbix

Login using the following credentials :

login: admin / password: zabbix

==================================================================================

Generate Fake CPU load :

perl -e 'while (--$ARGV[0] and fork) {}; while () {}' 4

top

==================================================================================

Generate Fake Disk-IO load and Free unused memory currently unavailable :


dd if=/dev/zero of=junk bs=1M count=1K (This will vreate a 1 GB file)

dd if=/dev/zero of=junk bs=1M count=10K (This will vreate a 10 GB file)

This is an useful command for when your OS is reporting less free RAM than it actually has. In case terminated processes did not free their variables correctly, the previously allocated RAM might make a bit sluggis over time.

This command then creates a huge file made out of zeroes and then removes it, thus freeing the amount of memory occupied by the file in the RAM.
In this example, the sequence will free up to 1GB(1M * 1K) of unused RAM. This will not free memory which is genuinely being used by active processes.

rm -rf junk

==================================================================================

Monitor Disk IO performance using iostat:

yum -y install sysstat

iostat -x 10 > /tmp/iostat.txt

tail -f /tmp/iostat.txt

==================================================================================

Test network speed without wasting disk

dd if=/dev/zero bs=1M count=1M | ssh root@192.168.160.161 'cat > /dev/null'

The above command will send 1GB of data from one host to the next over the network, without consuming any unnecessary disk on either the client nor the host. This is a quick and dirty way to benchmark network speed without wasting any time or disk space.


ENJOY !