Tuesday, June 4, 2013

To calculate the size of selected list of files in a directory

>>ls -l /mnt/dd_data/rawlogs/2013052004/federation_production/acc*
31216UTC.gz
-rw-r--r--  1 yahoo dev 38158188 May 20 11:43 /mnt/dd_data/rawlogs/2013052004/federation_production/access.fed1064.yss.ac2.yahoo.com.20130520041216UTC.gz
-rw-r--r--  1 yahoo dev 40594305 May 20 11:33 /mnt/dd_data/rawlogs/2013052004/federation_production/access.fed1064.yss.sk1.yahoo.com.20130520032713UTC.gz
-rw-r--r--  1 yahoo dev 39197951 May 20 11:34 /mnt/dd_data/rawlogs/2013052004/federation_production/access.fed1064.yss.sk1.yahoo.com.20130520042713UTC.gz
-rw-r--r--  1 yahoo dev 42180531 May 20 11:50 /mnt/dd_data/rawlogs/2013052004/federation_production/access.fed1065.yss.ac2.yahoo.com.20130520033334UTC.gz
-rw-r--r--  1 yahoo dev 35726770 May 20 11:53 /mnt/dd_data/rawlogs/2013052004/federation_production/access.fed1065.yss.ac2.yahoo.com.20130520043334UTC.gz
.
.
.
.
.
.

Now, to calculate the total size of these files, we can do as following:
>> ls -l /mnt/dd_data/rawlogs/2013052004/federation_production/acc* | awk '{SUM=SUM+$5; } END {printf "%10.0f\n", SUM}'
>>7962647332

This will give you size in bytes.
- If you want it in MBs, divide by 1024
- If you wanr it in GBs, divide by 1024*1024

Thursday, May 30, 2013

To get process name OR command name using process id.

For example I want to find out the command name or process name for the process id 16874 

>> ps ax -o pid,cmd | grep 16874 | grep -v grep | cut -d " " -f2-
               1                           2                 3                 4
output : 
>> /usr/local/bin/perl -w /home/y/bin/yroot hdfs_proxy --user dkuldeep

Explanation:    

1) To get the list of PIDs and relevant commands:
>> ps ax -o pid,cmd  

>>
PID CMD
    1 init [3]
    2 [migration/0]
    3 [ksoftirqd/0]
    4 [migration/1]
    5 [ksoftirqd/1]
    6 [events/0]
    7 [events/1]
    8 [khelper]
    9 [kacpid]
   33 [kblockd/0]
   2465 /usr/sbin/snmpd -u nobody -g nobody -I -smux -Lsd -Lf /dev/null -p /var/run/snmpd.pid
   2511 /usr/sbin/automount --timeout=600 /mnt file /etc/auto.mnt
   2525 /usr/sbin/smartd
   2534 /usr/sbin/acpid
   2552 /usr/sbin/sshd-2222 -f /etc/ssh/sshd_config.2222
   16874 /usr/local/bin/perl -w /home/y/bin/yroot hdfs_proxy --user dkuldeep


2) To filter for required PID (16874)
>>ps ax -o pid,cmd | grep 16874

>>
552 grep 16874
16874 /usr/local/bin/perl -w /home/y/bin/yroot hdfs_proxy --user dkuldeep


3)  To remove the grep command line
>> ps ax -o pid,cmd | grep 16874 | grep -v grep 

>>
16874 /usr/local/bin/perl -w /home/y/bin/yroot hdfs_proxy --user dkuldeep


4) To extract the exact command line
>> ps ax -o pid,cmd | grep 16874 | grep -v grep | cut -d " " -f2-

>>
/usr/local/bin/perl -w /home/y/bin/yroot hdfs_proxy --user dkuldeep



Wednesday, May 29, 2013

How to sort the list of values in UNIX on command prompt

Use SORT command of Unix:

For example:
>grep 'Copied successfully' /tmp/dd_grid_data_log | awk {'print $10'} | sort -n

output:
>2
2
2
2
2
2
2
2
2
2
2
2
3
3
46
47
50
56
57
61
63
66
68
70
72
73
73
111
113
122
126
127
130