Category Archives: Handy Commands

Handy commands, Useful commands with explanation

rsync usage in different situations

Posted by Suneil on

rsync (Remote Sync) is fast, remote and local file transferring tool. It’s used for copying as well as synchronizing files/directories in UNIX/Linux systems.

Pros:
– Supports copying links, devices, owners, groups and permissions.
– It’s faster than scp (Secure Copy) because rsync uses remote-update protocol which allows to transfer just the differences between two sets of files. First time, it copies the whole content of a file or a directory from source to destination but from next time, it copies only the changed blocks and bytes to the destination.
– Rsync consumes less bandwidth as it uses compression and decompression method while sending and receiving data both ends.

Some common options used with rsync commands:

-v, –verbose increase verbosity
-H, –hard-links preserve hard links
-x, –one-file-system don’t cross filesystem boundaries
-z, compress file data
-h, human-readable, output numbers in a human-readable format

-P same as –partial –progress
–partial keep partially transferred files
–progress show progress during transfer

-r, –recursive copies data recursively (but don’t preserve timestamps and permission while transferring data
-a, –archive archive mode, archive mode allows copying files recursively and it also preserves symbolic links, file permissions, user & group ownerships and timestamps

NOTE: You can use either -r or -a to transfer a directory.

Usages:

1) Normal usage

rsync -avHx source destination

2) From remote server to local

rsync -avHx user@server:/absolute/path/source /absolute/path/destination

3) From local to remote server

rsync -avHx /absolute/path/source user@server:/absolute/path/destination

4) Exclude particular file/directory

rsync -avHx source –exclude=”/path/to/file” –exclude=”/path/to/folder/*” destination

5) Connect to remote server using absolute SSH command:

rsync -PavHx source -e “ssh -i /path/to/PrivateKey.pem” user@server:/absolute/path/destination

Command to determine the public IP in Linux OS

Posted by Suneil on

If you are behind a router, then your computer will not know about the public IP address as the router does a network address translation.

If you are not behind a router, you can find it out using ifconfig. Else you could request this website using curl or wget and extract the information you need from it, e.g. using a perl script.

curl -s checkip.dyndns.org | sed -e ‘s/.*Current IP Address: //’ -e ‘s/<.*$//’
or shorter

wget http://ipinfo.io/ip -qO –

should do the trick.

Delete / truncate the entries in a file

Posted by Suneil on

The following command is the efficient one to delete/truncate all the entries from a file which is being actively used:

truncate -s 0 filename

Refer: http://linux.die.net/man/1/truncate

 

On the other hand, truncate would likely be slower than > filename when run from a script since running the truncate command requires the system to open the executable, load it, and the run it.

Delete files of certain size range using FIND command

Posted by Suneil on

Delete files of certain size range using FIND command:

find /path -type f -name “*.tmp” -size +500M  -size –100G -exec rm -fv {} \;

–> size is a numeric argument that can optionally be prefixed with + and –

Numeric arguments can be specified as:
+n    for greater than n,
-n     for less than n,
n      for exactly n.

 

Usage:

-size n[cwbkMG]
File uses n units of space.  The following suffixes can be used:

`b’    for 512-byte blocks (this is the default if no suffix is used)

`c’    for bytes

`w’    for two-byte words

`k’    for Kilobytes (units of 1024 bytes)

`M’    for Megabytes (units of 1048576 bytes)

`G’    for Gigabytes (units of 1073741824 bytes)

 

 

Kill a Zombie process

Posted by Suneil on

To kill a zombie process:

1) Get the process ID (PID) of zombie process

[USER@HOSTNAME~]# ps aux | grep Z
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
user      7823  0.0  0.0      0     0 ?        Z    Apr04   0:00 [check_disk] <defunct>

[USER@HOSTNAME~]#

 

2) Get the Parent Process ID (PPID) of zombie process:

cat /proc/PID/status | grep -i ppid

* Replace PID with the actual Process ID (PID) that you’ve got from Step 1

 

3) Kill the Parent Process ID

kill -9 PPID

* Replace PPID with the actual Parent Process ID (PPID) that you’ve got from Step 2.

 

AUTOMATED:

As an alternative, you can combine all the above manual steps and put into a script. I’ve created a sample script for you:

ps aux | grep Z | grep -v “pts/0” | awk ‘{print $2}’ | grep -v PID > file-pid.txt
for i in `cat file-pid.txt`
do
cat /proc/$i/status | grep -i ppid | awk ‘{print $2}’ | xargs kill -9
done

 

 

 

How to exclude directories/files while running zip command

Posted by Suneil on

Commands:

1) The following command will exclude all the files/subdirectories under ‘work’ subdirectory, but retaining the subdirectory ‘work’:

zip -r foo.zip foodir/ -x foodir/work/**\*

-x = exclude the files/directories

**\* = exclude all files and directories

2) If you want to exclude whole subdirectory itself, use the following argument:

-x foodir/work\*

 

3) If you want to exclude certain files, mention the exact file name

-x foodir/work/foo.txt

 

4) If you want to exclude all .txt files from the directory and all the subdirectories, use the following argument:

-x foodir/work/\*.txt

 

5) If you want to exclude .txt files only from the directory (not from subdirectories); use the following argument:

-x foodir/*.txt

 

P.S. I’ve used .txt files as the example. I’ve just outline the command pattern. You can use any file extensions and customize the command.