Category Archives: UNIX/Linux/Apache/PHP/MySQL

UNIX, Linux, Apache, PHP, MySQL

rsync usage in different situations

Posted by Suneil on

rsync (Remote Sync) is fast, remote and local file transferring tool. It’s used for copying as well as synchronizing files/directories in UNIX/Linux systems.

Pros:
– Supports copying links, devices, owners, groups and permissions.
– It’s faster than scp (Secure Copy) because rsync uses remote-update protocol which allows to transfer just the differences between two sets of files. First time, it copies the whole content of a file or a directory from source to destination but from next time, it copies only the changed blocks and bytes to the destination.
– Rsync consumes less bandwidth as it uses compression and decompression method while sending and receiving data both ends.

Some common options used with rsync commands:

-v, –verbose increase verbosity
-H, –hard-links preserve hard links
-x, –one-file-system don’t cross filesystem boundaries
-z, compress file data
-h, human-readable, output numbers in a human-readable format

-P same as –partial –progress
–partial keep partially transferred files
–progress show progress during transfer

-r, –recursive copies data recursively (but don’t preserve timestamps and permission while transferring data
-a, –archive archive mode, archive mode allows copying files recursively and it also preserves symbolic links, file permissions, user & group ownerships and timestamps

NOTE: You can use either -r or -a to transfer a directory.

Usages:

1) Normal usage

rsync -avHx source destination

2) From remote server to local

rsync -avHx user@server:/absolute/path/source /absolute/path/destination

3) From local to remote server

rsync -avHx /absolute/path/source user@server:/absolute/path/destination

4) Exclude particular file/directory

rsync -avHx source –exclude=”/path/to/file” –exclude=”/path/to/folder/*” destination

5) Connect to remote server using absolute SSH command:

rsync -PavHx source -e “ssh -i /path/to/PrivateKey.pem” user@server:/absolute/path/destination

How to create a SFTP user

Posted by Suneil on


1) Log into SFTP server.

2) Create the SFTP user with no shell.
useradd -s /sbin/nologin -d /home/UserName -m UserName

3) Craete the directory to upload files.
mkdir -p /home/UserName/ftpsite/upload

4) Change the permission of user’s home directory.
chown root:root /home/UserName
chown root:root /home/UserName/ftpsite

5) Change the permission of user’s upload directory.
chown UserName:sftp upload

6) Set the group ‘sftp’ for the ‘UserName’.
usermod -g sftp UserName

7) Add the new user to SSH configuration
File: /etc/ssh/sshd_config

Configuration:

Subsystem sftp internal-sftp
AllowUsers UserName UserName1 UserName2
Match Group sftp

ChrootDirectory /home/%u/ftpsite
ForceCommand internal-sftp
AllowTcpForwarding no

8) Restart the SSH daemon.
/etc/init.d/sshd restart

Connect to FTP server and upload files to check the functionality.

Port: 22

 

Command to determine the public IP in Linux OS

Posted by Suneil on

If you are behind a router, then your computer will not know about the public IP address as the router does a network address translation.

If you are not behind a router, you can find it out using ifconfig. Else you could request this website using curl or wget and extract the information you need from it, e.g. using a perl script.

curl -s checkip.dyndns.org | sed -e ‘s/.*Current IP Address: //’ -e ‘s/<.*$//’
or shorter

wget http://ipinfo.io/ip -qO –

should do the trick.

Delete / truncate the entries in a file

Posted by Suneil on

The following command is the efficient one to delete/truncate all the entries from a file which is being actively used:

truncate -s 0 filename

Refer: http://linux.die.net/man/1/truncate

 

On the other hand, truncate would likely be slower than > filename when run from a script since running the truncate command requires the system to open the executable, load it, and the run it.

Delete files of certain size range using FIND command

Posted by Suneil on

Delete files of certain size range using FIND command:

find /path -type f -name “*.tmp” -size +500M  -size –100G -exec rm -fv {} \;

–> size is a numeric argument that can optionally be prefixed with + and –

Numeric arguments can be specified as:
+n    for greater than n,
-n     for less than n,
n      for exactly n.

 

Usage:

-size n[cwbkMG]
File uses n units of space.  The following suffixes can be used:

`b’    for 512-byte blocks (this is the default if no suffix is used)

`c’    for bytes

`w’    for two-byte words

`k’    for Kilobytes (units of 1024 bytes)

`M’    for Megabytes (units of 1048576 bytes)

`G’    for Gigabytes (units of 1073741824 bytes)

 

 

Kill a Zombie process

Posted by Suneil on

To kill a zombie process:

1) Get the process ID (PID) of zombie process

[USER@HOSTNAME~]# ps aux | grep Z
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
user      7823  0.0  0.0      0     0 ?        Z    Apr04   0:00 [check_disk] <defunct>

[USER@HOSTNAME~]#

 

2) Get the Parent Process ID (PPID) of zombie process:

cat /proc/PID/status | grep -i ppid

* Replace PID with the actual Process ID (PID) that you’ve got from Step 1

 

3) Kill the Parent Process ID

kill -9 PPID

* Replace PPID with the actual Parent Process ID (PPID) that you’ve got from Step 2.

 

AUTOMATED:

As an alternative, you can combine all the above manual steps and put into a script. I’ve created a sample script for you:

ps aux | grep Z | grep -v “pts/0” | awk ‘{print $2}’ | grep -v PID > file-pid.txt
for i in `cat file-pid.txt`
do
cat /proc/$i/status | grep -i ppid | awk ‘{print $2}’ | xargs kill -9
done