Playing with Chmod!

What happens when you accidently change the permission of the all important ‘chmod’ command? 

Let’s try it out….

1.            # ls -l /bin/chmod

-rwxr-xr-x. 1 root root 48712 Oct 15  2014 /bin/chmod

2.   # rpm -qf `which chmod`

      coreutils-8.4-37.el6.x86_64

  

3.   # chmod 444 /bin/chmod

4.   # ls -l /bin/chmod

-r--r--r--. 1 root root 48712 Oct 15  2014 /bin/chmod

So, the permission is changed!

5.   # chmod 655 /bin/chmod

  -bash: /bin/chmod: Permission denied

Oooops……Screwed  !!!!! :(

Now is the time to do some rescue stuff…

6. Lets restore it back;

7.   #    install -m 0755 /bin/chmod ./chmod.tmp

      #   ./chmod.tmp 0755 /bin/chmod

8.   # ls -l  /bin/chmod

 -rwxr-xr-x. 1 root root 48712 Oct 15  2014 /bin/chmod

Your all important 'chmod' command is back!

Linux just works ! ;)

Recovering deleted files in Linux ! The ext4magic ...

I was wondering if I could find any tool that would undelete files from ext3/ext4 partitions. While searching I came across this tool which did the job I wanted. I tested it on an old Fedora-14 (32 bit) system;

[Source:   

http://source.kohlerville.com/2013/02/ext4-recover-deleted-files-undelete-using-ext4magic-on-centos-6/ 

]

Following are the steps;

1. Download URL;
http://rpms.plnet.rs/plnet-centos6-i386/RPMS.plnet-downloaded/ext4magic-0.3.1-1.2.i686.rpm
http://rpms.plnet.rs/plnet-centos6-x86_64/RPMS.plnet-downloaded/ext4magic-0.3.1-1.2.x86_64.rpm

2. Install;
# yum install ext4magic-0.3.1-1.2.i686.rpm --nogpgcheck
(I could not find the gpg key so I avoided checking the same while installing)

3. I deleted some of the files from "/LVM-1" directory which was acting as the mount point for
        /dev/mapper/vg1-lv--1

4. Stop using the filesystem that needs recovery and unmount it;
# umount /LVM-1

5. Copy partition;
(dd if=/dev/DEVICE of=/BACKUPPATH/DEVICE.img)
# dd if=/dev/mapper/vg1-lv--1 of=/BACKUP/vg1-lv--1.img

6. Examine the directory structure once. I created a separate directory for recovered files;
# tree BACKUP/
BACKUP/
├── RECOVERED
└── vg1-lv--1.img

7. Run ext4magic on the copy/dd-image to recover all deleted files (-m switch for files and 
        -M if entire filesystem needs recovery);

(ext4magic /BACKUPPATH/DEVICE.img -m -j /BACKUPPATH/journal.copy)

# ext4magic /BACKUP/vg1-lv--1-NEW.img -M -d /BACKUP/RECOVERED/
Warning: Activate magic-scan or disaster-recovery function, may be some command line options
         ignored
"/BACKUP/RECOVERED/"  accept for recoverdir
Filesystem in use: /BACKUP/vg1-lv--1-NEW.img

Using  internal Journal at Inode 8
Activ Time after  : Wed Jul 15 14:03:20 2015
Activ Time before : Wed Jul 15 14:07:04 2015
Inode 2 is allocated
-------- /BACKUP/RECOVERED//lost+found
-------- /BACKUP/RECOVERED//Dir-1
-------- /BACKUP/RECOVERED//Dir-2
-------- /BACKUP/RECOVERED//file-1
-------- /BACKUP/RECOVERED//file-2
-------- /BACKUP/RECOVERED//
MAGIC-1 : start lost directory search
MAGIC-2 : start lost file search
MAGIC-2 : start lost in journal search
MAGIC-3 : start ext4-magic-scan search
ext4magic : EXIT_SUCCESS


Bang! Files are back :) 

DRBD (Distributed Replicated Block Device)

DRBD (Distributed Replicated Block Device)
---------------------------------------------------------

DRBD refers to block devices which acts as to form high availability (HA) clusters on Linux systems. The process involves mirroring a whole block device over network. This is somewhat similar to network based RAID-1 where data is copied on two storage devices, failure of one results in activating the other one. 

But, there is marked difference between Network based RAID-1 and DRBD. In case of network based RAID-1, there exists a single application which access any one of the two storage devices at any point in time. The RAID layer chooses to read the other device when one fails. There is an abstraction between the application and RAID devices thus the application is not aware of which of the devices it is interacting at any given point. 
But, this is not the case with DRBD where exists two instances of the application, and each can read only from one of the two storage devices. Read more here...

Requirements

– Two disks  (preferably same size)
– Networking between machines (node1 & node2)
– Working DNS resolution  (/etc/hosts file)
– NTP synchronized times on both nodes
– Selinux Permissive (!)
– Iptables ports (7788) allowed

A. Test Environment:

– Two CentOS-6.6 systems with IP 172.16.20.46 and 172.16.20.47 designated as 
           "node1" and "node2" respectively 

– On both systems Disks are partitioned as;
  /dev/vdb1               1        2082     1049296+  83  Linux
  /dev/vdb2            2083      6241     2096136   83  Linux

– DNS resolution is done through /etc/hosts on both the systems by setting hostname <--> IP address 
    instead of FQDN  as;
 172.16.20.46 node1
 172.16.20.47 node2

B. SELinux and IPTABLES:

B1. Setting SELinux to permissive (Not a very good idea though!! :) )
# setenfore 0

B2. Allow DRBD port to accept connection from all (test only, change it to specific IPs 
       to your need)
# iptables -I INPUT 4 -p tcp --dport 7788 -j ACCEPT
# service iptables save
iptables: Saving firewall rules to /etc/sysconfig/iptables: [  OK  ]
 
     # service iptables restart
iptables: Setting chains to policy ACCEPT: filter           [  OK  ]
iptables: Flushing firewall rules:                         [  OK  ]
iptables: Unloading modules:                               [  OK  ]
iptables: Applying firewall rules:                         [  OK  ]

C. Configuration:
Note:  Carry out the following steps on both the systems unless mentioned otherwise;

1. Setup ELRepo (not EPEL, as I could not find DRBD packages for CentOS-6.6 release) on both 
    systems;
  # wget http://www.elrepo.org/elrepo-release-6-6.el6.elrepo.noarch.rpm .
# rpm -ivh elrepo-release-6-6.el6.elrepo.noarch.rpm

2. Check the yum repositories once;
# yum repolist
repo id                 repo name                               status
base                  CentOS-6 - Base                                       6,518
datastax              DataStax Repo for Apache Cassandra                         158
elrepo                ELRepo.org Community Enterprise Linux Repository - el6        334
epel                  Extra Packages for Enterprise Linux 6 - x86_64                   11,735
extras                CentOS-6 - Extras                                         38
updates               CentOS-6 - Updates                                         1,336

3. Install the packages;
# yum install drbd83-utils kmod-drbd83

4. Now, insert DRBD module manually into Kernel on both systems (or reboot to make it 
    effective);
  # modprobe drbd
# lsmod | grep drbd
      drbd       332493  0 

5. Create the Distributed Replicated Block Device resource file (/etc/drbd.d/clusterdb.res):
resource clusterdb
{
startup {
wfc-timeout 30;
outdated-wfc-timeout 20;
degr-wfc-timeout 30;
}

net {
cram-hmac-alg sha1;
shared-secret sync_disk;
}

syncer {
rate 10M;
al-extents 257;
on-no-data-accessible io-error;
}

on node1 {
device /dev/drbd0;
disk /dev/vdb2;
address 172.16.20.46:7788;
flexible-meta-disk internal;
}
 
on node2 {
device /dev/drbd0;
disk /dev/vdb2;
address 172.16.20.47:7788;
meta-disk internal;
}
}

6. Copy DRBD configuration files (viz. /etc/drbd.d/clusterdb.res) and entries in /etc/hosts file to    
     "node2"

7. Initialize the DRBD meta data storage on both machines;
# drbdadm create-md clusterdb

--==  Thank you for participating in the global usage survey  ==--
The server's response is:

you are the 25728th user to install this version
Writing meta data...
initializing activity log
NOT initialized bitmap
New drbd meta data block successfully created.
success

8. Start the DRBD service on both systems;
# service drbd start

Starting DRBD resources: [ d(clusterdb) s(clusterdb) n(clusterdb) ]..........

       DRBD's startup script waits for the peer node(s) to appear.
       - In case this node was already a degraded cluster before the
        reboot the timeout is 30 seconds. [degr-wfc-timeout]
        - If the peer was available before the reboot the timeout will
         expire after 30 seconds. [wfc-timeout]
        (These values are for resource 'clusterdb'; 0 sec -> wait forever)
        To abort waiting enter 'yes' [  29]:
.

9. Check status as;
# service drbd status
or,
# cat /proc/drbd

10. As you can see, at the beginning both nodes are secondary, which is normal. We need to figure out   
      which one would act as a primary that will initiate the first 'full sync' between the two nodes; 

      In our case, we choose "node1", thus;
A. On node1:
# drbdadm -- --overwrite-data-of-peer primary clusterdb

11. Put a watch on the sync in progress;
# watch 'cat /proc/drbd'
Every 2.0s: cat /proc/drbd                                 Tue Jul 14 08:59:44 2015

version: 8.3.16 (api:88/proto:86-97)
GIT-hash: a798fa7e274428a357657fb52f0ecf40192c1985 build by phil@Build64R6, 2014-11-24 
      14:51:37
     0: cs:SyncSource ro:Primary/Secondary ds:UpToDate/Inconsistent C r-----
     ns:429056 nr:0 dw:0 dr:429720 al:0 bm:26 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:1666980
[===>................] sync'ed: 20.8% (1666980/2096036)K
finish: 0:03:38 speed: 7,600 (9,128) K/sec

12. Once full sync is achieved, check the status on both the nodes;
A. Node1
# cat /proc/drbd
version: 8.3.16 (api:88/proto:86-97)
GIT-hash: a798fa7e274428a357657fb52f0ecf40192c1985 build by phil@Build64R6, 2014-11-
            24 14:51:37
           0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r-----
           ns:2096036 nr:0 dw:0 dr:2096700 al:0 bm:128 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0

B. Node2
[root@node2 ~]# cat /proc/drbd 
version: 8.3.16 (api:88/proto:86-97)
GIT-hash: a798fa7e274428a357657fb52f0ecf40192c1985 build by phil@Build64R6, 2014-11-       
            24 14:51:37
0: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r-----
           ns:0 nr:2096036 dw:2096036 dr:0 al:0 bm:128 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0

13. Create filesystem on Distributed Replicated Block Device device;
On the primary node i.e. node1:
# mkfs.ext4 /dev/drbd0

14. Now, mount DRBD device on primary node;
# mkdir /my-DRBD-Data
# mount /dev/drbd0 /my-DRBD-Data

15. Please note:  
One does not need need to mount the disk from secondary systems explicitly. All data that are 
       written on "/my-DRBD-Data" folder will be synced to the other system i.e. "node2".

16. Lets test that out;
16a. Unmount the "/my-DRBD-Data" folder from the primary node "node1". 
16b. Make the secondary node as primary node.  
16c. Mount back the "/my-DRBD-Data" on the second machine "node2", you will see the same 
                 contents in /my-DRBD-Data  folder.
17. Actual test;

17a. Create some data on Node1:
# cd /my-DRBD-Data
# mkdir bijit
# echo "hi" > test1.txt
# echo "hello" > test2.txt
 
                 [root@node1 my-DRBD-Data]# ll
total 28
drwxr-xr-x. 2 root root  4096 Jul 14 09:20 bijit
drwx------. 2 root root 16384 Jul 14 09:15 lost+found
-rw-r--r--. 1 root root     3 Jul 14 09:20 test1.txt
-rw-r--r--. 1 root root     6 Jul 14 09:21 test2.txt

17b. Unmount the data on the primary Node:
[root@node1 /]# umount /my-DRBD-Data/

17c. Make secondary node as the Primary:
[root@node1 /]# drbdadm secondary clusterdb
[root@node2 ~]# drbdadm -- --overwrite-data-of-peer primary clusterdb

17d. Mount back the "/my-DRBD-Data" on the second machine "node2";
[root@node2 ~]# mkdir /my-DRBD-Data
[root@node2 ~]# mount /dev/drbd0 /my-DRBD-Data/
 
  17e. Check the contents on Node2:
[root@node2 ~]# ll /my-DRBD-Data/
total 28
drwxr-xr-x. 2 root root  4096 Jul 14 09:20 bijit
drwx------. 2 root root 16384 Jul 14 09:15 lost+found
-rw-r--r--. 1 root root     3 Jul 14 09:20 test1.txt
-rw-r--r--. 1 root root     6 Jul 14 09:21 test2.txt

Now, delete and add some contents on the "node2" which now acts as primary. 
Unmount it and make it secondary. Switch to "node1" and make it as primary and mount it back. Check if the changes are replicated !

MySQL Cluster Setup on CentOS-6x

MySQL Cluster is a technology providing shared-nothing clustering and auto-sharding for the MySQL database management system. It is designed to provide high availability and high throughput with low latency, while allowing for near linear scalability.   
--- From Wikipedia


Note(s):
1. Tested on Centos-6.6 (64 bit) openstack instances.
2. Iptables is ON (specific iptables rules are given below).
3. SeLinux is Permissive (setenforce 0)
4. SeLinux Bollean values turned on for MySQL user;
# setsebool -PV allow_user_mysql_connect on
# setsebool -PV mysql_connect_any on


Architecture:
I. Management Node  
IP: 172.16.20.21

II. Node - 1   (Data and SQL Node)
IP: 172.16.20.22  

III.        Node - 2   (Data and SQL Node)
IP: 172.16.20.23


A. Downloads:
For all nodes download the following package;
http://dev.mysql.com/get/Downloads/MySQL-Cluster-7.4/MySQL-Cluster-gpl-7.4.4-1.el6.x86_64.rpm-bundle.tar .


B. Installation on Management Node (IP: 172.16.20.21):

1. Install the needed packages;
yum install perl libaio -y

2. Remove the following mysql library as it would conflict with cluster packages;
yum remove mysql-libs

3. Extract the compressed package and install it;
i. tar -xvf MySQL-Cluster-gpl-7.4.4-1.el6.x86_64.rpm-bundle.tar
ii. rpm -ivh MySQL-Cluster-server-gpl-7.4.4-1.el6.x86_64.rpm

4. Take a note of the initial root password of MySql
You will find that password in '/root/.mysql_secret'.

C. Installation on Data Node (IP: 172.16.20.22, 172.16.20.23):
1. Install the needed packages;
yum install perl libaio -y

2. Remove the following mysql library as it would conflict with cluster packages;
yum remove mysql-libs

3. Extract the compressed package and install it;
i. tar -xvf MySQL-Cluster-gpl-7.4.4-1.el6.x86_64.rpm-bundle.tar
ii. rpm -ivh MySQL-Cluster-server-gpl-7.4.4-1.el6.x86_64.rpm

4. Take a note of the initial root password of MySql:
You will find that password in '/root/.mysql_secret'.

5. Install the addtional MySQL client package on SQL Nodes:
rpm -ivh MySQL-Cluster-client-gpl-7.4.4-1.el6.x86_64.rpm


D. Installation on SQL NOdes (IP: 172.16.20.22, 172.16.20.23):

1. Install the needed packages;
yum install perl libaio -y

2. Remove the following mysql library as it would conflict with cluster packages;
yum remove mysql-libs

3. Extract the compressed package and install it;
i. tar -xvf MySQL-Cluster-gpl-7.4.4-1.el6.x86_64.rpm-bundle.tar
ii. rpm -ivh MySQL-Cluster-server-gpl-7.4.4-1.el6.x86_64.rpm

4. Take a note of the initial root password of MySql:
You will find that password in '/root/.mysql_secret'.

5. Install the addtional MySQL client package on SQL Nodes:
rpm -ivh MySQL-Cluster-client-gpl-7.4.4-1.el6.x86_64.rpm


E. Configurations:
A. Data and SQL Nodes;

# mkdir -p /usr/local/mysql/data
# chown -R mysql:mysql /usr/local/mysql/data

vi /etc/my.cnf
[mysqld]
# Options for mysqld process:
ndbcluster                      # run NDB storage engine

[mysql_cluster]
# Options for MySQL Cluster processes:
ndb-connectstring=172.16.20.21  # location of management server


B. Management node:
# mkdir /var/lib/mysql-cluster
# chown -R mysql:mysql /var/lib/mysql-cluster
# cd /var/lib/mysql-cluster

vi config.ini

[ndbd default]
# Options affecting ndbd processes on all data nodes:
NoOfReplicas=2    # Number of replicas
DataMemory=80M    # How much memory to allocate for data storage
IndexMemory=18M   # How much memory to allocate for index storage
 # For DataMemory and IndexMemory, we have used the
 # default values. Since the "world" database takes up
 # only about 500KB, this should be more than enough for
 # this example Cluster setup.

ServerPort=50501  # This is to allocate a fixed port through which one node is connected to the other node within the cluster.
 # By default, this port is allocated dynamically in such a way as to ensure that no two nodes on the same host
 # computer receive the same port number.
 # To open specific ports in a firewall to permit communication between data nodes and API nodes (including SQL
 # nodes), you can set this parameter to the number of the desired port in an [ndbd] section or (if you need to do
 # this for multiple data nodes) the [ndbd default] section of the config.ini file, and then open the port having
 # that number for incoming connections from SQL nodes, API nodes, or both.

[tcp default]
# TCP/IP options:
#portnumber=1186   # This the default; however, you can use any
 # port that is free for all the hosts in the cluster
 # Note: It is recommended that you do not specify the port
 # number at all and simply allow the default value to be used
 # instead

[ndb_mgmd]
# Management process options:
hostname=172.16.20.21           # Hostname or IP address of MGM node
datadir=/var/lib/mysql-cluster  # Directory for MGM node log files

[ndbd]
# Options for data node "A":
hostname=172.16.20.22           # Hostname or IP address
datadir=/usr/local/mysql/data   # Directory for this data node's data files

[ndbd]
# Options for data node "B":
hostname=172.16.20.23           # Hostname or IP address
datadir=/usr/local/mysql/data   # Directory for this data node's data files

[mysqld]
# SQL node options:
hostname=172.16.20.22           # Hostname or IP address
# (additional mysqld connections can be
# specified for this node for various
# purposes such as running ndb_restore)
[mysqld]
# SQL node options:
hostname=172.16.20.23           # Hostname or IP address
# (additional mysqld connections can be
# specified for this node for various
# purposes such as running ndb_restore)



F. IPtables rules;
(Here, I have allowed connection from any source on the destination ports as I have another set of                 firewall on top of the cloud and thus port are not exposed to the outside world, but in an ideal
                condition, one needs to specify the connection source)

I. On Management Node (INPUT chain):

# iptables -I INPUT -i eth0 -p tcp --dport 1186 -j ACCEPT
# iptables -I INPUT -i eth0 -p tcp --dport 3306 -j ACCEPT
# iptables -I INPUT -i eth0 -p tcp --dport 50501 -j ACCEPT
# service iptables save
# service iptables restart

II. On Data and SQL Nodes (INPUT chain):
# iptables -I INPUT -i eth0 -p tcp --dport 3306 -j ACCEPT

G. Start the services (in a sequence);

1. Management Node:
# ndb_mgmd -f /var/lib/mysql-cluster/config.ini

2. Data Node:
# ndbd

3. MySQL
# service mysql start

On the SQL Node,
Change the default mysql root password as;
# mysqladmin -u root -p'oldpassword' password newpass
(Check the old password in '/root/.mysql_secret')

H. Connect the MySQL Cluster Management Console and check the status (if everything        
         works fine then you should see similar to the following);

ndb_mgm> show
Cluster Configuration
---------------------
[ndbd(NDB)]     2 node(s)
id=2    @172.16.20.22  (mysql-5.6.23 ndb-7.4.4, Nodegroup: 0, *)
id=3    @172.16.20.23  (mysql-5.6.23 ndb-7.4.4, Nodegroup: 0, *)

[ndb_mgmd(MGM)] 1 node(s)
id=1    @172.16.20.21  (mysql-5.6.23 ndb-7.4.4)

[mysqld(API)]   2 node(s)
id=4    @172.16.20.22  (mysql-5.6.23 ndb-7.4.4)
id=5    @172.16.20.23  (mysql-5.6.23 ndb-7.4.4)

ndb_mgm>


I. Log files:
1. Management Node:
/var/lib/mysql-cluster

2. Data/SQL Nodes:
/usr/local/mysql/data/

NFS - Network File System

What is NFS?

Network File System or NFS is a client/server system. It allows users to access files across network and the shared files/directories from remote are treated as if they reside on local system.

Works on both TCP and UDP protocol.

The stateless UDP connection under normal conditions has less Protocol overhead than TCP which can translate into better performance on very clean, non-congested networks.

Note (From CentOS official docs):

NFSv4 has no interaction with portmapper, rpc.mountd, rpc.lockd, and rpc.statd, since protocol support has been incorporated into the v4 protocol. NFSv4 listens on the well known TCP port (2049) which eliminates the need for the portmapper interaction. The mounting and locking protocols have been incorpated into the V4 protocol which eliminates the need for interaction with rpc.mountd and rpc.lockd.

Salient features:

* A NFS file system mounted is very similar to a local file system on the machine
* NFS does not disclose the location of a file on the network
* An NFS server can be made of a completely different architecture and operating system
        than the client
* It also never discloses the underlying file system on the remote machine

Versions:
Currently there are three versions available viz. NFSv2, NFSv3 & NFSv4

Setup on CentOS-6.5 systems:
Following are the IP details of the server and the Client for this example setup;

i. Server: 172.16.20.8
ii. Client: 172.16.20.9

A. NFS Server (IP 172.16.20.8):

1. Install the packages;
yum install nfs*

2. Start the NFS service;
# service nfs start
Starting NFS services:      [  OK  ]
Starting NFS mountd:        [  OK  ]
Starting NFS daemon:        [  OK  ]
Starting RPC idmapd:        [  OK  ]

[NOTE:  Although, it is mentioned that rpcbind (earlier Portmapper) services are incorporated into NFS-V4 and there is no need to start the rpcbind service explicitly, I encountered NFS services failed to start without explicitly starting the "rpcbind" service; thus if you encounter the same, start the rpcbind service as; 

# service rpcbind start
Starting rpcbind:                                          [  OK  ]

]

3. Ports it listen to;

111/tcp  open  rpcbind
2049/tcp open  nfs

4. Open the relevant ports in the firewall so that they are accessible;

# iptables -I INPUT 4 -p tcp --dport 111 -j ACCEPT
# iptables -I INPUT 4 -p tcp --dport 2049 -j ACCEPT

Note: Number '4' is the position in the INPUT chain where these records are inserted, could be different in your case. Find that out using "iptables -L INPUT --line-numbers"

5. Create a NFS share;
mkdir /nfsshare

6. Share the directory with NFS client (172.16.20.9):

vi /etc/exports
/nfsshare 172.16.20.9(rw,sync,no_root_squash)


B. On the Client (172.16.20.9):

1. Install the packages
        #yum install nfs*

2. Start NFS service;
        # service nfs start

3. You may check all the rpcservices information on the server using;
         # rpcinfo -p 172.16.20.8

2. Create the mount point to mount the NFS share;
# mkdir /opt/nfs_share

3. Mount the NFS share directory from the NFS server to the client;
# mount -t nfs 172.16.20.8:/nfsshare/ /opt/nfs_share/

4. Check with mount command;
# mount
/dev/vda1 on / type ext4 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw,rootcontext="system_u:object_r:tmpfs_t:s0")
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
nfsd on /proc/fs/nfsd type nfsd (rw)
172.16.20.8:/nfsshare/ on /opt/nfs_share type nfs (rw,vers=4,addr=172.16.20.8,clientaddr=172.16.20.9)

5. On the client, one may also check with 'nfsstat -m' command (displays details about nfs mounts)

# nfsstat -m
/opt/nfs_share from 172.16.20.8:/nfsshare/
Flags:
rw,relatime,vers=4,rsize=262144,wsize=262144,namlen=255,hard,proto=tcp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=172.16.20.9,minorversion=0,local_lock=none,addr=172.16.20.8

Now, create files or directories under /opt/nfs/ (on the client) or /nfsshare on the server to see if they are visible/accessible to each other.

C. NFS Options:

1. The following would provide statistics about NFS server and client respectively;
nfsstat -s
nfsstat -c

2. Important configuration files (all man page info);

/etc/sysconfig/nfs
/etc/nfsmount.conf ---->  Configuration  file  for  NFS  mounts  that  allows options to be set
                                             globally, per server or per mount point.
/etc/exports     ---->  Contains  a table of local physical file systems on an NFS server that
                                             are accessible to NFS clients.

3. Any changes to the /etc/exports would only require to have NFS services reloaded to
        make the changes effective. This does not require clients to unmount NFS share or the
        any service restarts on clients and server.

eg.

Thus, an entry like the following in /etc/exports;
/nfsshare 172.16.20.9(rw,sync,no_root_squash)

when changed to ('rw' to 'ro');
/nfsshare 172.16.20.9(ro,sync,no_root_squash)

It only needs 'service nfs restart' on the NFS Server to make the changes effective.


D. NFS share options:

1. ro:    With the help of this option we can provide read only access to the shared files i.e client
                will only be able to read.

2. rw:   This option allows the client server to both read and write access within the shared
                directory.

3. sync:   Sync confirms requests to the shared directory only once the changes have been
                  committed.

4. no_subtree_check:   This option prevents the subtree checking. When a shared directory
                                       is the subdirectory of a larger file system, nfs performs scans of every
                                       directory above it, in order to verify its permissions and details.

                   Disabling the subtree check may increase the reliability of NFS, but reduce security.

5.     no_root_squash:    This phrase allows root to connect to the designated directory.
                                     Thus, remote  "root" user on the client would be treated a                                  
                                     local "root" user on the server.
                                      The opposite is "root_squash".

Issues:
1. # showmount -e
clnt_create: RPC: Unknown host

Solution: Check using;

                # showmount -e localhost
         Export list for localhost:
        /nfsshare 172.16.20.9

         Now, provide a hostname entry with IP in the /etc/hosts as;
         172.16.20.8 ins-1

         Where, ins-1 is the hostname as revealed by;
           # hostname
           ins-1

Installing Java-7 and Apache Tomcat/8.0.12 on CentOS-6.5

I. JAVA:

1. Search JDK in the yum repositories;

# yum search java | grep -i --color JDK
ldapjdk-javadoc.x86_64 : Javadoc for ldapjdk
icedtea-web.x86_64 : Additional Java components for OpenJDK - Java browser
java-1.6.0-openjdk.x86_64 : OpenJDK Runtime Environment
java-1.6.0-openjdk-demo.x86_64 : OpenJDK Demos
java-1.6.0-openjdk-devel.x86_64 : OpenJDK Development Environment
java-1.6.0-openjdk-javadoc.x86_64 : OpenJDK API Documentation
java-1.6.0-openjdk-src.x86_64 : OpenJDK Source Bundle
java-1.7.0-openjdk.x86_64 : OpenJDK Runtime Environment
java-1.7.0-openjdk-demo.x86_64 : OpenJDK Demos
java-1.7.0-openjdk-devel.x86_64 : OpenJDK Development Environment
java-1.7.0-openjdk-javadoc.noarch : OpenJDK API Documentation
java-1.7.0-openjdk-src.x86_64 : OpenJDK Source Bundle
ldapjdk.x86_64 : The Mozilla LDAP Java SDK

2. Once found, install the required version (if the search does yield any result then please download
         and install manually).

# yum install java-1.7.0-openjdk java-1.7.0-openjdk-devel

3. Check if there are already installed versions of Java;

# alternatives --config java

There is 1 program that provides 'java'.

Selection    Command
-----------------------------------------------
*+ 1           /usr/lib/jvm/jre-1.7.0-openjdk.x86_64/bin/java

Enter to keep the current selection[+], or type selection number:

4. Set JAVA_HOME varaiable and include Java binaries in the system PATH;

# vi /etc/profile

## Set JAVA_HOME
JAVA_HOME="/usr/lib/jvm/jre-1.7.0-openjdk.x86_64"
export JAVA_HOME

## Set JAVA Path
PATH=$PATH:/usr/lib/jvm/jre-1.7.0-openjdk.x86_64/bin

5. Execute the file so that the new changes are reflected on the system;
# source /etc/profile

6. Test Java:
A. # echo $JAVA_HOME
/usr/lib/jvm/jre-1.7.0-openjdk.x86_64

B. # cat > HelloWorld.java
  public class HelloWorld {
public static void main(String[] args) {
         System.out.println("Hello, World! ");
    }
  }

# javac HelloWorld.java
# java HelloWorld
Hello, World!

II. Apache Tomcat

1. Download Apache Tomcat/8.0.12:
# wget http://apache.mirrors.pair.com/tomcat/tomcat-8/v8.0.12/bin/apache-tomcat-8.0.12.tar.gz

2. Extract:
# tar -xvzf apache-tomcat-8.0.12.tar.gz

3. I moved the extracted directory under /usr/local;
# mv apache-tomcat-8.0.12 /usr/local/

4. # /usr/local/apache-tomcat-8.0.12/bin/startup.sh
Using CATALINA_BASE:   /usr/local/apache-tomcat-8.0.12
Using CATALINA_HOME:   /usr/local/apache-tomcat-8.0.12
Using CATALINA_TMPDIR: /usr/local/apache-tomcat-8.0.12/temp
Using JRE_HOME:        /usr/lib/jvm/jre-1.7.0-openjdk.x86_64
Using CLASSPATH:       /usr/local/apache-tomcat-8.0.12/bin/bootstrap.jar:/usr/local/apache-
        tomcat-8.0.12/bin/tomcat-juli.jar
Tomcat started.

Setting up OpenVPN Server on CentOS-6.5 (running as a Openstack cloud instance)

I. Setting up OpenVPN Server on CentOS-6.5 (running as a Openstack cloud instance). The setup may just work the same on other *nix based systems.

A. OpenVPN Server

1. Download and Install the EPEL repo which would provide OpenVPN rpm;
(Epel repo: http://dl.fedoraproject.org/pub/epel/6/x86_64/)
# wget http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm .
# rpm -ivh epel-release-6-8.noarch.rpm

2. Install openvpn and easy-rsa (RSA key management package)
# yum -y install openvpn easy-rsa

3. # mkdir -p /etc/openvpn/easy-rsa/keys
# cp -rf /usr/share/easy-rsa/2.0/* /etc/openvpn/easy-rsa/

B. Generate CA Certificate and CA key

4. Open /etc/openvpn/easy-rsa/vars and set the following attributes which we would use later on Certificates and keys;
# vi /etc/openvpn/easy-rsa/vars

export KEY_COUNTRY="IN"
export KEY_PROVINCE="WB"
export KEY_CITY="KOL"
export KEY_ORG="IT"
export KEY_EMAIL="tu.bijit@gmail.com"
export KEY_OU="MyOrganizationalUnit"

5. Copy the required OpenSSL configuration file which OpenVPN might fail to detect;
# cd /etc/openvpn/easy-rsa/
# cp openssl-1.0.0.cnf openssl.cnf

6. Now, lets build our Certificate Authority (CA) based on the information provided in /etc/openvpn/easy-rsa/vars.
# source ./vars
# ./clean-all
# ./build-ca

7. Generate certificate for the OpenVPN server. Respond "Yes" when prompted for build-key-server to commit.

# ./build-key-server server

8. Generate the same for Client (which we would be copying to client later):
# ./build-key client

9. Generate Diffie Hellman key exchange files using the build-dh script;
# ./build-dh

10. In the above step, the necessary keys and certificates will be generated in the /etc/openvpn/easy-rsa/keys/ directory. Copy the following certificate and key files to the /etc/openvpn/ directory.

ca.crt
dh2048.pem  (please make changes in the server.conf when needed)
server.crt
server.key

# cd /etc/openvpn/easy-rsa/keys/; cp ca.crt dh2048.pem server.crt server.key /etc/openvpn/

11. Copy all client certificates and keys to the remote VPN clients in order to authenticate to the VPN server. Since, we have generated certificates and keys for only one client, so we have to copy the following files (or its contents, keep the name exactly the same) to the VPN client.

ca.crt
client.crt
client.key

C. Configuring VPN Server

12. OpenVPN comes with a sample configuration file, copy that file to /etc/openvpn;

# cp /usr/share/doc/openvpn-2.3.2/sample/sample-config-files/server.conf /etc/openvpn/

15. Open /etc/openvpn/server.conf and make the follwoing changes;

Do the following;
i. Uncomment ( This is route all the traffic from Client through VPN):
push "redirect-gateway def1 bypass-dhcp"

ii. Use VPN to use opendns servers:
push "dhcp-option DNS 8.8.8.8"
push "dhcp-option DNS 8.8.4.4"

iii. Uncomment the following to reduce the OpenVPN daemon's privileges after initialization.

user nobody
group nobody

D. IP forwarding and routing Configuration:

13. vi /etc/sysctl.conf
Find the following line and set value “1” to enable IP forwarding.

# Controls IP packet forwarding
net.ipv4.ip_forward = 1

Run the following command to apply the sysctl changes.
# sysctl -p

14. iptables rules:

i.    Allow proper routing of our VPN subnet,assuming 10.8.0.0/24 is used in /etc/openvpn/server.conf
# iptables -t nat -A POSTROUTING -s 10.8.0.0/24 -o eth0 -j MASQUERADE

ii. Allow port 1194 of openvpn to let traffic come in (iptables -L --line-numbers; iptables -I INPUT -p udp --dport 1194 -j ACCEPT)
# iptables -I INPUT 4 -p udp --dport 1194 -j ACCEPT
# service iptables save

15. # service openvpn start

16.  Check if the OpenVPN is listening on port 1194 (UDP port by default, you can change that to TCP in /etc/server.conf)
# netstat -uapn | grep openvpn
udp        0      0 0.0.0.0:1194                0.0.0.0:*                               11577/openvpn


17. Check interface VPN is using (tun0 is in our case, "dev tun" will create a routed IP tunnel,);
# ip addr show
1: lo: mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
  valid_lft forever preferred_lft forever
2: eth0: mtu 1450 qdisc pfifo_fast state UP qlen 1000
link/ether fa:16:3e:7e:11:4a brd ff:ff:ff:ff:ff:ff
inet 172.16.20.8/24 brd 172.16.20.255 scope global eth0
inet6 fe80::f816:3eff:fe7e:114a/64 scope link
  valid_lft forever preferred_lft forever
3: tun0: mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 100
link/[65534]
inet 10.8.0.1 peer 10.8.0.2/32 scope global tun0

18. In certain cases, once VPN client connects to the VPN server, one might face loss of internet connectivity (was in my case); Following IPTABLES rules would be required to proper routing (as client changes the Gateway to VPN, which should be properly forwarded to access the LAN on the VPN server (you may use the specific LAN IP in the destination, here I used entire network in the destination)

    # Allow traffic initiated from VPN to access LAN
 iptables -I FORWARD -i tun0 -o eth0 -s 10.8.0.0/24 -m conntrack --ctstate NEW -j ACCEPT

    # Allow established traffic to pass back and forth
iptables -I FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT

    # service iptables save


II. OpenVPN Client (On Windows)

1. On Windows, one can download OpenVPN GUI for Windows;
http://openvpn.se/

2. Make sure we copied the following files from the server which were generated using "./build-key" earlier.

ca.crt
client.crt
client.key

3. Create a new file called client.ovpn, where "client" should match the name of the client being deployed (from build-key) with following contents;

remote x.x.x.x 1194
# Path to certificates (in windows as follows)
ca "C:\\MyVPN\\ca.crt"
cert "C:\\MyVPN\\client.crt"
key "C:\\MyVPN\\client.key"

4. Save and connect to VPN server using client config file.


III. OpenVPN Client (On Linux, CentOS release 6.5 (Final): 1. Download and install OpenVPN package. You would be required to setup EPEL repo as discussed 
before. 2. Copy the sample Openvpn client configuration to your desired locaton; # cp /usr/share/doc/openvpn-2.3.2/sample/sample-config-files/client.conf . 3. Copy the following files from the server which were generated using "./build-key" earlier. ca.crt client.crt client.key 4. Provide the following information in "client.conf"; remote x.x.x.x 1194 ca ca.crt cert client.crt key client.key 5. Connect to the OpenVPN server as; # openvpn client.conf &