RHCS cluster using ISCSI

  • PDF
  • Print
  • E-mail

In this case I want to create a redhat cluster using iscsi disks and RHCS.
The goal is to have a service which depend on IP and disk to failover between 2 nodes

 

Following IP adresses where used:
iscshi-host: 192.168.10.31
cluster node 1: 192.168.10.32
cluster node 2: 192.168.10.33

First I setup the iscsi host
I setup 1vcpu, 385mb ram and a root disk of 8G using free vmware server
Install a standard centos node (in my case ver 5.5 64 bit)
during install choose to install gnome, server and servergui packages

After install I disabled firewall and SElinux

when the system is ready do the following to share a 512mb file as an iscsi disk to the local network
for data and a 25mb file used for quorum in the cluster

1. install the needed software
   # yum install iscsi-initiator-utils
   # yum install openssl openssl-devel
   # yum install scsi-target-utils
2. Enable and start iscsi target software
   # chkconfig tgtd on
   # service tgtd start
3. Create the 2 files to be shared as iscsi disks
   # mkdir /iscsishares
   # dd if=/dev/zero of=/iscsishares/LUN0 bs=1M count=512
   # dd if=/dev/zero of=/iscsishares/LUN1 bs=1M count=25
4. Create the targets
   # tgtadm --lld iscsi --op new --mode target --tid 1 -T iqn.2010-11.iscsihost:storage.disk1.sys1.xyz
   # tgtadm --lld iscsi --op new --mode target --tid 2 -T iqn.2010-11.iscsihost:storage.disk2.sys1.xyz
5. Add your LUN to the target
   # tgtadm --lld iscsi --op new --mode logicalunit --tid 1 --lun 1 -b /iscsishares/LUN0
   # tgtadm --lld iscsi --op new --mode logicalunit --tid 2 --lun 1 -b /iscsishares/LUN1
6. Make LUN available from all initiators
   # tgtadm --lld iscsi --op bind --mode target --tid 1 -I ALL
  
# tgtadm --lld iscsi --op bind --mode target --tid 2 -I ALL
7. check it works by issuing
   # tgtadm --lld iscsi --op show --mode target
8. make it sticky by inserting the three lines in /etc/rc.local
   tgtadm --lld iscsi --op new --mode target --tid 1 -T iqn.2010-11.iscsihost:storage.disk1.sys1.xyz
   tgtadm --lld iscsi --op new --mode target --tid 2 -T iqn.2010-11.iscsihost:storage.disk2.sys1.xyz
   tgtadm --lld iscsi --op new --mode logicalunit --tid 1 --lun 1 -b /iscsishares/LUN0
   tgtadm --lld iscsi --op new --mode logicalunit --tid 2 --lun 1 -b /iscsishares/LUN1
   tgtadm --lld iscsi --op bind --mode target --tid 1 -I ALL
   tgtadm --lld iscsi --op bind --mode target --tid 2 -I ALL

Install both clusternodes - make sure you choose following packages: gnome, server, servergui, clustering, storage clustering
My test servers have this footprint: 1vcpu, 385mb ram and a root disk of 8G

Check that ALL 3 servers are using ntp and that they agree on time/date

Install and configure iscsi initiator software on both clusternodes
   # yum install iscsi-initiator-utils
   # service iscsi start
   # service iscsid start

Discover iscsi devices (you should see your 2 devices):
   # iscsiadm --mode discovery --type sendtargets --portal 192.168.10.31

import the iscsi devices
   # iscsiadm -m node -T iqn.2010-11.iscsihost:storage.disk1.sys1.xyz -p 192.168.10.31 -l
   # iscsiadm -m node -T iqn.2010-11.iscsihost:storage.disk2.sys1.xyz -p 192.168.10.31 -l

When that succeeds make sure it happens automatically.
   # iscsiadm -m node -T iqn.2010-11.iscsihost:storage.disk1.sys1.xyz -p 192.168.10.31 -o update -n node.startup -v automatic
   # iscsiadm -m node -T iqn.2010-11.iscsihost:storage.disk2.sys1.xyz -p 192.168.10.31 -o update -n node.startup -v automatic

You will be able to see your new devices in /proc/partitions

Now make sure that both clusternodes have themselves and each other in /etc/hosts eg:
   # Do not remove the following line, or various programs
   # that require network functionality will fail.
   127.0.0.1       localhost.localdomain localhost
   192.168.10.32   cluster1
   192.168.10.33   cluster2
   ::1             localhost6.localdomain6 localhost6


Now you are ready to play around with cluster software ... finally

Start by creating a basic /etc/cluster/cluster.conf (needed for the daemons to start)
I have attached a real basic one with a cluster called skau-cluster and with both my clusternodes.
this file must be EXACTLY the same on both clusternodes

<?xml version="1.0" ?>
<cluster config_version="3" name="skau-cluster">
        <fence_daemon post_fail_delay="0" post_join_delay="3"/>
        <clusternodes>
                <clusternode name="cluster1" nodeid="1" votes="1">
                        <fence/>
                </clusternode>
                <clusternode name="cluster2" nodeid="2" votes="1">
                        <fence/>
                </clusternode>
        </clusternodes>
        <cman expected_votes="1" two_node="1"/>
        <fencedevices/>
        <rm>
                <failoverdomains/>
                <resources/>
        </rm>
</cluster>


Start the necessary cluster daemons
   # chkconfig cman on
   # chkconfig rgmanager on
   # chkconfig clvmd on
   # chkconfig qdiskd on
   # service cman start
   # service rgmanager start
   # service clvmd start
   # service qdiskd start

What we now have are 3 devices on each cluster node - sda which is the / device, sdb 512MB for data and sdc
25mb for quorum

Lets start out by setting up quorum
Now the reason you need quorum is to maintain a host as cluster master - it works by using votes, in my setup
both nodes have 1 vote, in order to get vote majority they race to get ownership of the quorum device which also
holds a vote.
quorum devices are only necessary in two node clusters.
Read more

first initialize the quorumdisk for qdisk use (only do this on 1 node)
   # mkqdisk -c /dev/sdc -l qdisk

Check on both nodes that you see the quorum device
   # mkqdisk -L

Because our cluster daemon cman depends on a quorum device, we need to make sure that quorum daemon starts before cman, I don't know why redhat has'nt done this themselves, but simply change order on both nodes like this:

   #  mv /etc/rc3.d/S22qdiskd /etc/rc3.d/S20qdiskd

Now make your /etc/cluster/cluster.conf reflect the usage of quorum device

<?xml version="1.0"?>
<cluster alias="skau-cluster" config_version="6" name="skau-cluster">
        <quorumd interval="3" tko="23" label="qdisk" votes="3"/>
        <fence_daemon post_fail_delay="0" post_join_delay="3"/>
        <clusternodes>
                <clusternode name="cluster1" nodeid="1" votes="1">
                        <fence/>
                </clusternode>
                <clusternode name="cluster2" nodeid="2" votes="1">
                        <fence/>
                </clusternode>
        </clusternodes>
        <cman expected_votes="1" two_node="1"/>
        <fencedevices>
                <fencedevice agent="fence_scsi" name="qdisk"/>
        </fencedevices>
        <rm>
                <failoverdomains/>
                <resources/>
        </rm>
</cluster>

Check that you have 2 cluster members:

[root@cluster1 ~]# clustat
Cluster Status for skau-cluster @ Wed Nov 17 05:54:23 2010
Member Status: Quorate

 Member Name                             ID   Status
 ------ ----                             ---- ------
 cluster1                                    1 Online, Local
 cluster2                                    2 Online
 /dev/disk/by-id/scsi-1IET_00020001          0 Online, Quorum Disk

From now on you can use the system-config-cluster tool - It is a really easytouse frontend for /etc/cluster/cluster.conf
file
I will do my work in the GUI, but I will also show the XML generated.


 

 
 Iyi günler sevgili sayin sehirciler. Sizin için ugrasta oldugumuz isin ne oldugunu tabikide sizlerde biliyorsunuz. Biz sizlerin saga sola saldirmamasi için bu porno isini yapmaktayiz. Herkes porno isine düsman olmus durumda. Fakat siz kendinizi rahatlatmazsaniz ve disarda kizlara salça olursaniz ya dayak yersiniz yada fitilden ceza yatarsiniz. Bizim amacimiz siz sevgili izleyenlerin isteklerine göre pornolar yayinlayarak elizabeth seklinde bosalmanizdir. Türk porno kategorilerinde zaten güzel türk kizlarinin pornolarini bulabilirsiniz. Iyi Seyirler. Bol katkilar. escort istanbul istanbul escort ankara escort izmir escort Hedefimiz mutlu bir yasam. Mutlu bir yasam derken tabi ki hangi manada söyledigimi anladiginizi düsünüyorum. Bu Mananin içerigi tabiki cinselliktir. Cinsellik hiç bir zaman insanoglu için vazgeçilir bir olay olmamistir. Beynin sikine hükmedemedgi anlarda herkes cinsellik yapmak ister. Beyin fonksiyonlari adeta durur ve yaninda kadin olmayan erkekler veya yaninda erkek olmayan kizlar porno izlemeye karar verirler. Bu pornolarin genelinde ise kizlar ve erkekler türk olduklari için ve amacimiz türkleri mutlu etmek oldugu için türk porno kategorisinden türk porno videolarini izlerler. Simdi bu paylastigimiz siteden sizde porno izleyebilirsiniz. www.polisoyunlar.com www.kamyonoyunlar.comEn kaliteli escort bayanlar icin sitemize bekleriz porno