Get your server issues fixed by our experts for a price starting at just 25 USD/Hour. Click here to register and open a ticket with us now!

Author Topic: How to Install Distributed File System "GlusterFS" and setup  (Read 3145 times)

0 Members and 1 Guest are viewing this topic.


  • Guest
How to Install Distributed File System "GlusterFS" and setup
« on: February 03, 2014, 11:49:37 am »
The Gluster File System (GlusterFS) is an open source distributed file system that can scale out in building-block fashion to store multiple petabytes of data.

The clustered file system pools storage servers over TCP/IP or InfiniBand Remote Direct Memory Access (RDMA), aggregating disk and memory and facilitating the centralized management of data through a unified global namespace. The software works with low-cost commodity computers.

Use cases for GlusterFS include cloud computing, streaming media and content delivery.


In my setup I am using below configuration  or setup.

Storage server:-
Node1.linux.vs (
Node2.linux.vs (

Client to use gluster file system:-
Client.linux.vs (

On each server create the lvm partition and format them using xfs filesystem, however we can use the another filesystem as well (ext3,4) and Name resolution should be working.

Lvm creation process:-
Code: [Select]
# fdisk /dev/sdb
# pvcreate /dev/sdb1
#vgcreate vg_storage /dev/sdb1
# lvcreate –L 10G –n lv_home vg_storage
#mkfs.xfs –i size=512 /dev/vg_storage/lv_home
#mkdir /mnt/data/node1
#mount /dev/vg_storage/lv_home /mnt/data/node1

Note:- Same process follow on other storage nodes.

Installation of glusterFs

Enable EPEL Repository

Before Installing GlusterFS on the servers we need to install epel repository on our system depending on our operating system version using following command.

Code: [Select]
CentOS/RHEL 5 , 32 bit:
# rpm -Uvh

CentOS/RHEL 5 , 64 bit:
# rpm -Uvh

CentOS/RHEL 6 , 32 bit:
# rpm -Uvh

CentOS/RHEL 6 , 64 bit:
# rpm -Uvh

Now we have to install the GlusterFS on all nodes.

Code: [Select]
#yum install glusterfs*

Below list of installed packages


Note:- Install the gluster packages on each storage node.
After installation, it's time to start the glusterd service on each storage node.

Code: [Select]
#/etc/init.d/glusterd start         
#chkconfig glusterd on

Setting up trusted storage server pools
Code: [Select]
#gluster peer status
No peers present

Command show the status of trusted storage pool
Because still we do not add any storage server. So the will be empty.

Now we need to add the other nodes

Code: [Select]
#gluster peer probe node2.linux.vs
#gluster peer probe node3.linux.vs
#gluster peer probe node4.linux.vs

Note:- We don’t need to add node1 because we are on this node and it will automatically part of storage cluster.
Code: [Select]
#gluster peer status
Number of Peers: 3
Hostname: node2
Uuid: 5e987bda-16dd-43c2-835b-08b7d55e94e5
State: Peer in Cluster (Connected)
Hostname: node3
Uuid: 1e0ca3aa-9ef7-4f66-8f15-cbc348f29ff7
State: Peer in Cluster (Connected)
Hostname: node4
Uuid: 3e0caba-9df7-4f66-8e5d-cbc348f29ff7
State: Peer in Cluster (Connected)

Setting Up GlusterFS server Volume

Code: [Select]
#gluster volume create Vol1 node1:/mnt/data/node1 node2:/mnt/data/node2 node3:/mnt/data/node3 node4:/mnt/data/node4
#gluster volume info
Volume Name: Vol1
Type: Distribute
Status: Started
Number of Bricks: 4
Transport-type: tcp
Brick1: node1:/mnt/data/node1
Brick2: node2:/mnt/data/node2
Brick3: node3:/mnt/data/node3
Brick4: node4:/mnt/data/node4
#gluster volume start Vol1

Now our gluster storage server ready now we can mount it on client side.
Information about peer,node,vols available
Code: [Select]
# cd /var/lib/glusterd
Configure on GlusterFS Client to use a volume on GlusterFS servers

Code: [Select]
#yum install glusterfs-fuse glusterfs

Code: [Select]
#showmount –e node1
It will show you volume name which we shared from gluster storage servers.
Code: [Select]
#mkdir /storage
#mount.glusterfs node1:/Vol1  /storage
#touch /storage/file{1,2,3,4}

Now the file will available on node1,node2,node3,node4
because it is distributed storage volume.