gluster.infra

The gluster.infra role allows the user to deploy a GlusterFS cluster. It has sub-roles which can be invoked by setting the variables. The sub-roles are

1. firewall_config: Set up firewall rules (open ports, add services to zone)
2. backend_setup: Create VDO volume (If vdo is selected) Create volume groups, logical volumes (thinpool, thin lv, thick lv) Create xfs filesystem Mount the filesystem

Requirements

Ansible version 2.5 or above GlusterFS version 3.2 or above VDO utilities (Optional)

Role Variables

These are the superset of role variables. They are explained again in the respective sub-roles directory.

firewall_config

---

Name	Choices	Default value	Comments
gluster_infra_fw_state	enabled / disabled / present / absent	UNDEF	Enable or disable a setting. For ports: Should this port accept(enabled) or reject(disabled) connections. The states "present" and "absent" can only be used in zone level operations (i.e. when no other parameters but zone and state are set).
gluster_infra_fw_ports		UNDEF	A list of ports in the format PORT/PROTO. For example 111/tcp. This is a list value.
gluster_infra_fw_permanent	true/false	true	Whether to make the rule permanenet.
gluster_infra_fw_zone	work / drop / internal / external / trusted / home / dmz / public / block	public	The firewalld zone to add/remove to/from
gluster_infra_fw_services		UNDEF	Name of a service to add/remove to/from firewalld - service must be listed in output of firewall-cmd --get-services. This is a list variable

backend_setup

---

Name	Choices	Default value	Comments
gluster_infra_vdo		UNDEF	Mandatory argument if vdo has to be setup. Key/Value pairs have to be given. See examples for syntax.
gluster_infra_disktype	JBOD / RAID6 / RAID10	UNDEF	Backend disk type.
gluster_infra_diskcount		UNDEF	RAID diskcount, can be ignored if disktype is JBOD
gluster_infra_volume_groups			Mandatory variable, key/value pairs of vgname and pvname. pvname can be comma-separated values if more than a single pv is needed for a vg. See below for syntax.
gluster_infra_stripe_unit_size		UNDEF	Stripe unit size (KiB). DO NOT including trailing 'k' or 'K'
gluster_infra_lv_poolmetadatasize		16G	Metadata size for LV, recommended value 16G is used by default. That value can be overridden by setting the variable. Include the unit [G|M|K]
gluster_infra_thinpools			Thinpool data. This is a dictionary with keys vgname, thinpoolname, thinpoolsize, and poolmetadatasize. See below for syntax and example.
gluster_infra_lv_logicalvols		UNDEF	This is a list of hash/dictionary variables, with keys, lvname and lvsize. See below for example.
gluster_infra_thick_lvs		UNDEF	Optional. Needed only if thick volume has to be created. This is a dictionary with vgname, lvname, and size as keys. See below for example
gluster_infra_mount_devices		UNDEF	This is a dictionary with mount values. path, vgname, and lvname are the keys.
gluster_infra_cache_vars		UNDEF	This variable contains list of dictionaries for setting up LV cache. Variable has following keys: vgname, cachedisk, cachethinpoolname, cachelvname, cachelvsize, cachemetalvname, cachemetalvsize, cachemode. The keys are explained in more detail below

VDO Variable

---

If the backend disk has to be configured with VDO the variable gluster_infra_vdo has to be defined.

Name	Choices	Default value	Comments
gluster_infra_vdo		UNDEF	Mandatory argument if vdo has to be setup. Key/Value pairs have to be given. gluster_infra_vdo supports the keys explained below, see examples for syntax.

For Example:
gluster_infra_vdo:
   - { name: 'hc_vdo_1', device: '/dev/vdb' }
   - { name: 'hc_vdo_2', device: '/dev/vdc' }
   - { name: 'hc_vdo_3', device: '/dev/vdd' }

The gluster_infra_vdo variable supports the following keys:

VDO Key	Default value	Comments
state	present	VDO state, if present VDO will be created, if absent VDO will be deleted.
activated	'yes'	Whether VDO has to be activated upon creation.
running	'yes'	Whether VDO has to be started
logicalsize	UNDEF	Logical size for the vdo
compression	'enabled'	Whether compression has to be enabled
blockmapcachesize	'128M'	The amount of memory allocated for caching block map pages, in megabytes (or may be issued with an LVM-style suffix of K, M, G, or T). The default (and minimum) value is 128M.
readcache	'disabled'	Enables or disables the read cache.
readcachesize	0	Specifies the extra VDO device read cache size in megabytes.
emulate512	'off'	Enables 512-byte emulation mode, allowing drivers or filesystems to access the VDO volume at 512-byte granularity, instead of the default 4096-byte granularity.
slabsize	'2G'	The size of the increment by which the physical size of a VDO volume is grown, in megabytes (or may be issued with an LVM-style suffix of K, M, G, or T). Must be a power of two between 128M and 32G.
writepolicy	'sync'	Specifies the write policy of the VDO volume. The 'sync' mode acknowledges writes only after data is on stable storage.
indexmem	'0.25'	Specifies the amount of index memory in gigabytes.
indexmode	'dense'	Specifies the index mode of the Albireo index.
ackthreads	'1'	Specifies the number of threads to use for acknowledging completion of requested VDO I/O operations. Valid values are integer values from 1 to 100 (lower numbers are preferable due to overhead). The default is 1.
biothreads	'4'	Specifies the number of threads to use for submitting I/O operations to the storage device. Valid values are integer values from 1 to 100 (lower numbers are preferable due to overhead). The default is 4.
cputhreads	'2'	Specifies the number of threads to use for CPU-intensive work such as hashing or compression. Valid values are integer values from 1 to 100 (lower numbers are preferable due to overhead). The default is 2.
logicalthreads	'1'	Specifies the number of threads across which to subdivide parts of the VDO processing based on logical block addresses. Valid values are integer values from 1 to 100 (lower numbers are preferable due to overhead). The default is 1.
physicalthreads	'1'	Specifies the number of threads across which to subdivide parts of the VDO processing based on physical block addresses. Valid values are integer values from 1 to 16 (lower numbers are preferable due to overhead). The physical space used by the VDO volume must be larger than (slabsize * physicalthreads). The default is 1.

Volume Groups variable

---

Name	Choices	Default value	Comments
gluster_infra_volume_groups		UNDEF	This is a list of hash/dictionary variables, with keys, vgname and pvname. See below for example.

For Example:
gluster_infra_volume_groups:
   - { vgname: 'volgroup1', pvname: '/dev/sdb' }
   - { vgname: 'volgroup2', pvname: '/dev/mapper/vdo_device1' }
   - { vgname: 'volgroup3', pvname: '/dev/sdc,/dev/sdd'

Logical Volume variable

---

Name	Choices	Default value	Comments
gluster_infra_lv_logicalvols		UNDEF	This is a list of hash/dictionary variables, with keys, lvname, vgname, thinpool, and lvsize. See below for example.

For Example:
gluster_infra_lv_logicalvols:
   - { vgname: 'vg_vdb', thinpool: 'foo_thinpool', lvname: 'vg_vdb_thinlv', lvsize: '500G' }
   - { vgname: 'vg_vdc', thinpool: 'bar_thinpool', lvname: 'vg_vdc_thinlv', lvsize: '500G' }

Thick LV variable

---

Name	Choices	Default value	Comments
gluster_infra_thick_lvs		UNDEF	This is a list of hash/dictionary variables, with keys, vgname, lvname, and size. See below for example.

For Example:
gluster_infra_thick_lvs:
   - { vgname: 'vg_sdb', lvname: 'thick_lv_1', size: '500G' }
   - { vgname: 'vg_sdc', lvname: 'thick_lv_2', size: '100G' }

Thinpool variable

---

Name	Choices	Default value	Comments
gluster_infra_thinpools		UNDEF	This is a list of hash/dictionary variables, with keys: vgname, thinpoolname, thinpoolsize, and poolmetadatasize. See below for example.

gluster_infra_thinpools:
  - {vgname: 'vg_vdb', thinpoolname: 'foo_thinpool', thinpoolsize: '10G', poolmetadatasize: '1G' }
  - {vgname: 'vg_vdc', thinpoolname: 'bar_thinpool', thinpoolsize: '20G', poolmetadatasize: '1G' }

• poolmetadatasize: Metadata size for LV, recommended value 16G is used by default. That value can be overridden by setting the variable. Include the unit [G|M|K]
• thinpoolname: Can be ignored, a name is formed using the given vgname followed by '_thinpool'
• vgname: Name of the volume group this thinpool should belong to.

Variables for setting up cache

---

Name	Choices	Default value	Comments
gluster_infra_cache_vars		UNDEF	This is a dictionary with keys: vgname, cachedisk, cachethinpoolname, cachelvname, cachelvsize, cachemetalvname, cachemetalvsize, cachemode

vgname - The vg which will be extended to setup cache.
cachedisk - The SSD disk which will be used to setup cache. Complete path, for eg: /dev/sdd
cachethinpoolname - The existing thinpool on the volume group mentioned above.
cachelvname - Logical volume name for setting up cache, an lv with this name is created.
cachelvsize - Cache logical volume size
cachemetalvname - Meta LV name.
cachemetalvsize - Meta LV size
cachemode - Cachemode, default is writethrough.

For example:

   - { vgname: 'vg_vdb', cachedisk: '/dev/vdd', cachethinpoolname: 'foo_thinpool', cachelvname: 'cachelv', cachelvsize: '20G', cachemetalvname: 'cachemeta', cachemetalvsize: '100M', cachemode: 'writethrough' }

Variables for mounting the filesystem

---

Name	Choices	Default value	Comments
gluster_infra_mount_devices		UNDEF	This is a dictionary with mount values. path, vgname, and lvname are the keys.

For example:
gluster_infra_mount_devices:
        - { path: '/mnt/thinv', vgname: <vgname>, lvname: <lvname> }

Example Playbook

Configure the ports and services related to GlusterFS, create logical volumes and mount them.

---
- name: Setting up backend
  remote_user: root
  hosts: gluster_servers
  gather_facts: false

  vars:
     # Firewall setup
     gluster_infra_fw_ports:
        - 2049/tcp
        - 54321/tcp
        - 5900/tcp
        - 5900-6923/tcp
        - 5666/tcp
        - 16514/tcp
     gluster_infra_fw_services:
        - glusterfs

     # Set a disk type, Options: JBOD, RAID6, RAID10
     gluster_infra_disktype: RAID6

     # RAID6 and RAID10 diskcount (Needed only when disktype is raid)
     gluster_infra_diskcount: 10
     # Stripe unit size always in KiB
     gluster_infra_stripe_unit_size: 128

     # Variables for creating volume group
     gluster_infra_volume_groups:
       - { vgname: 'vg_vdb', pvname: '/dev/vdb' }
       - { vgname: 'vg_vdc', pvname: '/dev/vdc' }

     # Create thinpools
     gluster_infra_thinpools:
       - {vgname: 'vg_vdb', thinpoolname: 'foo_thinpool', thinpoolsize: '100G', poolmetadatasize: '16G'}
       - {vgname: 'vg_vdc', thinpoolname: 'bar_thinpool', thinpoolsize: '500G', poolmetadatasize: '16G'}

     # Create a thin volume
     gluster_infra_lv_logicalvols:
       - { vgname: 'vg_vdb', thinpool: 'foo_thinpool', lvname: 'vg_vdb_thinlv', lvsize: '500G' }
       - { vgname: 'vg_vdc', thinpool: 'bar_thinpool', lvname: 'vg_vdc_thinlv', lvsize: '500G' }

     # Mount the devices
     gluster_infra_mount_devices:
       - { path: '/mnt/brick1', vgname: 'vg_vdb', lvname: 'vg_vdb_thinlv' }
       - { path: '/mnt/brick2', vgname: 'vg_vdc', lvname: 'vg_vdc_thinlv' }

  roles:
     - gluster.infra

See also: https://github.com/gluster/gluster-ansible-infra/tree/master/playbooks

License

GPLv3