SlideShare a Scribd company logo
1 of 35
Download to read offline
GlusterFS & OpenStack
Vijay Bellur
Red Hat
Agenda
➢ What is GlusterFS?
➢ Architecture
➢ Concepts
➢ Algorithms
➢ Access Mechanisms
➢ Implementation
➢ OpenStack and GlusterFS
➢Resources
➢ Q&A
GlusterFS Deployment
Global namespace
Scale-out storage
building blocks
Supports
thousands of clients
Access using
GlusterFS native,
NFS, SMB and HTTP
protocols
Runs on
commodity
hardware
GlusterFS concepts

Trusted Storage Pool (cluster) is a collection of storage
servers.

Trusted Storage Pool is formed by invitation – you
“probe” a new member from the cluster and not vice
versa.

Logical partition for all data and management
operations.

Membership information used for determining quorum.

Members can be dynamically added and removed from
the pool.
GlusterFS concepts – Trusted Storage Pool
GlusterFS concepts – Trusted Storage Pool
Node1 Node2
Probe
Probe
accepted
Node 1 and Node 2 are peers in a trusted storage pool
Node2Node1
GlusterFS concepts – Trusted Storage Pool
Node1 Node2 Node3Node2Node1 Trusted Storage Pool
Node3Node2Node1
Detach

A brick is the combination of a node and an export directory – for
e.g. hostname:/dir

Each brick inherits limits of the underlying filesystem

No limit on the number bricks per node

Ideally, each brick in a cluster should be of the same size
/export3 /export3 /export3
Storage Node
/export1
Storage Node
/export2
/export1
/export2
/export4
/export5
Storage Node
/export1
/export2
3 bricks 5 bricks 3 bricks
GlusterFS concepts - Bricks
GlusterFS concepts - Volumes
➢ A volume is a logical collection of bricks.
➢ Volume is identified by an administrator provided name.
➢Volume is a mountable entity and the volume name is
provided at the time of mounting.
➢ mount -t glusterfs server1:/<volname> /my/mnt/point
➢ Bricks from the same node can be part of different
volumes
GlusterFS concepts - Volumes
Node2Node1 Node3
/export/brick1
/export/brick2
/export/brick1
/export/brick2
/export/brick1
/export/brick2 Videos
music
Volume Types
➢Type of a volume is specified at the time of volume
creation
➢ Volume type determines how and where data is placed
➢ Following volume types are supported in glusterfs:
a) Distribute
b) Stripe
c) Replication
d) Distributed Replicate
e) Striped Replicate
f) Distributed Striped Replicate
Distributed Volume
➢Distributes files across various bricks of the volume.
➢Directories are present on all bricks of the volume.
➢Single brick failure will result in loss of data availability.
➢Removes the need for an external meta data server.
How does a distributed volume work?
➢ Distribute works on the basis of a dht algorithm.
➢ A 32-bit hash space is divided into N ranges for N bricks
➢ At the time of directory creation, a range is assigned to each
directory.
➢ During a file creation or retrieval, hash is computed on the file
name. This hash value is used to locate or place the file.
➢Different directories in the same brick end up with different hash
ranges.
How does a distributed volume work?
How does a distributed volume work?
How does a distributed volume work?
Replicated Volume
➢Creates synchronous copies of all directory and file
updates.
➢Provides high availability of data when node failures
occur.
➢Transaction driven for ensuring consistency.
➢Changelogs maintained for re-conciliation.
➢Any number of replicas can be configured.
How does a replicated volume work?
How does a replicated volume work?
Distributed Replicated Volume
➢
Distribute files across replicated bricks
➢
Number of bricks must be a multiple of the replica count
➢
Ordering of bricks in volume definition matters
➢
Scaling and high reliability
➢
Improved read performance in most environments.
➢
Most preferred model of deployment currently.
Distributed Replicated Volume
Striped Volume
➢Files are striped into chunks and placed in various bricks.
➢Recommended only when very large files greater than
the size of the disks are present.
➢Chunks are files with holes – this helps in maintaining
offset consistency.
➢A brick failure can result in data loss. Redundancy with
replication is highly recommended
Translators in GlusterFS
➢Building blocks for a GlusterFS process.
➢Based on Translators in GNU HURD.
➢Each translator is a functional unit.
➢Translators can be stacked together for
achieving desired functionality.
➢ Translators are deployment agnostic – can
be loaded in either the client or server stacks.
VFS
Server
I/O Cache
Distribute / Stripe
POSIX
Ext4 Ext4Ext4
POSIX POSIX
Brick 1
ServerServer
Read Ahead
Brick 2 Brick n-1
Gluster Server
Replicate
Ext4
POSIX
Server
Brick n
Replicate
Customizable GlusterFS Client/Server Stack
Client
Gluster Server
Client
GigE, 10GigE – TCPIP / InfiniBand – RDMA
Gluster ServerGluster Server
Client Client
Dynamic Volume Management
Collectively refers to application transparent operations
that can be performed in the storage layer.
➢ Addition of Bricks to a volume
➢ Remove brick from a volume
➢ Rebalance data spread within a volume
➢ Replace a brick in a volume
➢ Adaptive Performance / Functionality tuning.
Access Mechanisms:
Gluster volumes can be accessed via the following
mechanisms:
➢ FUSE
➢ NFS
➢ SMB
➢ libgfapi
➢ ReST
➢ HDFS
ReST based access
UFO
Client
Proxy Account
Container
Object
HTTP Request
(REST)
Directory
Volume
FileClient
NFS or
GlusterFS Mount
Unified File and object view.
Entity mapping between file and object building blocks
Interface Possibilities
qemu
NFS
SMB
Hadoop
FUSE
Cinder
Swift (UFO)
Files Blocks
Objects
libgfapi
Whatever
IP RDMA
Transports
files BD
Back ends
DB
Interface Possibilities
OpenStack and GlusterFS – Current Integration
Glance
Images
Nova
Nodes
Swift
Objects
Cinder
Data
Glance
Data
Swift
Data
Swift API
Storage
Server
Storage
Server
Storage
Server…
KVM
KVM
KVM
…
● Separate Compute and Storage
Pools
● GlusterFS directly provides Swift
object service
● Integration with Keystone
● GeoReplication for multi-site
support
● Swift data also available via
other protocols
● Supports non-OpenStack use in
addition to OpenStack use
Logical View Physical View
OpenStack and GlusterFS - Future Direction
Hadoop
Guest
Other
Guest
...
Host
Gluster
Guest
Hadoop
Guest
Other
Guest
...
Host
Gluster
Guest
Nova
Compute
Nodes
Open Stack and GlusterFS - Future Direction
● POC based on proposed OpenStack FaaS (File as a
Service) proposal
● Cinder-like virtual NAS service
● Tenant-specific file shares
● Hypervisor mediated for security
● Avoid exposing servers to Quantum tenant network
● Optional multi-site or multi-zone GeoReplication
● FaaS data optionally available to non OpenStack nodes
● Initial focus on Linux guest
● Windows (SMB) and NFS shares also under consideration
Open Stack and GlusterFS - Summary
● Can address storage needs across the stack:
● Block
● Object
● File
● Hadoop/Big Data
Resources
Mailing lists:
gluster-users@gluster.org
gluster-devel@nongnu.org
IRC:
#gluster and #gluster-dev on freenode
Links:
http://www.gluster.org
http://hekafs.org
http://forge.gluster.org
http://www.gluster.org/community/documentation/index.php/Arch
Questions?
Thank You!

More Related Content

What's hot

VMware NSX 101: What, Why & How
VMware NSX 101: What, Why & HowVMware NSX 101: What, Why & How
VMware NSX 101: What, Why & HowAniekan Akpaffiong
 
Microsoft Windows Server 2022 Overview
Microsoft Windows Server 2022 OverviewMicrosoft Windows Server 2022 Overview
Microsoft Windows Server 2022 OverviewDavid J Rosenthal
 
Architecture patterns for distributed, hybrid, edge and global Apache Kafka d...
Architecture patterns for distributed, hybrid, edge and global Apache Kafka d...Architecture patterns for distributed, hybrid, edge and global Apache Kafka d...
Architecture patterns for distributed, hybrid, edge and global Apache Kafka d...Kai Wähner
 
[242]open stack neutron dataplane 구현
[242]open stack neutron   dataplane 구현[242]open stack neutron   dataplane 구현
[242]open stack neutron dataplane 구현NAVER D2
 
VMware Virtual SAN Presentation
VMware Virtual SAN PresentationVMware Virtual SAN Presentation
VMware Virtual SAN Presentationvirtualsouthwest
 
Open stack
Open stackOpen stack
Open stacksvm
 
Room 3 - 7 - Nguyễn Như Phúc Huy - Vitastor: a fast and simple Ceph-like bloc...
Room 3 - 7 - Nguyễn Như Phúc Huy - Vitastor: a fast and simple Ceph-like bloc...Room 3 - 7 - Nguyễn Như Phúc Huy - Vitastor: a fast and simple Ceph-like bloc...
Room 3 - 7 - Nguyễn Như Phúc Huy - Vitastor: a fast and simple Ceph-like bloc...Vietnam Open Infrastructure User Group
 
Room 3 - 6 - Nguyễn Văn Thắng & Dzung Nguyen - Ứng dụng openzfs làm lưu trữ t...
Room 3 - 6 - Nguyễn Văn Thắng & Dzung Nguyen - Ứng dụng openzfs làm lưu trữ t...Room 3 - 6 - Nguyễn Văn Thắng & Dzung Nguyen - Ứng dụng openzfs làm lưu trữ t...
Room 3 - 6 - Nguyễn Văn Thắng & Dzung Nguyen - Ứng dụng openzfs làm lưu trữ t...Vietnam Open Infrastructure User Group
 
Hashicorp Vault - OPEN Public Sector
Hashicorp Vault - OPEN Public SectorHashicorp Vault - OPEN Public Sector
Hashicorp Vault - OPEN Public SectorKangaroot
 
Kubernetes dealing with storage and persistence
Kubernetes  dealing with storage and persistenceKubernetes  dealing with storage and persistence
Kubernetes dealing with storage and persistenceJanakiram MSV
 
OpenShift-Technical-Overview.pdf
OpenShift-Technical-Overview.pdfOpenShift-Technical-Overview.pdf
OpenShift-Technical-Overview.pdfJuanSalinas593459
 
VMware VSAN Technical Deep Dive - March 2014
VMware VSAN Technical Deep Dive - March 2014VMware VSAN Technical Deep Dive - March 2014
VMware VSAN Technical Deep Dive - March 2014David Davis
 
Docker Introduction
Docker IntroductionDocker Introduction
Docker IntroductionSparkbit
 
Automate Your Kafka Cluster with Kubernetes Custom Resources
Automate Your Kafka Cluster with Kubernetes Custom Resources Automate Your Kafka Cluster with Kubernetes Custom Resources
Automate Your Kafka Cluster with Kubernetes Custom Resources confluent
 
Storage Requirements and Options for Running Spark on Kubernetes
Storage Requirements and Options for Running Spark on KubernetesStorage Requirements and Options for Running Spark on Kubernetes
Storage Requirements and Options for Running Spark on KubernetesDataWorks Summit
 
Azure Storage Services - Part 01
Azure Storage Services - Part 01Azure Storage Services - Part 01
Azure Storage Services - Part 01Neeraj Kumar
 
Azure key vault
Azure key vaultAzure key vault
Azure key vaultRahul Nath
 

What's hot (20)

VMware NSX 101: What, Why & How
VMware NSX 101: What, Why & HowVMware NSX 101: What, Why & How
VMware NSX 101: What, Why & How
 
Microsoft Windows Server 2022 Overview
Microsoft Windows Server 2022 OverviewMicrosoft Windows Server 2022 Overview
Microsoft Windows Server 2022 Overview
 
Architecture patterns for distributed, hybrid, edge and global Apache Kafka d...
Architecture patterns for distributed, hybrid, edge and global Apache Kafka d...Architecture patterns for distributed, hybrid, edge and global Apache Kafka d...
Architecture patterns for distributed, hybrid, edge and global Apache Kafka d...
 
[242]open stack neutron dataplane 구현
[242]open stack neutron   dataplane 구현[242]open stack neutron   dataplane 구현
[242]open stack neutron dataplane 구현
 
VMware Virtual SAN Presentation
VMware Virtual SAN PresentationVMware Virtual SAN Presentation
VMware Virtual SAN Presentation
 
Open stack
Open stackOpen stack
Open stack
 
Room 3 - 7 - Nguyễn Như Phúc Huy - Vitastor: a fast and simple Ceph-like bloc...
Room 3 - 7 - Nguyễn Như Phúc Huy - Vitastor: a fast and simple Ceph-like bloc...Room 3 - 7 - Nguyễn Như Phúc Huy - Vitastor: a fast and simple Ceph-like bloc...
Room 3 - 7 - Nguyễn Như Phúc Huy - Vitastor: a fast and simple Ceph-like bloc...
 
Room 3 - 6 - Nguyễn Văn Thắng & Dzung Nguyen - Ứng dụng openzfs làm lưu trữ t...
Room 3 - 6 - Nguyễn Văn Thắng & Dzung Nguyen - Ứng dụng openzfs làm lưu trữ t...Room 3 - 6 - Nguyễn Văn Thắng & Dzung Nguyen - Ứng dụng openzfs làm lưu trữ t...
Room 3 - 6 - Nguyễn Văn Thắng & Dzung Nguyen - Ứng dụng openzfs làm lưu trữ t...
 
Hashicorp Vault - OPEN Public Sector
Hashicorp Vault - OPEN Public SectorHashicorp Vault - OPEN Public Sector
Hashicorp Vault - OPEN Public Sector
 
Nfs
NfsNfs
Nfs
 
Kubernetes dealing with storage and persistence
Kubernetes  dealing with storage and persistenceKubernetes  dealing with storage and persistence
Kubernetes dealing with storage and persistence
 
OpenShift-Technical-Overview.pdf
OpenShift-Technical-Overview.pdfOpenShift-Technical-Overview.pdf
OpenShift-Technical-Overview.pdf
 
VMware VSAN Technical Deep Dive - March 2014
VMware VSAN Technical Deep Dive - March 2014VMware VSAN Technical Deep Dive - March 2014
VMware VSAN Technical Deep Dive - March 2014
 
Docker Introduction
Docker IntroductionDocker Introduction
Docker Introduction
 
Automate Your Kafka Cluster with Kubernetes Custom Resources
Automate Your Kafka Cluster with Kubernetes Custom Resources Automate Your Kafka Cluster with Kubernetes Custom Resources
Automate Your Kafka Cluster with Kubernetes Custom Resources
 
Storage Requirements and Options for Running Spark on Kubernetes
Storage Requirements and Options for Running Spark on KubernetesStorage Requirements and Options for Running Spark on Kubernetes
Storage Requirements and Options for Running Spark on Kubernetes
 
Why to Cloud Native
Why to Cloud NativeWhy to Cloud Native
Why to Cloud Native
 
Azure Storage Services - Part 01
Azure Storage Services - Part 01Azure Storage Services - Part 01
Azure Storage Services - Part 01
 
Azure key vault
Azure key vaultAzure key vault
Azure key vault
 
Kubernetes networking & Security
Kubernetes networking & SecurityKubernetes networking & Security
Kubernetes networking & Security
 

Viewers also liked

GlusterFS and Openstack Storage
GlusterFS and Openstack StorageGlusterFS and Openstack Storage
GlusterFS and Openstack StorageDeepak Shetty
 
GlusterFS Update and OpenStack Integration
GlusterFS Update and OpenStack IntegrationGlusterFS Update and OpenStack Integration
GlusterFS Update and OpenStack IntegrationEtsuji Nakai
 
Open stack swift architecture and monitoring
Open stack swift architecture and monitoringOpen stack swift architecture and monitoring
Open stack swift architecture and monitoringKavit Munshi
 
Scale out backups-with_bareos_and_gluster
Scale out backups-with_bareos_and_glusterScale out backups-with_bareos_and_gluster
Scale out backups-with_bareos_and_glusterGluster.org
 
Software Defined storage
Software Defined storageSoftware Defined storage
Software Defined storageKirillos Akram
 
Integrating gluster fs,_qemu_and_ovirt-vijay_bellur-linuxcon_eu_2013
Integrating gluster fs,_qemu_and_ovirt-vijay_bellur-linuxcon_eu_2013Integrating gluster fs,_qemu_and_ovirt-vijay_bellur-linuxcon_eu_2013
Integrating gluster fs,_qemu_and_ovirt-vijay_bellur-linuxcon_eu_2013Gluster.org
 
Award winning scale-up and scale-out storage for Xen
Award winning scale-up and scale-out storage for XenAward winning scale-up and scale-out storage for Xen
Award winning scale-up and scale-out storage for XenGlusterFS
 
Gluster Storage Platform Installation Guide
Gluster Storage Platform Installation GuideGluster Storage Platform Installation Guide
Gluster Storage Platform Installation GuideGlusterFS
 
How to Install Gluster Storage Platform
How to Install Gluster Storage PlatformHow to Install Gluster Storage Platform
How to Install Gluster Storage PlatformGlusterFS
 
[OpenStack Day in Korea] Keynote#2 - Bringing OpenStack to the Enterprise Dat...
[OpenStack Day in Korea] Keynote#2 - Bringing OpenStack to the Enterprise Dat...[OpenStack Day in Korea] Keynote#2 - Bringing OpenStack to the Enterprise Dat...
[OpenStack Day in Korea] Keynote#2 - Bringing OpenStack to the Enterprise Dat...Sungjin Kang
 
Introduction to GlusterFS Webinar - September 2011
Introduction to GlusterFS Webinar - September 2011Introduction to GlusterFS Webinar - September 2011
Introduction to GlusterFS Webinar - September 2011GlusterFS
 
Petascale Cloud Storage with GlusterFS
Petascale Cloud Storage with GlusterFSPetascale Cloud Storage with GlusterFS
Petascale Cloud Storage with GlusterFSThe Linux Foundation
 
Distributed Shared Memory Systems
Distributed Shared Memory SystemsDistributed Shared Memory Systems
Distributed Shared Memory SystemsAnkit Gupta
 
CephFS update February 2016
CephFS update February 2016CephFS update February 2016
CephFS update February 2016John Spray
 
OpenStack Cinder, Implementation Today and New Trends for Tomorrow
OpenStack Cinder, Implementation Today and New Trends for TomorrowOpenStack Cinder, Implementation Today and New Trends for Tomorrow
OpenStack Cinder, Implementation Today and New Trends for TomorrowEd Balduf
 
Bridging The Gap: Explaining OpenStack To VMware Administrators
Bridging The Gap: Explaining OpenStack To VMware AdministratorsBridging The Gap: Explaining OpenStack To VMware Administrators
Bridging The Gap: Explaining OpenStack To VMware AdministratorsKenneth Hui
 
Smb gluster devmar2013
Smb gluster devmar2013Smb gluster devmar2013
Smb gluster devmar2013Gluster.org
 
Lisa 2015-gluster fs-introduction
Lisa 2015-gluster fs-introductionLisa 2015-gluster fs-introduction
Lisa 2015-gluster fs-introductionGluster.org
 
Haskell勉強会 14.1〜14.3 の説明資料
Haskell勉強会 14.1〜14.3 の説明資料Haskell勉強会 14.1〜14.3 の説明資料
Haskell勉強会 14.1〜14.3 の説明資料Etsuji Nakai
 

Viewers also liked (20)

GlusterFS and Openstack Storage
GlusterFS and Openstack StorageGlusterFS and Openstack Storage
GlusterFS and Openstack Storage
 
GlusterFS Update and OpenStack Integration
GlusterFS Update and OpenStack IntegrationGlusterFS Update and OpenStack Integration
GlusterFS Update and OpenStack Integration
 
Introduccion al cluster
Introduccion al clusterIntroduccion al cluster
Introduccion al cluster
 
Open stack swift architecture and monitoring
Open stack swift architecture and monitoringOpen stack swift architecture and monitoring
Open stack swift architecture and monitoring
 
Scale out backups-with_bareos_and_gluster
Scale out backups-with_bareos_and_glusterScale out backups-with_bareos_and_gluster
Scale out backups-with_bareos_and_gluster
 
Software Defined storage
Software Defined storageSoftware Defined storage
Software Defined storage
 
Integrating gluster fs,_qemu_and_ovirt-vijay_bellur-linuxcon_eu_2013
Integrating gluster fs,_qemu_and_ovirt-vijay_bellur-linuxcon_eu_2013Integrating gluster fs,_qemu_and_ovirt-vijay_bellur-linuxcon_eu_2013
Integrating gluster fs,_qemu_and_ovirt-vijay_bellur-linuxcon_eu_2013
 
Award winning scale-up and scale-out storage for Xen
Award winning scale-up and scale-out storage for XenAward winning scale-up and scale-out storage for Xen
Award winning scale-up and scale-out storage for Xen
 
Gluster Storage Platform Installation Guide
Gluster Storage Platform Installation GuideGluster Storage Platform Installation Guide
Gluster Storage Platform Installation Guide
 
How to Install Gluster Storage Platform
How to Install Gluster Storage PlatformHow to Install Gluster Storage Platform
How to Install Gluster Storage Platform
 
[OpenStack Day in Korea] Keynote#2 - Bringing OpenStack to the Enterprise Dat...
[OpenStack Day in Korea] Keynote#2 - Bringing OpenStack to the Enterprise Dat...[OpenStack Day in Korea] Keynote#2 - Bringing OpenStack to the Enterprise Dat...
[OpenStack Day in Korea] Keynote#2 - Bringing OpenStack to the Enterprise Dat...
 
Introduction to GlusterFS Webinar - September 2011
Introduction to GlusterFS Webinar - September 2011Introduction to GlusterFS Webinar - September 2011
Introduction to GlusterFS Webinar - September 2011
 
Petascale Cloud Storage with GlusterFS
Petascale Cloud Storage with GlusterFSPetascale Cloud Storage with GlusterFS
Petascale Cloud Storage with GlusterFS
 
Distributed Shared Memory Systems
Distributed Shared Memory SystemsDistributed Shared Memory Systems
Distributed Shared Memory Systems
 
CephFS update February 2016
CephFS update February 2016CephFS update February 2016
CephFS update February 2016
 
OpenStack Cinder, Implementation Today and New Trends for Tomorrow
OpenStack Cinder, Implementation Today and New Trends for TomorrowOpenStack Cinder, Implementation Today and New Trends for Tomorrow
OpenStack Cinder, Implementation Today and New Trends for Tomorrow
 
Bridging The Gap: Explaining OpenStack To VMware Administrators
Bridging The Gap: Explaining OpenStack To VMware AdministratorsBridging The Gap: Explaining OpenStack To VMware Administrators
Bridging The Gap: Explaining OpenStack To VMware Administrators
 
Smb gluster devmar2013
Smb gluster devmar2013Smb gluster devmar2013
Smb gluster devmar2013
 
Lisa 2015-gluster fs-introduction
Lisa 2015-gluster fs-introductionLisa 2015-gluster fs-introduction
Lisa 2015-gluster fs-introduction
 
Haskell勉強会 14.1〜14.3 の説明資料
Haskell勉強会 14.1〜14.3 の説明資料Haskell勉強会 14.1〜14.3 の説明資料
Haskell勉強会 14.1〜14.3 の説明資料
 

Similar to Glusterfs and openstack

Introducing gluster filesystem by aditya
Introducing gluster filesystem by adityaIntroducing gluster filesystem by aditya
Introducing gluster filesystem by adityaAditya Chhikara
 
Performance characterization in large distributed file system with gluster fs
Performance characterization in large distributed file system with gluster fsPerformance characterization in large distributed file system with gluster fs
Performance characterization in large distributed file system with gluster fsNeependra Khare
 
GlusterFS Talk for CentOS Dojo Bangalore
GlusterFS Talk for CentOS Dojo BangaloreGlusterFS Talk for CentOS Dojo Bangalore
GlusterFS Talk for CentOS Dojo BangaloreRaghavendra Talur
 
GlusterFS Presentation FOSSCOMM2013 HUA, Athens, GR
GlusterFS Presentation FOSSCOMM2013 HUA, Athens, GRGlusterFS Presentation FOSSCOMM2013 HUA, Athens, GR
GlusterFS Presentation FOSSCOMM2013 HUA, Athens, GRTheophanis Kontogiannis
 
Gluster fs architecture_&amp;_roadmap-vijay_bellur-linuxcon_eu_2013
Gluster fs architecture_&amp;_roadmap-vijay_bellur-linuxcon_eu_2013Gluster fs architecture_&amp;_roadmap-vijay_bellur-linuxcon_eu_2013
Gluster fs architecture_&amp;_roadmap-vijay_bellur-linuxcon_eu_2013Gluster.org
 
Gluster fs hadoop_fifth-elephant
Gluster fs hadoop_fifth-elephantGluster fs hadoop_fifth-elephant
Gluster fs hadoop_fifth-elephantGluster.org
 
Gluster intro-tdose
Gluster intro-tdoseGluster intro-tdose
Gluster intro-tdoseGluster.org
 
Gluster intro-tdose
Gluster intro-tdoseGluster intro-tdose
Gluster intro-tdoseGluster.org
 
Red Hat Gluster Storage : GlusterFS
Red Hat Gluster Storage : GlusterFSRed Hat Gluster Storage : GlusterFS
Red Hat Gluster Storage : GlusterFSbipin kunal
 
GlusterFs: a scalable file system for today's and tomorrow's big data
GlusterFs: a scalable file system for today's and tomorrow's big dataGlusterFs: a scalable file system for today's and tomorrow's big data
GlusterFs: a scalable file system for today's and tomorrow's big dataRoberto Franchini
 
GlusterFS : un file system open source per i big data di oggi e domani - Robe...
GlusterFS : un file system open source per i big data di oggi e domani - Robe...GlusterFS : un file system open source per i big data di oggi e domani - Robe...
GlusterFS : un file system open source per i big data di oggi e domani - Robe...Codemotion
 
GlusterFs Architecture & Roadmap - LinuxCon EU 2013
GlusterFs Architecture & Roadmap - LinuxCon EU 2013GlusterFs Architecture & Roadmap - LinuxCon EU 2013
GlusterFs Architecture & Roadmap - LinuxCon EU 2013Gluster.org
 
Red Hat Storage Day Dallas - Storage for OpenShift Containers
Red Hat Storage Day Dallas - Storage for OpenShift Containers Red Hat Storage Day Dallas - Storage for OpenShift Containers
Red Hat Storage Day Dallas - Storage for OpenShift Containers Red_Hat_Storage
 
Deploying pNFS over Distributed File Storage w/ Jiffin Tony Thottan and Niels...
Deploying pNFS over Distributed File Storage w/ Jiffin Tony Thottan and Niels...Deploying pNFS over Distributed File Storage w/ Jiffin Tony Thottan and Niels...
Deploying pNFS over Distributed File Storage w/ Jiffin Tony Thottan and Niels...Gluster.org
 
Jaspreet webinar-cns
Jaspreet webinar-cnsJaspreet webinar-cns
Jaspreet webinar-cnsJaspreet Kaur
 
Module: Mutable Content in IPFS
Module: Mutable Content in IPFSModule: Mutable Content in IPFS
Module: Mutable Content in IPFSIoannis Psaras
 
OSDC 2010 | Use Distributed Filesystem as a Storage Tier by Fabrizio Manfred
OSDC 2010 | Use Distributed Filesystem as a Storage Tier by Fabrizio ManfredOSDC 2010 | Use Distributed Filesystem as a Storage Tier by Fabrizio Manfred
OSDC 2010 | Use Distributed Filesystem as a Storage Tier by Fabrizio ManfredNETWAYS
 

Similar to Glusterfs and openstack (20)

Introducing gluster filesystem by aditya
Introducing gluster filesystem by adityaIntroducing gluster filesystem by aditya
Introducing gluster filesystem by aditya
 
Performance characterization in large distributed file system with gluster fs
Performance characterization in large distributed file system with gluster fsPerformance characterization in large distributed file system with gluster fs
Performance characterization in large distributed file system with gluster fs
 
GlusterFS Talk for CentOS Dojo Bangalore
GlusterFS Talk for CentOS Dojo BangaloreGlusterFS Talk for CentOS Dojo Bangalore
GlusterFS Talk for CentOS Dojo Bangalore
 
GlusterFS Presentation FOSSCOMM2013 HUA, Athens, GR
GlusterFS Presentation FOSSCOMM2013 HUA, Athens, GRGlusterFS Presentation FOSSCOMM2013 HUA, Athens, GR
GlusterFS Presentation FOSSCOMM2013 HUA, Athens, GR
 
Gluster fs architecture_&amp;_roadmap-vijay_bellur-linuxcon_eu_2013
Gluster fs architecture_&amp;_roadmap-vijay_bellur-linuxcon_eu_2013Gluster fs architecture_&amp;_roadmap-vijay_bellur-linuxcon_eu_2013
Gluster fs architecture_&amp;_roadmap-vijay_bellur-linuxcon_eu_2013
 
Gluster fs hadoop_fifth-elephant
Gluster fs hadoop_fifth-elephantGluster fs hadoop_fifth-elephant
Gluster fs hadoop_fifth-elephant
 
GlusterFS And Big Data
GlusterFS And Big DataGlusterFS And Big Data
GlusterFS And Big Data
 
Gluster intro-tdose
Gluster intro-tdoseGluster intro-tdose
Gluster intro-tdose
 
Gluster intro-tdose
Gluster intro-tdoseGluster intro-tdose
Gluster intro-tdose
 
Red Hat Gluster Storage : GlusterFS
Red Hat Gluster Storage : GlusterFSRed Hat Gluster Storage : GlusterFS
Red Hat Gluster Storage : GlusterFS
 
Hdfs architecture
Hdfs architectureHdfs architecture
Hdfs architecture
 
GlusterFs: a scalable file system for today's and tomorrow's big data
GlusterFs: a scalable file system for today's and tomorrow's big dataGlusterFs: a scalable file system for today's and tomorrow's big data
GlusterFs: a scalable file system for today's and tomorrow's big data
 
GlusterFS : un file system open source per i big data di oggi e domani - Robe...
GlusterFS : un file system open source per i big data di oggi e domani - Robe...GlusterFS : un file system open source per i big data di oggi e domani - Robe...
GlusterFS : un file system open source per i big data di oggi e domani - Robe...
 
GlusterFs Architecture & Roadmap - LinuxCon EU 2013
GlusterFs Architecture & Roadmap - LinuxCon EU 2013GlusterFs Architecture & Roadmap - LinuxCon EU 2013
GlusterFs Architecture & Roadmap - LinuxCon EU 2013
 
Red Hat Storage Day Dallas - Storage for OpenShift Containers
Red Hat Storage Day Dallas - Storage for OpenShift Containers Red Hat Storage Day Dallas - Storage for OpenShift Containers
Red Hat Storage Day Dallas - Storage for OpenShift Containers
 
vBACD - Distributed Petabyte-Scale Cloud Storage with GlusterFS - 2/28
vBACD - Distributed Petabyte-Scale Cloud Storage with GlusterFS - 2/28vBACD - Distributed Petabyte-Scale Cloud Storage with GlusterFS - 2/28
vBACD - Distributed Petabyte-Scale Cloud Storage with GlusterFS - 2/28
 
Deploying pNFS over Distributed File Storage w/ Jiffin Tony Thottan and Niels...
Deploying pNFS over Distributed File Storage w/ Jiffin Tony Thottan and Niels...Deploying pNFS over Distributed File Storage w/ Jiffin Tony Thottan and Niels...
Deploying pNFS over Distributed File Storage w/ Jiffin Tony Thottan and Niels...
 
Jaspreet webinar-cns
Jaspreet webinar-cnsJaspreet webinar-cns
Jaspreet webinar-cns
 
Module: Mutable Content in IPFS
Module: Mutable Content in IPFSModule: Mutable Content in IPFS
Module: Mutable Content in IPFS
 
OSDC 2010 | Use Distributed Filesystem as a Storage Tier by Fabrizio Manfred
OSDC 2010 | Use Distributed Filesystem as a Storage Tier by Fabrizio ManfredOSDC 2010 | Use Distributed Filesystem as a Storage Tier by Fabrizio Manfred
OSDC 2010 | Use Distributed Filesystem as a Storage Tier by Fabrizio Manfred
 

More from openstackindia

Guts & OpenStack migration
Guts & OpenStack migrationGuts & OpenStack migration
Guts & OpenStack migrationopenstackindia
 
Copr HD OpenStack Day India
Copr HD OpenStack Day IndiaCopr HD OpenStack Day India
Copr HD OpenStack Day Indiaopenstackindia
 
Your first patch to OpenStack
Your first patch to OpenStackYour first patch to OpenStack
Your first patch to OpenStackopenstackindia
 
OpenStack Neutron Behind The Senes
OpenStack Neutron Behind The SenesOpenStack Neutron Behind The Senes
OpenStack Neutron Behind The Senesopenstackindia
 
OpenStack Storage Buddy Ceph
OpenStack Storage Buddy CephOpenStack Storage Buddy Ceph
OpenStack Storage Buddy Cephopenstackindia
 
State of Containers in OpenStack
State of Containers in OpenStackState of Containers in OpenStack
State of Containers in OpenStackopenstackindia
 
The OpenStack Contribution Workflow
The OpenStack Contribution WorkflowThe OpenStack Contribution Workflow
The OpenStack Contribution Workflowopenstackindia
 
Introduction to Cinder
Introduction to CinderIntroduction to Cinder
Introduction to Cinderopenstackindia
 
OpenStack NFV Edge computing for IOT microservices
OpenStack NFV Edge computing for IOT microservicesOpenStack NFV Edge computing for IOT microservices
OpenStack NFV Edge computing for IOT microservicesopenstackindia
 
OpenStack Tempest and REST API testing
OpenStack Tempest and REST API testingOpenStack Tempest and REST API testing
OpenStack Tempest and REST API testingopenstackindia
 
Deploying openstack using ansible
Deploying openstack using ansibleDeploying openstack using ansible
Deploying openstack using ansibleopenstackindia
 
Ceph openstack-jun-2015-meetup
Ceph openstack-jun-2015-meetupCeph openstack-jun-2015-meetup
Ceph openstack-jun-2015-meetupopenstackindia
 
Role of sdn controllers in open stack
Role of sdn controllers in open stackRole of sdn controllers in open stack
Role of sdn controllers in open stackopenstackindia
 
Outreachy with-openstack-zaqar
Outreachy with-openstack-zaqarOutreachy with-openstack-zaqar
Outreachy with-openstack-zaqaropenstackindia
 
Enhancing OpenStack FWaaS for real world application
Enhancing OpenStack FWaaS for real world applicationEnhancing OpenStack FWaaS for real world application
Enhancing OpenStack FWaaS for real world applicationopenstackindia
 
Openstack devops challenges
Openstack devops challenges Openstack devops challenges
Openstack devops challenges openstackindia
 
Demistifying open stack storage
Demistifying open stack storageDemistifying open stack storage
Demistifying open stack storageopenstackindia
 

More from openstackindia (20)

Guts & OpenStack migration
Guts & OpenStack migrationGuts & OpenStack migration
Guts & OpenStack migration
 
Copr HD OpenStack Day India
Copr HD OpenStack Day IndiaCopr HD OpenStack Day India
Copr HD OpenStack Day India
 
OPNFV & OpenStack
OPNFV & OpenStackOPNFV & OpenStack
OPNFV & OpenStack
 
Your first patch to OpenStack
Your first patch to OpenStackYour first patch to OpenStack
Your first patch to OpenStack
 
OpenStack Neutron Behind The Senes
OpenStack Neutron Behind The SenesOpenStack Neutron Behind The Senes
OpenStack Neutron Behind The Senes
 
OpenStack Storage Buddy Ceph
OpenStack Storage Buddy CephOpenStack Storage Buddy Ceph
OpenStack Storage Buddy Ceph
 
OpenStack Watcher
OpenStack WatcherOpenStack Watcher
OpenStack Watcher
 
State of Containers in OpenStack
State of Containers in OpenStackState of Containers in OpenStack
State of Containers in OpenStack
 
The OpenStack Contribution Workflow
The OpenStack Contribution WorkflowThe OpenStack Contribution Workflow
The OpenStack Contribution Workflow
 
Introduction to Cinder
Introduction to CinderIntroduction to Cinder
Introduction to Cinder
 
OpenStack NFV Edge computing for IOT microservices
OpenStack NFV Edge computing for IOT microservicesOpenStack NFV Edge computing for IOT microservices
OpenStack NFV Edge computing for IOT microservices
 
OpenStack Tempest and REST API testing
OpenStack Tempest and REST API testingOpenStack Tempest and REST API testing
OpenStack Tempest and REST API testing
 
Deploying openstack using ansible
Deploying openstack using ansibleDeploying openstack using ansible
Deploying openstack using ansible
 
Ceph openstack-jun-2015-meetup
Ceph openstack-jun-2015-meetupCeph openstack-jun-2015-meetup
Ceph openstack-jun-2015-meetup
 
Role of sdn controllers in open stack
Role of sdn controllers in open stackRole of sdn controllers in open stack
Role of sdn controllers in open stack
 
Outreachy with-openstack-zaqar
Outreachy with-openstack-zaqarOutreachy with-openstack-zaqar
Outreachy with-openstack-zaqar
 
Enhancing OpenStack FWaaS for real world application
Enhancing OpenStack FWaaS for real world applicationEnhancing OpenStack FWaaS for real world application
Enhancing OpenStack FWaaS for real world application
 
Openstack devops challenges
Openstack devops challenges Openstack devops challenges
Openstack devops challenges
 
Demistifying open stack storage
Demistifying open stack storageDemistifying open stack storage
Demistifying open stack storage
 
OpenStack Heat
OpenStack HeatOpenStack Heat
OpenStack Heat
 

Recently uploaded

AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
Spring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdfSpring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdfAnna Loughnan Colquhoun
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URLRuncy Oommen
 
Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?SANGHEE SHIN
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
GenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncGenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncObject Automation
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 

Recently uploaded (20)

AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
Spring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdfSpring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdf
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URL
 
Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
GenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncGenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation Inc
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 

Glusterfs and openstack

  • 2. Agenda ➢ What is GlusterFS? ➢ Architecture ➢ Concepts ➢ Algorithms ➢ Access Mechanisms ➢ Implementation ➢ OpenStack and GlusterFS ➢Resources ➢ Q&A
  • 3. GlusterFS Deployment Global namespace Scale-out storage building blocks Supports thousands of clients Access using GlusterFS native, NFS, SMB and HTTP protocols Runs on commodity hardware
  • 5.  Trusted Storage Pool (cluster) is a collection of storage servers.  Trusted Storage Pool is formed by invitation – you “probe” a new member from the cluster and not vice versa.  Logical partition for all data and management operations.  Membership information used for determining quorum.  Members can be dynamically added and removed from the pool. GlusterFS concepts – Trusted Storage Pool
  • 6. GlusterFS concepts – Trusted Storage Pool Node1 Node2 Probe Probe accepted Node 1 and Node 2 are peers in a trusted storage pool Node2Node1
  • 7. GlusterFS concepts – Trusted Storage Pool Node1 Node2 Node3Node2Node1 Trusted Storage Pool Node3Node2Node1 Detach
  • 8.  A brick is the combination of a node and an export directory – for e.g. hostname:/dir  Each brick inherits limits of the underlying filesystem  No limit on the number bricks per node  Ideally, each brick in a cluster should be of the same size /export3 /export3 /export3 Storage Node /export1 Storage Node /export2 /export1 /export2 /export4 /export5 Storage Node /export1 /export2 3 bricks 5 bricks 3 bricks GlusterFS concepts - Bricks
  • 9. GlusterFS concepts - Volumes ➢ A volume is a logical collection of bricks. ➢ Volume is identified by an administrator provided name. ➢Volume is a mountable entity and the volume name is provided at the time of mounting. ➢ mount -t glusterfs server1:/<volname> /my/mnt/point ➢ Bricks from the same node can be part of different volumes
  • 10. GlusterFS concepts - Volumes Node2Node1 Node3 /export/brick1 /export/brick2 /export/brick1 /export/brick2 /export/brick1 /export/brick2 Videos music
  • 11. Volume Types ➢Type of a volume is specified at the time of volume creation ➢ Volume type determines how and where data is placed ➢ Following volume types are supported in glusterfs: a) Distribute b) Stripe c) Replication d) Distributed Replicate e) Striped Replicate f) Distributed Striped Replicate
  • 12. Distributed Volume ➢Distributes files across various bricks of the volume. ➢Directories are present on all bricks of the volume. ➢Single brick failure will result in loss of data availability. ➢Removes the need for an external meta data server.
  • 13. How does a distributed volume work? ➢ Distribute works on the basis of a dht algorithm. ➢ A 32-bit hash space is divided into N ranges for N bricks ➢ At the time of directory creation, a range is assigned to each directory. ➢ During a file creation or retrieval, hash is computed on the file name. This hash value is used to locate or place the file. ➢Different directories in the same brick end up with different hash ranges.
  • 14. How does a distributed volume work?
  • 15. How does a distributed volume work?
  • 16. How does a distributed volume work?
  • 17. Replicated Volume ➢Creates synchronous copies of all directory and file updates. ➢Provides high availability of data when node failures occur. ➢Transaction driven for ensuring consistency. ➢Changelogs maintained for re-conciliation. ➢Any number of replicas can be configured.
  • 18. How does a replicated volume work?
  • 19. How does a replicated volume work?
  • 20. Distributed Replicated Volume ➢ Distribute files across replicated bricks ➢ Number of bricks must be a multiple of the replica count ➢ Ordering of bricks in volume definition matters ➢ Scaling and high reliability ➢ Improved read performance in most environments. ➢ Most preferred model of deployment currently.
  • 22. Striped Volume ➢Files are striped into chunks and placed in various bricks. ➢Recommended only when very large files greater than the size of the disks are present. ➢Chunks are files with holes – this helps in maintaining offset consistency. ➢A brick failure can result in data loss. Redundancy with replication is highly recommended
  • 23. Translators in GlusterFS ➢Building blocks for a GlusterFS process. ➢Based on Translators in GNU HURD. ➢Each translator is a functional unit. ➢Translators can be stacked together for achieving desired functionality. ➢ Translators are deployment agnostic – can be loaded in either the client or server stacks.
  • 24. VFS Server I/O Cache Distribute / Stripe POSIX Ext4 Ext4Ext4 POSIX POSIX Brick 1 ServerServer Read Ahead Brick 2 Brick n-1 Gluster Server Replicate Ext4 POSIX Server Brick n Replicate Customizable GlusterFS Client/Server Stack Client Gluster Server Client GigE, 10GigE – TCPIP / InfiniBand – RDMA Gluster ServerGluster Server Client Client
  • 25. Dynamic Volume Management Collectively refers to application transparent operations that can be performed in the storage layer. ➢ Addition of Bricks to a volume ➢ Remove brick from a volume ➢ Rebalance data spread within a volume ➢ Replace a brick in a volume ➢ Adaptive Performance / Functionality tuning.
  • 26. Access Mechanisms: Gluster volumes can be accessed via the following mechanisms: ➢ FUSE ➢ NFS ➢ SMB ➢ libgfapi ➢ ReST ➢ HDFS
  • 28. UFO Client Proxy Account Container Object HTTP Request (REST) Directory Volume FileClient NFS or GlusterFS Mount Unified File and object view. Entity mapping between file and object building blocks
  • 29. Interface Possibilities qemu NFS SMB Hadoop FUSE Cinder Swift (UFO) Files Blocks Objects libgfapi Whatever IP RDMA Transports files BD Back ends DB Interface Possibilities
  • 30. OpenStack and GlusterFS – Current Integration Glance Images Nova Nodes Swift Objects Cinder Data Glance Data Swift Data Swift API Storage Server Storage Server Storage Server… KVM KVM KVM … ● Separate Compute and Storage Pools ● GlusterFS directly provides Swift object service ● Integration with Keystone ● GeoReplication for multi-site support ● Swift data also available via other protocols ● Supports non-OpenStack use in addition to OpenStack use Logical View Physical View
  • 31. OpenStack and GlusterFS - Future Direction Hadoop Guest Other Guest ... Host Gluster Guest Hadoop Guest Other Guest ... Host Gluster Guest Nova Compute Nodes
  • 32. Open Stack and GlusterFS - Future Direction ● POC based on proposed OpenStack FaaS (File as a Service) proposal ● Cinder-like virtual NAS service ● Tenant-specific file shares ● Hypervisor mediated for security ● Avoid exposing servers to Quantum tenant network ● Optional multi-site or multi-zone GeoReplication ● FaaS data optionally available to non OpenStack nodes ● Initial focus on Linux guest ● Windows (SMB) and NFS shares also under consideration
  • 33. Open Stack and GlusterFS - Summary ● Can address storage needs across the stack: ● Block ● Object ● File ● Hadoop/Big Data
  • 34. Resources Mailing lists: gluster-users@gluster.org gluster-devel@nongnu.org IRC: #gluster and #gluster-dev on freenode Links: http://www.gluster.org http://hekafs.org http://forge.gluster.org http://www.gluster.org/community/documentation/index.php/Arch