Efficient Use of Cyberinfrastructure

The Efficient Use of Cyberinfrastructure
to Enable Data Analysis Collaboration

Summit ’09 October 15, 2009
Dave Fellinger
CTO
© 2009 DataDirect Networks, Inc. All Rights Reserved.

Company Data at a Glance
• Fast Growing Data Infrastructure Provider for Companies who
Demand Extreme Performance for their Large Content Files and
Unstructured Data
• Integrated Portfolio of Extreme Storage Platforms, Intuitive
Storage Management Software and Consulting Services
• Over 10 Years of Stability and Experience
• Over $130M in Annual Revenue
• Growing, Profitable and Hiring
• Over 200 Petabytes Installed Worldwide
• Clients include XBOX LIVE, Slide & Saudi Aramco
• Global Partners include IBM, Sony and Dell
• Expanding Globally, with Established Offices
• Europe, India, Australia, Asia Pacific and Japan

“DDN could find itself a market leading provider for the Internet computing era
in the same way that EMC did in the transactional era and NetApp did in the
distributed era”

Storage Designed for the Most
Extreme Environments in the World
• Sites That Have Zero Tolerance For Data Latency
• Sites That Demand Sub-Millisecond Application Response Times
• Sites that Stream Extreme HD Content on a Massive Scale
• Sites with Our Systems Delivering Over 100 Gigabytes Per Second of
Consistent Throughput
• Sites that Research International Scientific Data
• The Most Sophisticated and Demanding High Performance Computing
Environments
• Powering 8 of the Top 10 and
• Powering 50 of the Top 100
Supercomputer Sites


Data Distribution Evolution

• The Problem
• A large number of sites need simultaneous
access to local data for detailed analysis.

• The Solution
• Utilize a simplified object based file system to
provide simultaneous synchronous instances
of data based on policy.


Data Distribution Today

• The “Library Method”
• Large data warehouses are queried for data in
an iterative manner.

• MODIS as an example.
• All MODIS image data is stored at NASA
Goddard on both disk and tape.
• Inquiries generally spawn additional inquires
requiring time, study, and network bandwidth.

How is it Done Today?
URL explosion in objects

Map Objects to individual file systems Hundreds or thousands of building
blocks like this
Within file system files and folders are
named 1, 2, 3, 4…. for hashing and rapid No common management framework
index lookups Database
Huge investment in custom engineering
Must manage # of files per folder, # of to cobble together a global namespace
folders in each file system
Heavy reliance on expensive CDNs
Multiple IOPS to get a file limits # of file
reads per second and increases latency Internally developed replication schemes

LAN
NFS/HTTP/CIFS NFS/HTTP/CIFS NFS/HTTP/CIFS NFS/HTTP/CIFS NFS/HTTP/CIFS NFS/HTTP/CIFS

NFS/HTTP/CIFS NFS/HTTP/CIFS NFS/HTTP/CIFS NFS/HTTP/CIFS NFS/HTTP/CIFS NFS/HTTP/CIFS

Local File System Local File System Local File System Local File System Local File System Local File System

Local File System Local File System Local File System Local File System Local File System Local File System

RAID RAID RAID
Disk Disk Disk

© 2009 DataDirect Networks, Inc. All Rights Reserved. DataDirect Networks Confidential: Internal Use Only

Multi-site Content Delivery

SITE A SITE C

SITE D

Database
SITE B Tracking
File
Locations
Replication Software (x10)

Administrator (x5)

Storage/File System(s)
SITE E
RAID/LUNs/Fibre Channel (x10 – Thousands)

What’s within each site?
Complexity!
• Lots of things to manage
• No automation or coordination
Hundreds of filers • Must maintain a path to find every file
• Provisioning more capacity increases complexity

Dozens – Hundreds of File Systems!
Multiple File
Servers

Fibre Channel
Switching,
Cables, SFPs

Multiple
RAIDs


DDN’s Cloud Storage Initiative

! Take a clean look at the issues of massively scalable
file storage & distribution

! Understand best practices from leading edge
customers

! Develop a tailored solution
– Work with key users every step of the way
– Eliminate the need for complex multi-vendor integration
– Minimize customers having to write custom code
– Focus on file reads per second rather than IOPs

© 2009 DataDirect Networks, Inc. All Rights Reserved. DataDirect Networks Confidential: Internal Use Only9

Scalable File Delivery
with WOS Clouds
• Single Management Point
• Single Namespace for Billions of Files
• Fully Automated, Load Balancing & Self Healing
• Automated Best-Path File Retrieval
• Multi-site Policy-Based Replication
• Add Capacity Online in Seconds
• Easy!

Network
Connection

WOS Nodes

Database
Administrator Storing WOS
Object IDs


WOS Puts & Gets - 1

Users
Users

Customer
Application
Servers
WOS-B WOS-LIB WOS-LIB

Gigabit
Ethernet
Network

WOS Cluster

WOS Puts & Gets - 2

User uploads a
User uploads a
file to application
file to application
Users
Users server.
server.

Customer
Application
Servers
WOS-LIB WOS-LIB WOS-LIB

Gigabit
Ethernet
Network

WOS Cluster

WOS Puts & Gets - 3

Users
Users

Customer Application makes
Application makes
Application a call to the WOS-
a call to the WOS-
Servers LIB to store (PUT)
LIB to store (PUT)
a new object
a new object

Gigabit
Ethernet
Network

WOS Cluster

WOS Puts & Gets - 4

Users
Users

Customer WOS-LIB stores the object
WOS-LIB stores the object
Application on a node. Subsequent
on a node. Subsequent
Servers objects are automatically
objects are automatically
WOS-LIB WOS-LIB WOS-LIB load balanced across the
load balanced across the
cluster.
cluster.
Gigabit
Ethernet
Network

WOS Cluster

WOS Puts & Gets - 5

Users
Users

Customer
Application
Servers

If the storage
If the storage
Gigabit policy specifies
policy specifies
Ethernet replication, the
replication, the
Network object is replicated
object is replicated
to another node.
to another node.

WOS Cluster

WOS Puts & Gets - 6

Users
Users

WOS-LIB returns a unique
WOS-LIB returns a unique
Customer OID = 5718a36143521602 Object ID which the
Object ID which the
Application
application stores in lieu of
application stores in lieu of
Servers
WOS-LIB WOS-LIB WOS-LIB a file path.
a file path.

Gigabit
Ethernet
Network

WOS Cluster

WOS Puts & Gets - 7

Users
Users
A User needs
A User needs
the stored file.
the stored file.

Customer
Application
Servers

Gigabit
Ethernet
Network

WOS Cluster

WOS Puts & Gets - 8

Users
Users
Application makes a
Application makes a
call to WOS-LIB to
call to WOS-LIB to
read (GET) the
read (GET) the
object. The unique
object. The unique
Object ID is passed Customer
Object ID is passed
to WOS-LIB.
to WOS-LIB. Application
Servers

Gigabit
Ethernet
Network

WOS Cluster

WOS Puts & Gets - 9

Users
Users
WOS-LIB automatically
determines what
determines what
node(s) have the
node(s) have the
requested object,
requested object,
retrieves the object,
and returns it to the Customer
and returns it to the
application.
application. Application
Servers
OID = 5718a36143521602

Gigabit
Ethernet
Network

WOS Cluster

WOS Puts & Gets - 10

Users
Users
determines what
determines what
node(s) have the
node(s) have the
requested object,
requested object,
and returns it to the Customer
and returns it to the
application.
application. Application
Servers
OID = 5718a36143521602

Gigabit
Ethernet
Network

OID = 5718a36143521602

WOS Cluster

WOS Puts & Gets - 11

Application returns
Application returns
file to user.
file to user.
Users
Users

Customer
Application
Servers

Gigabit
Ethernet
Network

WOS Cluster

WOS Zones and Replication
Global Content Distribution - 1

Zones are logical groups of WOS
nodes in which you scope
replication, or data protection
Zone “West” policies.

In this example, we have two Zone “East”
zones which also map to
geographic data centers.

These data centers both serve the
public internet.

Data Protection via Replication
High Spe
ed WAN
Connecti
o n
User Defined Object
User Defined Object Data Protection via Replication
Storage Policies:
Storage Policies:
Wild: West=2, East=0
Wallstreet: East=2, West=0
Safe: West=1, East=1



Objects stored utilizing the “Wild”
Zone “West” policy will result in objects being
stored on two distinct nodes within
the “West” zone. Zone “East”
The server performing the store
action can be anywhere on the
network, including the “East” zone

High Spe
ed WAN
Connecti
o n
User Defined Object
Storage Policies:
Storage Policies:



Objects stored utilizing the
“Safe” policy will result in object
Zone “West” replicas being stored on one
node in the “West” zone and
one node in the “East” zone.
Zone “East”
Again, the server performing the
store action via the WOS-LIB
can be anywhere on the
network.

High Spe
ed WAN
Connecti
o n
User Defined Object
Storage Policies:
Storage Policies:



The “West” and “East” zones
utilize replication on high
Zone “West” performance WOS nodes for
the most demanding service
environments. Zone “East”
What if you wanted an extra
object copy for Disaster
Recovery?

High Spe
ed WAN
Connecti
o n
User Defined Object
Storage Policies:
Storage Policies:


Zone “DR”

AN
dW
ee
Sp
H igh
Hi
Data Protection via Replication gh
S

Zone “West”
pe
ed
W
AN

Zone “East”
The “DR” zone utilizes high
capacity WOS nodes.

High Spe
ed WAN
Connecti
o n
User Defined Object
Storage Policies:
Storage Policies:
CYA: West=1, East=1, DR=1


Zone “DR”

AN
dW
ee
Sp
H igh
Hi
S

Zone “West”
pe
ed
W
AN

Objects stored utilizing the “CYA” Zone “East”
storage policy will result in object
replicas on both coasts as well as
a copy in the “DR” zone.

High Spe
ed WAN
Connecti
o n
User Defined Object
Storage Policies:
Storage Policies:


Zone “DR”

AN
dW
ee
Sp
H igh
Hi
S

Zone “West”
pe
ed
W
AN

Regardless of replication Zone “East”
policy, any object (OID)
can be accessed from
any zone, whether the
object resides in that
zone or not!
High Spe
ed WAN
Connecti
o n
User Defined Object
Storage Policies:
Storage Policies:


Data Protection: Drive and Node
Failure Handling - 1

• Replication requires at least two copies of each
object to be stored for a given OID.
• With replication, for maximum performance,
individual objects are stored within 1 disk unit.


• Upon disk drive failure, all objects stored on the
failed drive are noted to be out of policy
compliance and recovery begins.
• Affected objects are copied in parallel to bring the
cluster back into full policy compliance.

!



• Policy restoration occurs on a per object basis,
NOT per node, hence only objects that resided on
the failed node will be replicated.
• When the failed node is replaced or returns online,
it simply becomes additional cluster capacity.


• Upon node failure, recovery of all objects stored
on the failed node begins.
• Affected objects are copied in parallel and
distributed to surviving nodes to bring the cluster
back into full policy compliance.



• Policy restoration occurs on a per object basis,
NOT per disk drive hence only used object space
will be replicated.
• When the failed disk drive is replaced, the
replacement simply becomes additional capacity.


Data Distribution Evolution

• The “Push Method”
• Data is automatically replicated based on
policy to scientists with specific requirements.
• ATLAS as an example;
• The LHC at CERN produces more than 1TB
per day.
• Relevant data is replicated to over 300 sites in
the US and Canada for analysis.
• Replication is based on specific interest and is
fully automated.

Conclusion

• To enable multi-point scientific study data must have
attributes of location (site) as well as traditional
ACLs.

• File systems distributing data must have redundant
automation to enable replication over geographies.

• Data distribution methods must be simplified to
reduce latency, maximize network efficiency allowing
improved processing efficiency.


Thank You

Dave Fellinger
dfellinger@ddn.com


Efficient Use of Cyberinfrastructure

Recommended

Recommended

More Related Content

What's hot

What's hot (19)

Viewers also liked

Viewers also liked (20)

Similar to Efficient Use of Cyberinfrastructure

Similar to Efficient Use of Cyberinfrastructure (20)

More from Cybera Inc.

More from Cybera Inc. (20)

Recently uploaded

Recently uploaded (20)

Efficient Use of Cyberinfrastructure