This document provides an introduction to Riak and Ripple. It summarizes that Riak is a distributed key-value store inspired by Amazon Dynamo that allows scaling by adding more machines. It stores data in buckets and uses JavaScript for MapReduce functions. The document demonstrates basic usage of the Riak client and RObject in Ruby, and explains how links enable lightweight relationships between objects and can be traversed through link walking. It also provides an example of a MapReduce function in JavaScript.

1. Introduction to
Riak & Ripple
Sean Cribbs
basho Developer Advocate

2. Introduction to
Riak & Ripple
Sean Cribbs
basho Developer Advocate
Rubyist, Evangelist,
Support Monkey

3. Riak @10,000 ft.

4. Riak @10,000 ft.
• Amazon Dynamo-inspired
replicated, distributed,
fault-tolerant, masterless,
scalable, operations-friendly

5. Riak @10,000 ft.
• Amazon Dynamo-inspired
replicated, distributed,
fault-tolerant, masterless,
scalable, operations-friendly
• Key-Value / Document

6. Riak @10,000 ft.
• Amazon Dynamo-inspired
replicated, distributed,
fault-tolerant, masterless,
scalable, operations-friendly
• Key-Value / Document
• Schema-less, content-agnostic

7. Riak @10,000 ft.
• Amazon Dynamo-inspired
replicated, distributed,
fault-tolerant, masterless,
scalable, operations-friendly
• Key-Value / Document
• Schema-less, content-agnostic
• Web-friendly
HTTP, JSON, Javascript

8. Dynamo-like
Scalability

9. Dynamo-like
Scalability
• To get...

10. Dynamo-like
Scalability
• To get...
• more storage,

11. Dynamo-like
Scalability
• To get...
• more storage,
• more throughput,

12. Dynamo-like
Scalability
• To get...
• more storage,
• more throughput,
• lower latency...

13. Dynamo-like
Scalability
• To get...
• more storage,
• more throughput,
• lower latency...
• ...add more machines.
aka horizontal & linear

14. Key-Value++

15. Key-Value++
• Data objects are identified by keys,
and have metadata

16. Key-Value++
• Data objects are identified by keys,
and have metadata
• Keys are grouped in buckets

17. Key-Value++
• Data objects are identified by keys,
and have metadata
• Keys are grouped in buckets
• Links enable lightweight
relationships

18. Key-Value++
• Data objects are identified by keys,
and have metadata
• Keys are grouped in buckets
• Links enable lightweight
relationships
• Query with MapReduce

19. Sans-Schema

20. Sans-Schema
• Buckets created on the fly

21. Sans-Schema
• Buckets created on the fly
• Values are opaque

22. Sans-Schema
• Buckets created on the fly
• Values are opaque
•Content-Type matters

23. Sans-Schema
• Buckets created on the fly
• Values are opaque
•Content-Type matters
•The application defines the
semantics:
more flexibility, more responsibility

24. Web-Friendly

25. Web-Friendly
•HTTP is primary interface

26. Web-Friendly
•HTTP is primary interface
•JSON is used for structured data

27. Web-Friendly
•HTTP is primary interface
•JSON is used for structured data
•Javascript is used for
MapReduce functions

28. Web-Friendly
•HTTP is primary interface
•JSON is used for structured data
•Javascript is used for
MapReduce functions
•Plays well with Varnish, Squid,
HAProxy, F5, etc.

29. Web-Friendly
•HTTP is primary interface
•JSON is used for structured data
•Javascript is used for
MapReduce functions
•Plays well with Varnish, Squid,
HAProxy, F5, etc.
[see also Webmachine]

30. Use Cases

31. Use Cases
•Document storage
(sparse structure)

32. Use Cases
•Document storage
(sparse structure)
•Distributed file storage
(small size, large soon)

33. Use Cases
•Document storage
(sparse structure)
•Distributed file storage
(small size, large soon)
•Session storage

34. Use Cases
•Document storage
(sparse structure)
•Distributed file storage
(small size, large soon)
•Session storage
•Distributed cache

35. Use Cases
•Document storage
(sparse structure)
•Distributed file storage
(small size, large soon)
•Session storage
•Distributed cache
• Browser-only/Mobile Apps

36. Getting Started

37. Getting Started
• Download a package from
http://downloads.basho.com

38. Getting Started
• Download a package from
http://downloads.basho.com
• Set the node name, IPs

39. Getting Started
• Download a package from
http://downloads.basho.com
• Set the node name, IPs
• Start up the node

40. Getting Started
• Download a package from
http://downloads.basho.com
• Set the node name, IPs
• Start up the node
• Join a cluster

41. Getting Started
• Download a package from
http://downloads.basho.com
• Set the node name, IPs
• Start up the node
• Join a cluster
• Start storing/retrieving values

42. Riak in Ruby

43. Riak in Ruby
• Three gems

44. Riak in Ruby
• Three gems
• riak-client (basic ops)

45. Riak in Ruby
• Three gems
• riak-client (basic ops)
• ripple (ODM)

46. Riak in Ruby
• Three gems
• riak-client (basic ops)
• ripple (ODM)
• riak-sessions
(Rack/Rails session stores)

47. Riak in Ruby
• Three gems
• riak-client (basic ops)
• ripple (ODM)
• riak-sessions
(Rack/Rails session stores)
• All HTTP - Protobuffs coming

48. require ‘riak’

49. require ‘riak’
• Make a client object
client = Riak::Client.new

50. require ‘riak’
• Make a client object
client = Riak::Client.new
• Get a bucket
bucket = client.bucket(‘foo’) # Riak::Bucket

51. require ‘riak’
• Make a client object
client = Riak::Client.new
• Get a bucket
bucket = client.bucket(‘foo’) # Riak::Bucket
• Get an object from the bucket
obj = bucket.get(‘bar’) # Riak::RObject

52. require ‘riak’
• Make a client object
client = Riak::Client.new
• Get a bucket
bucket = client.bucket(‘foo’) # Riak::Bucket
• Get an object from the bucket
obj = bucket.get(‘bar’) # Riak::RObject
• Initialize a new object
obj = bucket.new(‘baz’)

53. Riak::RObject

54. Riak::RObject
•Get/set object key
obj.key = “bar”

55. Riak::RObject
•Get/set object key
obj.key = “bar”
•Get/set content-type
obj.content_type = ‘application/json’

56. Riak::RObject
•Get/set object key
obj.key = “bar”
•Get/set content-type
obj.content_type = ‘application/json’
•Get/set the object body data
obj.data = {“name” => “Sean”}

57. Riak::RObject
•Get/set object key
obj.key = “bar”
•Get/set content-type
obj.content_type = ‘application/json’
•Get/set the object body data
obj.data = {“name” => “Sean”}
•Store the object
obj.store

58. More RObject

59. More RObject
•Get object’s bucket
obj.bucket

60. More RObject
•Get object’s bucket
obj.bucket
•Delete the object
obj.delete # freezes the object

61. More RObject
•Get object’s bucket
obj.bucket
•Delete the object
obj.delete # freezes the object
•Detect/extract siblings (more later)
obj.conflict? && obj.siblings
# Array<RObject>

62. More RObject
•Get object’s bucket
obj.bucket
•Delete the object
obj.delete # freezes the object
•Detect/extract siblings (more later)
obj.conflict? && obj.siblings
# Array<RObject>
•Follow/traverse links (more later)
obj.walk(:tag => “friend”)

63. Riak::Bucket

64. Riak::Bucket
•Set the replication factor
bucket.n_val = 5

65. Riak::Bucket
•Set the replication factor
bucket.n_val = 5
•Set default request quorums
bucket.r = 3 # w,dw,rw
# number or one/all/quorum

66. Riak::Bucket
•Set the replication factor
bucket.n_val = 5
•Set default request quorums
bucket.r = 3 # w,dw,rw
# number or one/all/quorum
•List keys
bucket.keys # pass block to stream

67. Riak::Bucket
•Set the replication factor
bucket.n_val = 5
•Set default request quorums
bucket.r = 3 # w,dw,rw
# number or one/all/quorum
•List keys
bucket.keys # pass block to stream
•Set consistency flag (allow sibling generation)
obj.allow_mult = true

68. Links: Lightweight
Relationships

69. Link Header
Link: </riak/demo/test1>; riaktag=”userinfo”

70. Link Header
bucket
Link: </riak/demo/test1>; riaktag=”userinfo”

71. Link Header
bucket
Link: </riak/demo/test1>; riaktag=”userinfo”
key

72. Link Header
bucket tag
Link: </riak/demo/test1>; riaktag=”userinfo”
key

73. Links in Ruby API

74. Links in Ruby API
• Create a link
Riak::Link.new(“/riak/bucket/key”,
“tag”)

75. Links in Ruby API
• Create a link
Riak::Link.new(“/riak/bucket/key”,
“tag”)
• Read an#object’s links
obj.links Set<Riak::Link>

76. Links in Ruby API
• Create a link
Riak::Link.new(“/riak/bucket/key”,
“tag”)
• Read an#object’s links
obj.links Set<Riak::Link>
• Convert<< obj2.to_link(“next”)
obj.links
an object to a link
obj.store

77. Link-Walking

78. Link-Walking
• Asks Riak to traverse a sequence of
links via special URL

79. Link-Walking
• Asks Riak to traverse a sequence of
links via special URL
• Filter by bucket/tag

80. Link-Walking
• Asks Riak to traverse a sequence of
links via special URL
• Filter by bucket/tag
• Can return results from
intermediate traversals

81. Link-Walking
• Asks Riak to traverse a sequence of
links via special URL
• Filter by bucket/tag
• Can return results from
intermediate traversals
• Response is nested multipart/mixed
(riak-client handles this for you)

82. L/W Example

83. L/W Example
GET /riak/demo/test1/_,friend,1
Start at demo/test1, follow all links
tagged “friend” and return the objects
obj = client[‘demo’][‘test1’]
obj.walk(:tag => “friend”,:keep => true)

84. L/W Example
GET /riak/demo/test1/_,friend,1
Start at demo/test1, follow all links
tagged “friend” and return the objects
obj = client[‘demo’][‘test1’]
obj.walk(:tag => “friend”,:keep => true)

85. L/W Example

86. L/W Example
GET /riak/demo/test1/_,_,_/_,_,1
Start at demo/test1, follow all links,
follow all links from those, return
everything from the last set
obj.walk({},{:keep => true})

87. Map-Reduce

88. list of keys
Map-Reduce

89. list of keys
map map map map map
Map-Reduce

90. list of keys
map map map map map
reduce
Map-Reduce

91. list of keys
map map map map map
reduce
results
Map-Reduce

92. Map
in Javascript
function(value, keyData, arg) {
var object = Riak.mapValuesJson(value)[0];
for(field in arg) {
if(object[field] != arg[field]) {
return [];
}
}
return [object];
}

93. Map
in Javascript
Riak object
function(value, keyData, arg) {
var object = Riak.mapValuesJson(value)[0];
for(field in arg) {
if(object[field] != arg[field]) {
return [];
}
}
return [object];
}

94. Map
in Javascript
Riak object
function(value, keyData, arg) {
var object = Riak.mapValuesJson(value)[0];
for(field in arg) {
{
if(object[field] != arg[field]) {
return []; bucket:”foo”,
} key:”bar”,
} vclock:”...”,
return [object];
} values:[
{metadata:{...},
data:”....”}
]
}

95. Map
in Javascript
function(value, keyData, arg) {
var object = Riak.mapValuesJson(value)[0];
for(field in arg) {
if(object[field] != arg[field]) {
return [];
}
}
return [object];
}

96. Map
in Javascript
Key-specific
data
function(value, keyData, arg) {
var object = Riak.mapValuesJson(value)[0];
for(field in arg) {
if(object[field] != arg[field]) {
return [];
}
}
return [object];
}

97. Map
in Javascript
function(value, keyData, arg) {
var object = Riak.mapValuesJson(value)[0];
for(field in arg) {
if(object[field] != arg[field]) {
return [];
}
}
return [object];
}

98. Map
in Javascript
Phase-
specific data
function(value, keyData, arg) {
var object = Riak.mapValuesJson(value)[0];
for(field in arg) {
if(object[field] != arg[field]) {
return [];
}
}
return [object];
}

99. Map
in Javascript
function(value, keyData, arg) {
var object = Riak.mapValuesJson(value)[0];
for(field in arg) {
if(object[field] != arg[field]) {
return [];
}
}
return [object];
}

100. Map
in Javascript
function(value, keyData, arg) { Parse JSON data
var object = Riak.mapValuesJson(value)[0];
for(field in arg) {
if(object[field] != arg[field]) {
return [];
}
}
return [object];
}

101. Map
in Javascript
function(value, keyData, arg) {
var object = Riak.mapValuesJson(value)[0];
for(field in arg) {
if(object[field] != arg[field]) {
return [];
}
}
return [object];
}

102. Map
in Javascript
function(value, keyData, arg) {
var object = Riak.mapValuesJson(value)[0];
for(field in arg) {
if(object[field] != arg[field]) {
return [];
}
} Check field equality
return [object];
}

103. Map
in Javascript
function(value, keyData, arg) {
var object = Riak.mapValuesJson(value)[0];
for(field in arg) {
if(object[field] != arg[field]) {
return [];
}
}
return [object];
}

104. Map
in Javascript
function(value, keyData, arg) {
var object = Riak.mapValuesJson(value)[0];
for(field in arg) {
if(object[field] != arg[field]) {
return []; Return nothing if unequal
}
}
return [object];
}

105. Map
in Javascript
function(value, keyData, arg) {
var object = Riak.mapValuesJson(value)[0];
for(field in arg) {
if(object[field] != arg[field]) {
return [];
}
}
return [object];
}

106. Map
in Javascript
function(value, keyData, arg) {
var object = Riak.mapValuesJson(value)[0];
for(field in arg) {
if(object[field] != arg[field]) {
return [];
}
}
return [object]; Return the object if all equal
}

107. Map
in Javascript
function(value, keyData, arg) {
var object = Riak.mapValuesJson(value)[0];
for(field in arg) {
if(object[field] != arg[field]) {
return [];
}
}
return [object];
}

108. Reduce
in Javascript
function(values,arg) {
return values.sort(function(a,b){
return a[arg] - b[arg];
});
}

109. Reduce
in Javascript
Map results
function(values,arg) {
return values.sort(function(a,b){
return a[arg] - b[arg];
});
}

110. Reduce
in Javascript
function(values,arg) {
return values.sort(function(a,b){
return a[arg] - b[arg];
});
}

111. Reduce
in Javascript
Phase-
specific data
function(values,arg) {
return values.sort(function(a,b){
return a[arg] - b[arg];
});
}

112. Reduce
in Javascript
function(values,arg) {
return values.sort(function(a,b){
return a[arg] - b[arg];
});
}

113. Reduce
in Javascript
function(values,arg) {
return values.sort(function(a,b){
return a[arg] - b[arg];
}); Custom sort on
}

114. Reduce
in Javascript
function(values,arg) {
return values.sort(function(a,b){
return a[arg] - b[arg];
});
}

115. Reduce
in Javascript
function(values,arg) {
return values.sort(function(a,b){
return a[arg] - b[arg];
});
}
Reduce potentially called multiple times!!

116. Example Query

117. Example Query
{“inputs”: “goog”,
“query”: [{“map”:{“language”:”javascript”,
“name”: “App.findHighGreater”,
“arg”: 600.0,
“keep”: false},
{“reduce”:{“language”:”javascript”,
“name”: “Riak.reduceMax”,
“keep”: true}]}

118. Example Query
{“inputs”: “goog”,
“query”: [{“map”:{“language”:”javascript”,
“name”: “App.findHighGreater”,
“arg”: 600.0,
“keep”: false},
{“reduce”:{“language”:”javascript”,
“name”: “Riak.reduceMax”,
“keep”: true}]}
Riak::MapReduce.new(c).add(‘goog’).
map(‘App.findHighGreater’, :arg => 600.0).
reduce(“Riak.reduceMax”, :keep => true).run

119. Built-in Functions

120. Built-in Functions
• Riak.mapValues

121. Built-in Functions
• Riak.mapValues
• Riak.mapValuesJson

122. Built-in Functions
• Riak.mapValues
• Riak.mapValuesJson
• Riak.mapByFields

123. Built-in Functions
• Riak.mapValues
• Riak.mapValuesJson
• Riak.mapByFields
• Riak.reduceSum

124. Built-in Functions
• Riak.mapValues
• Riak.mapValuesJson
• Riak.mapByFields
• Riak.reduceSum
• Riak.reduceSort

125. Built-in Functions
• Riak.mapValues
• Riak.mapValuesJson
• Riak.mapByFields
• Riak.reduceSum
• Riak.reduceSort
• Riak.reduceMin/reduceMax

126. Ripple - ODM

127. Document Models
class Person
include Ripple::Document
property :name, String, :presence => true
many :addresses
many :friends, :class => Person
end
class Address
include Ripple::EmbeddedDocument
property :street, String
end

128. Rails Setup

129. Rails Setup
# Gemfile
gem 'curb' # Faster HTTP
gem 'yajl-ruby' # Faster JSON
gem 'ripple'

130. Rails Setup
# Gemfile
gem 'curb' # Faster HTTP
gem 'yajl-ruby' # Faster JSON
gem 'ripple'

131. Rails Setup
# Gemfile
gem 'curb' # Faster HTTP
gem 'yajl-ruby' # Faster JSON
gem 'ripple'
$ gem install curb yajl-ruby ripple

132. Rails Setup (cont.)

133. Rails Setup (cont.)
# config/ripple.yml
development:
host: 127.0.0.1
port: 8098

134. Rails Setup (cont.)
# config/ripple.yml
development:
host: 127.0.0.1
port: 8098
# config/application.rb
require 'ripple/railtie'

135. Ripple Roadmap

136. Ripple Roadmap
• Testing server (edge on Github)

137. Ripple Roadmap
• Testing server (edge on Github)
• Protocol Buffers support

138. Ripple Roadmap
• Testing server (edge on Github)
• Protocol Buffers support
• Streaming MapReduce

139. Ripple Roadmap
• Testing server (edge on Github)
• Protocol Buffers support
• Streaming MapReduce
• Ripple-specific built-ins

140. Ripple Roadmap
• Testing server (edge on Github)
• Protocol Buffers support
• Streaming MapReduce
• Ripple-specific built-ins
• Better ActionView support
(form_for)

141. Happy Fun
Demo Time

142. Plug
basho

143. Plug
Interested in learning about support,
consulting, or Enterprise features?
basho

144. Plug
Interested in learning about support,
consulting, or Enterprise features?
Email info@basho.com or go to
http://www.basho.com/
contact.html to talk with us.
basho

145. Plug
Interested in learning about support,
consulting, or Enterprise features?
Email info@basho.com or go to
http://www.basho.com/
contact.html to talk with us.
basho

146. Plug
Interested in learning about support,
consulting, or Enterprise features?
Email info@basho.com or go to
http://www.basho.com/
contact.html to talk with us.
www.basho.com
sean@basho.com
@seancribbs
basho

147. Questions