SlideShare a Scribd company logo
1 of 99
Download to read offline
Writing DSLs
with Parslet
Jason Garber
Wicked Good Ruby Conf
D
D
T
D
D
T

TATFT
D
D
T
le!
gi
A

Continuous
Integration

TATFT
VIM

Continuo

us Delive

ir
Pa
ng
mi
ram
og
Pr

ro
do
o
om
P
ry

Scru

m
Parsing
DSLS
Domain-Specific Languages
<?xml version="1.0"?> <configuration><configSections><sectionGr
oup name="userSettings" type="System.Configuration.UserSettings
Group, System, Version=2.0.0.0, Culture=neutral, PublicKeyToken
=b77a5c561934e089"><section name="MSDNSampleSettings.My.MySetti
ngs" type="System.Configuration.ClientSettingsSection, System,
Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e0
89" allowExeDefinition="MachineToLocalUser" requirePermission="
false"/></sectionGroup></configSections> ... <userSettings><MSD
NSampleSettings.My.MySettings><setting name="Setting" serialize
As="String"><value>SomeDefaultValue</value></setting></MSDNSamp
leSettings.My.MySettings></userSettings> </configuration>
</xml>
I

DSLs
RssReader::Application.routes.draw do
devise_for :users
resources :feeds, only: [:edit, :update, :show, :index]
post '/feeds', to: 'feeds#create', as: 'create_feed'
resources :users do
get 'page/:page', action: :index, on: :collection
end
resources :posts do
member do
put :update_category
end
end
get '/my_profile', to: 'users#my_profile', as: :my_profile
root to: "home#home"
end
describe Stack do
context "stack with one item" do
let(:stack) { a_stack_with_one_item }
context "when popped" do
before { stack.pop }
it { should be_empty }
end
end
end
click_on "Sign Up"
fill_in "Email", with: account[:email]
fill_in "Password", with: account[:password]
fill_in "Confirmation", with:
account[:password_confirmation]
fill_in "Name", with: account[:name]
select account[:birthyear].to_s, from: "Year born"
check "Terms"
click_on "I'm ready to join!"
current_path.should eq "/accounts/#{account.id}/dashboard"
page.should have_content "Dashboard"
desc 'Generate markup and stylesheets and open browser preview'
task :preview => FileList['*.html'] + FileList['*.css'] do |t|
sh 'open -g index.html'
end
rule '.html' => '.haml' do |t|
puts "Rebuilding #{t.name}"
sh "haml #{t.source} #{t.name}"
end
rule '.css' => lambda { |cssfile| source(cssfile) } do |t|
puts "Rebuilding #{t.name}"
sh "sass #{t.source} #{t.name}"
end
get '/' do
@posts = Post.all(:order => [:id.desc], :limit => 20)
erb :index
end
get '/post/new' do
erb :new
end
get '/post/:id' do
@post = Post.get(params[:id])
erb :post
end
post '/post/create' do
post = Post.new(params)
status 201
redirect "/post/#{post.id}"
end
rule(:table) do
(str("table") >> attributes?.as(:attributes) >> str(".n")).maybe >>
table_row.repeat(1).as(:content) >> block_end
end
rule(:table_row) do
table_row_attributes >> table_row_content >> end_table_row
end
rule(:table_row_attributes) { (attributes?.as(:attributes) >> str(". ")).maybe }
rule(:table_row_content) { (table_header | table_data).repeat(1).as(:content) }
rule(:end_table_row) { str("|") >> (block_end.present? | (str("n"))) }
rule(:table_header) do
str("|_. ") >> table_content.as(:content)
end
rule(:table_data) do
str("|") >> str("n").absent? >> td_attributes? >> table_content.as(:content)
end
Internal DSLs
Fluent Interfaces
EXTERNAL
DSLS
upstream puma {
server unix:///tmp/sockets/puma.sock fail_timeout=0;
}
server {
listen 80 default deferred;
server_name promptworks.com www.promptworks.com;
root /srv/promptworks/public;
charset utf-8;
if (-f $document_root/system/maintenance.html) {
return 503;
}
error_page 503 @maintenance;
location @maintenance {
rewrite ^(.*)$ /system/maintenance.html last;
break;
}
}
<([A-Z][A-Z0-9]*)b[^>]*>(.*?)</1>
b(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?).){3}
(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)b
SELECT DISTINCT sc1.id
FROM (
SELECT DATE_ADD(entry_point.start_date, INTERVAL (digits_1.digit * 10 + d
(1 << (DAYOFWEEK(DATE_ADD(entry_point.start_date, INTERVAL (digits_1.digi
FROM (SELECT CAST('2012-03-01' AS date) AS start_date) AS entry_point
INNER JOIN (SELECT 0 AS digit UNION SELECT 1 UNION SELECT 2 UNION SELECT
ON (digits_1.digit * 10 + digits_2.digit) <= (DATEDIFF(DATE_ADD(entry_poi
) AS md
INNER JOIN schedules AS sc1
INNER JOIN negative_link_influences AS neg
on (sc1.call_type_id = neg.call_type_id or neg.call_type_id = -1)
WHERE sc1.schedule_on = md.date
AND neg.affected_shift = 0;
BEGIN	 TOTAL=0 }
{
	{ TOTAL = TOTAL + $1 }
END	 print TOTAL/NR }
{

{print "<li><a href="" $1 "">"
$1 "</a></li>"}
! doctype html
html
head
title Test
body
- unless items.empty?
ol
- items.each do |item|
li
= item
- else
p No items
Feature: Searching music
As a User
I want to be able to search music
So I can play it
Background:
Given I am logged in
Scenario: Search music
Given I am on the search screen
When I search for "mix"
Then I should see the mixtape
class erlang {
file { "/etc/apt/sources.list.d/esl-erlang.list":
ensure => present,
owner => root,
content => 'deb http://example.com/debian precise contrib';
}
exec { "apt-update":
command
=> "/usr/bin/apt-get update",
refreshonly => true;
}
package { "esl-erlang":
ensure => installed,
require => Exec['apt-update', 'import-key'];
}
}
include erlang
Heroku buildpack: Ruby
======================

This is a [Heroku buildpack](http://devcenter.heroku.com/article
It uses [Bundler](http://gembundler.com) for dependency manageme
Usage
----### Ruby
Example Usage:

$ heroku create --stack cedar --buildpack https://github.com
$ git push heroku master
title = "T??? Example"
[owner]
name = "Tom Preston-Werner"
organization = "GitHub"
bio = "GitHub Cofounder & CEOnLikes tater tots and
beer."
dob = 1979-05-27T07:32:00Z # First class dates? Why
not?
[database]
server = "192.168.1.1"
ports = [ 8001, 8001, 8002 ]
connection_max = 5000
enabled = true
global= {
time 4/4
key c major
tempo "Allegro" 4 = 132
set Score.skipBars = ##t
}
Violinone = new Voice { relative c''{
set Staff.midiInstrument = #"violin"
set tupletSpannerDuration = #(ly:make-moment 1 4)
% Jason solo
R1 * 9
r2 r4 times 2/3 { c4( b8)
gsave
1 0.5 scale
70 100 48 0 360 arc
fill
grestore
/Helvetica-Bold 14 selectfont
1.0 setgray
29 45 moveto
(Hello, world!) show
showpage
.accordion {
li {
border-top: 1px solid #e2e4e6;
&:first-child { border-top-color: transparent; }
}
a {
@include rem(padding, 5px 10px 6px);
&.icon {
padding-left: 40px;
position: relative;
}
&.icon img {
@include rem(left, 10px);
margin-top: -2px;
position: absolute;
}
}
}
h1. Give Textile a try!
A *simple* paragraph with a line break,
some _emphasis_ and a "link":http://redcloth.org
* an item
* and another
# one
# two
# three
h1. Give Textile a try!
A *simple* paragraph with a line break,
some _emphasis_ and a "link":http://redcloth.org
* an item
* and another
# one
# two
# three
A_HLGN = /(?:<(?!>)|<>|=|[()]+)/
A_VLGN = /[-^~]/
C_CLAS = '(?:([^)]+))'
C_LNGE = '(?:[[^]]+])'
C_STYL = '(?:{[^}]+})'
S_CSPN = '(?:d+)'
S_RSPN = '(?:/d+)'
A = "(?:#{A_HLGN}?#{A_VLGN}?|#{A_VLGN}?#{A_HLGN}?)"
S = "(?:#{S_CSPN}?#{S_RSPN}|#{S_RSPN}?#{S_CSPN}?)"
C = "(?:#{C_CLAS}?#{C_STYL}?#{C_LNGE}?|#{C_STYL}?
#{C_LNGE}?#{C_CLAS}?|#{C_LNGE}?#{C_STYL}?#{C_CLAS}?)"
PUNCT = Regexp::quote( '!"#$%&'*+,-./:;=?@^_`|~' )
HYPERLINK = '(S+?)([^ws/;=?]*?)(s|$)'
def links( text )
text.gsub!( /
([s[{(]|[#{PUNCT}])?
# $pre
"
# start
(#{C})
# $atts
([^"]+?)
# $text
s?
(?:(([^)]+?))(?="))?
# $title
":
(S+?)
# $url
(/)?
# $slash
([^w/;]*?)
# $post
(?=s|.[#{PUNCT}]+|$)
/x ) do |m|
pre,atts,text,title,url,slash,post = $~[1..7]
url = check_refs( url )
atts = pba( atts )
atts << " title="#{ title }"" if title
atts = shelve( atts ) if atts
"#{ pre }<a href="#{ url }#{ slash }"#{ atts }>" +
"#{ text }</a>#{ post }"
end
end
“Some people, when confronted
with a problem, think ‘I know, I'll
use regular expressions.’
Now they have two problems.”
— Jamie Zawinski
%%{
machine superredcloth_scan;
include superredcloth_common "ext/superredcloth_scan/superredcloth_common.rl";
action extend { extend = rb_hash_aref(regs, ID2SYM(rb_intern("type"))); }
# blocks
notextile_tag_start = "<notextile>" ;
notextile_tag_end = "</notextile>" CRLF? ;
notextile_block_start = ( "notextile" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ;
pre_tag_start = "<pre" [^>]* ">" (space* "<code>")? ;
pre_tag_end = ("</code>" space*)? "</pre>" CRLF? ;
pre_block_start = ( "pre" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ;
bc_start = ( "bc" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ; btype = ( "p" | "h1"...
block_start = ( btype >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ;
next_block_start = ( btype A C :> "."+ " " ) >A @{ p = reg - 1; } ;
pre_tag := |*
pre_tag_end
{ CAT(block); DONE(block); fgoto main; };
default => esc_pre;
*|;
notextile_tag := |*
notextile_tag_end
default => cat;
*|;
}%%

{ DONE(block); fgoto main; };
void rb_str_cat_escaped(VALUE str, char *ts, char *te, unsigned int opts);
void rb_str_cat_escaped_for_preformatted(VALUE str, char *ts, char *te, unsigned int opts);
VALUE superredcloth_inline(VALUE, char *, char *, VALUE);
VALUE superredcloth_inline2(VALUE, VALUE, VALUE);
#define CAT(H)
#define CLEAR(H)
#define INLINE(H, T)

rb_str_cat(H, ts, te-ts)
H = rb_str_new2("")
rb_str_append(H, rb_funcall(rb_formatter, rb_intern(#T), 1, regs))

VALUE superredcloth_transform(rb_formatter, p, pe, refs)
VALUE rb_formatter;
char *p, *pe;
VALUE refs;
{
if (RSTRING(block)->len > 0)
{
ADD_BLOCK();
}
if ( NIL_P(refs) && rb_funcall(refs_found, rb_intern("empty?"), 0) == Qfalse ) {
return superredcloth_transform(rb_formatter, orig_p, orig_pe, refs_found);
} else {
rb_funcall(rb_formatter, rb_intern("after_transform"), 1, html);
return html;
}
}
%%{
machine superredcloth_scan;
include superredcloth_common "ext/superredcloth_scan/superredcloth_common.rl";
action extend { extend = rb_hash_aref(regs, ID2SYM(rb_intern("type"))); }
# blocks
notextile_tag_start = "<notextile>" ;
notextile_tag_end = "</notextile>" CRLF? ;
notextile_block_start = ( "notextile" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ;
pre_tag_start = "<pre" [^>]* ">" (space* "<code>")? ;
pre_tag_end = ("</code>" space*)? "</pre>" CRLF? ;
pre_block_start = ( "pre" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ;
bc_start = ( "bc" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ; btype = ( "p" | "h1"...
block_start = ( btype >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ;
next_block_start = ( btype A C :> "."+ " " ) >A @{ p = reg - 1; } ;
pre_tag := |*
pre_tag_end
{ CAT(block); DONE(block); fgoto main; };
default => esc_pre;
*|;
notextile_tag := |*
notextile_tag_end
default => cat;
*|;
}%%

{ DONE(block); fgoto main; };
%%{
machine superredcloth_scan;
include superredcloth_common "ext/superredcloth_scan/superredcloth_common.rl";
action extend { extend = rb_hash_aref(regs, ID2SYM(rb_intern("type"))); }
# blocks
notextile_tag_start = "<notextile>" ;
notextile_tag_end = "</notextile>" CRLF? ;
notextile_block_start = ( "notextile" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ;
pre_tag_start = "<pre" [^>]* ">" (space* "<code>")? ;
pre_tag_end = ("</code>" space*)? "</pre>" CRLF? ;
pre_block_start = ( "pre" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ;
bc_start = ( "bc" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ; btype = ( "p" | "h1"...
block_start = ( btype >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ;
next_block_start = ( btype A C :> "."+ " " ) >A @{ p = reg - 1; } ;
pre_tag := |*
pre_tag_end
{ CAT(block); DONE(block); fgoto main; };
default => esc_pre;
*|;
notextile_tag := |*
notextile_tag_end
default => cat;
*|;
}%%

{ DONE(block); fgoto main; };
/*
%%{
* redcloth_scan.c.rl
*
machine superredcloth_scan;
* Copyright (C) 2009 Jason Garber
include superredcloth_common "ext/superredcloth_scan/superredcloth_common.rl";
*/
#define redcloth_scan_c
action extend { extend = rb_hash_aref(regs, ID2SYM(rb_intern("type"))); }
#define RSTRING_NOT_MODIFIED
# blocks
#include <ruby.h>
notextile_tag_start = "<notextile>" ;
#include "redcloth.h"
notextile_tag_end = "</notextile>" CRLF? ;
notextile_block_start = ( "notextile" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ;
VALUE mRedCloth, super_ParseError, super_RedCloth, super_HTML, super_LATEX;
pre_tag_start = "<pre" [^>]* ">" (space* "<code>")? ;
VALUE SYM_escape_preformatted, SYM_escape_attributes;
pre_tag_end = ("</code>" space*)? "</pre>" CRLF? ;
pre_block_start = ( "pre" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ;
#line 23 "ext/redcloth_scan/redcloth_scan.c"
bc_start = ( "bc" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ; btype = ( "p" | "h1"...
static const unsigned char _redcloth_scan_actions[] = {
block_start = ( btype >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ;
0, 1, 0, 1, 2, 1, 3, 1,
next_block_start = ( btype A C :> "."+ " " ) >A @{ p = reg - 1; } ;
4, 1, 5, 1, 6, 1, 7, 1,
9, 1, 10, 1, 11, 1, 12, 1,
pre_tag := |*
13, 1, 14, 1, 15, 1, 16, 1,
pre_tag_end
{ CAT(block); DONE(block); fgoto main; };
17, 1, 18, 1, 19, 1, 20, 1,
default => esc_pre;
21, 1, 22, 1, 23, 1, 24, 1,
*|;
25, 1, 26, 1, 27, 1, 28, 1,
29, 1, 30, 1, 34, 1, 35, 1,
notextile_tag := |*
36, 1, 38, 1, 40, 1, 42, 1,
notextile_tag_end
{ DONE(block); fgoto main; };
43, 1, 44, 1, 45, 1, 48, 1,
default => cat;
57, 1, 58, 1, 59, 1, 60, 1,
*|;
61, 1, 62, 1, 63, 1, 64, 1,
65, 1, 66, 1, 70, 1, 73, 1,
}%%
grammar Arithmetic
rule additive
multitive ( '+' multitive )*
end
rule multitive
primary ( [*/%] primary )*
end
rule primary
'(' additive ')' / number
end
rule number
'-'? [1-9] [0-9]*
end
end
require 'parslet'
class MiniParser < Parslet::Parser
rule(:integer) { match('[0-9]').repeat(1) }
root(:integer)
end
MiniParser.new.parse("1324321")
# => "1324321"@0
Parslet::Atoms
str('Boston')
Parslet::Atoms
str('Boston').parse('Boston')
Parslet::Atoms
str('Boston').parse('Boston')
# => "Boston"@0
Parslet::Atoms
str('Boston').parse('Boston')
# => "Boston"@0
match('[0-9a-f]')
Parslet::Atoms
str('Boston').parse('Boston')
# => "Boston"@0
match('[0-9a-f]')
any
Operators
str('Wicked') >> str('Good')
str('Ruby') | str('Elixir')
match('[Bb]') >> str('oston') |
match('[Mm]') >> str('assachusetts')
Repetition
str('foo').repeat
str('foo').repeat(1)
str('foo').repeat(1,3)
str('foo').repeat(0, nil)
str('foo').maybe
Presence
str('Java') >> str('Script').present?

str('0').repeat(1).absent? >>
match('[d]').repeat(1)
1. Create a grammar
What should be legal syntax?
2. Annotate the grammar:
What is important data?
3. Create a transformation:
How do I want to work with that data?
1. Create a grammar
What should be legal syntax?
2. Annotate the grammar:
What is important data?
3. Create a transformation:
How do I want to work with that data?
Capture
str('Common').parse('Common')
# => "Common"@0
str('Common').as(:park).parse('Common')
# => {:park=>"Common"@0}
Capture
str('a').repeat.as(:b)
# => {:b=>"aaa"@0}
str('a').as(:b).repeat
# => [{:b=>"a"@0}, {:b=>"a"@1}, {:b=>"a"@2}]
str('a').as(:a) >> str('b').as(:b) >> str('c')
# => {:a=>"a"@0, :b=>"b"@1}
rule(:table) do
(str("table") >> attributes?.as(:attributes) >> str(".n")).maybe >>
table_row.repeat(1).as(:content) >> block_end
end
rule(:table_row) do
table_row_attributes >> table_row_content >> end_table_row
end
rule(:table_row_attributes) { (attributes?.as(:attributes) >> str(". ")).maybe }
rule(:table_row_content) { (table_header | table_data).repeat(1).as(:content) }
rule(:end_table_row) { str("|") >> (block_end.present? | (str("n"))) }
rule(:table_header) do
str("|_. ") >> table_content.as(:content)
end
rule(:table_data) do
str("|") >> str("n").absent? >> td_attributes? >> table_content.as(:content)
end
table(#prices).
| Adults
| $5 |
| Children | $2 |

<table id="prices">
<tr>
<td>Adults</td>
<td>$5</td>
</tr>
<tr>
<td>Children</td>
<td>$2</td>
</tr>
</table>
rule(:table) do
(str("table") >> attributes?.as(:attributes) >> str(".n")).maybe >>
table_row.repeat(1).as(:content) >> block_end
end
rule(:table_row) do
table_row_attributes >> table_row_content >> end_table_row
end
rule(:table_row_attributes) { (attributes?.as(:attributes) >> str(". ")).maybe }
rule(:table_row_content) { (table_header | table_data).repeat(1).as(:content) }
rule(:end_table_row) { str("|") >> (block_end.present? | (str("n"))) }
rule(:table_header) do
str("|_. ") >> table_content.as(:content)
end
rule(:table_data) do
str("|") >> str("n").absent? >> td_attributes? >> table_content.as(:content)
end
def parenthesized(atom)
str('(') >> atom >> str(')')
end
parenthesized(match['d']).parse("(500)")
class HtmlTag < Parslet::Parser
root(:tag)
rule(:tag) { open_tag | close_tag | self_closing_tag | comment_tag }
rule(:open_tag) { str("<") >> tag_name >> attributes? >> str(">") }
rule(:close_tag) { str("</") >> tag_name >> str(">") }
rule(:tag_name) { match("[A-Za-z_:]") >> name_char.repeat }
...
end
class BlockHtmlTag < HtmlTag
rule(:tag_name) do
inline_tag_name.absent? >> any_tag_name
end
rule(:inline_tag_name) do
INLINE_TAGS.map {|name| str(name) }.reduce(:|)
end
end
> HtmlTag.new.open_tag.methods
=> [:name, :block, :try, :parslet, :to_s_inner, :parse, :apply, :cached?, ...]

> HtmlTag.new.open_tag.parse("<blockquote>")
=> "<blockquote>"@0
> HtmlTag.new.open_tag.parse("</blockquote>")
Parslet::ParseFailed: Failed to match sequence ('<' TAG_NAME ATTRIBUTES? '>') at line
1 char 2.
	 from /Users/jasongarber/.rvm/gems/ruby-2.0.0-p247/gems/parslet-1.5.0/lib/parslet/
cause.rb:63:in `raise'
	 from /Users/jasongarber/.rvm/gems/ruby-2.0.0-p247/gems/parslet-1.5.0/lib/parslet/
atoms/base.rb:46:in `parse'
	 from (irb):8
	 from /Users/jasongarber/.rvm/rubies/ruby-2.0.0-p247/bin/irb:16:in `<main>'
begin
RedClothParslet::Parser::BlockHtmlTag.new.tag.parse("<img>")
rescue Parslet::ParseFailed => failure
puts failure.cause.ascii_tree
end
Expected one of [OPEN_TAG, CLOSE_TAG, SELF_CLOSING_TAG, COMMENT_TAG] at line 1 char 1.
|- Failed to match sequence ('<' TAG_NAME ATTRIBUTES? '>') at line 1 char 2.
| `- Failed to match sequence ((!INLINE_TAG_NAME / &(INLINE_TAG_NAME NAME_CHAR{1, })) ANY_TAG_NAME)
at line 1 char 2.
|
`- Expected one of [!INLINE_TAG_NAME, &(INLINE_TAG_NAME NAME_CHAR{1, })] at line 1 char 2.
|
|- Input should not start with INLINE_TAG_NAME at line 1 char 2.
|
`- Input should start with INLINE_TAG_NAME NAME_CHAR{1, } at line 1 char 2.
|- Failed to match sequence ('</' TAG_NAME '>') at line 1 char 1.
| `- Expected "</", but got "<i" at line 1 char 1.
|- Failed to match sequence ('<' TAG_NAME ATTRIBUTES? SPACES? '/' '>') at line 1 char 2.
| `- Failed to match sequence ((!INLINE_TAG_NAME / &(INLINE_TAG_NAME NAME_CHAR{1, })) ANY_TAG_NAME)
at line 1 char 2.
|
`- Expected one of [!INLINE_TAG_NAME, &(INLINE_TAG_NAME NAME_CHAR{1, })] at line 1 char 2.
|
|- Input should not start with INLINE_TAG_NAME at line 1 char 2.
|
`- Input should start with INLINE_TAG_NAME NAME_CHAR{1, } at line 1 char 2.
`- Failed to match sequence ('<!--' (!COMMENT_TAG_END .){0, } COMMENT_TAG_END) at line 1 char 1.
`- Expected "<!--", but got "<img" at line 1 char 1.
I

Objects
describe RedClothParslet::Parser::HtmlTag do
it { should parse("<div>") }
it { should parse("<hr />") }
it { should parse("</div>") }
it { should parse("<!-- an HTML comment -->") }
describe "attribute" do
subject { described_class.new.attribute }
it { should parse(" class='awesome'") }
it { should_not parse(' 9kittens="cute"') }
end
end
$ rspec spec/parser/html_tag_spec.rb
1) RedClothParslet::Parser::HtmlTag tag
Failure/Error: it { should parse("</div>") }
expected TAG to be able to parse "</div>"
Expected one of [OPEN_TAG, CLOSE_TAG, SELF_CLOSING_TAG, COMMENT_TAG] at line 1 char 1.
|- Failed to match sequence ('<' TAG_NAME ATTRIBUTES? '>') at line 1 char 2.
| `- Failed to match sequence ([A-Za-z_:] NAME_CHAR{0, }) at line 1 char 2.
|
`- Failed to match [A-Za-z_:] at line 1 char 2.
|- Failed to match sequence ('<' TAG_NAME '>') at line 1 char 2.
| `- Failed to match sequence ([A-Za-z_:] NAME_CHAR{0, }) at line 1 char 2.
|
`- Failed to match [A-Za-z_:] at line 1 char 2.
|- Failed to match sequence ('<' TAG_NAME ATTRIBUTES? SPACES? '/' '>') at line 1 char 2.
| `- Failed to match sequence ([A-Za-z_:] NAME_CHAR{0, }) at line 1 char 2.
|
`- Failed to match [A-Za-z_:] at line 1 char 2.
`- Failed to match sequence ('<!--' (!COMMENT_TAG_END .){0, } COMMENT_TAG_END) at line 1 char 1.
`- Expected "<!--", but got "</di" at line 1 char 1.
# ./spec/parser/html_tag_spec.rb:9:in `block (3 levels) in <top (required)>'
13/13:

100% |==========================================| Time: 00:00:00

Finished in 0.01879 seconds
13 examples, 1 failure
1. Create a grammar
What should be legal syntax?
2. Annotate the grammar:
What is important data?
3. Create a transformation:
How do I want to work with that data?
Transformations
tree = {left: {int: '1'},
op: '+',
right: {int: '2'}}
class T < Parslet::Transform
rule(int: simple(:x)) { Integer(x) }
end
T.new.apply(tree)
# => {:left=>1, :op=>"+", :right=>2}
Transformations
tree = {left: {int: '1'},
op: '+',
right: {int: '2'}}
class T < Parslet::Transform
rule(int: simple(:x)) { Integer(x) }
rule(op: '+', left: simple(:l),
right: simple(:r)) { l + r }
end
T.new.apply(tree)
# => 3
Transformations
rule(:content => subtree(:c), :attributes => subtree(:a)) do |dict|
{:content => dict[:c], :opts => RedCloth::Ast::Attributes.new(dict[:a])}
end
rule(:table => subtree(:a)) do
RedCloth::Ast::Table.new(a[:content], a[:opts])
end
rule(:bq => subtree(:a)) do
RedCloth::Ast::Blockquote.new(a[:content], a[:opts])
end
Who you
callin’ slow?
class ErbParser < Parslet::Parser
rule(:ruby) { (str('%>').absent? >> any).repeat.as(:ruby) }
rule(:expression) { (str('=') >> ruby).as(:expression) }
rule(:comment) { (str('#') >> ruby).as(:comment) }
rule(:code) { ruby.as(:code) }
rule(:erb) { expression | comment | code }
rule(:erb_with_tags) { str('<%') >> erb >> str('%>') }
rule(:text) { (str('<%').absent? >> any).repeat(1) }

rule(:text_with_ruby) { (text.as(:text) | erb_with_tags).repeat.as(:t
root(:text_with_ruby)
end
class ErbParser < Parslet::Parser
rule(:ruby) { (str('%>').absent? >> any).repeat.as(:ruby) }
rule(:expression) { (str('=') >> ruby).as(:expression) }
rule(:comment) { (str('#') >> ruby).as(:comment) }
rule(:code) { ruby.as(:code) }
rule(:erb) { expression | comment | code }
rule(:erb_with_tags) { str('<%') >> erb >> str('%>') }
rule(:text) { (str('<%').absent? >> any).repeat(1) }

rule(:text_with_ruby) { (text.as(:text) | erb_with_tags).repeat.as(:t
root(:text_with_ruby)
end
class ErbParser < Parslet::Parser
rule(:ruby) { (str('%>').absent? >> any).repeat.as(:ruby) }
rule(:expression) { (str('=') >> ruby).as(:expression) }
rule(:comment) { (str('#') >> ruby).as(:comment) }
rule(:code) { ruby.as(:code) }
include Parslet::Accelerator
rule(:erb) { expression | comment | code }

optimized = apply( parser,
rule( (str(:x).absent? >> erb >> str('%>') }
rule(:erb_with_tags) { str('<%') >> any).repeat )
rule(:text) GobbleUp.new(x, 0) } >> any).repeat(1) }
{ { (str('<%').absent?
)

rule(:text_with_ruby) { (text.as(:text) | erb_with_tags).repeat.as(:t
root(:text_with_ruby)
end
Custom Atoms
•
“Who makes the best cheesesteaks in Boston?”
•
EngTagger: a corpus-trained, probabilistic tagger
•
Custom Parslet::Atom
•
Parse a limited set of natural-language queries
rule(:question) do
interrogative.maybe.as(:int) >>
verb_phrase.maybe.as(:verb) >>
superlative_phrase.maybe.as(:sup) >>
subject.as(:subj) >>
prepositional_phrase.repeat.as(:preps) >>
sentence_end.repeat.as(:punct)
end
rule(:verb_phrase)
rule(:verb_present)
rule(:verb_past)
rule(:verb_future)

{
{
{
{

(verb_present | verb_past | verb_future).as(:vp) }
word(:VBZ, :VB).as(:present) }
word(:VBD).as(:past) }
word(:MD).as(:future) >> word(:VB).maybe.as(:infinitive) }
Other Crazy Uses
Other Crazy Uses
•

User-supplied formulas / logic
Other Crazy Uses
•
Logs
•

User-supplied formulas / logic
Other Crazy Uses
•
Logs
•
Streaming text
•

User-supplied formulas / logic
Other Crazy Uses
•
Logs
•
Streaming text
•
The Right Reverend and Right Honourable the Lord
•
User-supplied formulas / logic

Bishop of London Richard John Carew Chartres
Who will write the next
awesome DSL?
Who
You will write the next
awesome DSL!.
Conclusions
•

Conclusions
DSLs make life better
•
•

Conclusions
DSLs make life better
internal_dsl > external_dsl if
internal_dsl.practicable?
Conclusions

•
•

DSLs make life better

•

Keep your parser clean

internal_dsl > external_dsl if
internal_dsl.practicable?
Conclusions

•
•

DSLs make life better

•
•

Keep your parser clean

internal_dsl > external_dsl if
internal_dsl.practicable?
“Situational awareness!”
jason@promptworks.com
@jasongarber

@promptworks

More Related Content

What's hot

Read data from Excel spreadsheets into R
Read data from Excel spreadsheets into RRead data from Excel spreadsheets into R
Read data from Excel spreadsheets into RRsquared Academy
 
PyCon Siberia 2016. Не доверяйте тестам!
PyCon Siberia 2016. Не доверяйте тестам!PyCon Siberia 2016. Не доверяйте тестам!
PyCon Siberia 2016. Не доверяйте тестам!Ivan Tsyganov
 
Refactoring using Codeception
Refactoring using CodeceptionRefactoring using Codeception
Refactoring using CodeceptionJeroen van Dijk
 
Parse Everything With Elixir
Parse Everything With ElixirParse Everything With Elixir
Parse Everything With ElixirGabriele Lana
 
SQL || overview and detailed information about Sql
SQL || overview and detailed information about SqlSQL || overview and detailed information about Sql
SQL || overview and detailed information about Sqlgourav kottawar
 
SQL querys in detail || Sql query slides
SQL querys in detail || Sql query slidesSQL querys in detail || Sql query slides
SQL querys in detail || Sql query slidesgourav kottawar
 
Why Go Scales
Why Go ScalesWhy Go Scales
Why Go ScalesEyal Post
 
Python 入門初體驗(程式語法)
Python 入門初體驗(程式語法)Python 入門初體驗(程式語法)
Python 入門初體驗(程式語法)政斌 楊
 
Zf2 how arrays will save your project
Zf2   how arrays will save your projectZf2   how arrays will save your project
Zf2 how arrays will save your projectMichelangelo van Dam
 
Business Rules with Brick
Business Rules with BrickBusiness Rules with Brick
Business Rules with Brickbrian d foy
 
Writing Readable Code with Pipes
Writing Readable Code with PipesWriting Readable Code with Pipes
Writing Readable Code with PipesRsquared Academy
 
Closure, Higher-order function in Swift
Closure, Higher-order function in SwiftClosure, Higher-order function in Swift
Closure, Higher-order function in SwiftSeongGyu Jo
 
Very basic functional design patterns
Very basic functional design patternsVery basic functional design patterns
Very basic functional design patternsTomasz Kowal
 
Database performance 101
Database performance 101Database performance 101
Database performance 101Leon Fayer
 

What's hot (20)

QA for PHP projects
QA for PHP projectsQA for PHP projects
QA for PHP projects
 
Read data from Excel spreadsheets into R
Read data from Excel spreadsheets into RRead data from Excel spreadsheets into R
Read data from Excel spreadsheets into R
 
PyCon Siberia 2016. Не доверяйте тестам!
PyCon Siberia 2016. Не доверяйте тестам!PyCon Siberia 2016. Не доверяйте тестам!
PyCon Siberia 2016. Не доверяйте тестам!
 
Refactoring using Codeception
Refactoring using CodeceptionRefactoring using Codeception
Refactoring using Codeception
 
Parse Everything With Elixir
Parse Everything With ElixirParse Everything With Elixir
Parse Everything With Elixir
 
SQL || overview and detailed information about Sql
SQL || overview and detailed information about SqlSQL || overview and detailed information about Sql
SQL || overview and detailed information about Sql
 
実践 memcached
実践 memcached実践 memcached
実践 memcached
 
SQL querys in detail || Sql query slides
SQL querys in detail || Sql query slidesSQL querys in detail || Sql query slides
SQL querys in detail || Sql query slides
 
Why Go Scales
Why Go ScalesWhy Go Scales
Why Go Scales
 
Unit testing zend framework apps
Unit testing zend framework appsUnit testing zend framework apps
Unit testing zend framework apps
 
Python 入門初體驗(程式語法)
Python 入門初體驗(程式語法)Python 入門初體驗(程式語法)
Python 入門初體驗(程式語法)
 
Zf2 how arrays will save your project
Zf2   how arrays will save your projectZf2   how arrays will save your project
Zf2 how arrays will save your project
 
Business Rules with Brick
Business Rules with BrickBusiness Rules with Brick
Business Rules with Brick
 
Les09 Manipulating Data
Les09 Manipulating DataLes09 Manipulating Data
Les09 Manipulating Data
 
Writing Readable Code with Pipes
Writing Readable Code with PipesWriting Readable Code with Pipes
Writing Readable Code with Pipes
 
Form tour person1
Form tour person1Form tour person1
Form tour person1
 
Closure, Higher-order function in Swift
Closure, Higher-order function in SwiftClosure, Higher-order function in Swift
Closure, Higher-order function in Swift
 
Very basic functional design patterns
Very basic functional design patternsVery basic functional design patterns
Very basic functional design patterns
 
Database performance 101
Database performance 101Database performance 101
Database performance 101
 
Pandas
PandasPandas
Pandas
 

Similar to Writing DSLs with Parslet

My First Rails Plugin - Usertext
My First Rails Plugin - UsertextMy First Rails Plugin - Usertext
My First Rails Plugin - Usertextfrankieroberto
 
Ruby Language - A quick tour
Ruby Language - A quick tourRuby Language - A quick tour
Ruby Language - A quick touraztack
 
Танки_в_Лунапарке: нагрузочное_тестирование_в_Яндексе
Танки_в_Лунапарке: нагрузочное_тестирование_в_ЯндексеТанки_в_Лунапарке: нагрузочное_тестирование_в_Яндексе
Танки_в_Лунапарке: нагрузочное_тестирование_в_ЯндексеYandex
 
Scala 3camp 2011
Scala   3camp 2011Scala   3camp 2011
Scala 3camp 2011Scalac
 
Refactoring to Macros with Clojure
Refactoring to Macros with ClojureRefactoring to Macros with Clojure
Refactoring to Macros with ClojureDmitry Buzdin
 
Fcontratos
FcontratosFcontratos
Fcontratoskarlloss
 
Use of django at jolt online v3
Use of django at jolt online v3Use of django at jolt online v3
Use of django at jolt online v3Jaime Buelta
 
The Ring programming language version 1.5.3 book - Part 44 of 184
The Ring programming language version 1.5.3 book - Part 44 of 184The Ring programming language version 1.5.3 book - Part 44 of 184
The Ring programming language version 1.5.3 book - Part 44 of 184Mahmoud Samir Fayed
 
The Ring programming language version 1.5.3 book - Part 54 of 184
The Ring programming language version 1.5.3 book - Part 54 of 184The Ring programming language version 1.5.3 book - Part 54 of 184
The Ring programming language version 1.5.3 book - Part 54 of 184Mahmoud Samir Fayed
 
Produce nice outputs for graphical, tabular and textual reporting in R-Report...
Produce nice outputs for graphical, tabular and textual reporting in R-Report...Produce nice outputs for graphical, tabular and textual reporting in R-Report...
Produce nice outputs for graphical, tabular and textual reporting in R-Report...Dr. Volkan OBAN
 
仕事で使うF#
仕事で使うF#仕事で使うF#
仕事で使うF#bleis tift
 
JavaOne2010 Groovy/Spring Roo
JavaOne2010 Groovy/Spring RooJavaOne2010 Groovy/Spring Roo
JavaOne2010 Groovy/Spring RooYasuharu Nakano
 
Wap to implement bitwise operators
Wap to implement bitwise operatorsWap to implement bitwise operators
Wap to implement bitwise operatorsHarleen Sodhi
 
Itsecteam shell
Itsecteam shellItsecteam shell
Itsecteam shellady36
 
Django - Framework web para perfeccionistas com prazos
Django - Framework web para perfeccionistas com prazosDjango - Framework web para perfeccionistas com prazos
Django - Framework web para perfeccionistas com prazosIgor Sobreira
 

Similar to Writing DSLs with Parslet (20)

My First Rails Plugin - Usertext
My First Rails Plugin - UsertextMy First Rails Plugin - Usertext
My First Rails Plugin - Usertext
 
Vcs23
Vcs23Vcs23
Vcs23
 
Borrador del blog
Borrador del blogBorrador del blog
Borrador del blog
 
Ruby Language - A quick tour
Ruby Language - A quick tourRuby Language - A quick tour
Ruby Language - A quick tour
 
Танки_в_Лунапарке: нагрузочное_тестирование_в_Яндексе
Танки_в_Лунапарке: нагрузочное_тестирование_в_ЯндексеТанки_в_Лунапарке: нагрузочное_тестирование_в_Яндексе
Танки_в_Лунапарке: нагрузочное_тестирование_в_Яндексе
 
Scala 3camp 2011
Scala   3camp 2011Scala   3camp 2011
Scala 3camp 2011
 
Refactoring to Macros with Clojure
Refactoring to Macros with ClojureRefactoring to Macros with Clojure
Refactoring to Macros with Clojure
 
Fcontratos
FcontratosFcontratos
Fcontratos
 
Use of django at jolt online v3
Use of django at jolt online v3Use of django at jolt online v3
Use of django at jolt online v3
 
The Ring programming language version 1.5.3 book - Part 44 of 184
The Ring programming language version 1.5.3 book - Part 44 of 184The Ring programming language version 1.5.3 book - Part 44 of 184
The Ring programming language version 1.5.3 book - Part 44 of 184
 
The Ring programming language version 1.5.3 book - Part 54 of 184
The Ring programming language version 1.5.3 book - Part 54 of 184The Ring programming language version 1.5.3 book - Part 54 of 184
The Ring programming language version 1.5.3 book - Part 54 of 184
 
Produce nice outputs for graphical, tabular and textual reporting in R-Report...
Produce nice outputs for graphical, tabular and textual reporting in R-Report...Produce nice outputs for graphical, tabular and textual reporting in R-Report...
Produce nice outputs for graphical, tabular and textual reporting in R-Report...
 
Groovy
GroovyGroovy
Groovy
 
Advance java
Advance javaAdvance java
Advance java
 
仕事で使うF#
仕事で使うF#仕事で使うF#
仕事で使うF#
 
JavaOne2010 Groovy/Spring Roo
JavaOne2010 Groovy/Spring RooJavaOne2010 Groovy/Spring Roo
JavaOne2010 Groovy/Spring Roo
 
Wap to implement bitwise operators
Wap to implement bitwise operatorsWap to implement bitwise operators
Wap to implement bitwise operators
 
Itsecteam shell
Itsecteam shellItsecteam shell
Itsecteam shell
 
Vcs16
Vcs16Vcs16
Vcs16
 
Django - Framework web para perfeccionistas com prazos
Django - Framework web para perfeccionistas com prazosDjango - Framework web para perfeccionistas com prazos
Django - Framework web para perfeccionistas com prazos
 

Recently uploaded

"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek SchlawackFwdays
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxNavinnSomaal
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationSlibray Presentation
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationRidwan Fadjar
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLScyllaDB
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii SoldatenkoFwdays
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett
 
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostLeverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostZilliz
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupFlorian Wilhelm
 
Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piececharlottematthew16
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 3652toLead Limited
 
My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024The Digital Insurer
 
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks..."LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...Fwdays
 
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr LapshynFwdays
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsMark Billinghurst
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Enterprise Knowledge
 
Training state-of-the-art general text embedding
Training state-of-the-art general text embeddingTraining state-of-the-art general text embedding
Training state-of-the-art general text embeddingZilliz
 
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsMemoori
 

Recently uploaded (20)

"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptx
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck Presentation
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 Presentation
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQL
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024
 
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostLeverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project Setup
 
Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piece
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365
 
My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024
 
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks..."LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
 
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024
 
Training state-of-the-art general text embedding
Training state-of-the-art general text embeddingTraining state-of-the-art general text embedding
Training state-of-the-art general text embedding
 
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial Buildings
 

Writing DSLs with Parslet

  • 1. Writing DSLs with Parslet Jason Garber Wicked Good Ruby Conf
  • 2.
  • 3.
  • 7.
  • 10.
  • 11. <?xml version="1.0"?> <configuration><configSections><sectionGr oup name="userSettings" type="System.Configuration.UserSettings Group, System, Version=2.0.0.0, Culture=neutral, PublicKeyToken =b77a5c561934e089"><section name="MSDNSampleSettings.My.MySetti ngs" type="System.Configuration.ClientSettingsSection, System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e0 89" allowExeDefinition="MachineToLocalUser" requirePermission=" false"/></sectionGroup></configSections> ... <userSettings><MSD NSampleSettings.My.MySettings><setting name="Setting" serialize As="String"><value>SomeDefaultValue</value></setting></MSDNSamp leSettings.My.MySettings></userSettings> </configuration>
  • 14. RssReader::Application.routes.draw do devise_for :users resources :feeds, only: [:edit, :update, :show, :index] post '/feeds', to: 'feeds#create', as: 'create_feed' resources :users do get 'page/:page', action: :index, on: :collection end resources :posts do member do put :update_category end end get '/my_profile', to: 'users#my_profile', as: :my_profile root to: "home#home" end
  • 15. describe Stack do context "stack with one item" do let(:stack) { a_stack_with_one_item } context "when popped" do before { stack.pop } it { should be_empty } end end end
  • 16. click_on "Sign Up" fill_in "Email", with: account[:email] fill_in "Password", with: account[:password] fill_in "Confirmation", with: account[:password_confirmation] fill_in "Name", with: account[:name] select account[:birthyear].to_s, from: "Year born" check "Terms" click_on "I'm ready to join!" current_path.should eq "/accounts/#{account.id}/dashboard" page.should have_content "Dashboard"
  • 17. desc 'Generate markup and stylesheets and open browser preview' task :preview => FileList['*.html'] + FileList['*.css'] do |t| sh 'open -g index.html' end rule '.html' => '.haml' do |t| puts "Rebuilding #{t.name}" sh "haml #{t.source} #{t.name}" end rule '.css' => lambda { |cssfile| source(cssfile) } do |t| puts "Rebuilding #{t.name}" sh "sass #{t.source} #{t.name}" end
  • 18. get '/' do @posts = Post.all(:order => [:id.desc], :limit => 20) erb :index end get '/post/new' do erb :new end get '/post/:id' do @post = Post.get(params[:id]) erb :post end post '/post/create' do post = Post.new(params) status 201 redirect "/post/#{post.id}" end
  • 19. rule(:table) do (str("table") >> attributes?.as(:attributes) >> str(".n")).maybe >> table_row.repeat(1).as(:content) >> block_end end rule(:table_row) do table_row_attributes >> table_row_content >> end_table_row end rule(:table_row_attributes) { (attributes?.as(:attributes) >> str(". ")).maybe } rule(:table_row_content) { (table_header | table_data).repeat(1).as(:content) } rule(:end_table_row) { str("|") >> (block_end.present? | (str("n"))) } rule(:table_header) do str("|_. ") >> table_content.as(:content) end rule(:table_data) do str("|") >> str("n").absent? >> td_attributes? >> table_content.as(:content) end
  • 22. upstream puma { server unix:///tmp/sockets/puma.sock fail_timeout=0; } server { listen 80 default deferred; server_name promptworks.com www.promptworks.com; root /srv/promptworks/public; charset utf-8; if (-f $document_root/system/maintenance.html) { return 503; } error_page 503 @maintenance; location @maintenance { rewrite ^(.*)$ /system/maintenance.html last; break; } }
  • 24. SELECT DISTINCT sc1.id FROM ( SELECT DATE_ADD(entry_point.start_date, INTERVAL (digits_1.digit * 10 + d (1 << (DAYOFWEEK(DATE_ADD(entry_point.start_date, INTERVAL (digits_1.digi FROM (SELECT CAST('2012-03-01' AS date) AS start_date) AS entry_point INNER JOIN (SELECT 0 AS digit UNION SELECT 1 UNION SELECT 2 UNION SELECT ON (digits_1.digit * 10 + digits_2.digit) <= (DATEDIFF(DATE_ADD(entry_poi ) AS md INNER JOIN schedules AS sc1 INNER JOIN negative_link_influences AS neg on (sc1.call_type_id = neg.call_type_id or neg.call_type_id = -1) WHERE sc1.schedule_on = md.date AND neg.affected_shift = 0;
  • 25. BEGIN TOTAL=0 } { { TOTAL = TOTAL + $1 } END print TOTAL/NR } { {print "<li><a href="" $1 "">" $1 "</a></li>"}
  • 26. ! doctype html html head title Test body - unless items.empty? ol - items.each do |item| li = item - else p No items
  • 27. Feature: Searching music As a User I want to be able to search music So I can play it Background: Given I am logged in Scenario: Search music Given I am on the search screen When I search for "mix" Then I should see the mixtape
  • 28. class erlang { file { "/etc/apt/sources.list.d/esl-erlang.list": ensure => present, owner => root, content => 'deb http://example.com/debian precise contrib'; } exec { "apt-update": command => "/usr/bin/apt-get update", refreshonly => true; } package { "esl-erlang": ensure => installed, require => Exec['apt-update', 'import-key']; } } include erlang
  • 29. Heroku buildpack: Ruby ====================== This is a [Heroku buildpack](http://devcenter.heroku.com/article It uses [Bundler](http://gembundler.com) for dependency manageme Usage ----### Ruby Example Usage: $ heroku create --stack cedar --buildpack https://github.com $ git push heroku master
  • 30. title = "T??? Example" [owner] name = "Tom Preston-Werner" organization = "GitHub" bio = "GitHub Cofounder & CEOnLikes tater tots and beer." dob = 1979-05-27T07:32:00Z # First class dates? Why not? [database] server = "192.168.1.1" ports = [ 8001, 8001, 8002 ] connection_max = 5000 enabled = true
  • 31. global= { time 4/4 key c major tempo "Allegro" 4 = 132 set Score.skipBars = ##t } Violinone = new Voice { relative c''{ set Staff.midiInstrument = #"violin" set tupletSpannerDuration = #(ly:make-moment 1 4) % Jason solo R1 * 9 r2 r4 times 2/3 { c4( b8)
  • 32. gsave 1 0.5 scale 70 100 48 0 360 arc fill grestore /Helvetica-Bold 14 selectfont 1.0 setgray 29 45 moveto (Hello, world!) show showpage
  • 33. .accordion { li { border-top: 1px solid #e2e4e6; &:first-child { border-top-color: transparent; } } a { @include rem(padding, 5px 10px 6px); &.icon { padding-left: 40px; position: relative; } &.icon img { @include rem(left, 10px); margin-top: -2px; position: absolute; } } }
  • 34.
  • 35.
  • 36. h1. Give Textile a try! A *simple* paragraph with a line break, some _emphasis_ and a "link":http://redcloth.org * an item * and another # one # two # three
  • 37. h1. Give Textile a try! A *simple* paragraph with a line break, some _emphasis_ and a "link":http://redcloth.org * an item * and another # one # two # three
  • 38.
  • 39. A_HLGN = /(?:<(?!>)|<>|=|[()]+)/ A_VLGN = /[-^~]/ C_CLAS = '(?:([^)]+))' C_LNGE = '(?:[[^]]+])' C_STYL = '(?:{[^}]+})' S_CSPN = '(?:d+)' S_RSPN = '(?:/d+)' A = "(?:#{A_HLGN}?#{A_VLGN}?|#{A_VLGN}?#{A_HLGN}?)" S = "(?:#{S_CSPN}?#{S_RSPN}|#{S_RSPN}?#{S_CSPN}?)" C = "(?:#{C_CLAS}?#{C_STYL}?#{C_LNGE}?|#{C_STYL}? #{C_LNGE}?#{C_CLAS}?|#{C_LNGE}?#{C_STYL}?#{C_CLAS}?)" PUNCT = Regexp::quote( '!"#$%&'*+,-./:;=?@^_`|~' ) HYPERLINK = '(S+?)([^ws/;=?]*?)(s|$)'
  • 40. def links( text ) text.gsub!( / ([s[{(]|[#{PUNCT}])? # $pre " # start (#{C}) # $atts ([^"]+?) # $text s? (?:(([^)]+?))(?="))? # $title ": (S+?) # $url (/)? # $slash ([^w/;]*?) # $post (?=s|.[#{PUNCT}]+|$) /x ) do |m| pre,atts,text,title,url,slash,post = $~[1..7] url = check_refs( url ) atts = pba( atts ) atts << " title="#{ title }"" if title atts = shelve( atts ) if atts "#{ pre }<a href="#{ url }#{ slash }"#{ atts }>" + "#{ text }</a>#{ post }" end end
  • 41. “Some people, when confronted with a problem, think ‘I know, I'll use regular expressions.’ Now they have two problems.” — Jamie Zawinski
  • 42.
  • 43. %%{ machine superredcloth_scan; include superredcloth_common "ext/superredcloth_scan/superredcloth_common.rl"; action extend { extend = rb_hash_aref(regs, ID2SYM(rb_intern("type"))); } # blocks notextile_tag_start = "<notextile>" ; notextile_tag_end = "</notextile>" CRLF? ; notextile_block_start = ( "notextile" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ; pre_tag_start = "<pre" [^>]* ">" (space* "<code>")? ; pre_tag_end = ("</code>" space*)? "</pre>" CRLF? ; pre_block_start = ( "pre" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ; bc_start = ( "bc" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ; btype = ( "p" | "h1"... block_start = ( btype >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ; next_block_start = ( btype A C :> "."+ " " ) >A @{ p = reg - 1; } ; pre_tag := |* pre_tag_end { CAT(block); DONE(block); fgoto main; }; default => esc_pre; *|; notextile_tag := |* notextile_tag_end default => cat; *|; }%% { DONE(block); fgoto main; };
  • 44. void rb_str_cat_escaped(VALUE str, char *ts, char *te, unsigned int opts); void rb_str_cat_escaped_for_preformatted(VALUE str, char *ts, char *te, unsigned int opts); VALUE superredcloth_inline(VALUE, char *, char *, VALUE); VALUE superredcloth_inline2(VALUE, VALUE, VALUE); #define CAT(H) #define CLEAR(H) #define INLINE(H, T) rb_str_cat(H, ts, te-ts) H = rb_str_new2("") rb_str_append(H, rb_funcall(rb_formatter, rb_intern(#T), 1, regs)) VALUE superredcloth_transform(rb_formatter, p, pe, refs) VALUE rb_formatter; char *p, *pe; VALUE refs; { if (RSTRING(block)->len > 0) { ADD_BLOCK(); } if ( NIL_P(refs) && rb_funcall(refs_found, rb_intern("empty?"), 0) == Qfalse ) { return superredcloth_transform(rb_formatter, orig_p, orig_pe, refs_found); } else { rb_funcall(rb_formatter, rb_intern("after_transform"), 1, html); return html; } }
  • 45.
  • 46.
  • 47. %%{ machine superredcloth_scan; include superredcloth_common "ext/superredcloth_scan/superredcloth_common.rl"; action extend { extend = rb_hash_aref(regs, ID2SYM(rb_intern("type"))); } # blocks notextile_tag_start = "<notextile>" ; notextile_tag_end = "</notextile>" CRLF? ; notextile_block_start = ( "notextile" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ; pre_tag_start = "<pre" [^>]* ">" (space* "<code>")? ; pre_tag_end = ("</code>" space*)? "</pre>" CRLF? ; pre_block_start = ( "pre" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ; bc_start = ( "bc" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ; btype = ( "p" | "h1"... block_start = ( btype >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ; next_block_start = ( btype A C :> "."+ " " ) >A @{ p = reg - 1; } ; pre_tag := |* pre_tag_end { CAT(block); DONE(block); fgoto main; }; default => esc_pre; *|; notextile_tag := |* notextile_tag_end default => cat; *|; }%% { DONE(block); fgoto main; };
  • 48. %%{ machine superredcloth_scan; include superredcloth_common "ext/superredcloth_scan/superredcloth_common.rl"; action extend { extend = rb_hash_aref(regs, ID2SYM(rb_intern("type"))); } # blocks notextile_tag_start = "<notextile>" ; notextile_tag_end = "</notextile>" CRLF? ; notextile_block_start = ( "notextile" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ; pre_tag_start = "<pre" [^>]* ">" (space* "<code>")? ; pre_tag_end = ("</code>" space*)? "</pre>" CRLF? ; pre_block_start = ( "pre" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ; bc_start = ( "bc" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ; btype = ( "p" | "h1"... block_start = ( btype >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ; next_block_start = ( btype A C :> "."+ " " ) >A @{ p = reg - 1; } ; pre_tag := |* pre_tag_end { CAT(block); DONE(block); fgoto main; }; default => esc_pre; *|; notextile_tag := |* notextile_tag_end default => cat; *|; }%% { DONE(block); fgoto main; };
  • 49. /* %%{ * redcloth_scan.c.rl * machine superredcloth_scan; * Copyright (C) 2009 Jason Garber include superredcloth_common "ext/superredcloth_scan/superredcloth_common.rl"; */ #define redcloth_scan_c action extend { extend = rb_hash_aref(regs, ID2SYM(rb_intern("type"))); } #define RSTRING_NOT_MODIFIED # blocks #include <ruby.h> notextile_tag_start = "<notextile>" ; #include "redcloth.h" notextile_tag_end = "</notextile>" CRLF? ; notextile_block_start = ( "notextile" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ; VALUE mRedCloth, super_ParseError, super_RedCloth, super_HTML, super_LATEX; pre_tag_start = "<pre" [^>]* ">" (space* "<code>")? ; VALUE SYM_escape_preformatted, SYM_escape_attributes; pre_tag_end = ("</code>" space*)? "</pre>" CRLF? ; pre_block_start = ( "pre" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ; #line 23 "ext/redcloth_scan/redcloth_scan.c" bc_start = ( "bc" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ; btype = ( "p" | "h1"... static const unsigned char _redcloth_scan_actions[] = { block_start = ( btype >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ; 0, 1, 0, 1, 2, 1, 3, 1, next_block_start = ( btype A C :> "."+ " " ) >A @{ p = reg - 1; } ; 4, 1, 5, 1, 6, 1, 7, 1, 9, 1, 10, 1, 11, 1, 12, 1, pre_tag := |* 13, 1, 14, 1, 15, 1, 16, 1, pre_tag_end { CAT(block); DONE(block); fgoto main; }; 17, 1, 18, 1, 19, 1, 20, 1, default => esc_pre; 21, 1, 22, 1, 23, 1, 24, 1, *|; 25, 1, 26, 1, 27, 1, 28, 1, 29, 1, 30, 1, 34, 1, 35, 1, notextile_tag := |* 36, 1, 38, 1, 40, 1, 42, 1, notextile_tag_end { DONE(block); fgoto main; }; 43, 1, 44, 1, 45, 1, 48, 1, default => cat; 57, 1, 58, 1, 59, 1, 60, 1, *|; 61, 1, 62, 1, 63, 1, 64, 1, 65, 1, 66, 1, 70, 1, 73, 1, }%%
  • 50.
  • 51.
  • 52.
  • 53. grammar Arithmetic rule additive multitive ( '+' multitive )* end rule multitive primary ( [*/%] primary )* end rule primary '(' additive ')' / number end rule number '-'? [1-9] [0-9]* end end
  • 54. require 'parslet' class MiniParser < Parslet::Parser rule(:integer) { match('[0-9]').repeat(1) } root(:integer) end MiniParser.new.parse("1324321") # => "1324321"@0
  • 60. Operators str('Wicked') >> str('Good') str('Ruby') | str('Elixir') match('[Bb]') >> str('oston') | match('[Mm]') >> str('assachusetts')
  • 63. 1. Create a grammar What should be legal syntax? 2. Annotate the grammar: What is important data? 3. Create a transformation: How do I want to work with that data?
  • 64. 1. Create a grammar What should be legal syntax? 2. Annotate the grammar: What is important data? 3. Create a transformation: How do I want to work with that data?
  • 66. Capture str('a').repeat.as(:b) # => {:b=>"aaa"@0} str('a').as(:b).repeat # => [{:b=>"a"@0}, {:b=>"a"@1}, {:b=>"a"@2}] str('a').as(:a) >> str('b').as(:b) >> str('c') # => {:a=>"a"@0, :b=>"b"@1}
  • 67. rule(:table) do (str("table") >> attributes?.as(:attributes) >> str(".n")).maybe >> table_row.repeat(1).as(:content) >> block_end end rule(:table_row) do table_row_attributes >> table_row_content >> end_table_row end rule(:table_row_attributes) { (attributes?.as(:attributes) >> str(". ")).maybe } rule(:table_row_content) { (table_header | table_data).repeat(1).as(:content) } rule(:end_table_row) { str("|") >> (block_end.present? | (str("n"))) } rule(:table_header) do str("|_. ") >> table_content.as(:content) end rule(:table_data) do str("|") >> str("n").absent? >> td_attributes? >> table_content.as(:content) end
  • 68. table(#prices). | Adults | $5 | | Children | $2 | <table id="prices"> <tr> <td>Adults</td> <td>$5</td> </tr> <tr> <td>Children</td> <td>$2</td> </tr> </table>
  • 69. rule(:table) do (str("table") >> attributes?.as(:attributes) >> str(".n")).maybe >> table_row.repeat(1).as(:content) >> block_end end rule(:table_row) do table_row_attributes >> table_row_content >> end_table_row end rule(:table_row_attributes) { (attributes?.as(:attributes) >> str(". ")).maybe } rule(:table_row_content) { (table_header | table_data).repeat(1).as(:content) } rule(:end_table_row) { str("|") >> (block_end.present? | (str("n"))) } rule(:table_header) do str("|_. ") >> table_content.as(:content) end rule(:table_data) do str("|") >> str("n").absent? >> td_attributes? >> table_content.as(:content) end
  • 70. def parenthesized(atom) str('(') >> atom >> str(')') end parenthesized(match['d']).parse("(500)")
  • 71. class HtmlTag < Parslet::Parser root(:tag) rule(:tag) { open_tag | close_tag | self_closing_tag | comment_tag } rule(:open_tag) { str("<") >> tag_name >> attributes? >> str(">") } rule(:close_tag) { str("</") >> tag_name >> str(">") } rule(:tag_name) { match("[A-Za-z_:]") >> name_char.repeat } ... end class BlockHtmlTag < HtmlTag rule(:tag_name) do inline_tag_name.absent? >> any_tag_name end rule(:inline_tag_name) do INLINE_TAGS.map {|name| str(name) }.reduce(:|) end end
  • 72. > HtmlTag.new.open_tag.methods => [:name, :block, :try, :parslet, :to_s_inner, :parse, :apply, :cached?, ...] > HtmlTag.new.open_tag.parse("<blockquote>") => "<blockquote>"@0 > HtmlTag.new.open_tag.parse("</blockquote>") Parslet::ParseFailed: Failed to match sequence ('<' TAG_NAME ATTRIBUTES? '>') at line 1 char 2. from /Users/jasongarber/.rvm/gems/ruby-2.0.0-p247/gems/parslet-1.5.0/lib/parslet/ cause.rb:63:in `raise' from /Users/jasongarber/.rvm/gems/ruby-2.0.0-p247/gems/parslet-1.5.0/lib/parslet/ atoms/base.rb:46:in `parse' from (irb):8 from /Users/jasongarber/.rvm/rubies/ruby-2.0.0-p247/bin/irb:16:in `<main>'
  • 73. begin RedClothParslet::Parser::BlockHtmlTag.new.tag.parse("<img>") rescue Parslet::ParseFailed => failure puts failure.cause.ascii_tree end Expected one of [OPEN_TAG, CLOSE_TAG, SELF_CLOSING_TAG, COMMENT_TAG] at line 1 char 1. |- Failed to match sequence ('<' TAG_NAME ATTRIBUTES? '>') at line 1 char 2. | `- Failed to match sequence ((!INLINE_TAG_NAME / &(INLINE_TAG_NAME NAME_CHAR{1, })) ANY_TAG_NAME) at line 1 char 2. | `- Expected one of [!INLINE_TAG_NAME, &(INLINE_TAG_NAME NAME_CHAR{1, })] at line 1 char 2. | |- Input should not start with INLINE_TAG_NAME at line 1 char 2. | `- Input should start with INLINE_TAG_NAME NAME_CHAR{1, } at line 1 char 2. |- Failed to match sequence ('</' TAG_NAME '>') at line 1 char 1. | `- Expected "</", but got "<i" at line 1 char 1. |- Failed to match sequence ('<' TAG_NAME ATTRIBUTES? SPACES? '/' '>') at line 1 char 2. | `- Failed to match sequence ((!INLINE_TAG_NAME / &(INLINE_TAG_NAME NAME_CHAR{1, })) ANY_TAG_NAME) at line 1 char 2. | `- Expected one of [!INLINE_TAG_NAME, &(INLINE_TAG_NAME NAME_CHAR{1, })] at line 1 char 2. | |- Input should not start with INLINE_TAG_NAME at line 1 char 2. | `- Input should start with INLINE_TAG_NAME NAME_CHAR{1, } at line 1 char 2. `- Failed to match sequence ('<!--' (!COMMENT_TAG_END .){0, } COMMENT_TAG_END) at line 1 char 1. `- Expected "<!--", but got "<img" at line 1 char 1.
  • 75. describe RedClothParslet::Parser::HtmlTag do it { should parse("<div>") } it { should parse("<hr />") } it { should parse("</div>") } it { should parse("<!-- an HTML comment -->") } describe "attribute" do subject { described_class.new.attribute } it { should parse(" class='awesome'") } it { should_not parse(' 9kittens="cute"') } end end
  • 76. $ rspec spec/parser/html_tag_spec.rb 1) RedClothParslet::Parser::HtmlTag tag Failure/Error: it { should parse("</div>") } expected TAG to be able to parse "</div>" Expected one of [OPEN_TAG, CLOSE_TAG, SELF_CLOSING_TAG, COMMENT_TAG] at line 1 char 1. |- Failed to match sequence ('<' TAG_NAME ATTRIBUTES? '>') at line 1 char 2. | `- Failed to match sequence ([A-Za-z_:] NAME_CHAR{0, }) at line 1 char 2. | `- Failed to match [A-Za-z_:] at line 1 char 2. |- Failed to match sequence ('<' TAG_NAME '>') at line 1 char 2. | `- Failed to match sequence ([A-Za-z_:] NAME_CHAR{0, }) at line 1 char 2. | `- Failed to match [A-Za-z_:] at line 1 char 2. |- Failed to match sequence ('<' TAG_NAME ATTRIBUTES? SPACES? '/' '>') at line 1 char 2. | `- Failed to match sequence ([A-Za-z_:] NAME_CHAR{0, }) at line 1 char 2. | `- Failed to match [A-Za-z_:] at line 1 char 2. `- Failed to match sequence ('<!--' (!COMMENT_TAG_END .){0, } COMMENT_TAG_END) at line 1 char 1. `- Expected "<!--", but got "</di" at line 1 char 1. # ./spec/parser/html_tag_spec.rb:9:in `block (3 levels) in <top (required)>' 13/13: 100% |==========================================| Time: 00:00:00 Finished in 0.01879 seconds 13 examples, 1 failure
  • 77. 1. Create a grammar What should be legal syntax? 2. Annotate the grammar: What is important data? 3. Create a transformation: How do I want to work with that data?
  • 78. Transformations tree = {left: {int: '1'}, op: '+', right: {int: '2'}} class T < Parslet::Transform rule(int: simple(:x)) { Integer(x) } end T.new.apply(tree) # => {:left=>1, :op=>"+", :right=>2}
  • 79. Transformations tree = {left: {int: '1'}, op: '+', right: {int: '2'}} class T < Parslet::Transform rule(int: simple(:x)) { Integer(x) } rule(op: '+', left: simple(:l), right: simple(:r)) { l + r } end T.new.apply(tree) # => 3
  • 80. Transformations rule(:content => subtree(:c), :attributes => subtree(:a)) do |dict| {:content => dict[:c], :opts => RedCloth::Ast::Attributes.new(dict[:a])} end rule(:table => subtree(:a)) do RedCloth::Ast::Table.new(a[:content], a[:opts]) end rule(:bq => subtree(:a)) do RedCloth::Ast::Blockquote.new(a[:content], a[:opts]) end
  • 82. class ErbParser < Parslet::Parser rule(:ruby) { (str('%>').absent? >> any).repeat.as(:ruby) } rule(:expression) { (str('=') >> ruby).as(:expression) } rule(:comment) { (str('#') >> ruby).as(:comment) } rule(:code) { ruby.as(:code) } rule(:erb) { expression | comment | code } rule(:erb_with_tags) { str('<%') >> erb >> str('%>') } rule(:text) { (str('<%').absent? >> any).repeat(1) } rule(:text_with_ruby) { (text.as(:text) | erb_with_tags).repeat.as(:t root(:text_with_ruby) end
  • 83. class ErbParser < Parslet::Parser rule(:ruby) { (str('%>').absent? >> any).repeat.as(:ruby) } rule(:expression) { (str('=') >> ruby).as(:expression) } rule(:comment) { (str('#') >> ruby).as(:comment) } rule(:code) { ruby.as(:code) } rule(:erb) { expression | comment | code } rule(:erb_with_tags) { str('<%') >> erb >> str('%>') } rule(:text) { (str('<%').absent? >> any).repeat(1) } rule(:text_with_ruby) { (text.as(:text) | erb_with_tags).repeat.as(:t root(:text_with_ruby) end
  • 84. class ErbParser < Parslet::Parser rule(:ruby) { (str('%>').absent? >> any).repeat.as(:ruby) } rule(:expression) { (str('=') >> ruby).as(:expression) } rule(:comment) { (str('#') >> ruby).as(:comment) } rule(:code) { ruby.as(:code) } include Parslet::Accelerator rule(:erb) { expression | comment | code } optimized = apply( parser, rule( (str(:x).absent? >> erb >> str('%>') } rule(:erb_with_tags) { str('<%') >> any).repeat ) rule(:text) GobbleUp.new(x, 0) } >> any).repeat(1) } { { (str('<%').absent? ) rule(:text_with_ruby) { (text.as(:text) | erb_with_tags).repeat.as(:t root(:text_with_ruby) end
  • 85. Custom Atoms • “Who makes the best cheesesteaks in Boston?” • EngTagger: a corpus-trained, probabilistic tagger • Custom Parslet::Atom • Parse a limited set of natural-language queries
  • 86. rule(:question) do interrogative.maybe.as(:int) >> verb_phrase.maybe.as(:verb) >> superlative_phrase.maybe.as(:sup) >> subject.as(:subj) >> prepositional_phrase.repeat.as(:preps) >> sentence_end.repeat.as(:punct) end rule(:verb_phrase) rule(:verb_present) rule(:verb_past) rule(:verb_future) { { { { (verb_present | verb_past | verb_future).as(:vp) } word(:VBZ, :VB).as(:present) } word(:VBD).as(:past) } word(:MD).as(:future) >> word(:VB).maybe.as(:infinitive) }
  • 90. Other Crazy Uses • Logs • Streaming text • User-supplied formulas / logic
  • 91. Other Crazy Uses • Logs • Streaming text • The Right Reverend and Right Honourable the Lord • User-supplied formulas / logic Bishop of London Richard John Carew Chartres
  • 92. Who will write the next awesome DSL?
  • 93. Who You will write the next awesome DSL!.
  • 96. • • Conclusions DSLs make life better internal_dsl > external_dsl if internal_dsl.practicable?
  • 97. Conclusions • • DSLs make life better • Keep your parser clean internal_dsl > external_dsl if internal_dsl.practicable?
  • 98. Conclusions • • DSLs make life better • • Keep your parser clean internal_dsl > external_dsl if internal_dsl.practicable? “Situational awareness!”