4. WHAT IS YQL
• Query, filter, and join data across web services
• Merge multiple data sources
• Expressive SQL-like language
• Uniform access
• No need to call different APIs
• Execute JS code to mutate request/response
• Decorate, annotate, create own data model
• Tools and documentation for rapid development
• Console, Table Editor, Guide, Blog, and Forum
6. YQL FORMATS
• Accepts JSON, XML, HTML, CSV
• Outputs XML, JSON, JSONP
• Provides data type transformers and mutators
• Tidies up data sources for you
• Lets you spend more time building your app
7. FAST, GLOBALLY AVAILABLE
• Deployed across the globe
• Both front and back facing caches
• Developers can control cache behavior
• _maxage parameter
• _stalewhilerevalidate parameter
• Rate limited for external developers
• 2000 calls per hour per IP on public endpoint
• 20000 on OAuth protected endpoint
8. APP DEVELOPMENT USING YQL
• Pick the open tables to build your app
• Build new ones for your API if needed
• Test the tables in the YQL console
• Use the YUI-YQL module to get data
• Or just call the REST query endpoint
9. DATA SECURITY USING YQL
• Public endpoint - /v1/public/yql
• Non-personalized data – securityLevel=“any”
• OAuth endpoint – v1/yql
• 2-legged – securityLevel=“app”
• 3-legged – securityLevel = “user”
17. TABLE EDITOR
• Rapid YQL table prototyping
• Create YQL tables
• Create YQL Environment files
• Create YQL Javascript execute files
• Sample templates for all three
• Nifty drag & drop support
• All the above stored in the Yahoo! cloud
Introduce selfWill be talking about YQL and what it does
Yahoo! Hosted serviceMaps familiar SQL syntax to APIs using YQL tables to bridge the gap – NOT a real database Powerful “execute” concept allows to build on top of data from various sources
Use YQL to read XML, JSON, HTML, or CSV data from various http/s sourcesAggregate, mutate, decorate this data within the execute phaseOutput your data in XML, JSON or JSONP
Various data formats as input/output as mentioned previouslyYQL provides an easy way of transforming between these formats – also special mutators within execute (xmlToJson) Quirks in conversion – jsonCompat modeInvokes Jtidy automatically to clean up – an optional HTML5 mode
Hosted in Yahoo! InfrastructureLeverages Squid & YTS caching technologiesProvides hooks to control cache behaviorNeed some controls – rate limited, but recently doubled
You can develop something very quickly with YQLChoose existing tables or create new ones using the console – try them outUse YUI library or plain http call to invoke table – integrate with your app
Console is a good tool to query, need actual http endpoints to use from within appsPublic for data that needs no auth – news feeds, weather, quotes, etc2-legged oauth for protecting app level tables3-legged oauth for personal data tied to a user
1 of 2 ways of playing with YQL – 2nd is pipes (graphical way of connecting/querying APIs)Starting point for most people, constant addition of nifty tools/utilities to the consoleAccess to a large number of existing tables – might be enough to start creating your hack !!
Check the Diagnostic box, get a <diagnostics> section with some good infoIs the table publicly callable, the urls accessed by the table, time spent fetching data, time sent executing within YQL engine, etc.Log messages also go to diagnostics
Another useful feature – check the Debug boxYQL captures all HTTP request/responses and stores them temporarily (5 mins)Gives you a link with a UUID that takes you to the request/response associated with a URLYou can see exactly what headers are being passed, what the response code is, etc.Debug is also how you skip caching on YQL
This is the page that shows you the request/response associated with a URLVery useful for debugging
Now that you have played around with the table, you want to start using a table from your app or from a command lineJust “curl” the REST query from the console – will work right away if the table is publicIf table has oauth, then you need to write some code on your app to sign the requestIMPORTANT – use hackevent.yql.yahooapis.com instead of query.yahooapis.com
Another great feature – around 1200 community generated tables – amazon, itunes, you name itAnybody that feels like exposing a data source as a YQL table can head over to our Github repoYou can easily import and execute these community tables within the console
Permalink – a link to the console page with the query and params pre-populated – shortener incidentally is built on top of YQLAlias provides an even smaller personal link you can share with others
Latest addition to the console – personalized tool to rapidly create/store/test tablesTied to your Yahoo! ID – login to YDN console before proceedingCompletes the tool set for someone writing a simple service that accesses third party data
Access the table editor by clicking on MY YQL in the right hand side of the consoleWill only work on the YDN console for now – use the YDN console to build your tables, change the hostname to use the REST querySee that once you go back to the YDN console the new table shows up under MY YQL on the right – click the refresh button if it does not
Same as creating a new tableWe use the Rhino interpreter, so not all JS methods/objects might work the sameInclude the newly created JS into an existing tables <execute> block … and the JS is executed as well
YQL Environment is a collection of tables and javascriptUseful to build multiple tables sharing common javascript – drag & drop tables/jsUse namespaces to organize your tables
Quick console demo – show tables, query answers table, diagnostics, debug, xml, json, built-in tables, community tables, rest query, permalink, query aliasQuick table demo – create crunchbase.product - http://api.crunchbase.com/v/1/product/iphone.js, add to env, run env
Develop on YDN console, change hostname for REST Query endpoint
