Presentation from Grant Ingersoll, LucidWorks #dataconf More at http://event.gigaom.com/structuredata/

1. THE DB AND THE INDEX: WHY OUR NEED FOR SEARCH IS SHAPING OUR TECHNOLOGY
SPEAKER: Grant Ingersoll
CTO
LucidWorks
Thursday, March 28, 13

2. The DB and the Index
Grant Ingersoll (@gsingers)
CTO
Confidential © Copyright 2012
Thursday, March 28, 13

3. Confidential and Proprietary
3 © 2012 LucidWorks
Thursday, March 28, 13

4. Search: NoSQL before NoSQL was cool
• Search is a system building block
- Text is only a part of the story
• If the algorithms fit,
use them!
• Embrace fuzziness!
• Scoring features are everywhere
Confidential and Proprietary
© 2012 LucidWorks
Thursday, March 28, 13

5. Reference Architecture
Access APIs
Search View Analytic Personalization &
1 2 Services Machine Learning
Shards 3 N
•Documents
Discovery & Document •Users
•Logs Classification Models
Enrichment Store In memory
Replicated
Multi-tenant
Content Acquisition
ETL, batch or near real-
time
Data
• LucidWorks Search
connectors
• Push
Confidential and Proprietary
© 2012 LucidWorks
Thursday, March 28, 13

6. Search (R)evolution
• Search use leads to search abuse
- Denormalization frees your mind
- Scoring is just a sparse matrix multiply
• Lucene/Solr evolution
- Non free text usages abound
- Many DB-like features
- Flexible indexing
- Finite State Transducers FTW!
• Scale
• “This ain’t your father’s relevance anymore”
Confidential and Proprietary
© 2012 LucidWorks
Thursday, March 28, 13

7. Resources
• http://www.lucidworks.com
• http://www.lucidworks.com/products/lucidworks-big-data
• grant@lucidworks.com
• @gsingers
Confidential and Proprietary
© 2012 LucidWorks
Thursday, March 28, 13

8. Thursday, March 28, 13