OSDC-2010 Database Full-text Search.... making it not suck
•
1 like•1,894 views
This document summarizes Hibernate Search, a full-text search module for Hibernate. It discusses how Hibernate Search integrates full-text search capabilities with Hibernate's persistent domain model by transparently synchronizing the index with the database. It also covers Hibernate Search's architecture, configuration and mapping, queries, and additional features like clustering and manual indexing. The goal of Hibernate Search is to enable powerful and flexible full-text search without significantly changing developers' workflows or paradigms when building applications.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (18)
Similar to OSDC-2010 Database Full-text Search.... making it not suck
Similar to OSDC-2010 Database Full-text Search.... making it not suck (20)
More from Aaron Walker
More from Aaron Walker (12)
Recently uploaded
Recently uploaded (20)
OSDC-2010 Database Full-text Search.... making it not suck
- 1. Database Full-text Search.... making it not suck Emmanuel Bernard > Aaron Walker Hibernate Search in Actio a.walker@base2services.com > blog.emmanuelbernard.c @aaronwalker > twitter.com/emmanuelbe www.base2services.com base2Services Pty Ltd 2010
- 2. Why Full-text Search • Add a rich search dimension to your persistent domain model • Standard SQL sucks for doing “Google like” searches base2Services Pty Ltd 2010
- 3. Integrate full-text search and persistent domain • SQL Search vs Full-text Search • Object model / Full-text Search mismatches • Hibernate Search architecture Configuration and Mapping • Demo • Full-text based object queries • Demo • Some more features base2Services Pty Ltd 2010
- 4. SQL search limits • Wildcard / word search • ‘%hibernate%’ • Approximation (or synonym) • ‘hybernat’ • Proximity • ‘Java’ close to ‘Persistence’ • Relevance or (result scoring) • multi-”column” search base2Services Pty Ltd 2010
- 5. Full Text Search • Search information • by word • inverted indices (word frequency, position) • In RDBMS engines • portability (proprietary add-on on top of SQL) • flexibility • Standalone engine • Apache Lucene(tm) http://lucene.apache.org base2Services Pty Ltd 2010
- 6. Mismatches with a domain model • Structural mismatch • full text index are text only • no reference/association between document • Synchronisation mismatch • keeping index and database up to date • Retrieval mismatch • the index does not store objects • certainly not managed objects base2Services Pty Ltd 2010
- 7. Hibernate Search • Under the Hibernate platform • LGPL • Built on top of Hibernate Core • Use Apache Lucene(tm) under the hood • In top 10 downloaded at Apache • Very powerful • Somewhat low level • easy to use it the “wrong” way • Solve the mismatches base2Services Pty Ltd 2010
- 8. Architecture • Transparent indexing through event system (JPA) • PERSIST / UPDATE / DELETE • Convert the object structure into Index structure • Operation batching per transaction • better Lucene performance • “ACID”-ity • (pluggable scope) • Backend • synchronous / asynchronous mode • Lucene, JMS base2Services Pty Ltd 2010
- 9. Architecture (Backend) • Lucene Directory Architecture (Backend) • standalone or symmetric cluster • immediate Directory • Lucene visibility • standalone or symmetric cluster • Can •affect front end runtime immediate visibility • Can affect front end runtime Hibernate + Hibernate Search Search request Index update Lucene Directory (Index) Database Search request Index update Hibernate + Hibernate Search Copyright Red Hat Middleware LLC. Do not redistribute unless authorized | v1.2 10 base2Services Pty Ltd 2010
- 10. Architecture (Backend) • JMS (Cluster)(Backend) Architecture • Search processed locally • JMS (Cluster) • •Search processedto a master node (JMS) Changes sent locally • • synchronous a master node (JMS) Changes sent to indexing (delay) • asynchronous indexing (delay) • •No No front extraextra cost front end end cost Slave Hibernate Database Lucene + Directory Search request Hibernate Search (Index) Copy Index update order Copy Hibernate Master Lucene + Directory Index update JMS Process Hibernate Search (Index) queue Master Copyright Red Hat Middleware LLC. Do not redistribute unless authorized | v1.2 11 base2Services Pty Ltd 2010
- 11. DEMO base2Services Pty Ltd 2010
- 12. Configuration and Mapping • Configuration • event listener wiring • transparent in Hibernate Annotations • Backend configuration • Mapping: annotation based • @Indexed • @Field(store, index) • @IndexedEmbedded • @FieldBridge • @Boost / @Analyzer base2Services Pty Ltd 2010
- 13. Mapping example Mapping example @Entity @Indexed(index="indexes/essays") public class Essay { ... @Id @DocumentId public Long getId() { return id; } @Field(name="Abstract", index=Index.TOKENIZED, store=Store.YES) public String getSummary() { return summary; } @Lob @Field(index=Index.TOKENIZED) public String getText() { return text; } @ManyToOne @IndexedEmbedded public Author getAuthor() { return author; } } Copyright Red Hat Middleware LLC. Do not redistribute unless authorized | v1.2 13 base2Services Pty Ltd 2010
- 14. Fluent Mapping API SearchMapping mapping = new SearchMapping(); mapping .analyzerDef( "ngram", StandardTokenizerFactory.class ) .filter( LowerCaseFilterFactory.class ) .filter( NGramFilterFactory.class ) .param( "minGramSize", "3" ) .param( "maxGramSize", "3" ) .entity(Address.class) .indexed() .property("addressId", METHOD) .documentId() .property("street1", METHOD) .field(); base2Services Pty Ltd 2010
- 15. Query • Retrieve objects, not documents • no boilerplate conversion code! • Objects from the Persistence Context • same semantic as a JPA-QL or Criteria query • Use org.hibernate.Query/javax.persistence.Query • common API for all your queries • pagination support • list / scroll / iterate support • Query on correlated objects • “JOIN”-like query base2Services Pty Ltd 2010
- 16. Query example Query example org.apache.lucene.search.Query luceneQuery; String queryString = "summary:Festina Or brand:Seiko" luceneQuery = parser.parse( queryString ); org.hibernate.Query fullTextQuery = fullTextSession.createFullTextQuery( luceneQuery ); fullTextQuery.setMaxResult(200); List result = fullTextQuery.list(); //return a list of managed objects queryString = "title:hybernat~" //Approximate search queryString = ""Hibernate JBoss"~10" //Proximity search queryString = "author.address.city:Atlanta" //correlated search Copyright Red Hat Middleware LLC. Do not redistribute unless authorized | v1.2 15 base2Services Pty Ltd 2010
- 17. More query features • Fine grained fetching strategy • Sort by property rather than relevance • Projection (both field and metadata) • metadata: SCORE, ID, BOOST etc • no DB / Persistence Context access • Filters • security / temporal data / category • Total number of results • New Type Safe Fluent Query API (Experimental) base2Services Pty Ltd 2010
- 18. DEMO base2Services Pty Ltd 2010
- 19. Some more features • Automatic index optimisation • Index sharding • Programmatic Mapping API • Manual indexing and purging • non event-based system • Mass Indexing/Re-indexing • Shared Lucene resources • Native Lucene access base2Services Pty Ltd 2010
- 20. Full-Text search without the hassle • Transparent index synchronisation • Automatic Structural conversion through Mapping • No paradigm shift when retrieving data • Clustering capability out of the box • Easier / transparent optimized Lucene use base2Services Pty Ltd 2010
- 21. Questions • http://search.hibernate.org • Hibernate Search in Action (Manning) • http://lucene.apache.org • a.walker@base2services.com • github.com/aaronwalker base2Services Pty Ltd 2010