Recommendations with Neo4j (FOSDEM 2015)
•
18 likes•5,183 views
Building a high performance Neo4j Recommendation Engine using the GraphAware Framework
Recommended
More Related Content
What's hot
What's hot (7)
Viewers also liked
Viewers also liked (20)
Similar to Recommendations with Neo4j (FOSDEM 2015)
Similar to Recommendations with Neo4j (FOSDEM 2015) (20)
More from Michal Bachman
Recently uploaded
Recently uploaded (20)
Recommendations with Neo4j (FOSDEM 2015)
- 1. GraphAware TM Michal Bachman graphaware.com @graph_aware Recommendations with Neo4j Building a high-performance recommendation engine
- 2. GraphAware TM Quick Intro Why Graphs? Business and Technical Challenges GraphAware Recommendation Engine About This Talk
- 3. GraphAware TM News you should read Books you should buy People you may know People you should date People you should market a product to … Recommendation Engines
- 4. GraphAware TM Content-based (features) Collaborative filtering (user <-> item relationships) Recommendation Engines
- 5. GraphAware TM Features as well as relationships can be naturally represented as a graph. Good News
- 6. GraphAware TM Example IS_OF_GENRE title: “Love Actually” Movie name: “Bob” User name: “Comedy” Genre RATED rating: 5 name: “Alice” User name: “Romance” Genre title: “American Pie” Movie IS_OF_GENRE IS_OF_GENRE RATEDrating: 5 INTERESTED_IN rating: 5 RATED
- 7. GraphAware TM Easy to understand Natural to model Flexible (schema-free) Fast to query Graphs (Neo4j)
- 8. GraphAware TM Great for a quick PoC Great for smaller data sets Great for relatively simple logic Cypher
- 10. GraphAware TM Requirements of real-world recommendation engines are often much more complex. The Reality
- 11. GraphAware TM Imagine you’re building the ”people you may know” feature on LinkedIn. Example
- 12. GraphAware TM After a brainstorming session, your team came up with the following ways of finding people one may know: Example
- 13. GraphAware TM Common contacts Facebook friends in common Email / mobile contacts in common Each others email / mobile contact Worked for the same company Studied at the same school Share the same interest Live in the same city People You May Know
- 14. GraphAware TM But that’s just the beginning! Let’s go back and re-visit. Example
- 15. GraphAware TM More contacts in common = better chance? Same city / school / company = does size matter? What about emails that don’t represent a person? What about people already connected? And pending… And rejected… And repeatedly ignored… People You May Know
- 16. GraphAware TM Finding things to recommend Serving the most relevant recommendations Measuring the quality of recommendations Time to market / cost of development Business Challenges
- 18. GraphAware TM So we came up with an open-source recommendation engine skeleton that will help you solve all the challenges. We’ve done it before
- 19. GraphAware TM plugin to Neo4j (uses GraphAware Framework) you have to use a JVM-language opinionated architecture very fast very flexible handles all the plumbing Recommendation Engine
- 20. GraphAware TM Engine per recommendation “reason” (core logic) Engine executes a graph traversal to find items Engines are assembled into higher-level engines Design Decisions
- 21. GraphAware TM Example IS_OF_GENRE title: “Love Actually” Movie name: “Bob” User name: “Comedy” Genre RATED rating: 5 name: “Alice” User name: “Romance” Genre title: “American Pie” Movie IS_OF_GENRE IS_OF_GENRE RATEDrating: 5 INTERESTED_IN rating: 5 RATED
- 22. GraphAware TM Engine per recommendation “reason” (core logic) Engine executes a graph traversal to find items Engines are assembled to higher-level engines Items discovered multiple times are more relevant Relevance depends on how was item discovered Design Decision
- 23. GraphAware TM Example IS_OF_GENRE title: “Love Actually” Movie name: “Bob” User name: “Comedy” Genre RATED rating: 5 name: “Alice” User name: “Romance” Genre title: “American Pie” Movie IS_OF_GENRE IS_OF_GENRE RATEDrating: 5 INTERESTED_IN rating: 5 RATED
- 24. GraphAware TM Engine per recommendation “reason” (core logic) Engine executes a graph traversal to find items Engines are assembled to higher-level engines Items discovered multiple times are more relevant Relevance depends on how was item discovered Items not to be recommended: “cross-cutting” concern Design Decisions
- 25. GraphAware TM Example IS_OF_GENRE title: “Love Actually” Movie name: “Bob” User name: “Comedy” Genre RATED rating: 5 name: “Alice” User name: “Romance” Genre title: “American Pie” Movie IS_OF_GENRE IS_OF_GENRE RATEDrating: 5 INTERESTED_IN rating: 5 RATED
- 26. GraphAware TM Input -> Engine -> Recommendations Scores and Score Transformers Blacklists Filters Post-processors Context (how many, how fast,…?) Loggers Architecture
- 27. GraphAware TM In 5 minutes, we’ll build a simple engine that recommends who you should be friends with. Let’s Build Something
- 31. GraphAware TM public class FriendsInCommon extends SomethingInCommon { @Override public String name() { return "friendsInCommon"; } @Override protected RelationshipType getType() { return FRIEND_OF; } @Override protected Direction getDirection() { return BOTH; } }
- 34. GraphAware TM public class FriendsInCommon extends SomethingInCommon { @Override protected ScoreTransformer scoreTransformer() { return new ParetoScoreTransformer(100, 10); } @Override public String name() { return "friendsInCommon"; } @Override protected RelationshipType getType() { return FRIEND_OF; } @Override protected Direction getDirection() { return BOTH; } }
- 36. GraphAware TM public class RewardSameLocation extends RewardSomethingShared { @Override protected RelationshipType type() { return LIVES_IN; } @Override protected Direction direction() { return OUTGOING; } @Override protected float scoreValue(Node reco, Node in, Node shared) { return 10; } @Override protected String scoreName() { return "sameLocation"; } }
- 37. GraphAware TM public class RewardSameLabels implements PostProcessor<Node, Node> { @Override public void postProcess(Recommendations<Node> out, Node in) { Label[] inLabels = toArray(in.getLabels()); for (Recommendation<Node> reco : out.get()) { if (Arrays.equals(inLabels, toArray(reco.getItem().getLabels()))) { reco.add("sameGender", 10); } } } }
- 39. GraphAware TM public final class FriendsContextFactory extends Neo4jContextFactory { @Override protected List<BlacklistBuilder<Node, Node>> blacklistBuilders() { return asList( new ExcludeSelf(), new ExistingRelationshipBlacklistBuilder(FRIEND_OF, BOTH) ); } @Override protected List<Filter<Node, Node>> filters() { return asList( new ExcludeSelf() ); } }
- 41. GraphAware TM public final class FriendsComputingEngine extends Neo4jTopLevelDelegatingEngine { public FriendsComputingEngine() { super(new FriendsContextFactory()); } @Override protected List<RecommendationEngine<Node, Node>> engines() { return asList( new FriendsInCommon(), new RandomPeople() ); } @Override protected List<PostProcessor<Node, Node>> postProcessors() { return asList( new RewardSameLabels(), new RewardSameLocation(), new PenalizeAgeDifference() ); } }
- 43. GraphAware TM public final class FriendsComputingEngine extends Neo4jTopLevelDelegatingEngine { public FriendsComputingEngine() { super(new FriendsContextFactory()); } @Override protected List<RecommendationEngine<Node, Node>> engines() { return asList( new FriendsInCommon(), new RandomPeople() ); } @Override protected List<PostProcessor<Node, Node>> postProcessors() { return asList( new RewardSameLabels(), new RewardSameLocation(), new PenalizeAgeDifference() ); } @Override public ParticipationPolicy<Node, Node> participationPolicy(Context<Node, Node> context) { return ParticipationPolicy.IF_MORE_RESULTS_NEEDED; } }
- 44. GraphAware TM public final class FriendsRecoEngine extends Neo4jTopLevelDelegatingEngine { public FriendsRecommendationEngine() { super(new FriendsContextFactory()); } @Override protected List<RecommendationEngine<Node, Node>> engines() { return asList( new Neo4jPrecomputedEngine(), new FriendsComputingEngine() ); } }
- 46. GraphAware TM public final class FriendsRecoEngine extends Neo4jTopLevelDelegatingEngine { public FriendsRecommendationEngine() { super(new FriendsContextFactory()); } @Override protected List<RecommendationEngine<Node, Node>> engines() { return asList( new Neo4jPrecomputedEngine(), new FriendsComputingEngine() ); } @Override protected List<Logger<Node, Node>> loggers() { return asList( new Slf4jRecommendationLogger<Node, Node>(), new Slf4jStatisticsLogger<Node, Node>() ); } }
- 48. GraphAware TM List<Recommendation<Node>> reco = recommendationEngine.recommend(getPersonByName("Adam"), Mode.REAL_TIME, 2); String expected = "(Vince {total:19.338144," + "ageDifference:-5.527864," + "friendsInCommon:14.866008," + "sameGender:10.0})," + "(Luanne {total:11.553411," + "ageDifference:-3.312597," + "friendsInCommon:14.866008})"; assertEquals(expected, toString(reco));
- 49. GraphAware TM List<Recommendation<Node>> reco = recommendationEngine.recommend(getPersonByName("Luanne"), REAL_TIME, 4); assertEquals("Daniela", reco.get(0).getItem().getProperty("name")); assertEquals(22, reco.get(0).getScore().getTotalScore(), 0.5); assertEquals("Adam", reco.get(1).getItem().getProperty("name")); assertEquals(12, reco.get(1).getScore().getTotalScore(), 0.5); assertEquals("Vince", reco.get(2).getItem().getProperty("name")); assertEquals(8, reco.get(2).getScore().getTotalScore(), 0.5); assertEquals("Bob", reco.get(3).getItem().getProperty("name")); assertEquals(-9, reco.get(3).getScore().getTotalScore(), 0.5);
- 50. GraphAware TM Finding things to recommend Serving the most relevant recommendations Measuring the quality of recommendations Time to market / cost of development Business Challenges
- 53. GraphAware TM Built-in ability to pre-compute recommendations Other built-in base-classes But we need your help! https://github.com/graphaware/neo4j-reco There’s More!
- 54. GraphAware TM Built-in algorithms Time-based ParticipationPolicy Integration with compute engines Machine learning Future
- 55. GraphAware TM GraphAware Framework makes it easy to build, test, and deploy generic as well as domain- specific functionality for Neo4j. GraphAware Framework
- 56. GraphAware TM GraphUnit & RestTest RelCount WarmUp Schema (wip) Recommendation Engine GraphAware Framework ChangeFeed UUID TimeTree Algorithms NodeRank
- 57. GraphAware TM Open Source (GPL) Active Production Ready Github: github.com/graphaware Our Web: graphaware.com Maven Central GraphAware Framework
- 58. GraphAware TM Try it Give us feedback Contribute Build your own modules Get in touch for support / consultancy GraphAware Framework
- 59. GraphAware TM GraphAware Events 31 Jan Recommendation Engines in Brussels (FOSDEM) 31 Jan GraphGen in Brussels (FOSDEM) 5 Feb Recommendation Engines Webinar 5 Feb Meetup at GraphAware (build your own Recommendation Engine) 10 Feb Neo4j Fundamentals in Manchester 10 Feb Neo4j Meetup in Manchester 17 Feb Neo4j Fundamentals in Edinburgh 17 Feb Neo4j Meetup in Edinburgh
- 61. GraphAware TM GraphConnect Europe 2015 When: Where: Tickets: Call for Papers: Sponsors: Thursday, 7th May, 2015 - main Conference Day Wednesday, 6th May 2015 - Training Day Etc venues, 155 Bishopsgate, London (next to Liverpool Street Station) now available on www.graphconnect.com 199$ early bird plus 100$ for training 499$ full price plus 100$ for training open now till 29th January all Neo4j community members, customers or general graph enthusiasts are invited to submit their talk open now till 29th January, email: gceurope@neotechnology.com