SlideShare a Scribd company logo
1 of 19
Amazon CloudSearch
& Future Search Architectures
Iain Fletcher
ifletcher@Searchtechnologies.com
The expert in the search space
Agenda
A brief introduction to Search Technologies
Search Application Architectures
Common challenges, emerging architectures

Our thoughts about Amazon CloudSearch

The expert in the search space
Search Technologies Overview
Ascot, UK
Karlsruhe, DE

Cincinnati, OH
Herndon, VA
San Diego, CA
San Jose, CR

• The leading IT Services company dedicated to
Enterprise Search & Search-based Applications
• Implementation, Consulting, Managed Services
• 120 employees and growing
• Independent, working with all of the leading
software vendors and open source alternatives
500+ Customers
Future Search Architectures

The expert in the search space
A Traditional Integrated Architecture
Integrated Search Engine

Content
Sources
Employee
Directory
CMS

File Share

Etc.

Connectors

Index Pipeline

Search
Index

UI

The year is 2013, but we still have:
Dissatisfaction with search results
Scalability issues (various)
High initial investment
Inflexibility in the model
Data integration issues…
The expert in the search space
Reasons for Change
Today’s leading “indexing engines” are very capable
The challenges to success almost always lie in
peripheral tasks. On the indexing side of the house,
this means:
Gathering and feeding data to the search engine
Preparing data for indexing (normalization, metadata
enrichment. Etc.)

The expert in the search space
Reasons for Change
The need to cope with continuous and rapid data
growth
Things usually break first on the data gathering side of
the house
E.g. The Government department and the 7TB File Server

The expert in the search space
A Traditional Integrated Architecture
Integrated Search Engine

Content
Sources
Employee
Directory
CMS

Connectors

Index Pipeline

Search
Index

UI

RE-INDEX

File Share

Etc.

The expert in the search space
A Better Search Architecture
Iterative
Development

Content
Sources
Employee
Directory
CMS

Etc.

Content
Processing

Connectors

Secure
xml
Cache

Search Engine
Index
Pipeline

Search
Index

RE-INDEX

Better operational agility

The expert in the search space
A Better Search Architecture
Iterative
Development

Content
Sources
Employee
Directory
CMS

Etc.

Content
Processing

Connectors

Secure
xml
Cache

Amazon CloudSearch
Index
Pipeline

Search
Index

RE-INDEX

Better operational agility
Much improved cost of
ownership
The expert in the search space
Let’s take a look at Amazon CloudSearch
If you’ve ever switched on an EC2 server, then
you can switch on a CloudSearch server
And of course, switch it off just as easily

Xml or JSON in, for indexing
RESTful / xml out, for searching
Has the bases covered in terms of core search
functionality

The expert in the search space
Amazon CloudSearch: Important Search Functions

Full-on Search Navigators
Results sorting by property
Plenty of relevancy tuning controls
Indexing speed is impressive
We’ve tested up to 500 docs/sec

Very long results lists can be served

The expert in the search space
Cost Effective & Flexible
The usual benefits of AWS
Only use what you need
Elastic scalability

Maintenance inclusive
Compare that to the cost of a Service Level Agreement for
an on-premise alternative
Then add the labour costs….

From less than £50 per month

The expert in the search space
What do you need to get started
Repository Connectors
Capture content
Convert to xml
XSLT. Etc.

Content Processing
Java code, Groovy. Etc.
A framework product, E.g. Aspire
Commercial software to suit the
purpose

Amazon S3
Atlassian Confluence
Documentum
eRoom
Secure File Systems
IBM Connections
JIVE
Lotus Notes
RDBMS
SharePoint 2010, 2013
SocialCast
TeamForge
Web Crawling

A User Interface
The expert in the search space
Search-specific UI/UX builders are available
The expert in the search space
What Applications Suit Amazon CloudSearch?

In principle any
CloudSearch is an option that everyone should
consider for search-based applications
Especially where elastic scalability is an advantage
Or where the data already lives in the cloud,
including S3 buckets

The expert in the search space
To conclude
If you like Amazon Web Services, then you’ll like
CloudSearch for the same reasons
It is a full-function indexing engine, ideally suited to
agile, flexible search architectures
If you are looking for a search-based application, it
should be on your shortlist
For many applications, CloudSearch will prove to be
highly cost effective
The expert in the search space
My email:

ifletcher@searchtechnologies.com

Useful URLs:
Connectors for Cloudsearch
www.searchtechnologies.com/cloudsearch-data-connectors.html

Aspire Content Processing Framework
www.searchtechnologies.com/aspire.html

A Java API for Amazon Cloudsearch
www.searchtechnologies.com/java-api-amazon-cloudsearch.html

Enterprise Search Surveys
www.searchtechnologies.com/enterprise-search-surveys.html

Blog: Searching Wikipedia with Amazon CloudSearch
www.searchtechnologies.com/cloudsearch-data-connectors.html

White Paper: A Big Data Architecture for Search
www.searchtechnologies.com/big-data-enterprise-search-architecture.html
The expert in the search space

More Related Content

More from Search Technologies

The Evolution of Search and Big Data
The Evolution of Search and Big DataThe Evolution of Search and Big Data
The Evolution of Search and Big DataSearch Technologies
 
Enterprise Search Summit Keynote: A Big Data Architecture for Search
Enterprise Search Summit Keynote: A Big Data Architecture for SearchEnterprise Search Summit Keynote: A Big Data Architecture for Search
Enterprise Search Summit Keynote: A Big Data Architecture for SearchSearch Technologies
 
Advanced Query Parsing Techniques
Advanced Query Parsing TechniquesAdvanced Query Parsing Techniques
Advanced Query Parsing TechniquesSearch Technologies
 
The things you need to know about SharePoint 2013 Search
The things you need to know about SharePoint 2013 SearchThe things you need to know about SharePoint 2013 Search
The things you need to know about SharePoint 2013 SearchSearch Technologies
 
Enterprise Search Best Practices Webinar 4.2013
Enterprise Search Best Practices Webinar 4.2013Enterprise Search Best Practices Webinar 4.2013
Enterprise Search Best Practices Webinar 4.2013Search Technologies
 

More from Search Technologies (7)

The Evolution of Search and Big Data
The Evolution of Search and Big DataThe Evolution of Search and Big Data
The Evolution of Search and Big Data
 
Enterprise Search Summit Keynote: A Big Data Architecture for Search
Enterprise Search Summit Keynote: A Big Data Architecture for SearchEnterprise Search Summit Keynote: A Big Data Architecture for Search
Enterprise Search Summit Keynote: A Big Data Architecture for Search
 
Advanced Query Parsing Techniques
Advanced Query Parsing TechniquesAdvanced Query Parsing Techniques
Advanced Query Parsing Techniques
 
Wikipedia Cloud Search Webinar
Wikipedia Cloud Search WebinarWikipedia Cloud Search Webinar
Wikipedia Cloud Search Webinar
 
The things you need to know about SharePoint 2013 Search
The things you need to know about SharePoint 2013 SearchThe things you need to know about SharePoint 2013 Search
The things you need to know about SharePoint 2013 Search
 
Enterprise Search Best Practices Webinar 4.2013
Enterprise Search Best Practices Webinar 4.2013Enterprise Search Best Practices Webinar 4.2013
Enterprise Search Best Practices Webinar 4.2013
 
Advanced Relevancy Ranking
Advanced Relevancy RankingAdvanced Relevancy Ranking
Advanced Relevancy Ranking
 

Recently uploaded

React Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkReact Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkPixlogix Infotech
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxLoriGlavin3
 
Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Kaya Weers
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityIES VE
 
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfSo einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfpanagenda
 
A Framework for Development in the AI Age
A Framework for Development in the AI AgeA Framework for Development in the AI Age
A Framework for Development in the AI AgeCprime
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxLoriGlavin3
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfNeo4j
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxLoriGlavin3
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity PlanDatabarracks
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxLoriGlavin3
 
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...itnewsafrica
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc
 
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...Wes McKinney
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxLoriGlavin3
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsNathaniel Shimoni
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesThousandEyes
 
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotesMuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotesManik S Magar
 
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better StrongerModern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better Strongerpanagenda
 

Recently uploaded (20)

React Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkReact Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App Framework
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
 
Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a reality
 
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfSo einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
 
A Framework for Development in the AI Age
A Framework for Development in the AI AgeA Framework for Development in the AI Age
A Framework for Development in the AI Age
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdf
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptx
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity Plan
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
 
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
 
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directions
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
 
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotesMuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
 
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better StrongerModern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
 

Amazon CloudSearch & Future Search Architectures

  • 1. Amazon CloudSearch & Future Search Architectures Iain Fletcher ifletcher@Searchtechnologies.com The expert in the search space
  • 2. Agenda A brief introduction to Search Technologies Search Application Architectures Common challenges, emerging architectures Our thoughts about Amazon CloudSearch The expert in the search space
  • 3. Search Technologies Overview Ascot, UK Karlsruhe, DE Cincinnati, OH Herndon, VA San Diego, CA San Jose, CR • The leading IT Services company dedicated to Enterprise Search & Search-based Applications • Implementation, Consulting, Managed Services • 120 employees and growing • Independent, working with all of the leading software vendors and open source alternatives
  • 5. Future Search Architectures The expert in the search space
  • 6. A Traditional Integrated Architecture Integrated Search Engine Content Sources Employee Directory CMS File Share Etc. Connectors Index Pipeline Search Index UI The year is 2013, but we still have: Dissatisfaction with search results Scalability issues (various) High initial investment Inflexibility in the model Data integration issues… The expert in the search space
  • 7. Reasons for Change Today’s leading “indexing engines” are very capable The challenges to success almost always lie in peripheral tasks. On the indexing side of the house, this means: Gathering and feeding data to the search engine Preparing data for indexing (normalization, metadata enrichment. Etc.) The expert in the search space
  • 8. Reasons for Change The need to cope with continuous and rapid data growth Things usually break first on the data gathering side of the house E.g. The Government department and the 7TB File Server The expert in the search space
  • 9. A Traditional Integrated Architecture Integrated Search Engine Content Sources Employee Directory CMS Connectors Index Pipeline Search Index UI RE-INDEX File Share Etc. The expert in the search space
  • 10. A Better Search Architecture Iterative Development Content Sources Employee Directory CMS Etc. Content Processing Connectors Secure xml Cache Search Engine Index Pipeline Search Index RE-INDEX Better operational agility The expert in the search space
  • 11. A Better Search Architecture Iterative Development Content Sources Employee Directory CMS Etc. Content Processing Connectors Secure xml Cache Amazon CloudSearch Index Pipeline Search Index RE-INDEX Better operational agility Much improved cost of ownership The expert in the search space
  • 12. Let’s take a look at Amazon CloudSearch If you’ve ever switched on an EC2 server, then you can switch on a CloudSearch server And of course, switch it off just as easily Xml or JSON in, for indexing RESTful / xml out, for searching Has the bases covered in terms of core search functionality The expert in the search space
  • 13. Amazon CloudSearch: Important Search Functions Full-on Search Navigators Results sorting by property Plenty of relevancy tuning controls Indexing speed is impressive We’ve tested up to 500 docs/sec Very long results lists can be served The expert in the search space
  • 14. Cost Effective & Flexible The usual benefits of AWS Only use what you need Elastic scalability Maintenance inclusive Compare that to the cost of a Service Level Agreement for an on-premise alternative Then add the labour costs…. From less than £50 per month The expert in the search space
  • 15. What do you need to get started Repository Connectors Capture content Convert to xml XSLT. Etc. Content Processing Java code, Groovy. Etc. A framework product, E.g. Aspire Commercial software to suit the purpose Amazon S3 Atlassian Confluence Documentum eRoom Secure File Systems IBM Connections JIVE Lotus Notes RDBMS SharePoint 2010, 2013 SocialCast TeamForge Web Crawling A User Interface The expert in the search space
  • 16. Search-specific UI/UX builders are available The expert in the search space
  • 17. What Applications Suit Amazon CloudSearch? In principle any CloudSearch is an option that everyone should consider for search-based applications Especially where elastic scalability is an advantage Or where the data already lives in the cloud, including S3 buckets The expert in the search space
  • 18. To conclude If you like Amazon Web Services, then you’ll like CloudSearch for the same reasons It is a full-function indexing engine, ideally suited to agile, flexible search architectures If you are looking for a search-based application, it should be on your shortlist For many applications, CloudSearch will prove to be highly cost effective The expert in the search space
  • 19. My email: ifletcher@searchtechnologies.com Useful URLs: Connectors for Cloudsearch www.searchtechnologies.com/cloudsearch-data-connectors.html Aspire Content Processing Framework www.searchtechnologies.com/aspire.html A Java API for Amazon Cloudsearch www.searchtechnologies.com/java-api-amazon-cloudsearch.html Enterprise Search Surveys www.searchtechnologies.com/enterprise-search-surveys.html Blog: Searching Wikipedia with Amazon CloudSearch www.searchtechnologies.com/cloudsearch-data-connectors.html White Paper: A Big Data Architecture for Search www.searchtechnologies.com/big-data-enterprise-search-architecture.html The expert in the search space

Editor's Notes

  1. Here’s the agenda for my part of the presentation.After a brief introduction to Search Technologies, I have some thoughts to share with you about how search architectures are evolving, and why.That will set the scene for taking a look at Amazon CloudSearch.