A presentation given by Ruben Sitbon, Lead Solutions Architect at Sipios, at our 2024 Austin API Summit, March 12-13. Session Description: ChatGPT has been a tidal wave, changing forever the way people and companies perceive the value of Artificial Intelligence. Many startups have launched products with ChatGPTI at its core, innovative SaaS players have all integrated Generative AI extensions or plugins, but it is now clear that users will be expecting more and more Generative AI to boost the features of products they use on a daily basis. In this talk, I will describe how a framework relying on Generative AI in-house APIs that allows: - Easily « boosting » any product feature with Generative AI - Improving the answers through a « trainer API » that allows experts to improve the accuracy and tone of the model - Bundling security and continuous compliance in the APIs to enjoy the benefits even within risk averse large corporates.

1. March 12, 2024
1

2. The stages of learning to mastery
2

3. The stages of learning to mastery
What this Jira
ticket means?
Is he sure he
wants this
feature?
Let’s suggest
him to do it this
way!
3

4. All new technologies take you back to the
SHU-zone
How on earth
does GenAI
work?
4

5. Who am I?
@rubensitbon
ruben-sitbon
● Started as Software Engineer
● Now Lead Solutions Architect
Client examples:
5

6. GenAI in 2023
Means Chatbots
Only a few game
changing products
6

7. GenAI had failures in 2023
7

8. How to avoid bad GenAI
products?
8

9. How to avoid bad GenAI
products?
Right
Solution
Right
Team
9

10. Output customization
Step 1 : Define the
dimensions needed
to solve the
problem
10

11. Output customization
Range
of
Knowledge
Step 1 : Define the
dimensions needed
to solve the
problem
11

12. Product
Manager
Skill set
Software
Engineer
Skill set
AI
Engineer
Skill set
Research
Realm
Output customization
Range
of
Knowledge
Step 1 : Define the
dimensions needed
to solve the
problem
Step 2 : Segment
with your team
skill set
12

13. Product
Manager
Skill set
Software
Engineer
Skill set
AI
Engineer
Skill set
Research
Realm
Output customization
Range
of
Knowledge
Step 1 : Define the
dimensions needed
to solve the
problem
Step 2 : Segment
with your team
skill set
Step 3 : Position the
tools on the chart
ChatGPT
GPTs
RAG
Functions
Fine
tuning
Custom
Models
13

14. Step 4 : Draw the
area for each tool
Product
Manager
Skill set
Software
Engineer
Skill set
AI
Engineer
Skill set
Research
Realm
Output customization
Range
of
Knowledge
Step 1 : Define the
dimensions needed
to solve the
problem
Step 2 : Segment
with your team
skill set
Step 3 : Position the
tools on the chart
ChatGPT
GPTs
RAG
Functions
Fine
tuning
Custom
Models
14

15. Boost existing products instead of
building new (bad) ones
Product
Ops
GenAI
API
SwE
15

16. Help the technical team
assess the security of
the product ideas
Facilitate the product
design by supplying the
right tools
Develop scalable GenAI
implementation with
LangChain
16

17. Help the technical team
assess the security of
the product ideas
Facilitate the product
design by supplying the
right tools
Develop scalable GenAI
implementation with
LangChain
17

18. Help the product design process
by providing the right tools
18

19. Help the product design process
by providing the right tools
19

20. Help the product design process
by providing the right tools
20

21. Help the product design process
by providing the right tools
21

22. Help the product design process
by providing the right tools
22

23. Help the technical team
assess the security of
the product ideas
Facilitate the product
design by supplying the
right tools
Develop scalable GenAI
implementation with
LangChain
23

24. Help the technical team assess
the security of the product ideas
Increase observability to
monitor security issues
Leverage on existing
best practices
Test a lot and limit the
attack surface
SQL/Prompt Injection
DDos/EDos Attacks
Supply Chain Vulnerability
Custom Middlewares
Monitor input
Monitor output
Manual testing
Integration test
API First architecture
24

25. Help the technical team
assess the security of
the product ideas
Facilitate the product
design by supplying the
right tools
Develop scalable GenAI
implementation with
LangChain
25

26. Level 1: 0-Shot Prompt
LLM
Prompt
Answer
26

27. Level 2: Few-Shot Prompt
LLM
Prompt Give context on
how to answer
Answer
27

28. Level 3: Simple Functions
LLM
Prompt
Give context on
how to answer
Functions
Develop/
Maintain
Answer
LLM
28

29. Level 4: Retrieval Augmented Generation
LLMs
Prompt
Answer
Vector
DB
Document
Processing
Upload Documents
Upload
Documents
Store Chucked
and vectorised
documents
Query
DB
29

30. Level 5: RAG + Functions
LLMs
+
Functions
Prompt
Answer
Vector
DB
Document
Processing
Store Chucked
and vectorised
documents
30
Upload Documents
Upload
Documents
Query
DB

31. Develop scalable GenAI
implementation with LangChain
Architecture level LangChain tools
Level 1 : 0-shot prompt Custom LLM
Level 2 : few-shot prompt Few-shots prompt templates
Level 3 : Simple functions Function and Output Parser
Level 4 : RAG Retrieval tools
Level 5 : RAG & Output functions LangSmith tools for observability
31

32. Did we master GenAI?
32

33. What’s next!
● Try this library at home and
show the demo to your PO/PM
● LangChain Cookbook
● Youtube Channels like : Nordic
APIs
33

34. Thank you
@rubensitbon
ruben-sitbon
34

35. 35