Securely Boosting Any Product with Generative AI APIs - Ruben Sitbon, Theodo Fintech

Editor's Notes

• #1: Hi Everyone Thanks for being here today and thanks for Nordic APIs to host this event So today we are going to talk about APIs of course and about AI since it’s the AI stage but first let me share something with you
• #2: Do you know what this means ? Maybe this would help ? I’ve heard some “ahhh” Shu ha ri are the 3 steps of learning to mastery of a skills
• #3: So why am I telling you all this ? Let’s introduce who I am I’m Ruben, I’m working at Sipios a french company which develop apps for Fintechs I started as a software engineer, now i’m Lead Solution Architect so my job is to find the right solution for a given problem of my prospective clients
• #4: First we need to understand the context of GenAI in 2023 We’ve seen a lot of chatbots And only a few game changing products
• #5: We also had the chance to witness the first failures of GenAI like this one Where AirCanada, had to refund a client because of a discount policy it’s chatbot imagined
• #7: In order to do that we need to understand the rules, understand the tools we have First let’s define the common axis on which we will make decisions when it comes to choose the right GenAI solution The first axis is the output customisation. Do you need only the raw output of a LLM which is a string ? or a formatted output ? or piece of code ready to be interpreted
• #8: Then you need to take a look at the range of knowledge you need to solve your problem Do you need “common” knowledge of a LLM or you need access to private documents or you need a LLM trained with very specific knowledge like medicine or law
• #9: Step 2 is to segment this complexe range of solution with teams skillset Product Manager skill set : knows how to use online solutions
• #10: Then you take your tools and put them on the workbench here on your mental model Chatgpt : very easy GPTs a bit harder Function is when you give your LLM the hability to call a function which can do an API call for example A RAG is a retrieval augmented generation : basically it’s a LLM with a database of documents Fine-tuning and finally building your own custom model
• #11: Finally decide de area of use for each tool For example : a simple GPT can be made by a product person, but if you want your GPT to be able to call APIs and return formatted outputs maybe you’ll need a little bit of help for a software engineer. Same for a RAG : a software engineer could build one. But if it’s a software systel with high criticity maybe you’ll eed a little bit of help from a AI Engineer
• #12: So why should we use these tools? to build another chatbot or to improve our existing products? I my opinion we should improve our existing products. And the only profile that can bring together The needs of the products The power of GenAI APis And the Ops : the fact to deploy those new features in production It’s us : the software engineers !!
• #13: So what does it mean for us. Well we need to work on 3 parts of our job : Contribute to the product design by providing the right tools Help the technical team asses the security of the product ideas Finally when it comes to actually build the solution implement it in a scalable way. And one solutions is the LangChain framework
• #14: I remember the day i’ve show my product manager a UI kit library like material UI. I saw a sparkle in his eyes. He told me “wow i didn’t knew we could do this, and this.” It’s the law of instrument : when you only have a hammer all problems look like a nail Well if the only genai applications you see are chatbots well you’ll keep building chatbots Here is an example of something you can do
• #21: Like for the API and web development the OWASP has decided to release the top 10 for LLMs
• #26: Vector DB like : Pinecone, Qdrant, Chroma
• #28: So let’s see ! Have we become master of GenAI ? Not yet but we are on a good path ! 1) We started from understanding the rules and mapping the GENAI environment the right way 2) We broke the rules of designing only chatbots and find a way to help product team, technical team in a scalable and secured way 3) For the Ri part ? Well, We need to stay curious and continue to make progress on the first two parts to create our own rules.