Ad
More Related Content
Similar to OutSystems User Group November 2024 (20)
Recently uploaded (20)
Ad
OutSystems User Group November 2024
- 1. zzzzzzz Turn OutSystems application data into AI Conversations Using Large Language Models Tools/Function Capability in OutSystems Developer Cloud Stefan Weber Senior Director Software Development Telelink Business Services OutSystems MVP – OutSystems Certified Trainer – AWS Community Builder
- 2. 2 Topics 1. Basics – Some basic information on LLMs and OpenAI API 2. Tool – What is a tool and how does tool calling work. 3. Tool Calling Demo – Using OpenAI API we take a closer look on tool calling flow in both Postman and OutSystems
- 3. OpenAI Chat Completions API became a de-facto standard and is used by multiple other runtimes ● AWS Bedrock Converse ● LLMStudio ● … The OpenAI API abstracts many features that otherwise would need to added to the prompt, like Conversations, Tools/Function Calling and more. LLM Parameters -> OpenAI API (Completion) Large Language Models Interactions have the same request-response behavior as website interactions. They do not store anything and they do not interact with anything else. ● Settings – Additional settings to manage the behavior of model, like creativity or stop sequences. ● Prompt – The prompt sent to the model for text inference. 3
- 4. Challenge Large Language Models (LLM) exhibit inconsistency. On occasion, they excel in providing accurate responses to inquiries, while at other times, they simply parrot unrelated facts extracted from their training corpus. Their occasional lapses into inconsistency are due to their systemic limitations. LLMs possess a statistical understanding of word relationships but lack genuine comprehension of meaning. 4 4
- 5. 5 Retrievable Augmented Generation (RAG) RAG is a technique for improving the quality of generated responses by an LLM. In this process, information from external knowledge sources, along with further instructions, is provided to generate fact-based results. Mitigations Model Fine-Tuning LLM fine-tuning is a process of adjusting and adapting a pre-trained large language model to perform specific tasks or to cater to a particular domain more effectively. While fine-tuning proves effective in emulating behaviors, it's not the best fit for cases that require extensive domain knowledge, such as legal or financial sectors. RAG and Model Fine-Tuning are not mutually exclusive but should be used in combination to ensure high-quality and uniform results.
- 6. ● Ingestion – Vectorize data using an embedding model and store them in memory. ● Retrieve – Get data from memory based on semantic vector similarity of user input. ● Augment – Embed retrieved data into a prompt. ● Generate – Send augmented prompt to model for text inference. Retrieval Augmented Generation 6 You are an assistant for question-answering tasks. Use the following pieces of retrieved context to answer the question. If you don't know the answer, just say that you don't know. Use three sentences maximum and keep the answer concise. Question: {{question}} Context: {{context / text_chunks}}
- 7. RAG Flow Turn information into data – Extract data from information sources and create semantic vector embeddings. ▪ Query – Perform semantic similarity search across vectorized data. ▪ Synthesize – Prepare one-shot or chain of thought prompt instructions and inject search results. ▪ Generate – Let LLM completions generate tailored results based on prompt. 7
- 8. Examples ▪ Web Search – To fetch real-time information from the internet. ▪ Data Fetch – To retrieve specific datasets or information from databases. ▪ Actions – Execute any sort of logic, e.g. sending an E-Mail or adding a database record What is a Tool? A tool refers to an external function or service that can be integrated to a LLM API call. The model analyzes the users input if a tool should be called and tries to identify necessary input parameters for the tool. The purpose of such tools is to add dynamic information into a conversation. It is not the model that executes a tool. It only returns if a tool is applicable to run. 8
- 9. ● Add tools_call message to message conversation. ● Call your tool. ● Add tools result message to conversation. ● Send API request. Tool Calling Flow ● Add Tool description and input parameters to API request. ● LLM reasons over tools and tries to identify input parameters from conversation. ● LLM returns either parameter clarification question or tools call message(s). 9
- 11. Stefan Weber Senior Director Software Development Telelink Business Services OutSystems MVP – OutSystems Certified Trainer – AWS Community Builder [email protected] +49 1590 1888452 https://www.tbs.tech https://without.systems https://lcnc.blog https://www.linkedin.com/in/stefanweber1/