Practical Applications of AI: Real World Examples
- 1. Practical Applications of AI - Real World Examples Nathan Dickerson Amir Kurtovic Lukas Neumann
- 2. Nathan Dickerson Amir Kurtovic Senior Developer, Bullhorn Senior Developer, Bullhorn Lukas Neumann Chief Architect, Invenias
- 4. Chatbot Ideation • Reminders • Scheduling • Screening • Follow Up
- 5. Redeployment Chatbot • Problem Statement: – Low redeployment correlates with not knowing when assignments end – Too high effort to check in with candidates constantly for end dates today • Features / Benefits: – Automatic engagement with candidates and hiring managers to find out when end dates actually are – Enables pipeline of redeployment activities thanks to accurate data Improve redeployment by fixing assignment length accuracy.
- 6. Demo
- 7. How We Built This Bullhorn ATS Bullhorn API {. .. } Cloud Functions Firestore DB Twilio Dialogflow Candidate1 3 4 5 6 7 2 8 9
- 8. How We Would Rebuild It Bullhorn ATS Bullhorn API {. .. } Cloud Functions Firestore DB Twilio Dialogflow Candidate1 3 4 5 7 2 8 9 6
- 9. Lessons Learned • Tech stack makes it easy to build out quickly • 10% model training and 90% customization of business rules • DialogFlow makes for a nice trainable ML text parser, but needs several layers of business logic before/after to be practical
- 10. Lessons Learned • Using random number to text from is easy, but ineffective • Difficulty finding a burning need that customers have that can be filled by chatbots
- 12. Problem Definition • Create a machine learning-driven system capable of automatically identifying entity relationships based on historical data
- 13. DEMO
- 14. ML Project Phases Data Collection ML Algorithm Infrastructure Integration Perception Reality
- 15. Problems We Faced Data Collection ML Algorithm Infrastructure Integration No existing labeled dataset Data warehouse exports Data Quality Preprocessing pipelines Storage Balancing dataset distributions Optimizing for business objective New API for interacting with hosted models ETL Pipeline Cloud Infrastructure configuration New UI Components Client onboarding process Cloud functions
- 16. Organizational Challenges • Hard to integrate ML projects into Agile/Lean development processes • Accepting less than perfect performance • Breaking down silos
- 18. Automated Invoice Parsing • GOAL: Eliminate manual data entry of incoming invoices • We built a fully-customizable AI engine which automates document ingestion – Users define which fields they want to extract – They mark the fields in sample documents to generate training data
- 19. Automated Invoice Parsing • The engine learns to automatically extract fields as specified by the user • If the system makes a mistake, user can provide instant feedback – We get more training data – Constant improvement in accuracy
- 20. System Overview Email / Document Invoice Parsing User Validation Scanner Machine Learning Accounting System API {. .. } OCR User Feedback
- 23. Lessons Learned • A lot of training data is required to reach good accuracy – 1,000s to 10,000s of training documents required to reach >95% accuracy – Might be challenging to manually create such volume of training documents
- 24. Lessons Learned • Looking at a specific document, two users might have two completely different answers which field it is (e.g. PO / Invoice #) – Generates lot of noise in the training data – We created “rule book” for users to address these ambiguities so they create consistent training data
- 25. Lessons Learned • Users do not expect AI to have perfect accuracy, but once they give feedback they expect AI won’t make the same mistake again – Infrastructure challenge as this would mean re- training the AI model on the spot to be ready instantly for the next processed document
- 26. Questions? Come visit us at the AI booth