Gartner Catalyst 2017: Image Recognition on Streaming Data
Download as PPTX, PDF•2 likes•488 views
This document discusses using MemSQL to perform real-time image recognition on streaming data. Key points include: - Feature vectors extracted from images using models like TensorFlow can be stored in MemSQL tables for analysis. - MemSQL allows querying these feature vectors to find similar images based on cosine similarity calculations. - This enables applications like detecting duplicate or illegal images in real-time streams.
Gartner Catalyst 2017: Image Recognition on Streaming Data
- 1. Image Recognition on Streaming Data Neil Dahlke, Senior Solutions Engineer 21 August 2017
- 2. AT MEMSQL Senior Solutions Engineer, San Francisco BEFORE MEMSQL I worked on Globus, a high performance data transfer tool for research scientists, out of the University of Chicago in coordination with Argonne National Lab. PREVIOUS TALKS Real Time, Geospatial, Maps (slides) Streaming in the Enterprise (slides) Real Time Analytics with Spark and MemSQL (slides) 2 Me at a Glance
- 3. The future of computing is visual…
- 4. 4
- 5. 5
- 6. 6
- 7. Mapping Social Imagery Handwriting and many more.
- 8. and it is also mathematical.
- 9. But first, let’s create a shared vocabulary.
- 10. Easy to setup real-time data pipelines with exactly-once semantics Streaming Data Ingest Memory optimized tables for analyzing real-time events Live Data Disk optimized tables with up to 10x compression and vectorized queries for fast analytics Historical Data 10 MemSQL at a Glance
- 11. Data Loading Query Latency Concurrency FAST LOW Vectorized queries Real-time dashboards Live data access Multi-threaded processing Transactions and Analytics Scalable performance HIGH Stream data Real-time loading Full data access 11
- 12. • Distributed, ANSI SQL, database • Full ACID features • Lock free, shared nothing • Compiled queries • Massively parallel • Geospatial and JSON • In-memory and on-disk • MySQL protocol • Streaming • HTAP (rowstore and columnstore) MemSQL in One Slide 12
- 13. Architecture: MemSQL Building Blocks memsqld 13
- 14. Architecture: Aggregators and Leaves Agg 1 Agg 2 Leaf 1 Leaf 2 Leaf 3 Leaf 4 14
- 15. Architecture: Aggregators Aggregate Agg 1 Agg 2 Leaf 1 Leaf 2 Leaf 3 Leaf 4 15
- 16. Architecture: Leaves Hold Partitions Agg 1 Agg 2 Leaf 1 Leaf 2 Leaf 3 Leaf 4 16
- 17. Architecture: It’s SQL All The Way Down Agg 1 Agg 2 agg1> select avg(price) from orders; leaf1> using memsql_demo_0 select count(1), sum(price) from orders; leaf2> using memsql_demo_12 select count(1), sum(price) from orders; ... Leaf 1 Leaf 2 Leaf 3 Leaf 4 17
- 18. Now, back to image recognition.
- 19. 19
- 21. How does it work?
- 22. 22 Real-Time Image Recognition Workflow ▪ Train a model with Spark and TensorFlow ▪ Use the Model to extract feature vectors from images • Model + Image => FV ▪ You can store every feature vector in a MemSQL table CREATE TABLE features ( id bigint(11) NOT NULL, image binary(4096) DEFAULT NULL, KEY id (id)USING CLUSTERED COLUMNSTORE );
- 23. 4,996 POINTS CLASSIFICATION DE-DUPLICATION MATCHING
- 26. 26 Working with Feature Vectors For every image we store an ID and a normalized feature vector in a MemSQL table called features. ID | Feature Vector x | 4KB To find similar images using cosine similarity, we use this SQL query: SELECT id FROM feature_vectors WHERE DOT_PRODUCT(image, 0xDEADBEEF) > 0.9
- 28. 28 Understanding Dot Product ▪ Dot Product is an algebraic operation • X = (x1, …, xN), Y = (y1, …, yN) • (X*Y) = SUM(Xi * Yi) ▪ With the specific model and normalized feature vectors DOT PRODUCT results in a similarity score. • The closer the score is to 1 the more similar are the images
- 29. 29 Performance numbers ▪ Memory speed: ~50GB/sec ▪ Vector size: 4KB ▪ 12.5 Million Images a second per node ▪ 1 Billion images a second on 100 node cluster
- 30. 30 Performance Enhancing Techniques Achieving best-in-class dot product implementation ▪ SIMD-powered ▪ Data compression ▪ Query parallelism ▪ Scale out ▪ Result: Processing at Memory Bandwidth Speed
- 31. 31 MemSQL gives us… ▪ Performance ▪ Scalability ▪ High concurrency ▪ Real-time (operational) ▪ Compatibility (BI, Spark, Kafka, ETL, etc) ▪ Hybrid deployment ▪ Robustness, durability, security ▪ And also…
- 32. Load Ingest from Apache Kafka, Amazon S3 or Kinesis Guarantee message delivery with exactly-once semantics Transform Map and enrich data with user defined or Apache Spark transformations MemSQL Streaming Extract 32
- 33. memsql> CREATE PIPELINE features_pipeline AS -> LOAD DATA KAFKA "public-kafka.memcompute.com:9092/features" -> INTO TABLE feature_vectors -> (id, feature); -> FIELDS TERMINATED BY ',’ -> (id, @image) -> SET image = UNHEX(SUBSTRING(@image, 3)); Query OK, (0.89 sec) memsql> START PIPELINE features_pipeline; Query OK, (0.01 sec) 33 Simple Streaming Setup with CREATE PIPELINE
- 34. 34 MemSQL Pipelines Sequence Data Sources MemSQL 1. Extract 2. Transform extracted data 3. Load into Database tables Pipelines
- 35. 35 MemSQL Pipelines Architecture: Kafka Kafka Broker MemSQL LeafPipelines Kafka Broker MemSQL LeafPipelines Kafka Broker MemSQL LeafPipelines MemSQL AggPipelines 1. Extract 2. Transform 3. Load Data reshuffle Metadata query 1. Extract 2. Transform 3. Load 1. Extract 2. Transform 3. Load
- 36. Real-Time Application New Image Stream Real-Time Processing Reference Image Store TensorFlow Streaming Real Time Image Recognition Workflow
- 37. Q&A
- 38. The Data Warehouse Blueprint for ML, AI and Hybrid Cloud GARY ORENSTEIN, SVP PRODUCT Tuesday, August 22 | 1:45p.m.-2:05p.m. | TechZone Theater at
Editor's Notes
- #26: Normalize each feature vector by dividing each element in the vector by the length of the vector, which gives us a length of 1 The similarity is higher if the dot product is close to one. is an algebraic operation that takes two equal-length sequences of numbers (usually coordinate vectors) and returns a single number. In Euclidean geometry, the dot product of the Cartesian coordinates of two vectors is widely used and often called inner product (or rarely projection product); see also inner product space. Algebraically, the dot product is the sum of the products of the corresponding entries of the two sequences of numbers.