AI/ML Infra Meetup | Scaling Vector Databases for E-Commerce Visual Search: Architectural Strategies for Millions of Products
0 likes353 views
AI/ML Infra Meetup Aug. 29, 2024 Organized by Alluxio For more Alluxio Events: https://www.alluxio.io/events/ Speaker: - Mahesh Pasupuleti (VP of DS, ML & Data Infra @ Poshmark) In the rapidly evolving world of e-commerce, visual search has become a game-changing technology. Poshmark, a leading fashion resale marketplace, has developed Posh Lens – an advanced visual search engine that revolutionizes how shoppers discover and purchase items. Under the hood of Posh Lens lies Milvus, a vector database enabling efficient product search and recommendation across our vast catalog of over 150 million items. However, with such an extensive and growing dataset, maintaining high-performance search capabilities while scaling AI infrastructure presents significant challenges. In this talk, Mahesh Pasupuleti shares: - The architecture and strategies to scale Milvus effectively within the Posh Lens infrastructure - Key considerations include optimizing vector indexing, managing data partitioning, and ensuring query efficiency amidst large-scale data growth - Distributed computing principles and advanced indexing techniques to handle the complexity of Poshmark's diverse product catalog
Ad
More Related Content
Similar to AI/ML Infra Meetup | Scaling Vector Databases for E-Commerce Visual Search: Architectural Strategies for Millions of Products (20)
More from Alluxio, Inc. (20)
Ad
Recently uploaded (20)
![Download Wondershare Filmora Crack [2025] With Latest](https://cdn.slidesharecdn.com/ss_thumbnails/neo4j-howkgsareshapingthefutureofgenerativeaiatawssummitlondonapril2024-240426125209-2d9db05d-250419-250428115407-a04afffa-thumbnail.jpg?width=560&fit=bounds)
Ad
AI/ML Infra Meetup | Scaling Vector Databases for E-Commerce Visual Search: Architectural Strategies for Millions of Products
- 1. POSHLENS
- 2. Scaling Vector Databases for E-Commerce Visual Search: Architectural Strategies for Millions of Products Mahesh Pasupuleti Poshmark VP - DS, ML & Data Eng
- 3. >_ Poshmark is a leading fashion resale marketplace powered by a vibrant, highly engaged community of buyers and sellers and real-time social experiences. Designed to make online selling fun, more social and easier than ever, Poshmark empowers its sellers to turn their closet into a thriving business and share their style with the world. Since its founding in 2011, Poshmark has grown its community to over 130 million users, helping sellers realize billions in earnings, delighting buyers with deals and one-of-a-kind items, and building a more sustainable future for fashion
- 4. Data & ML Architecture
- 5. Posh Lens ● Visual search feature that allows shoppers to search for desired merchandising using just a photo ● Enhanced product discovery by utilizing image data from listings, adding a powerful search method beyond text-based options.
- 6. Embedding ● Image embedding is a numerical representation that captures the key visual features of an image, allowing it to be compared and matched with other images in a multi-dimensional space. ● Image embedding is like turning an image into a 3D point based on its features (like color, shape, and texture), so similar images end up close together in that 3D space. Just that its in order of 250-1000 dimensions instead of just 3 dimension ● Poshmark uses proprietary model to compute image embeddings
- 8. Creating product ● Listing creation & Updates are captured via event bus system ● Clip model & Category classifier hosted on GPU enabled machines ● Queue worker converts Image to Embeddings ● Embedding are then indexed to Milvus Database in respective indexes Listing Create / Update Events Image Embedding
- 9. Querying ● App server sends image to ML search api ● ML search api calls: a. Object detection API, the biggest bounding box (B) along with the category that’s supported is picked for search. b. Image is cropped for B dimensions and retrieval api is called for generating embedding c. Milvus Search is performed against embedding generated ● App server will receive listing_ids and the similarity scores and then query elasticsearch for filtering/ business logic application. Sample Request Sample response
- 10. Why Milvus ? ● High-Performance Vector Search ○ Milvus is designed to handle large-scale vector searches efficiently. It can index and search through millions or even billions of vectors rapidly ● High Availability & Fault Tolerance ○ Milvus supports data replication across multiple nodes ○ In case of a node failure, Milvus automatically reroutes queries and tasks to other available nodes ● Load Balancing ○ Milvus can be deployed with load balancers that distribute incoming requests across multiple nodes ● Horizontal Scaling ○ Milvus employs a distributed architecture where compute and storage are separated ○ The separation of query, data, index nodes allows for better scaling. ● Open Source ○ Apache License Version 2.0 backed by Linux Foundation ● Rolling upgrades for each components allows for zero downtime
- 12. Milvus Configuration ● Indexes: ○ 10 : each per category ● Index Type ○ HSNW 64 ● Capacity ○ 150 M vectors of 256 Dimensions ● Coordinator ○ 2 Nodes * C6i.xlarge ● Data Nodes ○ 2 Nodes * M6i.large ● Etcd ○ 3 Nodes * C6i.xlarge ● Index ○ 4 Nodes * C6i.2xlarge ● Storage ○ MinIO ○ 4 Nodes * M6i.xlarge ● Message Storage ○ Pulsar ○ 3 nodes * R6i.xlarge ● Query Nodes ○ 21 Nodes * r6i.2xlarge
- 13. Challenges ● Filtering ○ Collections, Partitioning & Post Filtering ● Pagination ○ Choose a large value for top K parameter and filter results that are not required for current page and cache it in a different layer ○ Offset is supported but internally gets all N+Offset number of results and then performs a filter ● Higher QPS is expensive ○ We have observed that in cluster mode - the configuration provided based on inputs is usually good for 25QPS at p99 of 100ms. Increase in QPS requires scaling up of the query nodes/ increasing replicas as well. ● Parameter tuning ○ Nlist, NProbe ● Deletions are lazy ● Higher Cost of maintenance
- 14. Future ● Pagination ○ This can be done via iterator + Caching ● Different indices ○ We tried experiments with FLAT, IVF_FLAT, IVF_SQ8 and HNSW. ○ IVF_SQ8 provides a good balance between system requirements and performance of search. IVF_SQ8 was providing a recall of 90+% compared FLAT. ● GPU based index ○ It supports Higher QPS at lower costs ● Upgrades ○ Better support and improved performance ● Broaden Use cases ○ Complete the look, Trends, Similar Items ● Multimodal ○ Explore text + image embedding for richer results
- 15. Q & A
- 16. Thank You