September 2024

This month’s newsletter features insights on Amazon Research Awards' latest call for proposals, the technology behind our generative-AI-powered shopping assistant Rufus, the application and evaluation of large language models (LLMs), and more.

Deep dives

The technology behind Amazon’s GenAI-powered shopping assistant, Rufus: Trishul Chilimbi, Amazon VP and distinguished scientist, blogs about the scientific and engineering effort behind Rufus, Amazon’s new generative-AI-powered shopping assistant. Rufus is a custom LLM specialized for shopping that employs retrieval-augmented generation with a variety of evidence sources and leverages reinforcement learning to improve responses. To increase efficiency and reduce latency, Chilimbi and his team made advances in high-performance computing and implemented a new streaming architecture on Amazon Trainium and Inferentia chips.

How task decomposition and smaller LLMs can make AI more affordable: Burak Gozluklu, Amazon Web Services (AWS) machine learning architect, explains how decomposing generative-AI tasks into subtasks that can be assigned to smaller LLMs can reduce costs, and he analyzes the trade-off between cost and the creativity of the LLMs’ output, providing a “mental model” for when task decomposition makes sense and when it doesn’t.

Accounting for cognitive bias in human evaluation of large language models: To address the effects of cognitive bias on human evaluations of large language models, Amazon researchers propose ConSiDERS, a framework that emphasizes breaking content into easily verified facts, considering diverse evaluator perspectives, and incorporating contextual factors to improve accuracy and responsibility.

Quantifying images’ “conceptual similarity”: Training computer vision models depends on quantifying images’ similarity, but similarity can be a matter of concepts — not just pixel values. At CVPR 2024, Amazon researchers showed how to leverage vision-language models to quantify images’ conceptual similarity according to the length of the descriptions required to discriminate them: images easily discriminated by short descriptions are not very similar, but images that require a lot of text to distinguish them are similar.

Better-performing “25519” elliptic-curve cryptography: Elliptic-curve cryptography (ECC) is a public-key scheme that promises post-quantum security. With microarchitecture-specific optimization and automated reasoning, Amazon scientists made “25519” ECC (named for a large prime) more efficient and trustworthy.

The life of a prescription at Amazon Pharmacy: Pharmacies play a vital role in ensuring patients’ health, but the process of dispensing medications is far more complex than it may appear. At Amazon Pharmacy, researchers are using artificial intelligence to navigate this complexity and improve patients’ experiences — from pricing estimation and regulatory compliance to inventory management and chatbot assistants.

News and updates

How Amazon built Fire TV's new AI-powered search experience: To make Alexa and Fire TV search even smarter, a team of engineers and product experts focused on building a search system beyond keyword matching, harnessing Alexa’s vast knowledge base to enhance Fire TV search results and employing a new ranking machine learning model to improve the order and relevance of results.

How Amazon is using generative AI to improve product recommendations and descriptions: Leveraging generative AI, Amazon provides customers more personalized product recommendations and descriptions based on their shopping activity, reviews, and more. An LLM uses that information to generate new product titles, which highlight features believed to be important to specific customers. Another LLM, known as an evaluator LLM, challenges and improves the results, ensuring that the product information customers see is as useful as possible.

5 new generative-AI tools to accelerate seller growth and enhance the customer shopping experience: These include the personal AI assistant Project Amelia, which uses Amazon Bedrock and offers tailored business insights to boost productivity and drive seller growth, and the Amazon Ads Video generator, which uses generative-AI technology to make it easier for advertisers to create more interesting and relevant video ads for customers. 

Amazon Scholar named finalist for the Blavatnik National Awards for Young Scientists: Alexey Gorshkov was recognized for his research in advancing “the design of large interacting quantum systems through pioneering research at the intersection of quantum physics and information science with groundbreaking implications for quantum computers, sensors, and networks.”

Amazon researchers receive UBTECH Best Industry Application Award – Gold: Dylan Glas, senior applied scientist, and Bill Smart, Amazon Scholar, received the award for a paper presenting a model that enables social robots to identify non-intrusive parking locations in home environments, based solely on spatial geometry from 2-D occupancy maps. The model effectively captures 74% of user-preferred parking spots, demonstrating its potential for improving robot integration into social spaces without prior knowledge of user behaviors.

Academic collaborations

Amazon Research Awards (ARA) issues fall 2024 call for proposals: Now open until November 6, Amazon Research Awards will be seeking proposals in the following research areas: 

• AI for Information Security

• Automated Reasoning

• AWS AI: Generative AI, Governance and Responsible AI, ML Compilers, Distributed Training

• AWS Cryptography

• Sustainability

ARA provides grant recipients unrestricted funds and AWS promotional credits. Funded projects are assigned an Amazon research contact, and recipients also receive training resources, including AWS tutorials and hands-on sessions with Amazon scientists and engineers.

Amazon Science Campus career event series for masters and PhD students: Taking place November 5 - 14 on Tuesdays and Thursdays 5 - 7pm CT, the program provides masters and PhD students the opportunity to interact with Amazon scientists, learn about practical applications in machine learning, natural language processing and computer vision, and discover career opportunities at Amazon. 

Upcoming conferences

• IROS 2024, October 14 - 18, 2024

• NFORMS 2024, October 20 - 23

• CIKM 2024, October 21 - 25, 2024

• NABE 2024, October 27 - 29

New publications

• A framework for efficient model evaluation through stratification, sampling, and estimation

• A model explanation framework aligning Shapley contributions and permutation feature importance

• A shortcut to an optimal quantum linear system solver

• AbsInstruct: Eliciting abstraction ability from LLMs through explanation tuning with plausibility estimation

• Accurate customer address matching via weak supervision for geocode learning

• Address de-duplication using iterative k-core graph decomposition

• Aligning vision language models with contrastive learning

• An international study presenting a federated learning AI platform for pediatric brain tumors

• Annorama: Enabling immersive at-desk annotation experiences in virtual reality with 3D point cloud dioramas

• AutoDOM: Automated dimension overlay for generating measurement guides

• Automatically reducing privilege for access control policies

• Avoiding object damage in robotic manipulation

• Beauty beyond words: Explainable beauty product recommendations using ingredient-based product attributes

• CANDLE: Iterative conceptualization and instantiation distillation from large language models for commonsense reasoning

• Cloud resource protection via automated security property reasoning

• Consensus planning with primal, dual, and proximal agents

• Cost-effective hallucination detection for LLMs

• Data pruning via separability, integrity, and model uncertainty-aware importance sampling

• Data science projects development with Amazon SageMaker

• DiffSign: AI-assisted generation of customizable sign language videos with enhanced realism

• DPA-Net: Structured 3D abstraction from sparse views via differentiable primitive assembly

• Enhancing e-commerce product title translation with retrieval-augmented generation and large language models

• Evaluation of topic continuity using nonlinearlized naive bayes with attention mechanism

• Extending DRAT to SMT

• Free facts: An alternative to inefficient axioms in Dafny

• Label with confidence: Effective confidence calibration and ensembles in LLM-powered classification

• Large-scale indoor mapping with failure detection and recovery in SLAM

• Learning variant product relationship and variation attributes from e-commerce website structures

• MARec: Metadata alignment for cold-start recommendation

• Metapath of thoughts: Verbalized metapaths in heterogeneous graph as contextual augmentation to LLM

• Multi-task combinatorial bandits for budget allocation

• n-Sizes, attributes, and a priori sampling: A qualitative sampling model for large, heterogeneous populations

• Performing efficient and safe deformable package transport operations using suction cups

• Perspectivist approaches to natural language processing: A survey

• Pitfalls in link prediction with graph neural networks: Understanding the impact of target-link inclusion & better practices

• Playlist search reinvented: LLMs behind the curtain

• PoCo: Point context cluster for RGBD indoor place recognition

• PrivLM-Bench: A multi-level privacy evaluation benchmark for language models

• Ranking across different content types: The robust beauty of multinomial blending

• RMT-BVQA: Recurrent memory transformer based blind video quality assessment for enhanced video content

• Self supervised LLM customizer (SSLC): Customizing LLMs on unlabeled data to enhance contextual question answering

• Simulation-assisted learning for efficient bin-packing of deformable packages in a bimanual robotic cell

• SMT-D: New strategies for portfolio-based SMT solving

• Solving string constraints with concatenation using SAT

• Surf-Deformer: Mitigating dynamic defects on surface code via adaptive deformation

• TabRepo: A large scale repository of tabular model evaluations and its AutoML applications

• The Fuse platform: Integrating data from IoT and other sensors into an industrial spatial digital twin

• Transitivity-encoded graph attention networks for complementary item recommendations

• Understanding developer-analyzer interactions in code reviews

• Unsupervised text representation learning via instruction-tuning for zero-shot dense retrieval

• VLPG-Nav: Object navigation using visual language pose graph and object localization probability maps

• Weighted retriever ensembles for video-to-product ads curation

• Well-behaved (Co)algebraic semantics of regular expressions in Dafny

• What does the process look like?: A conceptual framework outlining a virtual simulation and simulator design process for instructional designers

• XCapsUTL: Cross-domain unsupervised transfer learning framework using a capsule neural network

LinkedIn | X/Twitter | Facebook | Instagram | GitHub | RSS

© 1996-2024 Amazon.com, Inc. or its affiliates | Privacy | Conditions of Use