A Well-Rounded Evaluation of Recommender Systems. The Second Edition of the workshop on well-rounding the evaluation of recommender systems, covering fairness, robustness and trustworthiness.
KDD, August 7th, Long Beach (CA), US
The ubiquity of personalized recommendations has led to a surge of interest in recommender systems research; however, evaluating model performance across use cases is still difficult.
Building on the success of EvalRS 2022 and Reclist, we are back with a broader scope and a new interactive format: our workshop focuses on multi-faceted evaluation for recommender systems, mixing a traditional research track (with prizes for innovative papers) and a hackathon , to live and breath the challenges with working code (with prizes for the best projects).
Important: Registration to KDD is required to attend the workshop, but submitting papers is not required to participate in the hackathon. The hackathon and social event is an affiliated event organized with the help of our generous sponsors, Snap , Costanoa and by Bauplan Labs .
We'll host a pizza hackathon night, where participants will pursue innovative projects for the rounded evaluation of recommender systems!
A new, open dataset, with notebooks, models, and tests is already available if you want to get into the workshop mood early: check the official repository to get started (yes, you can start your project before the workshop if you desire to do so!). As usual, everything goes back to the community as open source contributions.
Important: The hackathon and social event is an affiliated event organized with the help of our generous sponsors. To participate in the hackathon and win the prizes, you're not required to submit a paper to the workshop: if you can't be in Long Beach, write us an email, as we welcome hybrid participation from teams that can't come in person.
|Realistic but Non-Identifiable Synthetic User Data Generation||Link|
|Disentangling the Performance Puzzle of Multimodal-aware Recommender Systems||Link|
|Bridging Offline-Online Evaluation with a Time-dependent and Popularity Bias-free Offline Metric for Recommenders||Link|
|Evaluating Recommendation Systems Using the Power of Embeddings||Link|
|Metric@CustomerN: Evaluating Metrics at a Customer Level in E-Commerce||Link|
Bio : Joey is an engineering manager at Snap, where he leads the Web Ad Ranking team. His team owns the development of the CTR and CVR models used for predicting web engagements for Snapchat ads. Prior to Snap, he led the machine learning team at an AI startup (Scaled Inference) and studied computer science and machine learning at Carnegie Mellon University.
Abstract : Ranking and recommendation systems power a variety of Snapchat
features, including spotlight, discover, maps, search, lenses, ads, and friending. For all of
these products, model evaluation and analysis tooling play a central role at each stage of the
model development lifecycle. In this talk, we will discuss the evolution of Snapchat's unified
model evaluation system and the key design decisions that allowed it to grow to support more
than a dozen ML use cases at scale. We will also discuss the importance of extending model
evaluation beyond a single accuracy number, and how a flexible evaluation system can allow
engineers to ask and answer questions about their models that help to guide future model
Bio : Luca Belli is the founder of Sator Labs, a Visiting Nist AI Fellow and a UC Berkeley Tech Policy Fellow. Previously he was the co-founder and Research Lead for Twitter’s Machine learning Ethics, Transparency and Accountability (META) team where he guided industry leading approaches for responsible ML practices and product changes. Before that he operated as a Data Science and Machine Learning Engineer at Conversant and WolframAlpha. His research interests lie at the intersection of feedback loops, algorithmic amplification (with a special eye on politics), and algorithmic audits. He holds a Ph.D. in Math from Tor Vergata University in Rome.
Politecnico di Bari