Scaling AI Through Multi-Agent Collaboration
๐ฏ RecSys'25 Tutorial
Agentic Recommendation Systems
๐ Prague, Czech Republic โข September 26, 2025
๐
How Do We Scale Up AI?
โก
Traditional Scaling Laws Hit Limits
๐ก
Power Crisis
4-16 GW by 2030
Enough to power entire cities
Enough to power entire cities
๐
Data Depletion
Depleted by 2026-2028
Quality text data exhausted
Quality text data exhausted
๐ง
Performance Plateau
GPT-5 delayed >1 year
Early training runs failed
Early training runs failed
โ ๏ธ Inference-time scaling
No universal way to leverage improvements & address limitations
The New Paradigm:
Model โ Agent โ Multi-Agent Systems
๐ค
The Promise of Multi-Agent Collaboration
- Study Group Dynamics: Like humans collaborating on complex problems
- Cross-Ecosystem Integration: Bridge Claude, Gemini, GPT, Grok, and specialized coding agents
- Emergent Intelligence: Collective problem-solving beyond individual capabilities
- Real-time Intelligence Sharing: Agents learn and adapt from each other
The Promise of Collaborative Reasoning
๐๏ธ Built on AG2's foundational multi-agent research and community
๐
Proven Performance Gains
Grok-4 Standard
1
Single Agent Processing
38.6%
Last Human Exam Score
$30/month
Grok-4 Heavy
A1
A2
A3
Multi-Agent Collaboration
44.4%
Last Human Exam Score
$300/month
Gemini 2.5 DeepThink
๐
๐ฅ
Competition Gold Medals
IMO + ICPC
5/6 IMO Problems (2024)
10/12 ICPC Problems (2025)
First AI Gold Medals
๐ Multi-Agent Revolution
"Individual AI excellence + Multi-agent coordination = Next frontier of AI capabilities"
"Individual AI excellence + Multi-agent coordination = Next frontier of AI capabilities"
๐
MassGen Orchestrator
Task Distribution & Coordination
โ
๐๏ธ
Agent 1
Anthropic/Claude
๐จโ๐ป
Agent 2
Claude Code
๐
Agent 3
Google/Gemini
๐ค
Agent 4
OpenAI/GPT
โก
Agent 5
xAI/Grok
โ Real-time Collaboration โ
โ
๐
Shared Collaboration Hub
Real-time Notification & Consensus
๐
Key Features & Capabilities
๐
Iterative Refinement
The Reality of Reasoning
โ๏ธ
Multi-Backend Support
โก
Parallel Processing
๐ฅ
Intelligence Sharing
๐ฏ
Consensus Building
๐ง
Tech Deep Dive: Backend Abstraction Challenges
๐ฏ
Tech Deep Dive: Binary Decision Framework Solution
๐ฏ Key Innovation: Vote Invalidation Creates Dynamic Consensus
๐ฌ
Case Study: Success Through Peer Correction
Graduate-level physics question from GPQA-Diamond benchmark
๐ The Problem
A quasar shows a peak at 790 nm wavelength. Given Lambda-CDM cosmological parameters
(Hโ = 70 km/s/Mpc, ฮฉโ = 0.3, ฮฉฮ = 0.7), what is the comoving distance?
Options: A) 8 Gpc B) 7 Gpc C) 6 Gpc D) 9 Gpc
Options: A) 8 Gpc B) 7 Gpc C) 6 Gpc D) 9 Gpc
๐ฏ Final Result
โ
Correct Answer: A (8 Gpc)
Orchestration succeeded where individual agents initially failed
๐ค Round 1: Initial Answers
Claude: "I calculate ~6 Gpc โ Answer C"
GPT-5: "I get ~8.95 Gpc โ Answer D"
Gemini: "~6.1 Gpc โ Answer C"
GPT-5: "I get ~8.95 Gpc โ Answer D"
Gemini: "~6.1 Gpc โ Answer C"
๐ Self-Correction Process
Claude observes: "There is significant discrepancy in calculations:
Agent1 gets ~6.1 Gpc, Agent2 gets ~8.95 Gpc. Let me re-examine..."
โจ Breakthrough Moment
Claude revises: "Standard cosmological calculators yield 8000-8500 Mpc
for z=5.5. This equals 8.0-8.5 Gpc, closest to option A."
Result: 3/4 agents converge on correct answer
Result: 3/4 agents converge on correct answer
๐ก Success Mechanism:
Peer observation โ Discrepancy detection โ Self-correction โ Consensus
Peer observation โ Discrepancy detection โ Self-correction โ Consensus
๐
Benchmarking: Preliminary Results
Scientific evaluation across graduate-level reasoning, instruction-following, and narrative tasks
๐งช GPQA-Diamond
Graduate Physics/Chemistry
MassGen
87.4% ๐
Gemini
85.9%
Grok-4
85.4%
GPT-5
84.8%
Claude
68.2%
๐ IFEval
Instruction Following
MassGen
88.0% ๐
GPT-5
87.4%
Grok-4
84.7%
Gemini
66.0%
Claude
63.6%
๐ MuSR
Narrative Reasoning
Gemini
69.6% ๐
GPT-5
69.2%
MassGen
68.3%
Grok-4
67.6%
Claude
62.8%
๐ Overall Champion
MassGen: 81.2%
Wins 2/3 benchmarks โข Statistically significant
โ
Key Results:
โข Highest on 2/3 benchmarks
โข Best overall average
โข Consistent performance
โข Best overall average
โข Consistent performance
๐ Statistical:
โข vs Claude: p = 1.4e-07 โญโญโญ
โข vs Gemini: p = 1.1e-28 โญโญโญ
โข Not due to chance
โข vs Gemini: p = 1.1e-28 โญโญโญ
โข Not due to chance
๐ฌ Research Gap:
โข Oracle: 95.5% (GPQA)
โข Actual: 87.4%
โข Potential: 8.1 points
โข Actual: 87.4%
โข Potential: 8.1 points
๐ฏ
Agentic Recommendation Applications
Multi-Agent Personalization Pipeline
๐
Content Analysis
Multiple agents analyze user behavior, content features, and contextual signals simultaneously
โ๏ธ
Preference Fusion
Collaborative filtering, content-based, and deep learning agents debate optimal recommendations
๐ฏ
Dynamic Ranking
Real-time consensus building for personalized item ranking and diversity optimization
Cross-domain expertise, multi-objective optimization, explainable recommendations
๐ฌ
Live Demonstrations
๐ LLM Fun Facts Website (v0.0.14): Claude Code agents create interactive websites with enhanced logging and workspace isolation
Result: Conflict-free parallel development with comprehensive versioning
๐ Unified Filesystem (v0.0.16): Cross-backend collaboration between Gemini and Claude Code agents creating educational content with shared workspace management
Result: First-time cross-backend coordination producing comprehensive 25-slide presentations
๐ IMO 2025 Winner Research: Multi-agent fact-checking โ unanimous consensus on Google DeepMind victory
Result: Accurate identification despite conflicting information
๐ฐ Technical Analysis: Complex Grok-4 HLE pricing calculation through iterative refinement
Result: Accurate cost estimates through collaborative validation
โก
Get Started in 60 Seconds
# 1. Clone and setup
git clone https://github.com/Leezekun/MassGen
cd MassGen && pip install uv && uv venv
# 2. Configure API keys
cp .env.example .env # Add your API keys
# 3. Run single agent (quick test)
uv run python -m massgen.cli --model gemini-2.5-flash "When is your knowledge up to"
# 4. Run multi-agent collaboration
uv run python -m massgen.cli --config three_agents_default.yaml "Summarize latest news of github.com/Leezekun/MassGen"
git clone https://github.com/Leezekun/MassGen
cd MassGen && pip install uv && uv venv
# 2. Configure API keys
cp .env.example .env # Add your API keys
# 3. Run single agent (quick test)
uv run python -m massgen.cli --model gemini-2.5-flash "When is your knowledge up to"
# 4. Run multi-agent collaboration
uv run python -m massgen.cli --config three_agents_default.yaml "Summarize latest news of github.com/Leezekun/MassGen"
โ Supported Models & Providers
๐ข Major Providers:
Anthropic Claude & Claude Code โข Google Gemini โข OpenAI GPT โข xAI Grok โข ZAI GLM
๐ Local & Extended:
Cerebras โข Fireworks โข Groq โข LM Studio โข OpenRouter โข Together...
๐ ๏ธ Advanced Tools
Web Search โข Code Execution โข MCP Tools โข File Operations โข Browser Automation โข Advanced Permissions
๐
Build Agentic Recommendation Systems
Multi-Agent Personalization at Scale
๐ v0.0.3โv0.0.23 Latest | 10+ providers; Claude Code CLI; GPT-5 & GPT-OSS; Local Models; MCP Integration; Unified Filesystem
Thank you RecSys'25!
Questions & Discussion
Questions & Discussion