Still buzzing from the incredible #AgenticAI Summit at @UCBerkeley on 8/2 — 2,000+ joined in person, 30,000+ tuned in online. ⚡🌍 The energy was electric—visionaries, builders & researchers shaping the future of agentic AI! Missed it? Watch the recordings: Huge thanks to our speakers, sponsors, @BerkeleyRDI staff, volunteers, & everyone who joined! This momentum builds on our 25K+ LLM Agents MOOC community—and we’re launching a new edition, Agentic AI, in the series this Sep! Sign up here:

Really excited for the Agentic AI Summit 2025 at @UCBerkeley—2K+ in-person attendees and ~10K online! Building on our 25K+ LLM Agents MOOC community, this is the premier global forum for advancing #AgenticAI. 👀 Livestream starts at 9:15 AM PT on August 2—tune in!

Join us at Agentic AI Summit 2025 — August 2 at UC Berkeley, with ~2,000 in-person attendees and the leading minds in AI. Building on the momentum of the 25K+ LLM Agents MOOC community, this is the largest and most cutting-edge event on #AgenticAI. As 2025 emerges as the Year of the Agents, the summit offers a front-row seat to the breakthroughs shaping the future of #AgenticAI. Be part of the movement. 👀 Register for in-person or online attendance:

My group & collaborators have developed many popular benchmarks over the years, e.g., MMLU, MATH, APPS---really excited about our latest benchmark OMEGA Ω: 🔍Can LLMs really think outside the box in math? a new benchmark probing 3 axes of generalization: 1️⃣ Exploratory 2️⃣ Compositional 3️⃣ Transformative showing limitations of today's frontier AI and RL-training in these dimensions of generalization. Inspired by Boden’s typology of creativity, OMEGA advances beyond prior benchmarks with a programmatically generated dataset that combines precise control with rich diversity. Spanning a wide range of mathematical domains, it is explicitly designed to evaluate distinct axes of generalization and creative reasoning. By isolating and quantifying fine-grained failure modes, OMEGA provides a foundation for advancing LLMs toward genuine mathematical creativity—beyond mechanical proficiency. Huge thanks to my postdoc @YiyouSun @UCBerkeley leading the project, and amazing collaborators @nouhadziri @HannaHajishirzi @allen_ai and other co-authors!

1/ 🔥 AI agents are reaching a breakthrough moment in cybersecurity. In our latest work: 🔓 CyberGym: AI agents discovered 15 zero-days in major open-source projects 💰 BountyBench: AI agents solved real-world bug bounty tasks worth tens of thousands of dollars 🤖 Autonomously. A pivotal shift is underway — AI agents can now autonomously do what only elite human hackers could before.

Really excited to share our latest work on AgentSynth: A new paradigm for generating realistic, scalable, and long-horizon computer-use tasks and benchmarks! Our automated pipeline generates a dataset of 6,000+ tasks with two game-changing advantages: Dramatic Cost Savings 💰: AgentSynth produces tasks at just $0.60 per trajectory—orders of magnitude cheaper than traditional human annotation. High-Fidelity Complexity 🧗: These tasks mirror real-world long-horizon challenges, often requiring agents to coordinate across multiple applications like Chrome, VS Code, and LibreOffice to succeed.

🔐 Frontier AI is reshaping cybersecurity, raising critical new questions: 🔍 What is its current impact? ⚖️ Who stands to benefit more—attackers or defenders? 🛡️ How can we mitigate the risks? Addressing these challenges requires coordinated efforts across AI & security communities. In our recent paper, we explore the evolving landscape, analyze the dynamics between attackers and defenders, and call for proactive measures to ensure frontier AI tips the balance toward defense rather than offense. We predict that, in the short term, attackers are likely to gain more immediate advantages from AI capabilities than defenders. However, forecasting these dynamics is complex—and your perspective is vital to improving our collective understanding and response. We invite all AI and cybersecurity experts and practitioners to take our short survey and share your views—whether you agree or disagree with our predictions. #AI #CyberSecurity 🧵👇

🌟 Excited to announce our esteemed panel of judges for the #AgentX competition by @BerkeleyRDI @UCBerkeley. Huge thanks to @xinyun_chen_ @Chi_Wang_ @GoogleDeepMind; @KaiyuYang4 @Meta; @jayrodge15 @ZhidingYu @nvidia; @Somil_Agg @schmidtsciences; Samuel Barry @MistralAI; @bhawna_tweets, Austin Arensberg @Okta; @ben_burtenshaw @huggingface; @chuanli11 @ThomasBord50495 @LambdaAPI; @ozenhati Ben Ankiel @GroqInc; @waseem_s @Get_Writer; @SaraIttelson @Accel; @kevinzhang @BainCapVC; @ReddyVijayB @Mayfield; @maddiehfaulkner @NEA; @whoisnnamdi @David_Schmaier @LightspeedVP; @NextBigTeng @bhavikvnagda @BessemerVP; @divy93t @Google; @ysu_nlp @OhioState; Pushkar Nandkar @SambaNovaAI; Alok Tongaonkar @Cisco for joining us as judges, and more to be announced. 🚀 Close to 1,000 teams already joined #AgentX—building the future of Agentic AI across Entrepreneurship & Research tracks! 💰 Prize pool now $150K+, with total prizes/resources surpassing $450K! 🤖 Join NOW for the AgentX Competition and submit your projects by 5/31. ⏰

Excited to speak at @RSAConference 2025! Join my session "Safety and Security of LLM Agents: Challenges and Future Directions" on May 1 (12:20-1:10 PM PT). We'll explore safety and security challenges of LLM agents and discuss best practices for safe and secure LLM deployment. #RSAC #AI #CyberSecurity

Deeply humbled and honored to be elected to the American Academy of Arts and Sciences @americanacad! Excited to contribute to the Academy’s mission and advancing the common good!