Why the Best Product Managers Think in Systems, Not Features. A conversation with Siddharth Arora

Siddharth Arora has over 15 years of experience developing technology products. As Yelp's AI Product Manager for Experimentation & Analytics, he oversees how one of the world's most popular local discovery platforms makes data-driven decisions. His experience includes B2C, B2B, SaaS, and gaming, with leadership positions at global firms such as Zomato, RateGain, Snapdeal, and Games24x7. He also co-founded the logistics business Turannt and created JustAnotherPM, a popular newsletter for product managers.

In this new interview, we talked with Siddharth about how product management has evolved, what it takes to create a true experimental culture, and how AI is altering the way project managers work.

Hans India: Your career involved working in startups, global marketplaces, and now Yelp. Looking back, what experiences shaped the way you think about product management?

Siddharth:I believe that shifting between different stages of growing a company defined my career, with each step educating me something valuable.

During the early startup period, I learned to be action-oriented: ship fast, validate fast. When you work for a small business with limited resources, you don't have time for extensive planning. You build, learn, and iterate.

The experience I gained working in a global marketplace demonstrated to me the value of systems thinking. At businesses like Zomato and Snapdeal, I observed how simple levers such as price algorithms, ranking mechanisms, and trust signals can influence entire ecosystems. You are not just creating features; you are also inventing the rules of an economy.

Leading experimentation at Yelp taught me data humility. Intuition is useful, but evidence wins. The number of times data has startled or shown something I would not have predicted is humbling.

Building AI data products taught me that clarity outperforms cleverness. Products succeed when complexity is veiled beneath basic user value. Nobody cares how complex your model is if they don't understand what it does for them.

Working through these several stages underlined the fact that project management is context-dependent, rather than one-size-fits-all. What works for a 20-person startup will not work at a 5,000-person company..

Hans India: How would you say the role of product managers has evolved in the last 10 years, especially keeping in mind the rise of AI and data-driven decision-making?

Siddharth Arora: It has changed significantly. If ten years ago most decision-making was opinion-driven, now it is data-validated and AI-driven. We're entering this AI-augmented phase where machine learning models help us predict outcomes and automate decisions at scale.

Success now depends less on "what to build" and more on how to learn fast. The companies winning today are the ones who can iterate fastest and extract insights from every experiment.

AI specifically demands PMs who understand probability, feedback loops, and context engineering: how you frame problems for AI systems and validate their outputs. The modern PM blends storytelling, statistics, and systems thinking, not just roadmaps.

Hans India: You've led experimentation and analytics throughout your career. What would you call the biggest misconception PMs have about building a culture of experimentation?

Siddharth Arora: The first is thinking "more experiments equals better culture." Quantity doesn't equal learning. I've seen teams run dozens of tests without clear hypotheses or success metrics. What matters is disciplined learning: running fewer, better-designed experiments that actually teach you something.

Leadership fear kills experimentation. Without executive sponsorship or psychological safety, teams avoid bold tests that might fail. If none of your experiments are failing, you're not being ambitious enough.

HiPPO still rules in most organizations: the highest-paid person's opinion. Many companies claim to be data-driven but default to executive intuition when data challenges the prevailing wisdom. True experimentation culture means being willing to be proven wrong, regardless of your title.

Siloed experimentation when different teams run isolated tests without coordination is also a problem. Without optimization you get experiment pollution and conflicting metrics. It can seriously hurt the team's goals.

The experimentation also needs to be pretty smooth in terms of bureaucracy. Approval chains that don’t work fast slow down learning loops. If it takes you a couple of weeks to get approval for a test, you've lost the plot.

Hans India: How can experimentation evolve with the help of AI?

Siddharth Arora: Probabilistic outcomes break traditional testing frameworks. AI systems don't have binary results. PMs must define acceptable error rates and variance thresholds, not just declare winners.

Model drift is very real. What works today may degrade tomorrow as user behavior changes. Experimentation must evolve into continuous monitoring and re-validation. You can't just set it and forget it.

Metrics become multidimensional. You're tracking both model health (accuracy, bias, latency) and business KPIs like conversion and retention. And sometimes these are in tension.

Infrastructure is lagging behind the need. Few teams have mature MLOps pipelines for testing multiple model versions, logging AI behavior in production, and monitoring in real time.

There's also this confidence versus speed tension. Higher variance in AI outcomes often forces longer tests. More and more times methods like Bayesian testing or multi-armed bandits become important.

Explainability is essential. A "winning" model doesn't help much if stakeholders are uncertain why it won. You need to be in a position to interpret what the model is doing, not just that it's performing well.

And safety and bias must be baked in from the start. AI experiments can amplify injustice or cause real harm at scale. PMs must build in guardrails, bias audits, and live monitoring into every experiment.

Hans India: What advice would you give to product managers that try to stand out from the crowd in 2025?

Siddharth Arora: First, go beyond. Learn how incentives, data, and users interact. The best PMs think in systems. Companies look for PMs who understand not just roadmaps.

Master the fundamentals. Communication, prioritization, and product sense will always compound faster than chasing the latest framework. I see people collecting frameworks like Pokémon cards, but they can't write a clear strategy doc or prioritize a backlog.

Become data-fluent. You don't need to be a data scientist, but you must understand what metrics, experiments, and evidence mean and how they interact with each other. Also learn SQL and try to understand statistical significance.

Become an early adopter of AI and actually learn how AI products are built and depolyed. This is slowly becoming the new base literacy for PMs.

Write clearly, think clearly. Clarity is a massive differentiator. Good writing forces good thinking, and great PMs do both. Start a blog, write strategy memos, explain complex ideas simply.

Be curious, not performative. Depth beats noise. Spend time learning how things actually work rather than optimizing for visibility.

Show outcomes, not activity. Hiring managers and leaders value measurable impact over how busy you looked doing it. "I shipped five features" means nothing. "I increased retention by 15% through a series of onboarding experiments" means everything.

Finally, compound your learning. Treat your career like a portfolio of bets: write, teach, build, reflect. The PMs who grow fastest are the ones who deliberately invest in learning, not just doing.