Andrej Žukov is an expert in natural language processing and voice AI. Rod Rivera and Max Tee discuss with him the evolution of NLP, the current trends in voice AI, the challenges of adoption, and how businesses can implement voice AI effectively.
The conversation also delves into the impact of AI on pricing strategies, highlighting the importance of flexibility in billing and the changing landscape of consumer pricing.
We explore the evolving landscape of AI pricing models, the future of outcome-based pricing, Stripe’s strategic positioning in the AI billing space, and the transformative role of AI in market research and consulting.
We discuss how flexible billing systems are essential for adapting to new business models and how AI can enhance research capabilities while also reshaping traditional consulting roles.
Chapters
00:00 Introduction to AI and Voice Technology
01:10 The Evolution of Natural Language Processing
06:20 Current Trends in Voice AI
11:22 Challenges in Voice AI Adoption
19:22 Implementing Voice AI in Business
25:19 AI and Pricing Strategies
36:19 Evolving Pricing Models in Tech
41:05 The Future of Outcome-Based Pricing
45:21 Stripe’s Position in the AI Landscape
52:04 The Role of AI in Market Research and Consulting
Takeaways
Voice AI is approaching a breakthrough moment where technology and user experience finally converge: The combination of large language models handling idiosyncratic speech, effective tool-calling capabilities, and dramatic improvements in speech-to-text conversion are creating systems that people might actually enjoy using. While businesses are already adopting voice AI for low-stakes applications like debt collection, bookings, and basic customer service, high-stakes interactions still require human touch. The final barrier to widespread consumer adoption is believability – voice AI systems need to adapt naturally to conversational context rather than using artificial, one-size-fits-all voices and tones.
AI is fundamentally changing pricing strategies, but billing infrastructure often constrains innovation: As businesses explore outcome-based pricing for AI services, they’re constrained by what their billing systems can actually handle. For consumer AI, the market has oddly settled on $20/month as a standard price point, while business applications naturally gravitate toward usage-based models with committed bands and volume discounts. As billing systems evolve to handle more complex arrangements, we’ll see increasingly sophisticated pricing that better matches the unique value propositions of AI services, including tiered pricing, regional adjustments, and more flexible contracting terms.
The future of consulting isn’t extinction but transformation, as AI struggles with proactive information gathering: While deep research tools will replace entry-level analysis tasks traditionally handled by junior consultants, the core consulting function requires capabilities AI still lacks. Specifically, AI needs to become proficient at asking good questions, managing projects, and packaging deliverables in ways familiar to the market. The most valuable consulting work involves extracting ‘frontier knowledge’ locked in people’s minds within organizations – information that isn’t available online or in databases. This requires proactive questioning and relationship management that current AI can’t replicate. Rather than eliminating consulting, AI may increase the premium on high-quality human experiences and interactions while automating routine information gathering and analysis.
YouTube Episode
Spotify Podcast
Episode Transcript
Introduction and Guest Background
Max (00:01): Welcome to the next episode of the Chris Rodman show. Today we have a very special guest with us - someone with deep expertise in NLP who will also talk to us about voice AI. We’re very excited to have you, Andrej.
Andrej (00:22): Thank you so much for having me. Very excited for this episode.
Max (00:24): Awesome. Rod and I chatted about this yesterday and tried to understand what’s happening in the AI world. Your expertise will really help us understand better. For today’s episode, we’ll talk about what’s happening in voice AI, spend some time discussing AI and pricing, touch on Stripe’s recent announcement with all the AI activity within the company, and close with some cautionary tales about what deep research can and cannot do currently. Let’s start with a quick introduction about your background, Andrej.
Andrej (01:17): Sure. My background is originally in natural language processing. I did a PhD in the field pre-LLMs. Back then, we were still grappling with NLP subtasks. I like to think of it as a lot of highlighting words, connecting words, translation, etc. These were all subfields in natural language processing before LLMs really unified the field.
I worked in a couple of London-based startups, spent some time applying natural language processing at BlackRock. During the pandemic, I got into Y Combinator and started a pricing startup to help companies optimize their prices. More recently, I’m working on a voice AI product related to market research – speaking to people to gather information from them. That’s my background and story in short.
The Evolution of NLP and Generative AI
Max (02:23): That’s super interesting. Let’s start with your background. You mentioned something quite interesting to me – how generative AI has unified the NLP space. I wonder if you could elaborate on that. When you say “unify,” how does it do it, from the model itself? It’s quite interesting from a practitioner’s perspective.
Andrej (02:51): Sure. It used to be the case that natural language processing and machine learning in general was split into more manageable subtasks. Specifically in natural language processing, let me highlight three examples:
One would be recognizing named entities in a piece of text. For example, “the United Nations” is a named entity, and one of the use cases of recognizing named entities is that you can take them for further analysis – like determining the sentiment toward the United Nations in a piece of text.
That leads me to the second subtask, which might be sentiment analysis. Is a piece of text positive about a particular entity, or negative?
Maybe a third subtask would be translation – how do you go from English into German or vice versa?
If we double-click on those three examples: the first example, recognizing “the United Nations,” is almost a word highlighting problem. The second problem, sentiment analysis, is either a classification problem (positive or negative) or perhaps a regression problem if you’re trying to give a score between zero and one. And finally, with translation – that’s actually where historically a lot of generative AI and NLP draws its lineage from.
When you’re doing translation, you’re actually moving entities from one side to the other, making sure the sentiment of the text in English is the same as the sentiment in German. It’s a much more general problem. Translation was a very exciting precursor in NLP to modern generative models.
But what these modern models have done is remarkable – now you can go into ChatGPT and give it a piece of text, and it’ll recognize the United Nations for you, tell you the sentiment, or translate it into German. That’s what I mean by moving from small subtasks to a much more general model.
Max (05:34): Understood. That’s quite interesting. It’s almost like if I’m a human – I’m able to speak, eat, listen – they’re all different tasks, and then you combine it all together.
Andrej (05:46): And we’re seeing this story repeat itself. It’s a frequent, repeating historical pattern. We’ve pushed the limits of LLMs now, and we’ve moved from pre-training to test-time compute and multi-step reasoning, but maybe even that isn’t good enough. So we need to wrap it in layers of agentic processes and perhaps even layers of software. The hope is that maybe we’ll shake those shackles eventually as well.
Max (06:20): Interesting – it’s almost like you break it down to such small tasks, then build it back up and start combining with other elements to make even bigger tasks. It reminds me of the atomic level of adding different items to get to bigger things. That’s super cool. Thank you, that’s very helpful.
The Future of Voice AI
Max (06:20 continued): One of the things I wanted to check in on is around voice AI. Last year, if I remember correctly, a lot of larger venture firms in the valley had companies coming up with different types of voice AI. I wanted to get your thoughts on where you think voice AI is going based on your experience, understand how far we’ve come with voice AI, and where we are in the cycle. Perhaps we can do a little future-gazing.
Andrej (07:25): For sure. The TLDR of where we are now with voice AI is that we’re on the exponential up-ramp of voice AI getting really good. It reminds me of video calling, which only really took off with services like Skype, but you could orchestrate a video call in the 1980s – there were ways. Television networks were doing it when they had live news presenters in the field.
I remember speaking to the founder of Deepgram, which is famous for its speech-to-text models. He told this story about how in the early 2000s during his physics PhD, they created a voice AI model they could speak to in their lab. It wasn’t particularly good and was slow, and there weren’t many use cases for it back then.
But now, truly, there have been several big unlocks. First is large language models – the ability to have a general model that can handle the long tail of idiosyncrasies when people speak to voice AI. Names can be spelled in different ways, booking a table in a restaurant can be said in innumerably many ways. LLMs are very good at processing that.
The other unlock is tool calling – the ability for LLMs to call tools to action something and return it to the user in spoken form. Finally, the tech has just gotten really good in terms of speech-to-text models, text-to-speech models, and voice realism has improved massively. We now even have sound-in, sound-out models that aren’t as pipelined as previously.
But there are still bumps in the road. I expect in the next 12 months, those will be ironed out. There was interesting news just the other day about a new model from Sesame AI, which is pushing the boundary of reaching a voice AI model that you actually enjoy speaking to.
I think the crux is that speaking to an AI is actually pretty annoying. Sometimes it’s fun, but as soon as it makes its first mistake, you kind of drop it. I think we will hit a point where that isn’t the case anymore.
Max (10:18): That’s fascinating.
Voice Interfaces: Past Challenges and Adoption
Rod (10:40): Taking a step back, you mentioned that we’ve had this technology for decades. In the 80s and 90s, it was clunky and hard to use, and in recent years, it has gotten much better. However, we don’t really see that much voice as an interface. We tried a few years ago with Siri and all these assistants, and they didn’t catch on as expected. Do you have any thoughts on why that might be? Is it because we humans prefer to communicate some other way, or was the technology non-responsive, clunky, or ineffective?
Andrej (11:22): For sure. I would bifurcate this into business use cases and consumer use cases.
On the business side, the problem with voice AI historically has been achieving high enough accuracy to make it palatable for business use. Take the London-based company Poly AI, which works with large enterprises to automate customer support calls. One of their big use cases is bookings – for restaurants, pubs, gyms, etc. As I mentioned, there’s a really long tail of ways to book a place. You might call with a unique name for the local UK context, or say “I need a table for four, but there’s a child coming so I need a children’s seat.”
The way you tackled that pre-LLM was by building Byzantine pipelines to handle all possibilities. Now, the ease with which you can get extremely good accuracies has suddenly improved. On the business side, we’re seeing a large amount of adoption for low-stakes calls. What are low-stakes calls? For example, debt collection calls where you just need to continuously remind someone to pay, or slightly higher stakes like routing in freight, where a driver may call a centralized agent without having to speak to a human.
There are cases where the stakes are too high for voice AI to work well. In outbound sales, for instance, we’re seeing a spike in demand testing where people automate cold calls to gauge demand. But I don’t think we’re seeing actual applications of outbound sales calls by large enterprises trying to close sales – that’s seen as too high-stakes.
On the business side, there’s lots of application and traction already. On the consumer side, I think the biggest blocker is simply the believability of the voice. I encourage anyone listening to Google “Sesame AI” and check out probably the latest, best voice AI model available. It’s very, very good.
By believability, I mean that when we speak, the tone of our voice adapts to the current context. I’m not speaking to you in an ASMR voice now. I could convey the same information holding my microphone close and speaking in a sexy voice, but that would defeat the purpose of how podcasts are typically done – it would be a mismatch.
There’s often a mismatch between how a voice AI speaks to you – some “horrendous, agreeable, hyperactive American” voice – and your tired 11 PM self that wants to chat about whatever. On the consumer side, I think it’s much more about believability, and I think we’ll get there soon.
Rod (15:46): I just want to say that one thing I feel, especially on the consumer side, is that we’re already conditioned by decades of very limited systems. For my case, when I’m on the phone and notice I’m talking to a system, I’m always wondering what’s actually possible. I’m very cautious about what to say, wondering if the system will understand “yes” or “no.” I feel that maybe we’re used to ineffective, clunky systems, and therefore we’re very cautious about what to say.
Flow States and AI Interactions
Andrej (16:32): Yes, and there’s a parallel to be drawn with how coders used to feel or still feel. When coding, you’re typically conditioned to be very diligent – make sure the code compiles, runs correctly, has unit tests, and is properly architected. Now we’re moving into a world of “vibe coding.”
People are using tools like Cursor, Anthropic Code, or Claude Code, giving simple instructions: “do this, do that, next, next, next” until whatever compiles and runs is what they wanted. It’s more creative, more artistic.
Another good example is Adobe’s Firefly, which has incredible UX patterns that induce a flow state. One thing AI has unlocked is the ease with which we enter a flow state. We can discuss complex topics without having to walk to a bookshelf and open a calculus book to refresh our memory. It’s lowered the barrier to entry into a flow state, and voice AI is probably one of the last technologies to reach that point.
Max (18:32): That’s very interesting. When you talk about lowering barriers to flow, it’s almost like bringing the “natural” part back to natural language processing, making it easier to start tasks and get into that flow state.
Business Implementation of Voice AI
Max (18:32 continued): One thing you mentioned earlier is how businesses are outsourcing lower-stakes tasks to AI. For our listeners, especially business users, do you have a framework they can use to think about how they should implement voice AI? What would you consider “low stakes” for them?
Andrej (19:22): I think the most natural approach is to look at your existing call volumes of any sort – inbound, outbound, or internal. In those cases, you should ask yourself: is this automatable, or is a portion of it automatable?
Generally, what’s automatable is a function of risk, which is specific to your particular use case. You should ground that assessment in your context.
There are also interesting voice AI applications beyond existing call volume. With the barrier to making calls drastically lowered by AI, consider where you could leverage voice AI to improve information gathering or customer/prospect touchpoints.
For example, if you’re running an e-commerce business and an existing customer has added something to their basket but not checked out, would it make sense to give them a quick call to ask if they’re still interested? Or perhaps you want to gather feedback on their experience with a short 30-second conversation.
You’re now automating speaking to your customer in ways that were previously too expensive with human agents. So either look at existing call volume to see if you can automate a portion, or apply voice AI to entirely new use cases in your context.
Selecting Voice AI Systems for Enterprise
Rod (21:37): Thinking about that – many large organizations have call centers and customer support. I’m sure many are thinking about implementing voice systems to automate these processes. You mentioned the technology is getting quite good, with many new models in the market. How should companies decide which system to buy or which vendor to choose? Is there a checklist or best practices for making that decision?
Andrej (22:14): For sure. It depends on the size of your business. If you have existing call centers, the first question is whether it’s a BPO-style outsourced call center or something you already have in-house. Have you built some of your own infrastructure, or not?
You can divide checklists into two categories. There are the usual procurement checklists that vendors need to go through – I won’t comment on those because everyone has them. The AI-specific ones are perhaps more interesting.
One key metric to assess is the end-to-end resolution rate that a system can provide. In customer support specifically (though this can be generalized to other applications), what counts is complete resolution – not just the AI saying hello at the first touchpoint and then breaking down, requiring a handover to a human. That was one of the big problems in the 2010s with AI companies trying to automate customer support.
They would get you 60-80% of the way there, but not 100%, often introducing complexity into the flow rather than solving problems. There are other metrics that CX professionals track, but figuring out the end-to-end resolution capabilities of whatever AI system you’re adopting is crucial.
Also important is the impact distribution across use cases. It’s easy to hack end-to-end resolution scores if you only consider simple cases like “when are your opening times?” The real question is: is it actually automating the things you need most help with?
Max (24:55): The end-to-end resolution point is interesting. I can see this happening in gyms, for instance. If you attend classes once or twice, they want to upsell you to a yearly membership. Normally, staff at the front desk would call you, but you could automate that process.
AI and Pricing: Business Models and Trends
Max (24:55 continued): I want to shift gears to talking about AI and pricing. I think about this in two ways: first, how AI and pricing will change the way we price things; second, how AI companies are pricing themselves. There are stats showing companies growing from 1 to 100 million in record speed.
When it comes to revenue, it’s pricing multiplied by volume. Is this rapid growth because of huge volume or because they’re charging more for products? I’d love to get your thoughts, Andrej, given that you run AI companies. Let’s start with how AI companies are pricing themselves and reaching incredible revenue in a short time.
Andrej (26:36): I think the short timeframe, disregarding pricing for a second, is a function of three things. One is the value – undoubtedly, AI is providing value in many cases (not all, but many). The second factor is timing – we’re in an era where the tech industry’s total addressable market is higher than before.
When you see metrics on Twitter about how fast AI companies are growing, they’re often compared to SaaS companies started years ago when the market was smaller and dynamics were different. That’s hard to separate in business analysis.
The third factor, which is fascinating about the pace of adoption, is that this is one of those big secular changes. We saw something similar when the world moved to cloud computing, or when everyone became obsessed with data scientists. We’ve kind of forgotten about “big data,” but this is another wave – we all need to do AI now.
There are countless middle managers focusing on AI initiatives. I remember when I was at BlackRock, the world suddenly shifted from no one having heard of ESG to everyone doing ESG. You could tell because you’d sit in London cafes and within weeks, people in suits were talking about ESG. This is another one of those moments – for better or worse, we’re all in this together, and everyone feels they need AI. That’s what’s driving the industry growth.
Does Every Company Need AI?
Rod (29:10): On that matter, you mentioned that now we need to do AI, like before we needed to do data science. What’s the current state of play? We have those three somewhat tangential disciplines – data science, machine learning, and AI. Does every company need to be doing AI? Do they need to be doing data science or machine learning?
Andrej (29:39): No, I don’t think so. There are plenty of companies and use cases where you don’t need to be doing AI. If anything, I think the anti-cyclical play here, or what I expect to happen, is that there will be a premium on the human experience moving forward.
If AI takes over many functions and automates low-level tasks, then the human experience component becomes more important. You don’t need AI for your Pilates studio. Some of the SaaS tools you use might have AI components that are tangentially helpful, but what you’re optimizing for is the human experience of a well-led Pilates session.
Perhaps human-to-human support – white-glove style support – will become more important in a world where low-level FAQ-style questions are handled by AI.
One industry problem is that when these big shifts happen, there’s a proliferation of superficial projects led by middle managers in large enterprises who are angling for promotions. It’s important to step back and think about your company or industry as a whole, not getting too obsessed with whatever the current trend is.
Max (31:48): Absolutely. Everyone could use a little more research – as the crypto world says, “do your own research” before jumping into anything.
How AI Is Changing Pricing Models
Max (31:48 continued): Coming back to pricing, you’ve worked on pricing in AI. I’ve read online – I think it was Bain or BCG – about how AI could change pricing to make it flexible enough to respond to market conditions. I’d like to hear your thesis on how AI will affect pricing of different products and services going forward.
Andrej (32:31): I think one of the big limiting factors to pricing flexibility is actually billing, because billing is how you collect whatever you’ve priced. You cannot price something in a way your billing solution doesn’t support.
One thing I’m excited about is the unleashing of billing solution flexibility. For example, Stripe, at its core a payments company, is now also a billing company with Stripe Billing. Stripe was actually late to the billing game – companies like Zuora pioneered enterprise subscription billing, while companies like Recurly and ChargeBee handled it at smaller scales.
The flexibility of billing really limits how you price. One big talking point now is outcome-based pricing – the idea that as we move from “software as a service” to “services as software” (where AI automates what consultants do), you’ll be judged on the outcome or deliverable provided to the buyer.
What’s fascinating about outcome-based pricing is that it’s always somewhat a matter of interpretation. The outcome could be unsatisfactory, leading to disputes. If the world moves in that direction (though I don’t necessarily think it will), dispute management for delivered products or work will become really important for billing solutions.
Stepping back and bifurcating into B2B and B2C: On the B2C side, what’s fascinating is that the world has somehow decided $20 a month is the price point for AI services. I’m not sure why, but it is. Now with test-time compute and multi-step reasoning, we’re seeing higher price points because the costs of providing these services are higher.
On the consumer side, we’ll see more pricing maturity. Companies will go from flat $20/month to good-better-best models (which many already have, like $20 for basic, $200 for pro). We’ll see add-ons and regional pricing – tech pricing often starts US-biased before companies realize it makes sense to discount in certain geographies. But that’s not specific to AI; it’s a natural progression.
On the business side, what’s fascinating is you’re selling units of compute, so you naturally need a usage-based billing model. That’s exactly how AI companies have been pricing – with committed usage bands and volume discounts. Again, none of this is unique to AI, but that’s how these services are being priced.
What’s also interesting is the ratio between the actual cost of running these services, which is dramatically falling, and the price – but that takes us a bit away from the pricing conversation.
Max (37:36): I totally agree. I used to work for an old software company that sells to banks. One big reason we couldn’t change pricing was exactly as you said – our billing system couldn’t handle it. We had top software used by major banks, but we couldn’t change pricing not because clients wouldn’t pay, but because we had no idea how to bill it! It left quite an impression on me.
Andrej (38:10): Yes, and two things will happen: First, billing primitives will become more flexible. Stripe Billing and other providers will mature, making it possible to set up complex invoices with subscriptions, tiered usage, and discounts that can be edited when contracts are renegotiated.
Second, as primitives improve, there’s the question of how we’ll use this flexibility. Pricing is about matching what you’re selling to the value you provide – that’s the fairest setup in business relationships. When billing is no longer an obstacle, we may not immediately grasp how creative we can get.
Some companies have built highly flexible internal billing systems that allow their sales representatives tremendous flexibility in contracting. Without needing a deal desk to structure complex arrangements for high-value items, a seller of point-of-sale terminals to restaurants could craft custom deals – like “three months free, then discounted rates for six months, then full price by year-end, with refunds if SLAs aren’t met.”
These complex contracting situations were typically limited to high-value enterprise deals requiring deal desk involvement. We’re moving into a world where regular sales reps will be able to close deals with the same flexibility previously reserved for major contracts.
Challenges of Outcome-Based Pricing in AI
Rod (41:05): We’re moving towards outcome-based pricing. My experience with AI systems is that they often don’t get it right the first time. A classic case for outcome-based pricing might be with CRM: “I need AI expert leads based in London with specific characteristics.” Often, the results aren’t right.
However, the vendor has already incurred the computational cost of calling the model. For the vendor, the cost is already there, but the outcome was wrong. What can vendors and AI buyers do in cases where either the buyer is paying for incorrect results, or the vendor needs multiple attempts to get the right outcome before they can charge?
Andrej (42:07): Broadly two approaches: One is to say, “Look, there are pass-through costs here – the cost of using foundational models – which is the cost of providing the service. On top of that, I’m adding a margin. If this goes south and doesn’t work, at least pay for the pass-through costs.”
Obviously, customers might resist even that. In software, we lived in a world of “build once, sell a million times” where the actual cost of running SaaS was very low. Now we live in a world where margins are lower – at least for now – because the cost of goods sold is pretty high in AI.
In such cases, take a lesson from service businesses, which are incredibly good at setting expectations. Unlike software businesses where things sometimes don’t work but it’s manageable if customers cancel, service businesses need to be upfront about costs and expectations.
So either be very transparent about the pass-through costs and your role as a wrapper around existing foundational models, or get extremely good at setting expectations when contracting and anticipate situations where things don’t go well. In software, we did this with SLAs – service level agreements would define what happens if SLAs aren’t met, sometimes crediting customers to offset costs when services underperformed. We can take inspiration from that approach for AI products.
Max (44:30): I’m reminded of investment banking, where they charge based on capital raised for companies. You make more money that way, but you’re also taking on more risk because there will be days with fixed costs but little or no revenue. I like your thinking about passing on the base cost, similar to consulting where they charge for hours plus extras. Those are all service-based business models.
Stripe’s AI Strategy and Payment Innovation
Max (45:21): In our last few minutes, let’s cover a couple more topics. Stripe recently announced their latest results, with total volume reaching 1.4 trillion. Their goal has always been to grow the GDP of the internet. What caught my eye was that in their announcement, they highlighted four different AI initiatives and one thing on stablecoin. Given that we’ve discussed billing, what are your thoughts on how Stripe is tackling the AI world? They’re now one of the largest billing platforms, used by 300,000 companies with nearly 200 million active subscriptions.
Andrej (46:30): A few scattered thoughts on the Stripe annual letter: Stripe isn’t public but publishes an annual letter with high-level metrics. What’s interesting is that Stripe’s nearest competitor is Adyen, which typically services enterprises and won’t speak to you unless you’re doing significant annual revenue. Stripe’s bread and butter is getting you at the very beginning – Stripe Atlas for incorporation, Stripe Billing for YC companies.
Sometimes companies shift away from Stripe as things get more complex. Stripe’s goal has been to mature their billing systems and grow international payment coverage to keep up with larger customers, of which they have many.
One thing that struck me was the valuation multiples – Stripe is now valued much more highly than Adyen despite comparable results. That could be private market premiums, but it’s also a reflection of Stripe doubling down on the AI billing space. Most AI companies use Stripe, not other providers.
If AI takes over the world and we have AI agents shopping on our behalf, Stripe is extremely well-positioned to grow in that space. Another thing they don’t highlight much in the letter but is visible is Stripe Links – when you pay for OpenAI as a consumer, you’re setting up a small consumer Stripe account, tying your payment card for easy reuse in other Stripe situations. That’s the beginning of something potentially incredible for Stripe. Their annual letter is quite revealing in terms of strategy and direction.
Max (49:23): Absolutely. One thing I noticed is their agentic payment system, which ties to what you mentioned – if my payment details can follow me across websites, saving me from typing them repeatedly while merchants don’t store my information because it’s controlled by my agent. John Collison gave an example on a podcast about telling your agent to “go buy me some t-shirts.”
This will change how commerce works, making it more natural and lowering barriers to both flow state and low-level tasks – you just don’t need to worry about them. Combined with what they’re doing with stablecoin for cross-border payments, it’s quite exciting. From your perspective, building in the AI and market research space, how do you think about this? They’ll eventually be able to see where the world economy is heading as everything moves online.
Andrej (50:42): Yes, I caught either Patrick or John on a podcast talking about how they developed an inflation prediction model, which is pretty cool. There are many interesting possibilities assuming the world fully moves toward programmatic payments – easily programmable payments.
Just the other day, I was reflecting on something from my background – I’m half Slovenian, half Serbian, from ex-Yugoslavia. Companies there would submit their debts to a government agency regularly, and that agency would cancel out circular debts so that financial actors wouldn’t take commissions on those transactions. In a world of easily programmable payments, many interesting things could happen.
Deep Research: Capabilities and Limitations
Max (52:04): Great. In our final minute – Ben Evans, the former A16Z investor and prolific researcher, has been giving us reality checks about what AI can and cannot do. He recently wrote an article about deep research capabilities and limitations.
He noted that “AI models are very good at things computers can’t do well, but very bad at what computers do very well.” Computers excel at deterministic tasks, while AI is better at probabilistic tasks like pulling information from multiple sources. But when you need something very specific, AI becomes more unreliable, though it will improve. I’d love your thoughts on this, given your journey from relating different entities to seeing those subtasks combined.
Andrej (53:23): It’s true that AI excels at probabilistic tasks and struggles with more deterministic computer-like operations. However, I’d argue that you can always adopt intermediate representations to handle deterministic tasks. AI is incredible at coding, which translates into deterministic actions across data.
For example, if you’re doing data analysis with SQL queries, you can either give entire spreadsheets to an AI for immediate answers, or it can generate code that runs across your table to produce the answer. The trajectory of improvement in this area has been amazing – in the 2010s, people tried to generate SQL code with little success, but now it works well.
Regarding Benedict Evans’ point, one thing that’s often conflated with deep research – not by Ben himself, but by many on Twitter – is the idea that “this is the end of consulting.” I take major issue with this because it misunderstands what consulting is about.
It’s not just about middle managers needing justification for projects and bringing in McKinsey – that’s a cynical view. What consulting often involves is cases where an AI would need to be proactive in pulling out information.
I think about knowledge in tiers: there’s online public knowledge on the internet that’s scrapable; there’s near-field public knowledge available through subscriptions like Gartner reports; and there’s frontier knowledge locked in people’s heads and organizations.
Consultants often speak to internal stakeholders about how they’re tackling problems – essentially pulling frontier knowledge from their minds. To replicate that, you need an AI that’s very good at asking questions, and we don’t have that yet.
For anyone thinking about automating consulting, you need an AI that’s good at asking questions, managing projects (timelines, expectations), and packaging deliverables in ways familiar to the market. Consultancies have knowledge management teams with pre-existing decks, solutions, and guides for their consultants because they’re often repeating similar work across different clients. Taking inspiration from that approach makes more sense than just searching across Google.
Rod (58:02): One thing I find is that with deep research products like Perplexity or Gemini, they seem to replace what would often be given to the intern or entry-level analyst. While consulting might not disappear as an industry because of brand power and prestige, is it possible that many consulting-related roles will completely disappear?
Andrej (58:53): Absolutely. We’re moving into a world where low-level legal services can be answered by ChatGPT, medical scans can be analyzed by AI, and secondary research can be done using deep research tools.
But going back to Jevons paradox (which everyone started talking about a few weeks ago), what often happens is proliferation rather than replacement. More people engage with research because it’s more accessible. Someone on Twitter mentioned we need a flow state-inducing AI that your “lizard brain” uses before bed instead of scrolling through TikTok or Instagram. Someone else replied that they’re already doing this – hunched over their pillow chatting with ChatGPT about research while their partner watches TV.
So AI will definitely increase the aggregate amount of research, which is awesome.
Max (01:00:35): Yes, Jevons paradox – Satya Nadella mentions it too. Increasing the efficiency of resources can lead to increased consumption of that resource. When we improve research capabilities, we’ll likely just consume more of it.
One positive outcome might be that investment bankers and consultants, especially junior analysts, will have better work-life balance. In services businesses, the higher you move, life doesn’t actually get easier. It gets harder because you have to spot-check all the work your teams are doing, while still managing client relationships and asking the right questions.
I think there’s an opportunity with voice AI – you could easily call someone using voice AI, ask a few questions, and capture enough information to form an initial opinion before going deeper. I started my career asking people questions like “would you use this software?” which I think will change. I’m quite excited about it.
Andrej (01:01:58): Just to add one thing: Another fascinating topic to think about is what kind of information we’ll be extracting from experts in the future. If folks in an enterprise or industry have used services like Tegus, these are like a window into the future – they have open transcript libraries you can subscribe to. A lot of that data is very interesting and will have enormous potential when combined with AI.
Max (01:02:46): Yeah, and the question then becomes who owns that data and so on. We’ll get to that when the time comes. But I totally agree – there’s still a lot to unlock, especially with nuanced information in large organizations. I can only imagine what we’d find if you combined all the transcripts and internal filings of IBM.
Andrej (01:03:10): Mmm-hmm.
Max (01:03:13): Just imagine what you would see there. You could say the same for Microsoft, Google, and others. First off, Andrej, thank you so much for giving us extra time. We really appreciate it. This was a very good conversation. To wrap up – thank you again for sharing your expertise with us. Today we covered four different topics: voice AI, AI pricing, Stripe’s recent announcement, and what deep research can and cannot do, as well as the future of consulting.
Thank you everyone for tuning in and listening. If you like the episode, please share and subscribe. And Andrej, if people want to find out more about you, where can they go?
Andrej (01:04:03): Gosh, I guess two venues. One is, add me on LinkedIn. The other is come to London, Bermondsey Street, Watch House. I’m there all the time and you can spot me in the wild.
Max (01:04:15): Lovely. There you go, listeners. You know where to find Andrej. Thank you so much again for listening, and we’ll be back next week. Thank you.
Your Hosts
Christine Wang: https://www.linkedin.com/in/christinewang0/
Rod Rivera: https://www.linkedin.com/in/rodriveracom/
Maxson J.Y. Tee: https://www.linkedin.com/in/maxsontjy/
Social: X:
https://x.com/chrisrodmax
Instagram: https://instagram.com/chrisrodmax
LinkedIn: https://linkedin.com/company/chrisrodmax
Subscribe: https://chrisrodmax.com
Share this post