How the cost of generative AI is shaping the investment landscape

How investors and startups should navigate the high costs of generative AI was one of dominant themes at this year’s GCVI Summit in Monterey. With the costs of creating foundational AI models running into the hundreds of millions of dollars, it has become clear that only a handful of foundational models will dominate the market, Nagraj Kashyap, general partner at Touring Capital, told the Summit audience.

“We are heading for a commoditisation of similar models. The very high costs mean only large corporations are funding them,” he said.

See the full talk here:

Businesses are racing, however, to figure out how to how to use those models in their operations, faster and better than the competition. The rapid emergence of large language models last year has morphed into a “shift to how to integrate into the business use case,” said Albert Lee in corporate development at Google.

Full session here:

1. CVC can be a shortcut through AI experimentation costs

This is where corporate venture capital teams can play an outsized role, AI pioneer Andrew Ng told the Summit audience. Companies typically want to try out four or five generative AI ideas at a time, says Ng. The cost of building all these trials in-house is high— customising a large language model and integrating it with internal data can still run to tens of millions of dollars, according to McKinsey estimates. Venture investments and partnering with startups would be a useful shortcut in this period of experimentation, said Ng.

Full talk below:

2. The race to create the “smartphone of AI”

Sideline chat at the conference revealed that many investors are still somewhat skeptical about how soon generative AI can be used in highly regulated industries. But one of the hottest discussion points was Rabbit, the AI-powered handheld portable assistant that was the hit of CES. Even if generative AI can’t be yet quite trusted to write your insurance policy, it might do any number of smaller consumer tasks like making restaurant reservations, booking flights and organising events. The race to create the “smartphone of AI” is on.

3. Access to compute becomes a survival issue

The high cost of the compute power needed to train and run generative AI is something that startups, investors, corporations — and even national governments — will need to grapple. Nvidia’s Keith Strier pointed out that 85% of countries don’t have access to enough compute power to train a large language model. That raises serious geopolitical questions, but on the business level, investors will have to think about how their gen-AI based portfolio companies access the resources they need. “Compute is neither infinite nor easy,” said Strier.

There’s a growing trend, starting last year, he said, for startups to raise funds specifically for computing needs. Companies that don’t do this will be failing their investors, he said. The compute question is so acute that one investment firm even bought its own supercomputer for use by its portfolio companies.

See the full session below:

4. Semiconductor innovation is accelerating in response to the computer bottleneck

The race to source GPU power will undoubtedly swell Nvidia’s coffers even further, but at the same time it is spurring investment into alternatives — where there is a bottleneck, there is a business opportunity. Semiconductor innovation is being driven forward at a rapid pace, said Elana Lian, investment director at Intel Capital. And — in the way that tech often eats its own tail — semiconductor startups are drafting in AI tools to speed the development of better AI chips, said Raymond Chik, cofounder of UntetherAI, a startup creating AI processors.

The rest of the Summit:

Of course, the GCVI summit is not solely about generative AI, however dominant that theme might have been this year.

Other themes emerging from the Summit included corporate venture units shifting their focus towards creating strategic value for the parent companies as the opportunity for lucrative exits in the form of IPOs has dried up. CVC units have been tested, said Paul Asel, cofounder of NGP Partners, in a way that is making the industry stronger. “It has been healthy for the sector,” he said.

Asel, who has steered NGP through several leadership changes at parent company Nokia, has always said that CVC units must be able to do both — provide strategic value and good financial returns — to remain consistent, even as corporate priorities change.

We’ve been hearing a lot more CVC units talking about experimenting with different models, such as building their own ventures.

“CVC is very good at looking around corners, but it is never going to be a strategic thing for corporations. These corporations are making a billion dollars every couple of days,” said UP Labs founder and CEO, John Kuolt.

Instead, the UP Labs team is building startups that corporations Porche and Alaska Airlines have the option to spin in or buy after a couple of years. It means big companies can tap into entrepreneurs that a big company would never be ape to hire. “I just am flying back with one of our CEOs on his private plane. Porsche would never be able to hire him, he sold his last company to Apple,” said Kuolt.

See the full talk here:

There’s a lot more from the two Summit days here, encompassing everything from investing in Africa:

to deeptech:

to innovation in sport:

The full set of videos available from the two days is on our YouTube channel here.