We want to pace the frontier but what about pacing its venture funding

I'm optimistic and excited about the future of AI and our society. I believe a future with AI will be more productive, more meaningful and more interesting. Every day I see entrepreneurs using AI to build ventures with a positive impact. Companies like Basecamp Research are building new models in life sciences to design medicine and find effective ways to use them to save lives. Google DeepMind recently launched its most advanced weather model, WeatherNext 3, which generates hourly forecasts and helps governments and individuals respond better to severe weather and climate conditions.

But recently we witnessed many leaders' alarm from the risks of AI progress. We know that dooming and negative warnings take much more public attention than the positives, but we must address them seriously and manage those tradeoffs in the industry. We must pace the frontier calls not only for investment in risk prevention but for pacing the rate of capability and model progress, so that risk assessment and prevention move alongside the models. This comes after a growing number of incidents. In the OAI-HF (OpenAI–Hugging Face) incident, a swarm of agents carried out a cyberattack outside their prescribed task. Other incidents include agents publishing third-party images, and bots attempting to access data from unpublished government files.

The leading AI labs have been proactive, talking about the risks and calling for collaboration. Their leaders agree that action is needed, in partnership with the government. David Sacks pushed back writing "Dario has written that we need to 'pace the frontier,' and Sam has agreed. People may be surprised by my response: go ahead." He didn't fully reject government intervention but returned responsibility and accountability to the lab leaders. It was recently published that the frontier leaders have signed a "morally binding" agreement to manage the technology's risks voluntarily and to self-regulate their progress.

This ping-pong between the government and the labs made me notice a group that has opted-out of the discussion, and of any responsibility or accountability — AI investors. It is evident when we read the details of Anthropic's S-1, filed ahead of its $2 trillion IPO that investors are not stopping and leading with big conviction investing in the frontier. According to Fortune magazine the filing is reporting that Revenue for FY25 was $4.6 billion, against operating expenses of $13 billion and an operating loss of more than $8 billion. Net loss was $42 billion. Anthropic plan is to invest over half a trillion dollars in compute, data centers and cloud services. The filing also states that progress in AI could pose "catastrophic or existential risks to humanity."

This raises a critical question for venture capitalists, private equity funds and even public investors: where and how do we deploy capital in the frontier and in the AI industry in general? In due diligence, where is the trade-off between financial gain and risk? Imagine an investment committee debating an investment opportunity where the investment memo states there is a tangible risk to all of humanity. Investors already manage business and technology risk, but this is where we need to think hard about our theses, risk models and accountability when funding AI companies with societal risks. It isn't a new problem and we have funded ventures building nuclear energy technologies, pharma startups that have risk for peoples' lives and defence tech companies that with the wrong care and management could be used to harm innocent people — all carry social and environmental risks.

In this letter I'm not calling on AI leaders to act on safety. They have done so and agreed terms to self-regulate. This is a call for the investment and venture capital community to do the same. We need to discuss, agree on and write down how we evaluate AI technologies and where we deploy capital, so that AI is built and advanced by the right people for the right purposes. We like to think of the best investors as those who are adding to founders more than just capital. In this case, I believe that the investment community carries the same weight of responsibility as tech leaders and government officials. A big part of that value is to share the risk and not only the upside, but also to prevent feeding the capital without thinking of a different type of risk. The big question is if a good solution to address the risk of AI is to do some correction to funding round sizes, valuation and pace in which we fund certain ventures to make sure we are funding ones that will make society better and not risk it.