Funding Open Source Generative AI with Crypto

The intersection between generative artificial intelligence and Web3 has been one of the most active areas of research and development in crypto circles over the past few months. Decentralized compute, zero-knowledge AI, small foundation models, decentralized data networks, and AI-first chains are some of the recent trends aimed at enabling Web3-native Rails for AI workloads.

These trends are technological innovations that attempt to bridge the Web3 and AI worlds, representing a natural friction against the centralized nature of generative AI. While technological bridges with AI are fundamental to the evolution of Web3, they do not represent the only integration path for these technology trends.

What if the path to integrating Web3 and AI was purely financial rather than purely technical? This suggests that crypto's programmable finance and capital formation capabilities could be useful for one of the biggest challenges facing the creative AI market today.

What challenge are we talking about? Nothing but the funding challenges of open source generative AI.

Despite the recent level of innovation in decentralized AI, the gap with centralized AI tech is widening rather than narrowing. Many agree that blockchains represent the best technology alternative to the increasingly centralized AI control of large tech platforms. However, the challenges of adopting decentralized AI platforms are monumental.

Decentralized compute is an obvious pillar for decentralized AI but proves impractical for pre-training and fine-tuning workloads that require nearby GPUs with access to datasets that are often blocked by corporate firewalls. Sit behind. Zero-knowledge ML is too expensive to be practical in large foundation models and has not seen any real demand in the market. Decentralized data marketplaces need to overcome the same problems that have prevented data marketplaces from becoming big tech businesses.

While decentralized AI strives to overcome these frictions, centralized alternatives are rapidly creating a terrifying gap between the two. One trend that is keeping hopes alive for a world in which decentralized AI can succeed is the rapid evolution of open-source generative AI.

All decentralized AI trends rely on a healthy open-source generative AI ecosystem, yet that ecosystem may not be as healthy as it seems.

In the last few years, we have seen an explosion of innovation in open source large generative AI as alternatives to platforms such as OpenAI/Microsoft, Google or Anthropic. MetaLama has become the surprising undisputed champion of open source generative AI with the release of Models. Companies like Mistral have raised billions in venture funding, enterprise platforms like Databricks or Snowflake are pushing open source models, and there is an increasing number of open source generative AI releases on a weekly basis.

Although the momentum in open source generative AI is strong, a more detailed analysis reveals a different reality. Open-source generative AI faces a major funding problem. When it comes to large foundation models, only large companies like Databricks, Snowflake, Meta or well-funded startups like Mistral are keeping up with the performance of large closed models. Most releases from other labs like Databricks and Snowflake focus on optimized enterprise workloads, while much of the recent open source research focuses on complementary techniques rather than new models.

The reason behind this trend can be attributed to the astronomical costs of building large frontier models. Any pre-training cycle for a 20-billion-plus-parameter model can cost between ten and a hundred million dollars and involve a process of several months with multiple failed attempts. These costs are beyond the budget of most university labs. To make matters more interesting, many grants to AI university labs come from large tech giants, who are then immediate beneficiaries of the results.

Monetizing open source has historically been difficult, and monetizing open source generative AI is difficult at AI scale. As a result, open-source generative AI faces a huge funding gap that could create a serious gap with those responsible for AI.

Crypto capital formation startups seem like one of the few viable alternatives to address the funding shortfall in generative AI. Throughout their history, crypto tokens have been a primary vehicle for generating capital for Web3 projects through bull and bear market cycles. Can some of these principles be applied to open source generative AI? There are certainly more than one interesting option.

Gitcoin represents one of the most successful examples of funding open source innovation in the Web3. The quadratic funding mechanism pioneered by Gitcoin can be directly applied to generative AI. Bringing native generative AI capabilities to Web3 is paramount to the evolution of the space, so it's natural to expect that generative AI projects will gain community attention.

Let's say a university AI lab needs to raise $10 million to pre-train an LLM based on a novel architecture. Multiple DAOs and foundations can contribute to a Gitcoin grant that can be matched by grantees, creating a more efficient funding mechanism. This procedure is much more effective than the existing alternatives in the market.

Funding open source projects enables mechanisms in which the value generated by these projects can benefit the original funding community. When it comes to Web3 and open generative AI, an interesting idea is to set up a license in which any commercial application using the Web3 tokens financing model can use that particular token format. I should return part of this income. This mechanism can also be implemented through smart contracts.

Funding vehicles for open source AI is one of the most important challenges to address in the current creative AI landscape. Open source is traditionally difficult to finance, and open source generative AI is even more so given the expensive computational requirements.

Not enabling adequate funding channels to foster open-source innovation in generative AI could create a systemic risk for the entire space as the balance shifts to entirely closed commercial platforms. Crypto has established some of the most sophisticated and battle-tested channels for funding open source innovation. Perhaps, the first bridge between Web3 and generative AI will be financial and not necessarily technical.

Note: The views expressed in this column are those of the author and not necessarily those of CoinDesk, Inc. or reflect the views of its owners and affiliates.