One of the key discoveries that triggered the beginning of the AI revolution is that bigger is better. That is, as the number of parameters in a large language model (LLM) increases dramatically, new valuable behaviors emerge. The result has been a series of generations, each one ten times (10X) larger than the previous generation.

After several generations, we are now encountering bottlenecks — both technical and financial. The scale involved has become so great that we are entering uncharted territory. How we deal with these scale bottlenecks has significant implications. Implications for our financial system in particular, and society in general. The underlying changes are so dramatic that new approaches are likely to be needed to deal with them.

Following the 2017 publication of the key paper Attention is All You Need, people began experimenting with what became known as LLMs. By late 2022, a relatively small number of companies developing leading-edge GenAI systems began to observe the emergence of new, valuable behaviors as the size of LLM models (“brains” of GenAI systems) increased (generally measured by the number of parameters). Going from millions of parameters to tens of millions, hundreds of millions, billions, tens of billions and hundreds of billions. Numbers in the trillions of parameters started to present difficulties, although the pressures to continue to 10X didn’t abate.

The 10X problem

Recently, we started to hit limits. Someplace in the Trillions of parameters, people realized that with existing chip and data center infrastructure technology, there was not enough fab (semiconductor manufacturing) capacity on the planet to produce enough chips to make the next 10X generation. Two different approaches emerged to deal with this bottleneck: China’s and the US’s.

Shut out from the most powerful AI chips, China’s government started funding GenAI software development. Today, there are approximately 20 state-funded Chinese companies releasing open-weights Models (open-weights is similar but more extensive than open-source). China is seeking to overcome the limitations of fewer, less powerful chips by learning as much as possible from US proprietary models, supporting software innovation and sharing (through Open Weights).

The US frontier model companies are focusing on new, more powerful chips and more of them. While they are waiting for more capable hardware, they are using the time to refine their proprietary models (secret source code and secret weights). They are also following the Chinese open-weights models closely.

NVIDIA, an American technology company, has announced that it will have a new generation of more powerful chips. Other sources are doing the same. My best guess is that they will be available in mid to late 2027. But even with more powerful chips, larger numbers will be needed.

NVIDIA has also announced a new rack architecture (the organization of chips into support systems that are stacked to maximize chip density within limited space). The current rack architecture is operating at its power-handling and cooling limits. The new architecture proposed by Nvidia requires twice the power and twice the cooling of current racks. Nvidia is calling this new architecture AI Factory. Nvidia is announcing this now as part of a request to the ecosystem to develop solutions for these new racks. 

To increase the level of challenge, it is unclear whether these new racks will fit within existing data centers. It may be necessary to build new buildings to hold the new racks. In any case, whether new or existing buildings, they will need greatly increased power sources and communication feeds.

The costs involved have risen well into the trillions of dollars. We are entering the financial range where previously only the largest nation-states could operate financially. So, it has grown beyond the capacity of a single company to finance. US companies have dealt with this by forming partnerships. This has the effect of creating something similar to a zaibatsu (a Japanese business conglomerate before World War II) or a keiretsu (a Japanese conglomerate after World War II). It appears that the players think that not all the parties will survive, at least in the AI frontier segment. Accordingly, they are making partnerships with multiple players. The result is not yet a clear set of competing entities.

Diverging strategies in the US–China AI race

Thus, if you look at this as a horse race between the US and China, both are betting on different horses. The Chinese, at least in part because of restricted access to hardware, are betting on limited hardware and open-weights software. The US is betting on dramatically increasing hardware capability and proprietary weights software. Behind both horses are a set of adaptation challenges.

To many who look at US companies, these complex partnerships involving previously unheard of amounts of money are troubling. The fact that some of the people in these companies are entrepreneurs who have never previously worked under such intense public scrutiny has also troubled some. Some of those who are troubled have looked back at the Internet bubble burst. Is concern about systemic financial system risk warranted?

What are the systemic risks?

Expectations about the future drive financial markets. If enough people expect a bubble-burst market, a bubble-burst market will occur. So, setting expectations on a sound foundation is important. What follows is an attempt at such a foundation.

First, it is good to put this in context. For many, the changes that AI is creating are hard to recognize. These people are well-versed in financial analysis techniques that worked before AI’s emergence. For them, it is somewhat similar to the experience of people well-versed in agricultural economies being confronted with the beginning of the Industrial Revolution. In that transition, the amounts of money involved and the ways of assessing risk changed dramatically.

As we make the transition to the AI economy, there will be winners and losers. But it is unlikely that the whole AI ecosystem will collapse in a fashion similar to the Web Bubble Burst. There are two kinds of frontier model companies: 1) large existing businesses whose leaders think that they need a strong position in AI to stay relevant, and 2) pure, but very large start-ups.

The large frontier LLM companies are unlikely to fall into crippling financial situations overnight. They may have difficulty raising the enormous amounts of cash needed for the next 10X. Or they may run into technical/marketing problems. If either, they may shrink the way IBM did when it lost its position as the dominant computer company.

Pure start-ups may face financial difficulties. This can be the result of challenges raising enough cash, technical/marketing problems or mistakes in correctly estimating the exact timing of future revenues. In such a case, the most likely outcome is a forced merger with one of the winning companies. A significant reduction in valuation may cause some investors in the losing company to incur losses. The losses could be large. Most of the investors in such a company will have hedged their bets. But even if insufficient hedging is done, the reduction in valuation is not likely to have a drastic systemic effect.

The large semiconductor companies have a lot of experience in weathering changing market/competitive dynamics. They are public companies that have seen AI dramatically increase their share prices. Competitive rearrangement can lower one company’s stock price while increasing others’. Financial troubles among the semi companies’ customers and frontier model companies could impact quarterly revenue. A delay in sales resulting from one of the frontier companies falling into financial difficulty may affect stock prices.

In the past, there has been such substantial demand for AI chips that orders have been placed well in advance. If any order is canceled, there is a purchaser ready to step up and take it over. Based on the current outlook, it seems that there is a risk of one or more semi companies having a stock price reduction. But systemic risk seems unlikely.

In the rest of the infrastructure ecosystem, losers may not make dramatic departures. One possible outcome is that the industry becomes more vertically integrated. Weak companies in the infrastructure ecosystem are purchased by the very big players. For the smaller start-ups in the infrastructure ecosystem, the normal start-up failure rate may also characterize the space.

For companies in the application, customization, domain-focused and other spaces, the typical start-up failure rate may also apply.

Technical innovation may change some of the underlying forces driving the 10X phenomenon. Such changes move relatively slowly. In semiconductors, it takes approximately three years or more. In software, maybe faster, but still slow enough for the players to adjust enough to avoid a systemic financial shock.

There are two sources of systemic risk resulting from: 1) a market disruption and 2) a jobs-led economic downturn. 

First, the 10X increase in financing needs is straining the financial markets’ ability to provide the required amounts. Thus, there is a temptation to remove the guardrails put in place following the Great Depression and the Great Recession. One such effort is to open the private market to retail investors (low net worth individuals as opposed to professional investors such as insurance companies). Recently, the US administration issued an executive order that points in this direction. 

The systemic risk is that a large number of individuals, without the ability to effectively assess risks and an inability to hedge, will be drawn into very high-risk portions of the private equity market and incur crippling losses.  Losses in savings, retirement accounts, etc., will leave the individuals struggling with mortgages, credit card debt, auto loans and more. If the high-risk funds themselves run into trouble, it could be even worse. Our experience with the Great Recession is an indicator of how serious such a systemic situation could be.

Second, job loss through AI automation could cause an economic downturn. GenAI is still in its early stages of development. It will continue to grow in capability as the 10X generations proceed. On top of that, we are in the very early stages of learning how to apply AI. Over time, we will get much better at it. Given these facts, it is difficult to exactly predict the full effect and timing of AI’s impact on employment.

Some have argued that 20% of jobs will be eliminated, while new jobs will be created. Others have estimated that 80% or more will be automated, and very few new jobs will be created. The US consumer supports the US economy. The US economy supports the global economy. If enough consumers lose their jobs, or fear losing their jobs and stop buying, the result could be a very serious systemic economic disruption.

Concentration of power

What may be more significant is the concentration of power. Many believe that control of AI will mean control of the world. That is what motivates the race between the US and China. If, as appears likely, the US horse wins, there will be a great amount of power in the hands of a small number of interlinked companies. 

That power could be concentrated in a single company or keiretsu. If it doesn’t happen with this 10X step, the next 10X step is likely to force it. That is, the size of the financial resources needed will become so large that only one such organization will be able to exist. 

The leaders of the frontier model companies don’t talk about this potential concentration. Whether they feel it but can’t articulate it, or just don’t think it is politic to mention it, they are motivated by it. Some approach it by being cautious. Playing the long game. Others are tempted to take all-or-nothing approaches. Thus, raising the risk level.

Maximizing upside while minimizing downside

The 10X process leads to questions of nationally focused antitrust, natural monopolies and public utility regulation. There is also work from an international perspective. An international approach, if viable, might make sense because the impacts of these very large models will be global.

The UN has started studying the area. China has released an international cooperation proposal. Some of these international efforts point to what has been done around nuclear power. It is important to be knowledgeable about what has worked and not worked in the past. However, because these 10X-size endeavors are unprecedented, new approaches are likely to be necessary.

Thought is needed about both the power from the financial size and the power from the AI itself, what this means for society in general, financial markets in particular and for the growing wealth gap. With a good understanding of these factors, attention can then turn to maximizing benefits while minimizing adverse effects. In doing so, new approaches may emerge. Just as financial markets had to make dramatic changes as the world moved from an agrarian economy to the Industrial Revolution, the AI economy may require a similar set of dramatic changes. An open AI group is exploring these questions of AI and society.

Conclusion

For the last few years, AI has been in a process of growing model sizes 10X per generation. Recently, that 10X step has begun to run into bottlenecks — first chip availability, then financing. These bottlenecks and the industry’s response to them have some very important implications for financial markets and society. New approaches will be needed to deal with these implications.

[Kaitlyn Diana edited this piece.]

The views expressed in this article are the author’s own and do not necessarily reflect Fair Observer’s editorial policy.