Amazon Executive Dives into Big Model Entrepreneurship: AI Companies Will Be Saf
In August of this year, former Amazon Chief Scientist Li Mu published an article reviewing his progress and thoughts after a year of diving into the entrepreneurial journey with large models, which attracted industry attention. The startup he is part of is Boson AI. One month later, a journalist from First Financial Daily met with Boson AI's CEO, who is also the company's other co-founder and Li Mu's Ph.D. advisor at CMU (Carnegie Mellon University), Alex Smola, in Riyadh, Saudi Arabia.
Li Mu wrote in his article, "If there is anything you should try in your life, do it sooner rather than later, because once you really start, you find there is so much new to learn, and you always lament why you didn't start (entrepreneurship) earlier." Speaking of entrepreneurship, Alex Smola also told the reporter that he should have done it a few years ago, but he believes that now is always the right time.
Both Alex Smola and Li Mu are well-known figures in the AI field. Alex Smola is a renowned scientist in machine learning and has been working in the AI field for 30 years. In 2016, he joined Amazon as a Distinguished Scientist at the VP (Vice President) level. Li Mu is one of the authors of the deep neural network framework MXNet. Both left Amazon in 2023 to start their own ventures, becoming part of the wave of scientists turning to generative artificial intelligence entrepreneurship.
Unlike enterprises that access large models like ChatGPT through APIs, Boson AI's main business currently is to create customized models for clients. In the interview, Alex Smola discussed the company's business model, the insights gained from the past year of entrepreneurship, and the importance of maintaining a balance between income and expenditure for a startup. This may provide a perspective on observing a large model startup company.
Advertisement
How is the market sentiment?
In the absence of mature C-end LLM (Large Language Model) applications, the well-known revenue model for LLMs includes businesses paying for API calls to access commercial large models. Boson AI is also in the B-end business, providing customized large model services.
According to Li Mu, the company initially "couldn't move forward" with GPT-4, so they hoped to train models for specific needs. The company initially trained models from scratch, but as more high-quality models became open-source, the company shifted to enhancing model performance for business scenarios. The company first focused on technology and then gradually thought about its vision. Now the vision is set as "intelligent entities that accompany humans." According to the official website, in June this year, Boson AI launched the Higgs series of LLMs, which are optimized based on Meta's open-source model Llama-3. In July, they launched Higgs Llama V2, also based on Llama.
Perhaps because they are based on existing open-source large models without the need to train from scratch, and have found some applications in the enterprise sector, Li Mu stated in his article that the company has broken even in terms of income and expenditure over the past year. He expressed gratitude to the clients for "giving time to breathe," allowing him to not rush between investors recently.
In order to generate income for the company, Alex Smola's work includes finding clients. He attended the GAIN SUMMIT World Artificial Intelligence Summit in Riyadh, hosted by the Saudi Data and Artificial Intelligence Authority, with one goal being to help the company expand its customer base.
Since the beginning of the LLM entrepreneurship wave, it has been a common phenomenon for companies to seek applications with their technology. In the B-end market, a major domestic LLM manufacturer recently told reporters that after nearly a year of trials, they found that due to the limitations of LLM capabilities, the application of LLMs in traditional industries and industries requiring high levels of expertise is not fast, and they can only play a supportive role, with more time needed to penetrate into more core segments. Boson AI is also gradually confirming its own vision and exploring the capabilities of LLMs in the process of doing business.Even though Alex Smola acknowledges that Large Language Models (LLMs) still struggle to penetrate certain segments of traditional industries, he believes that LLMs can do more. "LLMs may not play a significant role in segments like steel manufacturing or battery production, but they can still find applications, for instance, in areas where white-collar workers may not have received proper training," Alex Smola told reporters. Looking at the company's customer base, the majority come from insurance, gaming, and education sectors, which share a commonality in being related to useful and engaging conversational systems. Recently, market demand related to voice has also been increasing, such as AI in call centers that can improve service quality and reduce service request delays caused by overburdened call centers.
"It's not about taking away jobs from certain individuals, but rather reducing the time to handle (call center) demands from a year to a week, expanding human productivity. It will take some time before AI reduces the need for human labor, and by then, humans will be thinking about what they can do with their creativity and productivity," Alex Smola told reporters.
Unlike general large models that provide API interfaces, Li Mu believes that customized models have the advantage of more optimal inference costs, with costs being 1/10th of calling an API. In addition, Alex Smola believes that customers using customized models can fully control and adjust the models, which is difficult to achieve with the method of accessing general large model APIs. Enterprises of a certain scale will need this autonomy of customized models.

Regarding the claim that LLM applications are not meeting expectations, Alex Smola thinks that LLM applications are still in their early stages. Moreover, LLMs themselves have limitations. He discussed at the summit that humans may gradually deplete the tokens (word elements) available for building LLMs. Intelligent systems that can learn and improve on their own and are universally capable of handling various tasks may not appear in the next 10 to 20 years.
"Most companies that adopted LLMs early on did so because their CEOs were very decisive, not intimidated by the high computational and human resource costs, and decisively pushed their internal teams to try new technologies," Li Mu said, suggesting that more companies will attempt to use LLMs in the future. However, as for whether there is already a strong market demand for customized models, Alex Smola believes it is hard to predict. Some unexpected small teams have begun to have such needs, while some large companies are just starting this journey, and many companies still have a considerable way to go.
"Making money is safer."
Should big model startups "burn money" or make money? It seems to be a common practice for many big model manufacturers to "burn money" first to make a name for themselves, then find market applications or seek further financing. But this also carries certain risks. Previously, AI startup Stability AI was reported to be in financial trouble due to insufficient revenue and inability to secure new funding. OpenAI also periodically reports losses and the need for further financing.
Alex Smola does not agree with the approach of relying solely on fundraising. "You can raise funds or make money yourself. If you make money, it's safer," Alex Smola told reporters. "You can also (raise funds) to become very large, very conspicuous, 'burn very brightly,' and then collapse. Like Stability, they were very famous and dazzling for a few years, with a lot of money, and I think they didn't focus much on revenue. If they had focused on revenue earlier, the situation might be different now."
Alex Smola told reporters that becoming very famous and waiting for acquisition is one path for startups, but it's a high-risk value proposition, and it's a bit strange to start a business just to be acquired. If a company cares about revenue and customers, the market will provide feedback on the company's products, helping the company to optimize its products.
In the current climate where big model startups are still hot and investment institutions are still interested in startup projects, startups including Boson AI are making choices about whether to maintain this balance of income and expenditure or to secure more financing and go all out. Li Mu mentioned in an article that the company might have been able to "have a billion in cash on hand" if it had continued to raise funds, but at the time, considering that too much financing would be difficult to exit or be "put on a pedestal," they did not do so. Now, thinking back, "starting a business is about defying fate, what retreat?" Alex Smola remains cautious to this day.Perhaps because I am older than some other founders, I pay more attention to overall finances and risks. I worked at Amazon for 7 years, which may have taught me some things to be mindful of. Li Mu is a bit younger than me, and he might be more inclined to take risks, while I might be more conservative, but we have a good working relationship. Alex Smola stated that, looking back, he should have started his own business two years earlier, but fortunately, he did not continue to wait. The company's business model is not perfect yet, but it has become "interesting." Beyond custom models, there may be other possibilities in the future.