It reminds me of the Welsh poet Dylan Thomas's "Do not go gentle into that good night"
"The gentle singularity" is the hand of fate that quietly descends, tender yet irresistible, just as a dying old man can gently walk into that "good night," quietly inviting humanity to step into the unknown abyss.
Humans may not rage against the tides of fate, but rather, like an old man in twilight, with a hint of curiosity and submission, they are gently consumed by the singularity.
Before reading Altman, we should silently recite this line from the poem:
Do not go gentle into that good night;
rage, rage against the dying of the light
The following is the full text of the blog, in both Chinese and English:
The Gentle Singularity
Sam Altman
Sam Ultraman
We are past the event horizon; the takeoff has started. Humanity is close to building digital superintelligence, and at least so far it’s much less weird than it seems like it should be.
We have crossed the event horizon; the takeoff has begun. Humanity is about to build digital superintelligence, and at least so far, its form is far less bizarre than we imagined.
Robots are not yet walking the streets, nor are most of us talking to AI all day. People still die of disease, we still can’t easily go to space, and there is a lot about the universe we don’t understand.
Robots are not yet ubiquitous on the streets, and most of us still do not spend our days conversing with artificial intelligence. People will still die from diseases, we still cannot easily enter space, and there are many mysteries in the universe that we do not understand.
And yet, we have recently built systems that are smarter than people in many ways, and are able to significantly amplify the output of people using them. The least-likely part of the work is behind us; the scientific insights that got us to systems like GPT-4 and o3 were hard-won, but will take us very far.
However, we have recently built systems that are smarter than humans in many ways, capable of significantly amplifying the output of users. The seemingly most impossible part of this work has been completed; the scientific insights that led to the emergence of systems like GPT-4 and o3 were hard-won, but they will take us very far.
AI will contribute to the world in many ways, but the gains to quality of life from AI driving faster scientific progress and increased productivity will be enormous; the future can be vastly better than the present. Scientific progress is the biggest driver of overall progress; it’s hugely exciting to think about how much more we could have.
人工智能将从多方面为世界造贡献,但由人工智能推动的更快科学进步和更高生产力所带来的生活质量提升将是巨大的; 未来可能远比现在更加美好。 科学进步是整体进步的最大驱动力; Thinking that I can have more is a very exciting thing.
In some big sense, ChatGPT is already more powerful than any human who has ever lived. Hundreds of millions of people rely on it every day and for increasingly important tasks; a small new capability can create a hugely positive impact; a small misalignment multiplied by hundreds of millions of people can cause a great deal of negative impact.
In a certain grand sense, ChatGPT is already more powerful than any living human. Hundreds of millions of people rely on it every day to handle increasingly important tasks; a tiny new feature can have a huge positive impact; and a small error, when amplified by hundreds of millions of people, can also lead to a significant negative effect.
2025 has seen the arrival of agents that can do real cognitive work; writing computer code will never be the same. 2026 will likely see the arrival of systems that can figure out novel insights. 2027 may see the arrival of robots that can do tasks in the real world.
2025年见证了能够从事真正认知工作的智能体的诞生; 编写计算机代码的方式将从此改变。 2026年可能会出现能够发现新颖见解的系统。 2027年可能会出现能够在现实世界中执行任务的机器人。
A lot more people will be able to create software, and art. But the world wants a lot more of both, and experts will probably still be much better than novices, as long as they embrace the new tools. Generally speaking, the ability for one person to get much more done in 2030 than they could in 2020 will be a striking change, and one many people will figure out how to benefit from.
更多的人将能够创造软件和艺术。 但世界对这两者的需求也会大大增加,只要专家们拥抱新工具,他们可能仍然会比新手出色得多。 总的来说,一个人在2030年能完成工作的量远超2020年,这将是一个显座的变化,许多人会找到从中受益的方法。
In the most important ways, the 2030s may not be wildly different. People will still love their families, express their creativity, play games, and swim in lakes.
In the most important aspects, the 2030s may not be drastically different. People will still love their families, express creativity, play games, and swim in the lake.
But in still-very-important-ways, the 2030s are likely going to be wildly different from any time that has come before. We do not know how far beyond human-level intelligence we can go, but we are about to find out.
But in some aspects that are still very important, the 2030s may be completely different from any era before. We do not know how far we can surpass human-level intelligence, but we are about to find the answer.
In the 2030s, intelligence and energy—ideas, and the ability to make ideas happen—are going to become wildly abundant. These two have been the fundamental limiters on human progress for a long time; with abundant intelligence and energy (and good governance), we can theoretically have anything else.
In the 2030s, intelligence and energy – that is, ideas and the ability to think about them – will be abundant. 这两者长期以来一直是人类进步的根本限制factor; With plenty of intelligence and energy (and good governance), I can have everything.
Already we live with incredible digital intelligence, and after some initial shock, most of us are pretty used to it. Very quickly we go from being amazed that AI can generate a beautifully-written paragraph to wondering when it can generate a beautifully-written novel; or from being amazed that it can make live-saving medical diagnoses to wondering when it can develop the cures; or from being amazed it can create a small computer program to wondering when it can create an entire new company. This is how the singularity goes: wonders become routine, and then table stakes.
We are already living in an incredible digital intelligence, and after the initial shock, most of us have become accustomed to it. We quickly transitioned from being amazed that AI could generate a beautifully written paragraph to wondering when it would be able to write a well-crafted novel; from being astonished by its ability to make life-saving medical diagnoses to pondering when it would develop cures; from marveling at its capacity to create a small program to considering when it would start a brand new company. This is how the singularity occurs: miracles become the norm and then become basic requirements.
We already hear from scientists that they are two or three times more productive than they were before AI. Advanced AI is interesting for many reasons, but perhaps nothing is quite as significant as the fact that we can use it to do faster AI research. We may be able to discover new computing substrates, better algorithms, and who knows what else. If we can do a decade’s worth of research in a year, or a month, then the rate of progress will obviously be quite different.
我们已经听到科学家们说,他们的生产力是使用人工智慧智能之前的两到三倍。 先进的人工智慧智能之所以引人入胜,原因There are many,但或许没有哪一点比我们能用它来Accelerate artificial intelligence research is more important。 我们或许能够发现新的计算基底、更好的算法,以及天知道还有什么。 如果能在一个月或一年内完成十才能完成研究,那么进步的速度显然将截然不同。
From here on, the tools we have already built will help us find further scientific insights and aid us in creating better AI systems. Of course this isn’t the same thing as an AI system completely autonomously updating its own code, but nevertheless this is a larval version of recursive self-improvement.
从现在开始,我们已经构建的工具将帮助我们发现更深入的科学见解,并协助我们创造更优秀的人工智能系统。 当然,这与一个人工智能系统完全自主地更新自己的代码并非一回事,但这已是递归式自我改进的雏形。
There are other self-reinforcing loops at play. The economic value creation has started a flywheel of compounding infrastructure buildout to run these increasingly-powerful AI systems. And robots that can build other robots (and in some sense, datacenters that can build other datacenters) aren’t that far off.
还有其他的自我强化循环正在发挥作用。 经济价值的创造已经启动了一个飞轮效应,不断推动基础设施建设,以运行这些日益强大的人工智慧智能系统。 而能够制造其他机器人机器人(在某种意义上,能够建其他数据中心的数据中心)也并非遥不可和。
If we have to make the first million humanoid robots the old-fashioned way, but then they can operate the entire supply chain—digging and refining minerals, driving trucks, running factories, etc .—to build more robots, which can build more chip fabrication facilities, data centers, etc, then the rate of progress will obviously be quite different.
如果我们必须用传统方式制造出最初的一百万个类人机器人,但之后它们能够运营整个供应链——挖掘和精炼矿物、驾驶卡车、运营工厂等等——来制造更多的机器人,而这些机器人又能建更多的chip片制造厂、数据中心等,那么进步的速度显然将截然不同。
As datacenter production gets automated, the cost of intelligence should eventually converge to near the cost of electricity. (People are often curious about how much energy a ChatGPT query uses; the average query uses about 0.34 watt-hours, about what an oven would use in a little over one second, or a high-efficiency lightbulb would use in a couple of minutes. It also uses about 0.000085 gallons of water; roughly one fifteenth of a teaspoon.)
随着数据中心生产的自动化,智能的成本最终应会趋近于电力的成本。 (People often wonder how much energy it takes to do a ChatGPT lookup; 平均一次查询大约消耗0.34 Wa时,相当于一个oven一秒多一点的用量,或一个高效灯泡几分钟的用量。 它还消耗大约0.000085加仑水;约equal to 19,茶spoon的15/1。 ）
技术进步的速度将持续加快,而人类适应几乎任何事物的能力也将一如既往。 过程中会有非常艰难的部分,比如整类工作的消失,但另一方面,世界将变得如此富且迅速,以於于我们能够认真考虑以前从未dare to think of the new policy concept。 我们可能不会一蹴而就地采纳新的社会契约,但几十后回望,这些渐进的变化将汇聚成巨大的变革。
If history is any guide, we will figure out new things to do and new things to want, and assimilate new tools quickly (job change after the industrial revolution is a good recent example). Expectations will go up, but capabilities will go up equally quickly, and we’ll all get better stuff. We will build ever-more-wonderful things for each other. People have a long-term important and curious advantage over AI: we are hard-wired to care about other people and what they think and do, and we don’t care very much about machines.
如果历史可为借鉴,我们将找到新事去,产生新需求,并迅速absorbing 新工具(工业revolution后的职业变迁是一个很好的一般的例子)。 期望会提高,但能力也会同样快速了提高,我们都会得到更好的东西。 我们将为彼此创造出越来越beautiful things。 与人工智能相比,人类拥有一种长期、重要且奇特的优势:我们的天性决定了我们在caring 人以及人的所思所为,而对机器则不那么在乎。
thousand years ago 的自给自足的农民看到我们许多人现在的工作,会说我们做的是"假工作",认为我们只是在玩游戏自娱自乐,因为我们有充足的食和难以想象的luxu。 我希望一千后我们看待未来的工作时,也会觉得它们是"非常假工作",但我毫不怀疑,从事这些工作的人会覺得它们无比重要和满足。
The rate of new wonders being achieved will be immense. It’s hard to even imagine today what we will have discovered by 2035; maybe we will go from solving high-energy physics one year to beginning space colonization the next year; or from a major materials science breakthrough one year to true high-bandwidth brain-computer interfaces the next year. Many people will choose to live their lives in much the same way, but at least some people will probably decide to “plug in”.
新奇迹实现的速度将是巨大的。 Today我们甚至难以想象到2035年会有什么发现; 也许我们会在一年内解决高能物理问题,然后在下年开启太空colony; 或者在年内得重大材料科学breakthrough,然后在下年实现真高band 宽脑机接口。 许多人会选择以大致相同的方式生活,但至少有些人可能会决定"接入"。
展望未来,这一切听起来似乎难以理解。 但亲身经历时,可能会覺得shock但尚可应对。 从相对论的视角看,奇点是点滴发生的,fusion is 缓慢进行的。 我们正攀登在指数级技术进步的漫长arc线上; 前看总是显得陡,后看则显得平,but 它是一条平滑的曲线。 (回想一下2020年,如果有人说2025年将出现接近通用人工智慧智能的东西,听起来会是什么感觉,再对比一下过去5年实际的经历。 ）
There are serious challenges to confront along with the huge upsides. We do need to solve the safety issues, technically and societally, but then it’s critically important to widely distribute access to superintelligence given the economic implications. The best path forward might be something like:
With the enormous benefits come severe challenges that need to be faced. We do need to address security issues from both technological and societal perspectives, but given its economic impact, it is also crucial to widely promote access to superintelligence. The best path forward may be as follows:
Solve the alignment problem, meaning that we can robustly guarantee that we get AI systems to learn and act towards what we collectively really want over the long-term (social media feeds are an example of misaligned AI; the algorithms that power those are incredible at getting you to keep scrolling and clearly understand your short-term preferences, but they do so by exploiting something in your brain that overrides your long-term preference).
解决对齐问题,意味着我们能够有力地保证人工智能系统学习并按照我们集体的长期真实意愿行事(社交媒体的信息流是未对齐人工智能的一个例子; 驱动它们的算法在让你不停滚动方面表现出色,并能clearly understand your 短期preferences,但它们是通过利用你大脑中某种压倒你长期偏好的东西来做到这一点的)。
Then focus on making superintelligence cheap, widely available, and not too concentrated with any person, company, or country. Society is resilient, creative, and adapts quickly. If we can harness the collective will and wisdom of people, then although we’ll make plenty of mistakes and some things will go really wrong, we will learn and adapt quickly and be able to use this technology to get maximum upside and minimal downside. Giving users a lot of freedom, within broad bounds society has to decide on, seems very important. The sooner the world can start a conversation about what these broad bounds are and how we define collective alignment, the better.
然后,专注于让超级智能变得廉价、普及,并且不过度集中于任何个人、公司或国家。 社会具有韧性、创造力并且适应快速。 如果我们能够驾驭人类的集体意愿和智慧,那么尽管我们会犯很多错误,有些事情会出错,但我们将迅速学习和适应,并能够利用这项技术来获得最大的好处和最小的坏处。 在社会必须决定的广泛boundary 内,给予用户大量的自由似乎非常重要。 世界越早开始就这些广泛boundary 是什么以及我们如何定义集体对齐展开对话,就越好。
我们(整个行业,不仅仅是OpenAI)正在为世界构建一个大脑。 它将是高度个性化的,并且对每个人来说都易于使用; 我们将受限于好的想法。 In the early days of the Qiqi Dynasty, the technical people of the early enterprise world have always made fun of those "Mr. Jizi"; 那些有想法却在寻找团队来实现它的人。 现在 in my eyes 来,he 们即将Welcome your own spring。
OpenAI is a lot of things now, but before anything else, we are a superintelligence research company. We have a lot of work in front of us, but most of the path in front of us is now lit, and the dark areas are receding fast. We feel extraordinarily grateful to get to do what we do.
Today, OpenAI wears many hats, but first and foremost, we are a superintelligent research company. There is still much work to be done in front of us, but the path ahead is mostly illuminated, and the dark areas are quickly receding. We are incredibly grateful to be engaged in this endeavor.
Intelligence too cheap to meter is well within grasp. This may sound crazy to say, but if we told you back in 2020 we were going to be where we are today, it probably sounded more crazy than our current predictions about 2030.
廉价到无法计量的智能已触手可。 这么说可能听起来很疯狂,但如果我们在2020年告诉你我们将达到今天的水平,那听起来可能比我们现在对2030年的预测更疯狂。
May we scale smoothly, exponentially and uneventfully through superintelligence.
May we step into the era of superintelligence smoothly, exponentially, and without turbulence.
