With the advent of machine intelligence purportedly on the near-term time horizon, many claims of hyper-accelerated societal progress are being bandied about. Such claims, however, are often only veneers of hazy techno-utopian futures, with little in the way of concrete claims. We are interested here in exploring not the difference between machine and human intelligence, but rather between the achievements that will likely be possible as a result of such intelligence.

In a previous discussion, I discussed the gap between intelligence and achievement, where achievement seems far more strongly correlated with curiosity and a persistent application of intelligence towards a goal rather than raw intellect. A natural question, therefore, is will the achievements of machine intelligence be limited if no intrinsic curiosity is built into machines? After all, this previous discussion appears to indicate so.

The surprising thing is that in-silico intelligence lacks some of the properties of biological intelligence that seem to render curiosity useful. In particular, curiosity is most critical in maintaining a psychological optimism to continue pursuing a direction of inquiry. Yet, in-silico intelligence likely will not have such psyches that need maintenance and can, therefore, indefinitely be directed towards pursuits, no matter how seemingly pointless such directions feel. In other words, the infinite persistence of machine intelligence will render curiosity unnecessary. A machine intelligence, in other words, should not only be able to achieve feats of intellectual discovery that humans are able to, but even go further and make achievements that are on even longer time horizons than those typical of human feats.

If such indefinite achievement of discovery is possible, how should the efforts of such intelligence be directed? One approach is to simply forgo any semblance of planning and explore the space of possible inquiries uniformly at random. In a world of infinite abundance, this may, in fact, be the optimal strategy, since it allows for the exploration of threads whose utility is on such a long time horizon that they would otherwise never be explored.

However, this is not the state of the world: machine intelligence will be embedded in a society of humans with finite economic resources, and, critically, such intelligence exhausts those resources, specifically money and energy. This means that achievement along particular directions will no longer be bottlenecked by the psychological fortitude of its explorers, but almost purely by the allocation of resources in those directions. While a seemingly novel phenomenon, analogous constructions have existed in history.

Organizations, akin to machine intelligence, can themselves be viewed as intelligent organisms, capable of learning through the adaptation of their constituent members. Organizations do exhibit psyches to a degree, where collective feelings of enthusiasm or despair about future prospects can permeate amongst their people and, as a result, into similar decisions of collective fortitude in pursuing a particular direction. Nonetheless, the psychological whims of an organization are an order of magnitude more dissipated compared to those of individual humans, making them a reasonable comparison point for machine achievement.

We, therefore, want to consider cases of the allocation of resources in organizations to reason through how this will manifest with machine intelligence. Two cases are in the allocation of resources in Bell Labs and that in Nixon’s “War on Cancer” directive, which represent contrasting cases of success. Over the course of its existence, Bell Labs created radar, the transistor, Unix, the C programming language, and foundational theories of communication, amongst others. In contrast, while efforts directed by the “War on Cancer” yielded treatments for childhood leukemia, there have been significantly fewer direct successes from this organizational effort. This is not the result of a talent mismatch: each effort had magnificently talented scientists and researchers working on it.

The core difference lies in the presence of a collective “taste.” Research taste is the intuition that a particular question is worth pursuing prior to its answering. Taste is the scientific equivalent of the intuition a chess grandmaster feels when he plays a good move: often, he will have no sense of why a particular move is good, just a feeling that it is. Formally, the quality of moves can be quantified by their value functions. Taste, then, is the value estimate of answering a particular question.

The success of Bell Labs, in part, was driven by a collective, clear sense internally of what research directions were promising; in other words, the organization had a well-calibrated taste. In contrast, while the objectives of the cancer efforts were clear, there was no clear approach to tackle this problem. Many analogous cases exist across fields: proving the Riemann Hypothesis, unifying quantum mechanics and gravity, or discovering room-temperature superconductors. Each of these problems is a concrete goal, yet none lends itself to a clear approach for progress, rendering a taste elusive for the field to acquire.

The curious implication, therefore, is that, in a world of finite resources, the capabilities of machine achievement will be bottlenecked by the acquisition of a taste. In the short term, this will likely be provided by humans, given that we own the resources that collectively run such intelligence. This, then, points to the answer of where people should spend their efforts in a world of constant flux: develop a taste for great work.