AI from fast to wise.
The performance is a function of motivation, let me explain.
Let’s start with the original meaning of performance. In the early days of computing, they had isolated mainframes with dummy terminals. The performance was limited by the processing power, and the memory available for the number of user tasks being executed. In the early era of the PC, the same was true, except most of the time, you had only a single-user with multiple tasks. You could run a text editor, a spreadsheet, and play the solitaire game at the same time. The Morse Law was a rule and the computational power doubled every 18 months, or so.
Then there came Internet and application programming interfaces (API) the specific tasks could be now done on hundreds of “cloud” servers in parallel.
Then there came NVidia GPU with massive parallelism of simple computational operation measured in hundreds of Giga (1,000,000,000), or trillions, of decimal math operations per second (GFLOPS).
The motivation until then was a single user experience, for example, gaming performance with 120 video frames per second.
Then there came machine learning via deep neural networks (DNN). You trained the predictive network on a single, or multiple GPU cards, which cut the training time on, say, 15 million images from a week to a few minutes. The DNN became as good, or better than humans in predicting very particular things.
People are impatient, yes you need your conversation translation now, a self-driving car to be responsive, and you want your video feed not to stumble, but the AI does not have to exist in the same time scale.
Humans form wise opinions over years of trial-and-error, and human brains are actually very slow comparing to computers. We would like to think we make snap decisions, but it is a fallacy, and, it is always better to “sleep on it”, too.
The Morse Law is irrelevant anymore for artificial intelligence (AI).
The brain, as I explained in a previous article, is composed of many centers, each highly specialized. The brain is limited to the physical size which is dictated by several factors such as how much food energy we can afford to spend on thinking and its initial size at birth.
The AI could be unlimited in size, limited only by the costs or operation, and even that, with solar power, is becoming less relevant. Let’s say we change the motivation from fast to wise. Let’s say, we do not care if it takes a week, or a year, to get the right answer, but that we do care the answer is right.
A good phone in 2021 has the capability for about 12 trillion math operations per second (12 TOPS).
An average size predictive deep neural network (DNN) today takes about half a trillion (500,000,000,000), or 0.5 TOPS, of operations per second to perform well enough for fast video gaming. On a desktop, this can be accomplished easily on a GPU, or a tensor processing unit (TPU). I can buy several inexpensive 4 TOPS Google Coral TPU and attach them via a USB hub.
You could now run multiple DNN in parallel, let’s say in order to understand the request and compose the answer. The same machine could offload the tasks to remote servers such as Google Translate API and thousands of more services that are, or will be available in the cloud. What’s more, the individual computers or phones could talk to each other to derive a swarm consensus.
The AI shall be defined not as a single box or even a warehouse of computers, but the Internet of interconnected deep neural networks, each specializing in a single, well-defined task, and all talking to each other via API. The AI answers do not always have to be instant, they just have to be wiser.
