The Great Gray Ravelled Knot:

Overview
    It is hard not to wax eloquent when describing the construction and the capabilities of the human brain. From a purely computational point of view, the human brain appears to be about 100 times faster and more complex then the upcoming Cray T3E supercomputer or about 1,000,000 times more elaborate than a top-of-the-line PC.

Complexity:
    The human brain contains about 50 billion to 200 billion neurons (nobody knows how many for sure), each of which interfaces with 1,000 to 100,000 other neurons through 100 trillion (10¹⁴) to 10 quadrillion (10¹⁶) synaptic junctions. Each synapse possesses a variable firing threshold which is reduced as the neuron is repeatedly activated. If we assume that the firing threshold at each synapse can assume 256 distinguishable levels, and if we suppose that there are 20,000 shared synapses per neuron (10,000 per neuron), then the total information storage capacity of the synapses in the cortex would be of the order of 500 to 1,000 terabytes. (Of course, if the brain's storage of information takes place at a molecular level, then I would be afraid to hazard a guess regarding how many bytes can be stored in the brain. One estimate has placed it at about 3.6 X 10¹⁹ bytes.)

Speed:
    Neurons require about a millisecond to discharge, followed by a 4 millisecond refractory period That could amount to as many as 2 X 10¹⁸ connection updates per second. In practice, the firing rate and the synaptic count probably isn't that high. There are 40-Hertz firing waves that sweep the entire brain from back to front. The 6,000,000,000 neurons in the visual cortex also fire about 40 times a second to give us our 20-frame-per-second visual update rate, so we might be looking at perhaps 240 billion firings per second in the visual cortex. Each neuron connects to a number of other neurons through dendrites and an axon (an average of 15,000 interconnections per neuron in the visual cortex), so we might be dealing with about 50 trillion (5 X 10¹³) synaptic junctions in the visual cortex. At 40 firings a second, the visual cortex should be able to perform about 2 quadrillion synaptic activations per second—2 X 10¹⁵ connection updates per second or 2,000,000 Gcups (giga-connection-updates-per-second)—compared to 10 Gcups for current neural networks. For the brain as a whole, assuming 10,000 interconnections per neuron, the number might be about 10 times this amount, or 20,000,000 Gcups.
    It has been estimated that computational speeds of 10⁹ calculations per second (1 Gigops) would be required to match the edge and motion detection capabilities of the first four layers of the human retina, and 10¹³ operations per second (10,000 Gigops) to 10¹⁶ operations per second (10,000,000 Gigops) would be necessary to emulate what is done in the brain overall.

Accuracy and Redundancy:
    Another computational parameter which may be of interest in assessing the brain's capabilities is its information storage capacity. However, my preliminary estimates of the storage we would actually need to store what we remember suggests numbers much lower than those above.)
    Some redundancy in cerebral memory storage may be needed to accommodate for the quantum unreliability of the brain's nanocircuitry. (The synaptic junctions measure only a few atomic diameters across.)

Power:
    Still another impressive parameter regarding the brain is how little power it dissipates (of the order of 100 watts). By comparison, a 500-MIPS (500-Million Instructions Per Second) DEC Alpha chip dissipates about 50 watts. Using 20,000 DEC Alpha chips, we would need a 1,000 kw behemoth to achieve the lower threshold of 10,000 Gigops, or about 20,000 times as much power.
    Only now are microprocessors appearing that offer the promise of reaching speeds and storage capacities at the lower end of this envelope. Intel is funded to develop a 1.8 terops (1.8 X 10¹² operations per second) supercomputer using 9,000 P6 chips (1999 Update: Already delivered), and has announced plans to develop a 10 terops (10¹³ operations per second) parallel processor by the end of this decade. Cray Computer Corporation is developing the T3E parallel processor which, using 2,048 DEC processors, should yield up to 1.2 terops, with up to 4 terabytes of RAM. The new $550 Texas Instruments TMS80C can deliver up to 2 gigops. The Sega Genesis Saturn game set (1999 Update: Anyone for Dreamcast?) utilizes video chips that operate at almost 1 gigops. With 12-gigabyte disk drives available for a retail price of about $400 each, a terabyte of on-line disk storage ought to be achievable today for less than $35,000 (1999 Update: Now less than $7,500). With disk drive capacities 100-folding every 10 years, a $250 3-terabyte disk drive ought to be available on desktop PCs (or in robots) by the year 2012 (1999 Update: now at 76 Gigabytes), and perhaps by the year 2010, together with 1 terops of 16-bit integer processing speed and 32 gigabytes of RAM. Alternatively, $75,000 (1999 Update: Already down to $750,000) should buy 100 terabytes of disk storage 15 years hence. Another $75,000 might purchase 100 terops of 16-bit integer processing power and for an additional $75,000, 3.2 terabytes of RAM (1999 Update: Current cost - $3,200,000).(
    It may seem to be blatant hubris to contemplate the implementation of human-caliber intelligence on a desktop computer but I'm not altogether convinced that it will require the above kinds of processing speed and storage capacity to approach human levels of intellectual capability in various relevant areas. Progress is being made too rapidly in developing such uniquely human capabilities as high-accuracy speech recognition, machine vision, facial recognition, optical character recognition, graphics manipulations, and robotic motion. We aren't there yet but in another 5, 10, or 15 years… In any case, it seems worth the effort to see how far we can go. We'll certainly learn a lot in the process.