Question 1: Tell us about yourself. What is your background and education, and what projects are you currently working on?

I am a 22 year-old senior undergraduate physics major at Whitman College in Walla Walla, Wa. While I was browsing the college library toward the end of my sophomore year, I happened on a book called “in the image of the brain”. I believe it is by Jim Jubak. The idea of brain function has always intrigued me, and the possibility of creating an artificial network based on biological computational principles drove me to look into the subject. This book got me started, and I have spent the last two years looking into the subject independently and taking any class with “neuro” as a pre-fix. I am continuing with the development of various physical neural network aspects while finishing my last year at Whittman as a full-time student.

Question 2: Tell us about Knowmtech.

Knowmtech [profile] is an Intellectual property holding company for ideas related to Knowm™ synapses. Although the company is fairly new, I have been thinking about the problem of artificial neural networks for quite some time. At first, I tried to think of alternate ways of building entire neural networks, neurons and all. With my limited experience in the field, I found this to be a quick dead-end. When I realized that the problems with building artificial neural network resided with the synapses and not the neurons, I was able to focus my search and really make progress. While on a flight from Salt Lake City to Santa Fe, I realized that physically achieving neural network-like functionality was all about forming a connection. A synapse is nothing more that a coupling between two neurons-a connection, in its simplest sense. I knew that whatever connection I came up with must be very, very small or it would be useless, and it must be modifiable. When I think of a transistor, I think of three terminals. One terminal controls what happens at the other two. This is a kind of modifiable connection, but I realized it wouldn’t work because a synapse only has two terminals. A synapse modifies itself as a result of how it is used, not what one does to control it. So then I figured I needed to find a connection that changed as a function of it being used. The only way a connection can be used (in an electronic device) is by passing some sort of current though it, or more generally applying an electric field. After reading about carbon nanotubes everywhere, I came up with the idea: Use an electric field to cause nanotubes to align and form a connection between two electrodes. I originally thought that it would work with straight DC. But then I started to wonder why other people hadn’t already done the same thing. If it was this easy, why hadn’t anybody done it? So I sat on it for a semester. One day, about 6 months later, I hit the library. It didn’t take very long to find research showing that nanotubes, gold nanowires, indium phosphide nanowires, carbon nanoparticles-a whole list of particles-had been aligned in electric fields. The only catch was that the field wasn’t DC-it was AC, and as the frequency increased, so did the speed of alignment. I didn’t realize it at the time, but soon concluded that frequency dependant connection formation is perfect for neural network learning. So what I found was better than what I expected. At that point, I picked up the phone and made a few telephone calls to set in motion a secure path to revealing my discovery. I guess you could say that was the beginning of Knowmtech.

Question 3: Describe how Knowmtech's approach is different than that of conventional neural net approaches.

Knowmtech’s approach is certainly different. Current approaches to neural networks are primarily based on software (which will never work for large networks), and/or analog/digital hardware. After looking into the subject, I was surprised to find that even hardware implementations are not very good at emulating large networks. From a naive view point, a network is nothing but a lot of neurons and a whole bunch of synapses. A neuron can be emulated by any electrical devise that outputs a signal when its input voltage goes above a certain threshold. This is very easy to accomplish with standard circuitry, so the problem certainly is not here. The problem is the synapses. Synapses have both a memory function and a processing function, and trying to emulate both of these functions with the basic electronic building blocks is just impractical. One can store the synapse values digitally and then shuttle 1’s and 0’s around, but this takes up time just moving data around and results in a network based on mathematical algorithms, which is what we are doing with software. This slows things down considerably. One could also store the synapse values as a charge on a capacitor, but this also does not work well because the capacitors are large and they have to be constantly refreshed. To fully understand the scope of the problem, just look at a human brain: we have about 100 billion neurons, each connected to each other at least 10,000 times. We cannot yet fit 1 billion transistors on a chip, let alone the trillions of synapses, which currently require more than just a transistor to emulate. And we cannot assume that a smaller network running faster will achieve the same results. The size of the network is crucial. Of course we could mathematically simulate the synapses and neurons instead of building them all. But even considering the relatively slow firing rate of a biological neuron (on the order of 100 Hz), we would need peta-flop computers to model a network the size of the brain. If nature has found a way to fit this computational power into a few pounds of goo (most of the goo is not even neurons), then I think we need to look for some very different neural computational approaches.

Knowmtech, hopefully, will put an end to all this madness. First, it is obvious that our brains our not the result of a massive supercomputer simulating a neural network. Our brains are the network. This means that feed-back within a brain must be responsible for learning, not a mindless mathematical algorithm crunching numbers on a CPU. Knowmtech is developing IP related to building networks that teach themselves through internal feed-back, not by a number-crunching CPU. All a CPU must do is present a Knowm™ network with the training data. The Knowm™ network will teach itself how to match inputs with desired outputs. Second, a Knowm™ synapse is very small, non-volatile, and its dependence on frequency makes it very easy to incorporate into a network that uses internal feed-back to modify synapse strengths.

One of the most helpful things one of my professors said to me was that “the connections will operate too slow… actual molecules have to physically move around. That just doesn’t compare to the nano-second speeds of current electronics”. It was then that I realized that most people don’t understand the enormous difference between how we (as a neural network) compute information and how our computers do. When researchers of the late 1990’s realized that nanotubes can be aligned gradually in accordance with an applied alternating electric field, they likely did not realize that it could also be adapted for neural computational systems. What they wanted was instant alignment. Send a pulse over two electrodes and immediately create a connection. When one decomposes the world into 1’s and 0’s, it is imperative that whatever mechanism you use to store and manipulate data be extremely fast, or you just cant crunch the numbers in time. But a neural network, even when simulated on a standard computer, only updates synapse values gradually. My best reality check is always “does my brain do that?”. I know that my synapse values can’t modify themselves billions of times per second, and yet I’m still pretty capable…more so than any computer (at certain tasks). The power of a neural network is obviously not in the speed of its connection formation. Recognizing the differences between standard computers and a neural network is seminal to building successful neural network hardware. More biologists need to talk with electronic engineers and visa versa. Hopefully, Knowmtech will foster this kind of communication. It is truly amazing how much can be accomplished when the blinders are taken off.

Question 4: What is the relationship between neural nets and Moore's law?

Moores law states that computing speed (or transistor density) doubles approximately every 2 years. This is great news for computer buffs, but what it really means is that current approaches to increasing computing speed will fail in 15 or 20 years. If we would like to continue to increase the computing speed of computers, we are going to have to move into more parallel architectures. I cannot think of a more parallel architecture than the brain-100 billion independent neurons. Neural networks seem like a logical approach to increasing computing speed. It is obvious, however, that neural networks like a brain cannot compute information in the same way as a computer. A computer is good for anything we can distill into an efficient algorithm. The brain is good for extracting meaningful information from massive amounts of noisy data (in real time). By building a computational system that combines both the standard CPU with a self-adapting neural network, incredibly efficient and capable processors could be built.

Question 5: Shouldn't massive tasks that can be handled by neural nets be handled by giant supercomputers?

There are a few answers to this question. First, if one considers the computational abilities of a human brain, it is immediately clear that a super computer is not much of a solution. Given that biological neurons fire at about 100Hz, and there are about 100 billion of them, with 10,000 synapses for every neuron, all being modified in real-time, one gets the idea of just how big the giant supercomputer would have to be. Perhaps tens of peta-flops? Second, considering that we can perform these calculations, why would we want a super-computer to do it? If we can figure out how to build large, self-learning networks, we could increase the neuron firing rate to a few million Hz. Then we would really have a capable neural network-and we could fit it on a small chip and put it anywhere.

Question 6: How wide a range of problems can your proposed neural nets handle? Can they theoretically handle any computer task?

Knowmtech’s proposed neural networks should be able to handle most of the problems at the boundary between humans and computers. Just as an individual cannot find the 10 billionth decimal of pi, I don’t think a Knowm™ network will be able to handle any computer task. What standard computers are good at, they will remain good at. Large-scale pattern recognition, such as robust speech recognition and facial recognition, will become the realm of artificial neural networks. The combination of traditional computing with artificial neural networks is where exciting things start to happen. The biggest mistake being made right now is assuming that a bigger computer will save the day. We need a different kind of computer, not a faster one.

Question 7: When does Knowmtech plan on having its first product on the market? How much will it cost?

Hopefully very soon, but this is dependant on many factors. Knowmtech was not organized for the manufacture of Knowm™ chips. Given my current student status, and my desire to continue my education, it might be impractical to go the venture capital route. Most of the technology to produce a Knowm™ chip is already in place with large corporations like Intel, IBM, HP, Fujitsu, etc., so I believe that it would be a step in the wrong direction to pursue more than just IP licensing and take on more risk than I need too. My greatest passion is not running a company. I hope that through industry licensing and cooperative R&D, a Knowm™ chip will be available within a couple years. As far as the cost, I could foresee Knowm™ chips embedded in, for example, cell phones costing a few bucks, or in association with more capable chips running parallel with CPU’s, although likely costing much more. Perhaps a fair price approximation of a Knowm™ can be arrived at by benchmarking against current CPU and memory prices. The inherent fault-tolerance of a Knowm™ chip would result in very high-yields and therefore much lower prices.

Question 8: What is your assessment of artificial intelligence? Will your systems eventually lead to genuine sentience and consciousness?

I am a strong believer in looking to biology for answers to AI. If we consider ourselves genuinely conscious, and we are composed of billions of independent neurons, it seems silly to look for alternate forms of intelligence before we can copy what we know works. Although I do believe humans will create conscious technology within the next 20 or 30 years, I cannot say it will come from Knowm™ networks. I definitely think Knowmtech is a step in the right direction, but only the future will tell.

Question 9: What is your assessment of molecular nanotechnology? Do you ever think that the concept of molecular assemblers will ever be feasible?

Molecular nanotechnology is the logical path to take right now. I certainly can’t think of a way to build smaller, faster computers without resorting to atoms and molecules. Working with atoms and molecules to build precise structures is looking very difficult right now, but I believe solutions will show themselves shortly. I don’t think it is possible to deny the future availability of molecular assemblers. After all, we are surrounded with them right now. We are molecular assemblers. Building molecular assemblers is just a matter of reverse engineering.

Question 10: It would seem that your technology would attract the attention of many investors. Have you had difficulty obtaining venture-capital funding?

As an intellectual property (i.e., primarily patents) holding company, we are not actively looking for venture capital, although we have received offers. We expect that our future licensing revenues would probably limit the need for venture capital. Of course, we are keeping all doors open, but the only foreseeable need for venture capital is with patent infringement litigation and foreign patent filing. If all goes well, the need for venture capital would probably not be a problem.

Question 11: Could Knowmtech's technologies be effectively applied to robotics? Has Knowmtech done any research in this area?

Let me put it this way…If a lobster gets around better than a super-computer controlling a bunch of servos, then perhaps we should try looking into another approach…Neural networks are an obvious approach to robotics, but until now have been held back by neural-network hardware and software limitations. An independent, self-teaching neural network could definitely open up new possibilities in robotics. Knowmtech’s only interest is in developing the neural network systems. The applications are certainly everywhere, but we are focusing on the base technology.

Question 12: Does Knowmtech have any prototype systems currently operating?

No. I have developed my ideas to a point where they can be legally patented and where I believe a qualified manufacturer can take over a fabricate Knowm™ devices. My status as an undergraduate Physics major at a Liberal Art college has made it difficult to assemble the parts, but I am currently working on building a device as a part of my thesis. If all goes well, by the time this semester is over, somebody else will have built a basic Knowm™ TM chip. What I would like to do is to foster research in this area. The more people think about the idea, the more that can be accomplished. I know my limitations, and I know there are many people out there who can take my idea much farther than I have.

Question 13: What are your plans for the future?

I plant to pursue graduate work in some combination of VLSI design, computational neural science and emergent systems. I would love nothing more than to be an independent inventor, licensing my ideas to companies. I have plans on the drawing board for creating a network of independent scientists, engineers, and lawyers called the Tinkertank. Tinkertank would gather the resources to support teams of scientists and lawyers so that they could competitively pursuer research leading to intellectual property and obtain majority ownership of their own ideas, thus creating the equivalent of a “free-lance scientist”.