With On Intelligence, I find myself in the unique position of having heavily evangelized a book before I’ve even finished it. I read half of it and started buying copies for friends. This is something I’ve never done before, so if you’re busy, you can take a quick tl;dr, and assume that if you’re interested in how intelligence works, namely how the brain functions at a high level (learning patterns, predicting the future, forming invariant representations of things) and how we might functionally simulate that with computers, do not pass go, do not collect $200, go buy a copy (Amazon, Powells) and read it.
Still here? Good, because I have a lot to say. This isn’t really a book review, it’s more of a book summary and an exhortation to activity. You’ve been warned.
A Little Backstory
Earlier this year I went to OSCON, and at OSCON the keynote that impressed me the most was by Jeff Hawkins, creator of the PalmPilot and founder of HandSpring. Here’s the video:
As appropriate for an Open Source conference, Jeff’s company, Numenta was announcing that they were open sourcing their neocortical simulator library, NuPIC, and throwing it out there for people to hack on. NuPIC was based on the work Numenta had done on neocortical simulations since he wrote the book, On Intelligence, in 2005. NuPIC is software that simulates the neocortex, the sheet of grey matter on the outside of your brain where all your experiences live. 3 years of French? It’s in the neocortex. The ability to figure out that two eyes and a nose equals a face? The neocortex. The neocortex even has the ability to directly control your body, so that muscle memory you rely on to do that thing you do so well, like riding a bike or painting or driving a car? That’s all in your neocortex. It’s the size of a large dinner napkin (the largest in humans, but every mammal has one), is about as thick as 7 business cards, and wraps around the outside of your head. It is you.
Intrigued, I went to the full length session that the Numenta team presented…
One of their main demos was an electrical consumption predictor for a gymnasium. When initialized, the NuPIC system is empty, like a baby’s brain. Then you start to feed it data, and it starts to try to predict what comes next. At first, its predictions fall a little behind the data it’s receiving, but as the days of data go by, it starts to predict future consumption an hour out (or whatever you’ve configured), and it gets pretty good at it. Nobody told NuPIC what the data was, just like our DNA doesn’t tell our brains about French verbs, the structure is there and with exposure it gets populated and begins to predict.
At the end of the talk, their recommendation for learning more about this stuff was to read On Intelligence. So, eventually, that’s what I did.
A Little Hyperbole
The simulation, in software and silicon, of the biological data handling processes, and building software off of that simulation, is the most interesting thing I’ve seen since Netscape Navigator. Everything up to your iPhone running Google Maps is progressive enhancement and miniaturization of stuff I’ve seen before. Building brains feels different.
I have a Newton Messagepad 2000 around here somewhere. It had mobile email over packet radio with handwriting recognition in 1997. In 2001 I was using a cell phone with a color screened to look up directions and browse web sites in Japan. It’s all iterating, getting better bit by bit, so that when we look back in 10 years we think that we’ve made gigantic leaps. Have we? Maybe, but software is still stubbornly software-like. If I repeat the same error 10 times in a row, it doesn’t rewrite itself. My computer doesn’t learn about things, except in the most heavy handed of ways.
Writing and shipping working software is hard, there’s rarely time to focus on the things that aren’t necessities. In video games this has given us nicer and nicer graphics, since for the last few generations, graphics have sold games. Now developers are starting to realize there’s a huge swath of interesting stuff you can do that doesn’t require a 500 person art department and a million dollar budget. Sir, You Are Being Hunted has procedural map generation. Rimworld has AI storytellers that control events in the game world to create new experiences.
Looking beyond the game space, a few weeks ago I was talking with a large networking company about some skunkworks projects they had, and one of them was a honey pot product for catching and investigating hack attempts. The connections between deep simulations like Dwarf Fortress or the AI Storyteller in Rimworld and how a fake sysadmin in a honey pot should react to an intruder are obvious. If it’s all scripted and the same, if the sysadmin reboots the server exactly 15 seconds after the attacker logs in, it’s obviously fake. For the product to work, and for the attacker to be taken, it has to feel real, and in order to fool software (which can pick up on things like that 15 second timer), it has to be different every time.
One thing that these procedural and emergent systems have in common is that they aren’t rigidly structured programs. They are open to flexibility, they are unpredictable, and they are fun because unexpected things happen. They’re more like a story told by a person, or experiencing a real lived-in world.
I believe that to do that well, to have computers that surprise and delight us as creators, is going to require a new kind of software, and I think software like Numenta’s NuPIC neocortical simulator is a huge step in that direction.
Let’s Deflate That a Bit
Ok, so NuPIC isn’t a whole brain in a box. It’s single threaded, it’s kind of slow to learn, and it can be frustratingly obtuse. One of the samples I tried did some Markov chaining style text prediction, but since they fed each letter into the system as a data point instead of whole words, the system would devolve into returning ‘the the the the the’, because ‘the’ was the most common word in the data set I trained it with.
Neocortical simulators are a new technology in the general developer world. We’ve had brute force data processing systems like Hadoop, methods developed to deal with the problems of the Google’s of the world, and now we have NuPIC. The first steps towards Hadoop were rough, the first steps towards neocortical simulators are going to be rough.
It’s also possible that we’re entering another hype phase, brought on by the rise of big data as the everywhere-buzzword. We had the decades of AI, the decade of Expert Systems, the decade of Neural Networks, but without a lot to show for it. This could be the decade of the neocortex, where in 10 years it’ll be something else, but it’s also possible that just like the Web appeared once all the pieces were in place, the age of truly intelligent machines could be dawning.
Oh, This Was a Book Review?
It’s hard to review On Intelligence as a book, because how well it’s written or how accessible the prose may be is so much less important than the content. Sandra Blakeslee co-wrote the book, and undoubtedly had a large hand in hammering Jeff’s ideas into consumable shape. It isn’t an easy read due to the ideas presented, but it’s fascinating, and well worth the effort.
In the book Jeff describes the memory-prediction framework theory of the brain. The theory essentially states that the neocortex is a big non-specialized blob that works in a standard, fairly simple way. The layers in the sheet of the neocortex (there are 7 of them), communicate up and down, receiving inputs from your sensory organs, generalizing the data they get into invariant representations, and then pushing predictions down about what they will receive data about next. For instance, the first layer may get data from the eye and say, there’s a round shape here, and a line shape next to it. It pushes ’round shapes’ and ‘line shapes’ up to the next level, and says “I’ll probably continue to see round shapes and line shapes in the future”. The bouncing around of your natural eye movements gets filtered out, and the higher levels of the brain don’t have to deal with it. The next level up says, “This kind of round shape and line shapes seem to be arranged like a nose, so I’m going to tell the layer up from me that I see a nose”. The layer up from that gets the ‘I see a nose’ and two ‘I see an eye’ reports and says, “Next layer up, this is a face”. If it gets all the way to the top and there’s no mouth, which doesn’t match the invariant representation of ‘face’, error messages get sent back down and warning flags go off and we can’t help but stare at poor Keanu…
These layers are constantly sending predictions down (and across, to areas that handle other related representations) about what they will experience next, so when we walk into a kitchen we barely notice the toaster and the microwave and the oven and the coffee maker, but put a table saw onto the counter and we’ll notice it immediately.
As we experience things, these neurons get programmed, and as we experience them more, the connections to other things strengthen. I figure this is why project based learning works so much better than rote memorization, because you’re cross connecting more parts of your brain, and making it easier for that information to pop up later. Memory palaces probably work the same way. (I’m also half way through Moonwalking with Einstein, about that very thing.)
So, Have We Mentioned God Yet?
This is where things start to get weird for me. I grew up in a very religious family, and a large part of religion is that it gives you an easy answer to the ‘what is consciousness’ question when you’re young. Well, God made you, so God made you conscious. You’re special, consciousness lets you realize you can go to heaven, the dog isn’t conscious and therefor can’t, etc.
About a third of the way into On Intelligence I started having some minor freakouts, like you might have if someone let you in on the Truman Show secret. It was like the fabric of reality was being pulled back, and I could see the strings being pulled. Data in, prediction made, prediction fulfilled. Consciousness is a by-product of having a neocortex. (Or so Jeff postulates at the end of the book.) You have awareness because your neocortex is constantly churning on predictions and input. Once you no longer have predictions, you’re unconscious or dead, and that’s that.
That’s a heavy thing to ponder, and I think if I pondered it too much, it would be a problem. One could easily be consumed by such thoughts. But it’s like worrying about the death of the solar system. There are real, immediate problems, like teaching my daughter how stuff (like a Portal Turret) works.
Let’s Wrap This Thing Up With A Bow
I’m sorry this post was so meandering, but I really do think that neocortical simulators and other bio processing simulations are going to be a huge part of the future. Systems like this don’t get fed a ruleset, they learn over time, and they can continue to learn, or be frozen in place. Your self-driving car may start with a car brain that’s driven simulated (Google Street View) roads millions of miles in fast-forward, and then thousands of miles in the real world. Just like everyone runs iOS, we could all be running a neocortex built on the same data. (I imagine that really observant people will be able to watch Google’s self driving cars and by minor variations in their movements, tell what software release they’re running.) Or we could allow ours to learn, adjust its driving patterns to be faster, or slower, or more cautious.
The power of software is that once it is written, it can be copied with nearly no cost. That’s why software destroys industries. If you write one small business tax system, you can sell it a million times. If you grow a neocortex, feed it and nurture it, you’ve created something like software. Something that can be forked and copied and sold like software, but something that can also continue to change once it’s out of your hands. Who owns it? How can you own part of a brain? Jeff writes in the book about the possibilities of re-merging divergent copies. That’s certainly plausible, and starts to sound a whole lot like what I would have considered science fiction 10 years ago.
I finally finished On Intelligence. I have Ray Kurzweil’s book, How to Create a Mind on my nightstand. I’ve heard they share a lot of similar ideas. Ray’s at Google now, solving their problem of understanding the world’s information. He’s building a brain, we can assume. Google likes to be at the fore-front.
We could throw up our hands and say we’re but lowly developers, not genius computer theorists or doctors or what have you. The future will come, but all we can do is watch. The problem with that is that Google’s problems will be everyone’s problems in 5 years, so for all the teeth gnashing about Skynet and Bigdog with a Google/Kurzweil brain, it’s much more productive to actually get to work getting smarter and more familiar with this stuff. I wouldn’t be surprised if by 2020 ‘5+ Years Experience Scaling Neocortical Learning Systems in the Cloud’ was on a lot of job postings. And for the creative, solving the problem of how the Old Brain’s emotions and fears and desires interfaces with the neocortex should be rife with experimental possibilities.
NuPIC is on github. They’re putting on hackathons. The future isn’t waiting. Get to it.
Updates
Here’s a video from the Goto conference where Jeff talks about the neocortex and the state of their work. This video is from October 1st of 2013, so it’s recent. If you have an hour, it’s really worth a watch.
One thought on “Book Review: On Intelligence by Jeff Hawkins”