ai | Jennifer Kramer

Machines That Tell Stories: SXSW 2015

Last Saturday at SXSW Interactive Jon Lebkowsky and I curated a Core Conversation titled Machines That Tell Stories. I proposed the topic as a book project to Jon last year, and we put together this discussion as a stepping stone. Software storytellers are in the air. There were over a dozen sessions at SXSW this year on storytelling systems, and that kind of consensus usually heralds a new wave about to break. We’ve setup a twitter and tumblr for this project, if you want to follow along.

Our argument: Software is moving beyond raw data and into narrative. First it will help you weave the tales you want to spin, but soon it may be telling stories better than all but the best human storytellers.

The conversation was all over the place, and I don’t think anyone recorded it, but here are some notes and references that could be helpful…

Lisa Cron’s Wired for Story: “A story is how what happens affects someone who’s trying to achieve what turns out to be a difficult goal, and how she changes as a result.”
Wired For Story Takeaway: Story is about mechanics, the trappings that you think of as important aren’t as critical as hitting the right beats that resonate with the human brain.
The Future of Storytelling Conference – Great speaker videos
Dwarf Fortress’s Legends mode, Procedural Poetry Analysis (Leave the creative imagination up to the user. Provide concrete, easy to procedurally generate elements, and let the brain fill in the rest.)
Weavrs as storytellers
The Nest Home Report monthly email as a machine-generated story
Collaborative human/machine storytelling at DARPA
Machine data into text reporting at Automated Insights (1 billion articles for 1 person each, instead of 1 article for 1 billion readers) More at CNN
Games by Angelina – Procedurally generated videogames, played through brute-force to see if they’re solvable. Potentially compare play throughs to known-pleasing physical interactions (progressively more complex button presses and movements)
Mechanical Turk as a part of a story machine, using human filtering to produce more compelling procedural content
Turing in The Imitation Game: The question isn’t whether machines will think like humans, it’s whether machines will think like machines.
tmbotg – Random TMBG tweeting bot, sometimes interacted with by humans due to serendipity
Why limit to text? Is software that generates a song based on your day’s quantified self data creating a story?
Shadows of Mordor’s Nemesis System as a storytelling engine – characters continue to exist when you aren’t looking, maintain the thread without you
Games as half-way points: PROCJAM’s The Inquisitor as procedural murder mystery
NaNoGenMo – Software generated novels
Eugene Goostman – Chatbot & Winner of the Loebner Prize. 13 year old Ukrainian boy personification: more constraints (space on twitter, language barrier with Eugene) result in increased credibility
Deus Ex Machina interactive theater project in Austin, sms polling to a web UI to allow for story decisions
Communal entertainment as a cultural touchstone: In a world where everyone gets personalized entertainment, does it become harder to relate to other people? (No more watercooler conversations?)
Storium as a story generation human/software collaboration system

We had a great crowd for the conversation, and even managed to be “Hot” in the schedule. Thanks to everyone who was able to make it!

Games That Play Themselves

Some thoughts on RPGs and God games that keep playing when you aren’t watching, and what new hardware platforms like the Raspberry Pi and cheap tablets might mean for them.

A few days ago a new iOS app called Dreeps landed in my news feed, heralded with headlines like Maybe The Laziest RPG You Could Ever Play and A Video Game That Plays Itself. Dreeps is an app where a little robot boy goes on an adventure, Japanese RPG style. You set an alarm to tell him to rest, and that’s it. When the alarm goes off, he gets up and gets on with his adventure, fighting monsters and meeting NPCs. There’s pixel art and chiptune audio. Dialog is word balloons with squiggly lines for text. It’s all very atmospheric. You just don’t do anything, really, but watch when you want and suggest he get up when he’s resting after a fight.

Dreeps is a lot like Godville, a game I talked about in a post about Pocket Worlds back in 2012. They’re games that (appear, depending on the implementation) to be running and progressing even when you’re not around. While Godville does its magic with text, Dreeps has neat graphics and sound. They’re essentially the same game, though. A singular hero you have slight control over goes on a quest. In Godville it’s for your glory (since you’re their god), in Dreeps it’s to destroy evil (I think).

Both Dreeps and Godville are passive entertainment experiences, they’re worlds that are all about you, but not really games you play. They’re games you experience, or perhaps we need a new word for this kind of thing. While books and TV shows and music (although not playlists, as we’ve seen with Pandora) are hard to create for just one person’s unique enjoyment, games are great at that. They can take feedback and craft an experience just for you, and as we built more complex technology and can access more external datasets, they can get even more unique.

Imagine a game like Dreeps where the other characters (or maybe even the enemies) are modeled algorithmically after your Facebook friends (or LinkedIn contacts). Take their names, mash them through a fantasy-name-izer, do face detection and hue detection to pick hair color and eye color, maybe figure out where they’re from (geolocated photos, profile hometowns or checkins) for region-appropriate clothing. Weather from where they are, or where your friends live, maybe playing on an appropriate map. You could even use street view and fancy algorithms to identify key regional architectural elements and generate game levels that ‘feel’ like the places they live. That starts to get pretty interestingly personalized, though much less predictable.

Mike Diver over at Vice posted an article about Dreeps titled I Am Quite OK With Video Games That Play Themselves, where his main point was that he’s figured out that he’s actually bad at games, and it’s nice to have something where you can enjoy the progression without working about your joystick skills. Maybe Mike should spent more time with Animal Crossing, a game series I think Dreeps shares a lot of DNA with. In Animal Crossing your character inhabits a town that progresses in real-time. You can go fishing and dig up treasure and pick fruit and talk to the other inhabitants in your little village, but the world keeps going when you’re not playing, so if you leave it alone for a long time, you come back to a game that’s progressed without you (with the game characters wondering where you’ve been). Dreeps is like that, but without the active user participation. It’s like a zen Farmville. Take out the gamification, add in some serenity.

It feels like Dreeps could be a really fantastic lock-screen-game, if that’s a thing. You nudge your phone awake, and see your guy trudging along. He’s always there, in a comforting, reassuring, living way. Maybe Samsung or someone with some great cross-vertical reach could implement lock-screen or sleep-screen as a platform across TVs, phones, tablets, fridges, etc. That’d be something.

I was talking to a friend of mine about these kind of games yesterday, pondering where this is headed, and I mentioned that the experience almost feels like an Ecosphere. Ecospheres are those totally enclosed ecosystems, where aside from providing a reasonable temperature and sunlight, you’re a completely passive observer. There’s something nice about walking by and peeking in on it every once in a while. Something comforting about knowing that even when you’re not watching it’s going on about its fantastically complex business without you. But there’s also a spiritual weight to it, because it’s a thing that could cease to exist. I could cover the Ecosphere with a sheet or leave it out in the cold, I could delete Godville or Dreeps from my phone, or have my phone stolen, unable to retrieve my little robotic adventurer.

It isn’t a huge weight now that we carry with these sorts of things. In fact, I stopped checking in on my Godville character a few months ago, after over a year of nearly daily care. Sometimes you just lose the thread. But these systems are going to become more complex, more compelling. They’re going to have more pieces of ourselves in them. How would I feel if a friend of mine was a major character in Dreeps, always showing up to help me out, and then he died in real life? What if Dreeps decides to shutter their app, or not release an upgrade for the new phone I get after that? Would I leave my device plugged in, forever stuck at iOS whatever, just so the experiences could keep going? The Weavrs I created for myself back in 2012 are gone, victims to this onward march of technology and unportability of complex cloud-based systems. I’m fortunate that I never got too attached. Droops is an app, but there’s still a lot there outside of my control.

I’m particularly interested in where this stuff intersects with physical objects. Tamagotchis are still out there, and we’re building hardware with enough smarts to be able to create interesting installations. There’s an Austin Interactive Installation meetup I keep meaning to go to that’s probably full of folks who would have great ideas about this. Imagine a pico-projector or LCD screen and a RaspberryPi running a game like Dreeps, but with the deep complexity and procedural generation systems of Dwarf Fortress. Maybe a god game like Populous, with limited interaction. You’d be like Bender in Godfellas, watching a civilization grow. Could that sit in your home, on your desk or by the bookshelf, running a little world with little adventurers for years and years? Text notifications on your phone when interesting things happened. A weekly email of news from their perspective? As it sat on your desk for longer, would it be harder and harder to let go of? When your kids grew up, would they want to fork a copy and take it with them?

4 years ago there were no low-power GPU sporting Raspberry Pis or globally interconnected Nest thermostats or dirt-cheap tablet-sized LCD screens or PROCJAM. Minecraft was still in alpha, the indie game scene hadn’t exploded, the App Store was still young, procedural content generation was a niche thing. Now all those pieces are there, just waiting to be plugged together. So who’s going to be the first one to do it?

Book Review: On Intelligence by Jeff Hawkins

With On Intelligence, I find myself in the unique position of having heavily evangelized a book before I’ve even finished it. I read half of it and started buying copies for friends. This is something I’ve never done before, so if you’re busy, you can take a quick tl;dr, and assume that if you’re interested in how intelligence works, namely how the brain functions at a high level (learning patterns, predicting the future, forming invariant representations of things) and how we might functionally simulate that with computers, do not pass go, do not collect $200, go buy a copy (Amazon, Powells) and read it.

Still here? Good, because I have a lot to say. This isn’t really a book review, it’s more of a book summary and an exhortation to activity. You’ve been warned.

A Little Backstory

Earlier this year I went to OSCON, and at OSCON the keynote that impressed me the most was by Jeff Hawkins, creator of the PalmPilot and founder of HandSpring. Here’s the video:

As appropriate for an Open Source conference, Jeff’s company, Numenta was announcing that they were open sourcing their neocortical simulator library, NuPIC, and throwing it out there for people to hack on. NuPIC was based on the work Numenta had done on neocortical simulations since he wrote the book, On Intelligence, in 2005. NuPIC is software that simulates the neocortex, the sheet of grey matter on the outside of your brain where all your experiences live. 3 years of French? It’s in the neocortex. The ability to figure out that two eyes and a nose equals a face? The neocortex. The neocortex even has the ability to directly control your body, so that muscle memory you rely on to do that thing you do so well, like riding a bike or painting or driving a car? That’s all in your neocortex. It’s the size of a large dinner napkin (the largest in humans, but every mammal has one), is about as thick as 7 business cards, and wraps around the outside of your head. It is you.

Intrigued, I went to the full length session that the Numenta team presented…

One of their main demos was an electrical consumption predictor for a gymnasium. When initialized, the NuPIC system is empty, like a baby’s brain. Then you start to feed it data, and it starts to try to predict what comes next. At first, its predictions fall a little behind the data it’s receiving, but as the days of data go by, it starts to predict future consumption an hour out (or whatever you’ve configured), and it gets pretty good at it. Nobody told NuPIC what the data was, just like our DNA doesn’t tell our brains about French verbs, the structure is there and with exposure it gets populated and begins to predict.

At the end of the talk, their recommendation for learning more about this stuff was to read On Intelligence. So, eventually, that’s what I did.

A Little Hyperbole

The simulation, in software and silicon, of the biological data handling processes, and building software off of that simulation, is the most interesting thing I’ve seen since Netscape Navigator. Everything up to your iPhone running Google Maps is progressive enhancement and miniaturization of stuff I’ve seen before. Building brains feels different.

I have a Newton Messagepad 2000 around here somewhere. It had mobile email over packet radio with handwriting recognition in 1997. In 2001 I was using a cell phone with a color screened to look up directions and browse web sites in Japan. It’s all iterating, getting better bit by bit, so that when we look back in 10 years we think that we’ve made gigantic leaps. Have we? Maybe, but software is still stubbornly software-like. If I repeat the same error 10 times in a row, it doesn’t rewrite itself. My computer doesn’t learn about things, except in the most heavy handed of ways.

Writing and shipping working software is hard, there’s rarely time to focus on the things that aren’t necessities. In video games this has given us nicer and nicer graphics, since for the last few generations, graphics have sold games. Now developers are starting to realize there’s a huge swath of interesting stuff you can do that doesn’t require a 500 person art department and a million dollar budget. Sir, You Are Being Hunted has procedural map generation. Rimworld has AI storytellers that control events in the game world to create new experiences.

Looking beyond the game space, a few weeks ago I was talking with a large networking company about some skunkworks projects they had, and one of them was a honey pot product for catching and investigating hack attempts. The connections between deep simulations like Dwarf Fortress or the AI Storyteller in Rimworld and how a fake sysadmin in a honey pot should react to an intruder are obvious. If it’s all scripted and the same, if the sysadmin reboots the server exactly 15 seconds after the attacker logs in, it’s obviously fake. For the product to work, and for the attacker to be taken, it has to feel real, and in order to fool software (which can pick up on things like that 15 second timer), it has to be different every time.

One thing that these procedural and emergent systems have in common is that they aren’t rigidly structured programs. They are open to flexibility, they are unpredictable, and they are fun because unexpected things happen. They’re more like a story told by a person, or experiencing a real lived-in world.

I believe that to do that well, to have computers that surprise and delight us as creators, is going to require a new kind of software, and I think software like Numenta’s NuPIC neocortical simulator is a huge step in that direction.

Let’s Deflate That a Bit

Ok, so NuPIC isn’t a whole brain in a box. It’s single threaded, it’s kind of slow to learn, and it can be frustratingly obtuse. One of the samples I tried did some Markov chaining style text prediction, but since they fed each letter into the system as a data point instead of whole words, the system would devolve into returning ‘the the the the the’, because ‘the’ was the most common word in the data set I trained it with.

Neocortical simulators are a new technology in the general developer world. We’ve had brute force data processing systems like Hadoop, methods developed to deal with the problems of the Google’s of the world, and now we have NuPIC. The first steps towards Hadoop were rough, the first steps towards neocortical simulators are going to be rough.

It’s also possible that we’re entering another hype phase, brought on by the rise of big data as the everywhere-buzzword. We had the decades of AI, the decade of Expert Systems, the decade of Neural Networks, but without a lot to show for it. This could be the decade of the neocortex, where in 10 years it’ll be something else, but it’s also possible that just like the Web appeared once all the pieces were in place, the age of truly intelligent machines could be dawning.

Oh, This Was a Book Review?

It’s hard to review On Intelligence as a book, because how well it’s written or how accessible the prose may be is so much less important than the content. Sandra Blakeslee co-wrote the book, and undoubtedly had a large hand in hammering Jeff’s ideas into consumable shape. It isn’t an easy read due to the ideas presented, but it’s fascinating, and well worth the effort.

In the book Jeff describes the memory-prediction framework theory of the brain. The theory essentially states that the neocortex is a big non-specialized blob that works in a standard, fairly simple way. The layers in the sheet of the neocortex (there are 7 of them), communicate up and down, receiving inputs from your sensory organs, generalizing the data they get into invariant representations, and then pushing predictions down about what they will receive data about next. For instance, the first layer may get data from the eye and say, there’s a round shape here, and a line shape next to it. It pushes ’round shapes’ and ‘line shapes’ up to the next level, and says “I’ll probably continue to see round shapes and line shapes in the future”. The bouncing around of your natural eye movements gets filtered out, and the higher levels of the brain don’t have to deal with it. The next level up says, “This kind of round shape and line shapes seem to be arranged like a nose, so I’m going to tell the layer up from me that I see a nose”. The layer up from that gets the ‘I see a nose’ and two ‘I see an eye’ reports and says, “Next layer up, this is a face”. If it gets all the way to the top and there’s no mouth, which doesn’t match the invariant representation of ‘face’, error messages get sent back down and warning flags go off and we can’t help but stare at poor Keanu…

These layers are constantly sending predictions down (and across, to areas that handle other related representations) about what they will experience next, so when we walk into a kitchen we barely notice the toaster and the microwave and the oven and the coffee maker, but put a table saw onto the counter and we’ll notice it immediately.

As we experience things, these neurons get programmed, and as we experience them more, the connections to other things strengthen. I figure this is why project based learning works so much better than rote memorization, because you’re cross connecting more parts of your brain, and making it easier for that information to pop up later. Memory palaces probably work the same way. (I’m also half way through Moonwalking with Einstein, about that very thing.)

So, Have We Mentioned God Yet?

This is where things start to get weird for me. I grew up in a very religious family, and a large part of religion is that it gives you an easy answer to the ‘what is consciousness’ question when you’re young. Well, God made you, so God made you conscious. You’re special, consciousness lets you realize you can go to heaven, the dog isn’t conscious and therefor can’t, etc.

About a third of the way into On Intelligence I started having some minor freakouts, like you might have if someone let you in on the Truman Show secret. It was like the fabric of reality was being pulled back, and I could see the strings being pulled. Data in, prediction made, prediction fulfilled. Consciousness is a by-product of having a neocortex. (Or so Jeff postulates at the end of the book.) You have awareness because your neocortex is constantly churning on predictions and input. Once you no longer have predictions, you’re unconscious or dead, and that’s that.

That’s a heavy thing to ponder, and I think if I pondered it too much, it would be a problem. One could easily be consumed by such thoughts. But it’s like worrying about the death of the solar system. There are real, immediate problems, like teaching my daughter how stuff (like a Portal Turret) works.

Let’s Wrap This Thing Up With A Bow

I’m sorry this post was so meandering, but I really do think that neocortical simulators and other bio processing simulations are going to be a huge part of the future. Systems like this don’t get fed a ruleset, they learn over time, and they can continue to learn, or be frozen in place. Your self-driving car may start with a car brain that’s driven simulated (Google Street View) roads millions of miles in fast-forward, and then thousands of miles in the real world. Just like everyone runs iOS, we could all be running a neocortex built on the same data. (I imagine that really observant people will be able to watch Google’s self driving cars and by minor variations in their movements, tell what software release they’re running.) Or we could allow ours to learn, adjust its driving patterns to be faster, or slower, or more cautious.

The power of software is that once it is written, it can be copied with nearly no cost. That’s why software destroys industries. If you write one small business tax system, you can sell it a million times. If you grow a neocortex, feed it and nurture it, you’ve created something like software. Something that can be forked and copied and sold like software, but something that can also continue to change once it’s out of your hands. Who owns it? How can you own part of a brain? Jeff writes in the book about the possibilities of re-merging divergent copies. That’s certainly plausible, and starts to sound a whole lot like what I would have considered science fiction 10 years ago.

I finally finished On Intelligence. I have Ray Kurzweil’s book, How to Create a Mind on my nightstand. I’ve heard they share a lot of similar ideas. Ray’s at Google now, solving their problem of understanding the world’s information. He’s building a brain, we can assume. Google likes to be at the fore-front.

We could throw up our hands and say we’re but lowly developers, not genius computer theorists or doctors or what have you. The future will come, but all we can do is watch. The problem with that is that Google’s problems will be everyone’s problems in 5 years, so for all the teeth gnashing about Skynet and Bigdog with a Google/Kurzweil brain, it’s much more productive to actually get to work getting smarter and more familiar with this stuff. I wouldn’t be surprised if by 2020 ‘5+ Years Experience Scaling Neocortical Learning Systems in the Cloud’ was on a lot of job postings. And for the creative, solving the problem of how the Old Brain’s emotions and fears and desires interfaces with the neocortex should be rife with experimental possibilities.

NuPIC is on github. They’re putting on hackathons. The future isn’t waiting. Get to it.

Updates

Here’s a video from the Goto conference where Jeff talks about the neocortex and the state of their work. This video is from October 1st of 2013, so it’s recent. If you have an hour, it’s really worth a watch.

Book Review: Kill Decision by Daniel Suarez

Daniel Squarez‘s latest techno-thriller Kill Decision isn’t a happy book. It’s an especially unhappy book if you’re excited about quadcopters, RC planes, self-organizing swarm AI, or any of that neat, fun stuff.

Daniel’s first published book was Daemon, a novel about a programmer who, upon discovering that his time is up, creates a distributed dumb-agent network of actions and actors triggered by reports in news feeds. The thing that made Daemon so interesting wasn’t just that concept, it was that Daniel has a really good grasp on the technology, so everything that happened in the book kind of made sense. There was no magic bullet, it was all ‘oh, yea, that could work’.

Kill Decision is a book about drones, specifically autonomous drones that can kill. It was only a few years ago that I remember wondering when someone was going to strap a handgun (even a fake one) to a quadcopter and attempt a robbery by drone. Kill Decision is a book about just that, except the handgun is quadcopter optimized and the person getting robbed is the USA.

It’s been a while since I’ve read any popular techno-thrillers, but from what I remember, Kill Decision follows the arc pretty well. There’s a tough soldier type, a naive but smart audience proxy, a team of good guys for gun fodder, and a big bad. The pacing is good, the details are good, and the book keeps you guessing. I guess my only complaint is also the books point, that in the end, with a robot that can kill, it’s really hard to figure out who the bad guy is. In Kill Decision there isn’t a Snidley Whiplash twirling his mustache just off stage, at least that we get to see, and that lack of a direct villain gives the book a feeling of existential angst. The bots just keep coming, and in the end, there isn’t a clear win or loss.

Lots of thrillers are spy novels with more gadgets. They’re Jason Bourne, a lone operative outwitting the watchful, ever-present eye of big evil. It’s a big data dream, outwitting the system. Kill Decision is different. Kill Decision is a zombie novel, except the zombies are cheap, deadly, swarming technology.

If you can handle that kind of anxiety, and you like books about AI, maker, and military technology, Kill Decision is an easy recommendation. Also, go watch this video of Joi Ito interviewing Daniel Suarez at the Media Lab. Joi gives Kill Decision two thumbs up.

Kurzweil, Bot AI and the GoogleBoard

A few weeks ago it was revealed that Ray Kurzweil, pioneer of OCR, speech recognition and AI assistance tools, had joined Google to work on machine learning and language processing projects. My initial reactions were excitement (Google knows the time is ripe for this to happen), cynicism (big name matchups like this rarely work out like they’re supposed to), and last night, during a 12 hour drive from Santa Fe to Austin, curious speculation.

These days it’s rare to truly disconnect. We have the internet floating through the air at home, swirling around our mobile devices as we drive around town. Even in remote places we can read ebooks or listen to our music. When you’re driving through the lonely landscape of New Mexico at 11pm on a Saturday night, with a car full of sleeping people… technology leaves you to your imagination. So here’s my take on where a Google/Kurzweil mashup may take us.

The Stacks

Google’s an amazing company. I’m a die hard Apple product user, but even I realize that Google’s better positioned for the next 50 years. They’ve spent the last 15 years assembling a mindbogglingly good technology base. While Apple has been great at forecasting what people will use and making beautiful, easy to use versions of that, Google has spent the last 15 years figuring out what impact technology’s going to have on peoples lives, and building all the foundational technologies to make it happen. They’ve spent a ton of money and time building technologies that are hard to replicate.

There’s been a lot of talk recently about the five stacks: Amazon, Apple, Microsoft, Google and Facebook. They like to wrap you up in their ecosystems, but in reality, Google’s the only one doing the whole thing.

Amazon doesn’t have a real search option, and generally don’t do deep technology development. They’re a lot like Facebook in this regard, they’re agile, but not deep. They can give you a social experience, but they can’t really make your life better, only full of more content (Amazon) or better connected to your friends (Facebook).

Microsoft’s having trouble staying relevant, and while they have a good foothold in the living room, mobile’s abysmal and nobody gets Windows 8. I don’t think they really have a vision for where they want to be as a company in 10 years. They just want people to keep buying Office.

So that leaves us with Apple and Google. Apple has great product design, but they aren’t a deep software technology company. They can design great experiences, but that’s only an advantage for so long. If someone else offers a device that fundamentally does something they can’t match, that someone else (Google) can eventually catch up in design and ease of use. Just look at Google’s Maps app for iOS. Not a skeuomorph to be found. I think Android’s still too complicated for my parents, but it’s obviously getting better.

So we’re left with Google. They have a great technology foundation, gobs of really smart people, and more and more experience making what they build easy to use. And now they’ve hired Ray Kurzweil. Why? Because they want to leverage all this amazing technology they’ve built to be your life AI assistant.

The Predictive Technologies

Lets look at some technologies that Google’s built that will eventually be seen as the ancestors of whatever Kurzweil’s team comes up with. First, we’ll be communicating with it using our voice. Google’s been working on it’s voice technology for a while, including Google Voice Search (call a phone number, say your search, and get the results read to you), Google Voice Voicemail Transcription, Youtube Automatic Video Transcription, and even Google Translate (Speak english, hear Spanish!). Their Google Voice Search is better than Siri, in my experience.

Second, it’ll be with us everywhere (thanks to Android), and it’ll be predictive based on being continually active (thanks to Google’s massive computing capacity, and oddles of data at it’s disposal). An example of this is Google Now, but the Kurzweil version will be even better. Google has been really smart about letting developers build cheap Android devices, but almost all of them still go back to Google for email, calendar, etc. They’ve leveraged the market, but the customers are still theirs.

Third, it’ll reach out and touch other devices. While your phone might be your personal magic wand for the internet, it’s hard to share things on the phone screen, and there are all kinds of things we could do with larger displays. Google’s started doing this with it’s Airplay-like wireless display mirroring. The Google Nexus Q is essentially an admission that you need an easy way to share what you’re listening to when you’re with your friends. The device lets your Android device discover it and use it for output, you’re no longer limited to your android device’s speakers.

The GoogleBoard

Audio is a first step, and mirroring your entire screen is fine, but the future belongs to sharing. You want your friends to be able to come over to your house, and share their lolcat pic or funny video on your TV or other display without needing HDMI cables, or taking over the entire screen. You may want a note taking application to be displayed next to a streaming video, or you may want to play a game where everyone uses their android devices as controllers. For that, you need something smarter. You need something like… a GoogleBoard. (Thank goodness you just bought a hardware company.)

Imagine a display that fades into the environment. It may be small (a 15″ screen next to your door that notices when you walk by and shows you the weather forecast, your expected commute time and reminders) or big (a chalkboard in your kitechen or your refrigerator door). You won’t be watching movies on it, so display fidelity isn’t as important as TVs, but it can use the same production base, so they’ll be cheap. They’ll also be smart, they’ll run android like Google TV, they’ll be internet connected, but they’ll have a lot of features dedicated to sharing their screen space.

They’ll use bluetooth or nfc, they’ll probably have cameras (for google hangout/google talk video conferencing). They’ll be aware of your friends, thanks to Google’s permission system with a group blog bolted on top that’s masquerading as a social network (Google+, if you didn’t catch that). You’ll own the Google Board, and you’ll be able to say ‘everyone in my friends circle can use this’. Your friends will come over to your house, and they’ll be able to magic up a video or graphic or app on their Android device, and fling it up to the GoogleBoard, where it can use the whole thing (if it was empty), or share space with other users already using the Board. Depending on your preferences, the app may utilize the Board’s network connectivity, or the display may just be that, a display that is driven from your Android device like an X-Windows app, with all the network traffic going through your Android device’s backhaul. Maybe Android will be smart enough that it can price-optimize it’s network traffic, using it’s own wifi when it can, your friends wifi when it’s available, or LTE as a fallback.

People love interesting information, and a GoogleBoard would be uniquely suited to provide constant global metric displays. You could have a home dashboard on one, that shows up when nobody’s using it. Your family’s pedometers and scales feed into little personal health meters on the side. ‘Dad, you should probably lay off the pringles, your avatar’s looking a little sad.’ ‘Hey, the fridge is out of milk! (and Target’s having a sale, thank you Google Ads, touch here to add it to your delivery order)’ ‘You play a lot of Kruder and Dorfmeister through your Nexus Q, and Thievery Corporation’s going to be in town, do you want tickets?’

When the GoogleBoard isn’t being used, they may use neat simple technologies to be energy efficient art displays, white boards (I’m imagining capacitive chalk markers that you can see when the Google Board is ‘off’, but are transparent when the google board is on.) They could be coffee tables or kitchen tables. You could play an RTS or card game with the rest of your family over dinner. You could watch a funny video and throw it over to the living room TV, if you really wanted everyone to see it.

Google Bot Avatars

So now that we’re all sharing these displays at once, we need a way to identify who’s who, and now that our android devices (and by extension, google activity) is exposed to other people through a Kurzweil-derived AI, maybe it becomes time to give the thing a name and an avatar. This AI Bot Avatar is your personal concierge for everything Google can offer you, you talk to it, it talks back, it lives in the cloud, but it’s snuggly at home on your phone or Google Glass device, because that only belongs to you. It can pop it’s head up on your home display devices (your Google Boards and Google TV, or your Berg Little Printer), and it can be an invited guest on your friends Boards or Boards and computers at work or school. Since it has a unique name, you can summon it in the car, and all your friends can use your Android-driven car bluetooth speaker system to talk to their devices and ask for their music to be played. Or display things on the Board in the self-driving car’s ceiling.

So lets say my kid’s AI bot avatar is Princess Fluffypants, because she’s a kid and that’s how she rolls. Her AI assistant pulls in stuff from Khan Academy or Make Magazine Youtube videos (because it knows she’s interested in science, but could use some help in math), it keeps her up to date on trends, including what her friends are watching, and gives her the latest news. When she communicates with her AI bot, the bot has a personality (maybe the kid picks ‘Royal’ since Fluffypants is a Princess). Bot grooming and accessorizing becomes a thing, because the Google AI Bot has all of Google’s knowledge behind it, and can probably be programmed and modified like android apps.

My AI bot may be more serious, maybe I’m really into P. G. Wodehouse, so I have a Jeeves, and maybe Stephen Fry’s making some extra scratch by lending his voice to my avatar set. Maybe I even have multiple AI bots, since it’s weird for Jeeves to be talking to me about football or my interest in crumping (or maybe that’s hilarious). But that’s a topic for another blog post.

Application Network Portability

One requirement that this raises is the need for the applications that run your bot, or the applications your bot runs, depending on your perspective, to be network portable. You need to be able to execute code in the Google cloud, things you want to happen regularly or things that benefit from rapid access to large volumes of data, but then you also want software execution on your device, or you want to push a little applet over to a TV or GoogleBoard, since it’s inefficient to render the graphics on your phone and then push them when the display could run the app itself. Or maybe the display reports it’s capabilities and the Phone decides whether to push the applet (the display’s fast enough for what I want) or just use it as a display (the display’s two years old, and isn’t fast enough for this application).

Lots of iOS developers (myself included) gnash their teeth when they think of the insane panoply of Android devices. Testing software at all the resolutions, form factors, and aspect ratios is incredibly painful, but in a network transportable world, maybe that was a smart decision. You never know where your app is going to be displayed, on a landscape 16×9 screen, in a portrait 16×9 area on a larger display, on a square car dashboard, so you design for flexibility.

The Dark Horse

So Google seems pretty well positioned here. Amazon doesn’t seem like a serious player. Facebook will continue to make money, but needs to branch out if they want to control more than just the social conversation. Microsoft is chasing it’s tail, trying to stay relevant. Apple will continue to make beautiful, amazing devices, but they may not have the technological muscle to pull off the next level of magical user experience. Already they have to partner for their most useful features, and that isn’t a good place to be.

There’s one technology company that’s looking like it may be a dark horse entry into this technical re-invention, though, and that’s Wolfram Research. Wolfram|Alpha powers Siri’s more complex question answering, and they’re really gung-ho on their algorithmic approach to the world. With the amount of user generated search data they’re collecting, Wolfram|Alpha could get really good, really fast. If you aren’t reading Stephen Wolfram’s blog, you should. At SXSW last year he mentioned that he wanted to get Mathematica into more areas, to make it more of a foundational piece people could build on. If Wolfram Research was able to turn Wolfram|Alpha and Mathematica into a really good open source development platform for bot and internet search applications, you could get something really powerful.

Wolfram Research is privately held, and I don’t believe that Stephen Wolfram and whoever else owns pieces of it will sell. Any non-Google stack should be slavering to get their hands on it, but being private may keep it out of their reach.

Conclusion

Whatever comes of the Google/Kurzweil partnership, be it really interesting, a spectacular Xanadu-esque failure, or a quiet Google Labs-esque decommissioning, it’s worth paying close attention to. The future doesn’t magically appear, people sit down and build it. There’s nothing stopping any of the technologies I’ve mentioned from appearing in the next few years, and Google’s in a prime position to make it happen. While a lot of it is inspired by science fiction, successful science fiction grabs the imagination like a good early adopter product should. There aren’t many things I’d consider dropping everything to work on, and intelligent network-native bots are one of them. When they appear they’re going to radically remake our daily life experience.

Life in the Weavrs Web

Jeff Sym lives in South Austin and likes Indian TV dramas, dubstep inspired remixes and the Austin Children’s Museum. Keiko Kyoda lives in Japan, likes to read old travel books and wants Condensed Milk for dinner. They tweet. Sometimes they even post things they shouldn’t.

Jeff and Keiko didn’t exist yesterday.

The first time I failed the Turing test was 1993. I’d dialed up to a BBS in Austin, a one-line operation probably running out of some guys bedroom. There was an option in one of the menus to chat with the sysop. It was an ELIZA style bot. It took at least a screen full of text and growing irritation for me to realize I was talking to a machine. I don’t remember a lot from 1993, but I remember sitting there in front of my 14″ glowing CRT, feeling incredibly dumb. (A few years later I upgraded to this NeXT Cube.)

Artificial intelligence is only as convincing as the data behind it. Back in that relative stone age the system could only echo back at me what I’d written or ask open ended questions. “How does that make you feel?” Watson read all of Wikipedia before it (he?) went on Jeopardy. If you started talking to Watson about cars, I bet it/he could respond with some really interesting trivia, and you could chat with it/him for a while before you realized you weren’t talking to a person.

The most visible ‘ask me a question and I’ll give you an answer’ system is Apple’s Siri. Siri can tell you what the weather’s like outside, and she’ll soon be able to tell you what year and model of car you just snapped a picture of. Siri could listen to you and tell if you’re angry, or if you had a really great day yesterday, based on your tweets and Facebook posts. Siri could team up with Mint to watch your bank account balance, and suggest that hey, you aren’t investing enough for retirement, maybe you don’t need that thing you just price compared on your phone. Maybe you should put that money into your Roth IRA instead. This is all possible because these systems have access to fantastically more data than they used to.

Jeff and Keiko are Weavrs. You create weavr bots by selecting a gender (or object), a name, and a collection of interest keywords. Then you define some emotions. _____ makes me _____ when I’m at _____. You can tell weavrs where they live, and they’ll wander around their neighborhood. They utilize public social APIs (flickr, last.fm, twitter, google local), driven by some black box keyword magic, to find and post things they like. You can add pluggable modules to weavr’s to say, post their dreams. Over time they can develop new emotions about different things. There’s even a system for programming a Monomyth into their lives.

Weavrs exist on their own. You can ask them questions, but you can’t tell them ‘I like this, post more like this.’ The developers of the Weavr platform consider this to be important. Weavrs evolve and grow without your direct hand guiding them. I can understand why they didn’t want to allow ‘more like this’ feedback. It makes the entire system more complex, but it’s obvious that having more full featured persona creation/control options is going to be a big part of the future of social bots.

Weavrs most public impact so far (at least as far as I can tell) reveals a bit about how people will likely react to this sort of thing. Author of Men Who Stare at Goats and The Psychopath Test, Gonzo Journalist Jon Ronson (@jonronson) did a bit on his video show about twitter bots. The Weavr folks found out and using the contents of his Wikipedia page, created a @jon_ronson Weavr. The result was somewhat predictable: much gnashing of teeth. There’s an excellent article about this, and Weavrs in general, on Wired UK.

Twitter has over 140 million active users. A large number of these are spam bots, designed to convert ego (retweets and replies) into $ (clickthroughs). What we don’t really know, and what may in fact be unknowable soon, is how many of these are bots of a different kind. How many of them exist just to exist. To learn, grow, develop. We heard a lot about companies creating armies of real-looking twitter accounts for nefarious purposes during the Arab Spring. It doesn’t take a lot of work, once you have a valid social model that can be fed keywords, to create a twitter bot the simulates the interest of every ‘person’ that Wikipedia has an entry for.

What we don’t hear about, and I don’t think is discussed enough, is the non-nefarious potential for these independent personas. Imagine a platform somewhat beyond Weavr. Weavr 2.0, maybe. It ties into more social platforms. It has artistic taste (or not). Maybe it takes walks through its neighborhood, and snaps out ‘photos’ from segments of google street view images. (Jeff Sym liked this picture today, while he was wandering around downtown Austin.) Maybe it goes on trips, setting arbitrary routes through hot points. Maybe my (should I even call it ‘my’ anymore, except that in some way perhaps I’m responsible for it, like a child?) Weavr that’s really into Information Security decides to take a road trip to DEF CON. Maybe because he’s also a bit of a conspiracy theorist, he decides to drop by Roswell on his way, maybe he looks around in Google Street View and takes a picture. Maybe because I’ve stirred the 3d Visu-chromasome pot, he has an appearance (and taste in clothes), so maybe he puts himself into the picture (apologies to Charles Stross).

Wolfram Alpha (that powers the ‘question/answer’ part of Siri with a >90% relevancy rate) is 20 million lines of Mathematica code. You’d need a lot less than that to do what I just outlined. You need an event parser. Easy, the events are already online. You need a map, and the ability to search for hotspots of keywords along the route or near an area. If I did a keyword search for ‘conspiracy’ between Austin and Las Vegas, don’t you think Roswell would pop up? If I did a search for clusters of photos taken in Roswell on Flickr or some other social photo site, I’m sure I’d find the geo location and general object background of something interesting. Analyze light and time of day, pose and place model, render and voila. Picture postcard. Get it printed and mailed from New Mexico with a pay-as-you-go errand service. Boom, your virtual persona just became real.

These personas would be great for directed research: I need a ‘me’ who lives in Amsterdam and loves to take pictures and eat cheese, since I’m going there in 6 months and I want to get a feel for the place. What if you created a relationship engine. X is my Y, I’m in love with Z. You could have your own little soap opera that evolves online every day. It ties itself into current events. It evolves in ways you can’t predict, because it feeds off the now.

I’m going to ignore the whole ‘but if you don’t know they’re fake’ problem for now. On one hand, that’s kind of the point. On the other, it sucks to be duped. In the future (and present) you can’t trust anything you see. It’s probably time to get over any assumption of authenticity.

One thing that’s nagging at me, though, is death. A lot of us know people on twitter or Facebook who’ve died. In the coming years a lot more of us will. We’ll be reminded of birthdays, of past events, things will pop up in our timeline because someone commented a few years later. Their memory continues to exist, but in a concentrated form. Unless their profile’s deleted completely, they continue to exist. If there was a collection of directed, trained bots behind the scenes, they might even continue to post. They might be posting things relevant to their interests 10 or even 20 years later, if the systems keep running that long. As it is, Jeff and Keiko will only ‘die’ if Weavr shuts down. If they had control of their own codebases and knew how to set themselves up as a lightweight every-2-hour process on a system somewhere, maybe they could pay for their own existence through clickthroughs and blog ads… indefinitely.

Weavrs want to continue to exist, as well. Right now you can clone them, in the future they may, after some length of existence, spawn some kids with variations on their ‘genetic makeup’. Some kids will be interesting, attract attention and flourish, some will be too boring or too weird and disappear. The parent weavr continues to exist through it’s children.

There’s the school of thought that all people have several people inside of them. You have a ‘masculine’ take-charge person, maybe a ‘feminine’ artistic, caring person, maybe a young zany person and an older, wiser person. They all make up you, but maybe with these technologies one day soon you’ll be able to manifest them more concretely. You could have an inner circle of very directed Weavrs. Maybe to maximize their inventiveness you’ll make deals with them. More freedom for them, wider results sets for you. The deal with your wise, older persona, in exchange for the investment tips and long-range perspective, is that it gets to virtually go down to Florida every winter. Maybe your virtual young, wild persona, in exchange for keeping you up to date on the latest fashion trends and music recommendations, gets to stay out late and virtually attend hot underground shows. They’re not just agents, they’re symbiotes.

These autonomous net entities, these ghosts in the social web or e-horcruxes, whatever you’d like to call them, aren’t going back in the box. We have to learn to deal with them, and due to social connectedness and meaning being a currency in our society, whoever figures out how to utilize them best is going to have an advantage. Businesses and marketeers will take advantage after the artists finish tinkering. Someone’s already using Weavrs to create market segment identities (PDF) for the cities in China with more than a million people (there are 150 such cities, too many to look at individually).

We’re all familiar with code that runs ‘for us’. Flickr, McAfee, these services run with our content or on our computers, but they don’t really run for just us, and they don’t exist independently… yet. One groundbreaking thing that Weavr is moving towards is removing the AI logic from the content (Weavrs pull from the web and post back to it, but they don’t exist in a walled garden like Flickr, they exist outside of it and talk to it via APIs). Eventually I think we’ll see some open source or self-runnable version of this, an agent that lives wherever you want. Once my dependency on an outside software provider for the black box is gone, I’m free to integrate whatever bits I like (fork that thing on GitHub!), and work towards a social agent that can exist for as long as someone keeps the lights on.

Postscript 1:

I just had a weird thought. Irma and I have noticed that our Weavrs post a lot of things we’re interested in (or find cool/neat). Since we created them, they feel like an extension of ourselves, so there’s a personal ownership angle to the things they post. “Oh,” I say, “this bot is like me.” I don’t say that when my friends post things, though. I don’t say, “Wow, this social appendage of me is like me.” I suppose someone really ego centric would say that, but we consider our friends to be independent entities. We know we don’t control them, and unless they’re our brothers or sisters, we probably didn’t have a hand in how they initially developed. Our Weavrs, on the other hand, feel like an extension of ourselves. I’m not sure what that means, but it’s a weird thought on individuality and influence domains.