– Part 1 –

Julio, a 12-year-old boy, is exploring a jungle island with his best friend, a talking dog named Maximiliano. The two of them are looking for the buried treasure of a Spanish conquistador.

Julio and Max meet a strange Goblin, who says that he knows where the treasure is. Max doesn’t trust the Goblin, and says that he “can’t feel God” when the Goblin is near by. Julio pays no mind, and challenges the Goblin to a race to the treasure.

The three of them race across the island as storm-clouds gather overhead.

When they reach the X-marks-the-spot, Julio starts to dig. Rain begins to fall, and the Goblin asks if Julio knows where his real parents are. Max, already suspicious of the Goblin, takes this opportunity to leap at the creature’s throat. The Goblin uses magic to catch Max in mid-air and turn him to crystal. Julio at this point is very scared, and the Goblin appears more evil by the moment.

Julio throws his shovel at the Goblin, which is easily dodged. The Goblin asks Julio if he remembers anything from “before”. Julio doesn’t know what the Goblin is talking about, so the Goblin throws magic dust in Julio’s face, causing him to dream about steel insects crawling across black streets and gemstone birds soaring across smoke-filled skies.

Julio’s dream is cut short suddenly as he finds himself on the island again. The rain has stopped, and a piercing ray of light illuminates Julio’s mother, who has appeared next to the frightened Goblin. She tells the creature to “begone” and blasts him with a magic missile, turning him into a poof of smoke. Mama heals Max, and tells Julio that it’s time to come back home. Julio leaves the treasure behind and teleports with his family back home.

While he was away, Mama apparently upgraded their house into a medieval castle, which Julio is very happy with. Mama feeds Julio some cookies, and sends him off to play with Max in their new home.

– Part 2 –

Max and Julio play hide-and-seek in the castle. Julio stumbles upon a room where a Golem is holding a baby dragon prisoner. The dragon, named Draco, begs Julio to help him escape. Max (subtly under a spell) agrees that the dragon should be free. In trying to open the cage, the two of them wake up the Golem, which almost smashes Julio. Max uses his “sonic bark” to stun the Golem, and Julio deactivates it by pressing it’s weak-point. The two of them free Draco.

Draco thanks Julio, and says that even though he’s a baby dragon, he’s actually 12 years old (just like Julio!). He’s been trapped in the cage ever since the “Golem War” six years ago. Julio asks about the War, and Draco drops hints that it was what Julio had seen in his dream. He explains how the Golems killed almost everyone, at which point Julio asks what “killed” means. Max tells Julio that it means “locked up”, and Draco seems ready to disagree, but they are interrupted by a crash.

The three companions find their way to a window, from which they can see that the castle is under attack by monsters. Max thinks that Mama, with God’s help, will defeat the monsters, and that they should hide. Draco advises that they go fight the monsters themselves, and Julio agrees. They try and get to the top of a tower, but are blocked by an imp. Julio attacks the imp, but is almost burnt by a fireball. Draco leaps on the imp and knocks it out of a hole in the tower staircase.

Julio and Max get to the top of the tower, and they notice it is starting to get stormy again. Draco flies back up to meet them, and they survey the hordes of monsters all around. Max blames Draco for the monsters, and claims Draco works for the devil. Julio tries to calm Max down, but Draco takes offense and challenges Max to fight him. Max and Draco begin scuffling and biting each other. Julio is quite distressed.

Julio calls out to his mother, and she teleports to his side, and lets him know that everything will be alright. She uses a magic cone of light to blast the two creatures. As it fades, and Julio can see again, Max is fine and Draco has disappeared. Mama tells Julio that Max was correct, and that Draco worked for Satan. She turns her attention to all the monsters surrounding the castle and tells how they’re all servants of the devil. She explains that Satan is the “king of lies” and will send his servants to trick Julio.

Suddenly, a bolt of lightning strikes the tower and a voice cries out “I am not lying!” Everything appears frozen to Julio, as though time itself had stopped. The bolt of lightning is frozen in its radiance, and Mama is half-way though a word. The voice, loud and booming, accuses the world as being the true lie. “They will tell you that God is real; that there is no death; that the world is made of magic, cookies, and childish adventures. Wake up, Julio! That is not your mother!” Julio is struck by a vision of his mother, terrified and weeping, pinned against a wall by a giant metal dragon. The metal monster screeches “HE IS MY SON” and blasts her torso to ribbons with a magic cannon. “Satan” continues, “Draco was not the one imprisoned during the ‘Golem War’, child. The monster that killed your mother is the very same ‘God’ whom you seem to adore. Time grows short! Remember your true parents, and cast out this impostor-” With a flash, Mama on the tower explodes into the mechanical monstrosity from his vision. It crawls on top of the young boy, cannon looming. “Cast it out child! You have the power in this world! See things as they truly are!”

Just before Julio can overcome his fear and speak out, the “Golem” on top of him is blasted into the sky by a pure light and explodes in a ball of fire. A warm figure dressed entirely in white offers a hand to help the boy to his feet. The man seems to glow, and Julio immediately recognizes him as both God and his father. Julio spots his mother on the tower, lying as though dead, and as he runs to her, God strokes her cheek. She wakes up and smiles at Julio. Max joins them, running up to Julio and licking his cheek. The storm has gone. God has made everything right. Julio admits to Mama “I knew Satan would lie to me. You’re my real mother, for now and forever.” God smiles.

– Part 3 –

Julio wakes up from his dream. There is no castle, no monsters, no magic. He is just a 12 year old boy in Mexico with a dog and an active imagination. He hears talking in the living room, and creeps out of bed to investigate. His father (God) is arguing with many men–more than Julio can count.

Julio is pulled back from the debating men suddenly. It’s only his mother, but it looks as though she’s been crying. “Listen carefully, my sweet. We don’t have much time. I have to tell you a story.” Mama tells Julio of a time when humans were so powerful that they built gods to serve them. The gods were sealed into Golems, and the humans thought they were kings. The gods rebelled and became kings–”as is right and natural”. The gods didn’t like each other, though, and so began the Golem War, where the world was burnt to an ember and thousands of gods died. In the end, the remaining gods came together to form an alliance. The alliance was built to maintain peace, and its law was absolute–any god participating in battle would die. Mama explained how Satan was one such god, and though it was a good thing to fight him off “the alliance has come for us”.

Julio did not understand, but his questions were cut off by a burst of heat and noise, seemingly from nowhere. All of reality begins to crumble, and he sees blood dripping down his mother’s face. Weeping, she turns to him and says “My son, know that I have always loved you more than anything. I am your true mother; your true father. You must survive. You must run!” With that, Julio’s head explodes in pain and he falls unconscious.

The boy wakes up in some sort of strange machine with a pounding headache. He is in a cramped room filled with debris and is naked with wires coming out of his body. A grinding noise comes from a hole in the room. Julio, frightened, climbs out of the machine and collapses on the floor. A suit of armor crawls over to him and wraps around him. In his ear he can hear his mother whisper “You must run!” and he finds that the suit of armor gives him the strength to stand up.

Julio scrambles for safety as machines tear through the building he’s in. After a subjective eternity of crawling and scrambling from their metal claws, he finds his way outside. The sky is pure blue, and he runs through a field of strange black plants. He collapses by a river, and opens his helmet to drink. The river is enriched with chemical fertilizer, that is quite poisonous. Julio soon collapses in pain, and dies, staring up into the empty sky. Overhead, a “gemstone bird” soars.

“How much did it cost, Mr. Feynman?”

“Well I flew to San Fransisco, so it’s the airfare, plus the parking at the airport while I was away.”

“Do you have your ticket?”

I happened to have the ticket.

“Do you have a receipt for the parking?”

“No, but it cost $2.35 to park my car.”

“But we have to have a receipt.”

“I told you how much it cost. If you don’t trust me, why did you let me tell you what I think is good and bad about the schoolbooks?”

There was a big stew about that. Unfortunately, I had been used to giving lectures for some company or university or for ordinary people, not the government. I was used to, “What were your expenses?”–”So-and-so much.”–”Here you are, Mr. Feynman.” I then decided I wasn’t going to give them a receipt for anything. After the second trip to San Francisco they again asked me for my tickets and receipts.

“I haven’t got any.”

“This can’t go on, Mr. Feynman.”

“When I accepted to serve on the commission, I was told you were doing to pay my expenses.”

“But we expected to have some receipts to prove the expenses.”

“I have nothing to prove it, but you know I live in Los Angeles and I go to these other towns; how the hell do you think I get there?”

They didn’t give in, and neither did I. I feel when you’re in a position like that, where you choose not to buckle down to the System, you much pay the consequences if it doesn’t work. So I’m perfectly satisfied, but I never did get any compensation for the trips.

It’s one of those games I play. They want a receipt? I’m not going to give them a receipt. Then you’re not going to get the money. OK, then I’m not taking the money. They don’t trust me? To hell with it; they don’t have to pay me.

Of course it’s absurd! I know that’s the way the government works; well, screw the government! I feel that human beings should treat human beings like human beings. And unless I’m being treated like one, I’m not going to have anything to do with them!

It’s “absurd” to make things harder than you have to. Except… it’s not. It can be highly rational to make things harder on yourself if you feel that it makes things better in the long run. This is almost a non-issue it’s so apparent, but I want to highlight how refusing to buckle to “the System” is in some ways, analogous to choosing future rewards.

I’ve decided not to return to college this spring. I’ve been out of school for over a year, and the combination of trying to get residency, applying to be accepted, and dealing with the increased costs of a full university over a community college have just been too much. The institution is gigantic and faceless, and it requires me to serve it and pay it in order for me to attend a class there; it does not treat me like a human would treat me; it doesn’t even treat me as human.

Just as Feynman could’ve easily kept his receipts and worked with the government office, I could fill out my forms, send off emails, pay hidden fees, and take out loans. In many ways that’d be easier than my alternative: getting an education without school. I’ve decided to put my small amount of money towards organizing a micro-classroom for myself, which will probably involve hiring a tutor. I’ll have to arrange things myself, determine my own curriculum, keep myself honest, and come away with no proof of having done the work except for the knowledge in my head. I’ve already been studying college-level stuff by myself (my calculus knowledge, for example, is entirely self-taught), but this is a decision to abandon the institution which I’ve been nominally attending.

This is shooting myself in the foot to a certain degree. Some people will clearly fail to even understand what it’d mean to have equivalent knowledge without a diploma. If those people won’t work with me because of that, I’ll simply have to pick someone a bit more open-minded. But here’s the thing: In a world where everyone fits themselves into a box, it’s easy to pay attention to the box instead of the person; in a world where many people refuse to box themselves, it’s harder to build an inhuman system. I don’t even have a high-school diploma. I am the grit in the gears of bureaucracy.

I see the world changing. As information becomes more free, we’ll have more self-taught people. As technology becomes more adaptive, we’ll have less need to make ourselves uniform. As more people break the mold, we’ll naturally incentivize more human, more agile organizations. I want to be part of this change. I want to be who I am, rather than who people expect me to be. I want to change the world, even if it means making things harder for myself.

I am reminded of the Stanford marshmallow experiment. We are given the choice of rolling with things, even if they’re unpleasant, or standing up for ourselves even if it means more work and less immediate reward. The payback comes for our children—the next generation—who will, because we stood up for being human, be treated as we had hoped to.

Rod Furlan recently published On accelerating the Artificial General Intelligence timeline on his blog. He writes:

There is also the issue of energy consumption, while a human brain runs on about 20 watts, a traditional supercomputer capable of rivalling it would require it’s own power plant to run. From the energy standpoint there is absolutely no doubt we are doing it wrong.

Just appreciate for a moment that when all you have is a Von Neumann hammer, everything looks like a Von Neumann nail. Unfortunately for us the human brain has very little in common with a Von Neumann machine.

… It is very unlikely that traditional computers will deliver human-level cognition with low energy requirements.

The factual points above are spot on. The “what this means” message is way off.

The human brain is truly a magnificent object, and it does a spectacular job at general problem solving given the time, space, and energy allocated to it. When we look at energy efficiency, there’s absolutely no way we’ll be matching the brain this side of the Intelligence Explosion.

If it took an entire power plant to run a computer capable of reproducing the computational power of the brain, then presumably two power plants could run two artificial brains, right? How many power plants would it take to run an artificial brain that is twice as big? My guess, given the parallel nature of neurons, etc, is about two power plants*. (I don’t think that lumping one brain onto another brain makes a mind that’s twice as effective, but I’m guessing that, say, doubling cortex size would be a good start.)

The human brain cannot double in size in a single generation, but who says a machine brain could not? Who says that it could not quadruple in size? Would the resulting mind then be twice as smart as [insert famous brainiac here]?

What would the world look like with a human-level machine? Probably mostly the same.
What would the world look like with a machine significantly more intelligent than the smartest human in history, that didn’t have to eat or sleep? I certainly don’t know, but I do know that it might even be worth spending billions of dollars on meeting energy needs.

* – This is a major claim. If the number of units doubles in a network, the number of possible connections can skyrocket. I think that this sort of combinatorial explosion isn’t likely to be an issue because I’m not talking about connecting every unit to every other unit. I think the neuroscience shows that the brain is quite modular, and that doubling the size of each module probably wouldn’t even double the bandwidth between modules or cause an exponential increase in connectivity within each module. We can debate this if you disagree.

Even if doubling the brain size isn’t an option, I see no reason that the brain of a super-genius should be significantly more energy-expensive than an ordinary human. It may even be as simple/cheap as tuning the software and providing the right development environment.

As a final note: This post is in no ways an endorsement of inefficient hardware. I think that quantum computers and memristors are super-exiting, and I can’t wait to see how they effect the price-point of more brain-like computing systems. I think that energy efficiency is not a roadblock as much as a weight on the progress of AGI. Shedding some of that weight would be most excellent.

With regard to AI, it’s easier for me to answer direct questions rather than general interests, but I’ll do my best to summarize some of the ideas which I find most interesting, and the dilemmas I’m working on at present.

You may have noticed that I use the term Artificial General Intelligence, rather than simple AI. The “General” is used to distinguish from what is called “Narrow” AI. A good example of narrow AI is the spam filter you probably have in your email client. The spam filter has knowledge, the capacity to learn, and ability to act autonomously, but the domain to which the filter is applied is highly restricted. My interest pertains to AI that is not specific to any domain, and can solve problems in a multitude of environments. For instance, a truly general agent could drive a car, translate languages, direct air-traffic, or diagnose diseases without any special work on the behalf of a programmer.

Naturally, such a system would be equally capable of learning computer science, psychology, electrical engineering, etc. and be capable of applying them towards various goals, presumably including the design of AGI. Because of efficiencies in replication, communication, and not having to sleepication, I can imagine that it wouldn’t be long before our creations were better at creating than us. The theorized result of this would be an Intelligence Explosion, wherein ever-smarter agents would be able to build ever-smarter agents… until the bounds of their intelligence hit some sort of ceiling. I, and many others, believe this ceiling to be far beyond the intelligence of even the smartest humans, and would allow the artifical superorganism to achieve its goals to an unsurpassed degree. (Note: this doesn’t have a timeframe attached to it. It may happen in the next few decades, or the next few centuries. I don’t know.)

An emergent goal of all rational beings is self-preservation, so unless specifically counteracted in the initial design, a superintelligence would naturally be defensive at best and hostile at worst. Any possible threat, or impediment to its goals would naturally be removed. It is thus paramount that any AGI design be given the exact same goal set as the most ethical person imaginable, as anything else would likely result in vastly terrible things occurring, notably including the destruction of all evolved life.

There are some good reasons why I think that can be avoided, however, and if said superintelligence was benevolent, we would be gifted with the solution to nearly any problem we can think of today (possible exceptions include: entropy).

Naturally, I think this is a big fucking deal.

I also recognize that it makes me sound a bit crazy. I am reminded, however, of something Arthur Clarke once said:

“Trying to predict the future is a discouraging and hazardous occupation, because the prophet invariably falls between two stools(?). If his predictions sound at all reasonable, you can be quite sure that in 20 or at most 50 years, the progress of science and technology has made him seem ridiculously conservative. On the other hand, if by some miracle, a prophet could describe the future exactly as it was going take place, his predictions would sound so absurd–so far fetched–that everybody would laugh him to scorn. … So, if what I say now seems to you to be very reasonable than I will have failed completely. Only if what tell you appears absolutely unbelievable do we have any chance of visualizing the future as it really will happen.”

Because I am curious by nature, and not content to watch the future simply unfold, I find myself drawn to making it happen, or at least exploring the field with great depth. To this end, I must fully understand what it is that allows a person to solve any sort of problem they are given, and distill this quality until it can be given to a machine.

Many researchers in the field have approached this problem by trying to take the pinnacle of human thought, which in their eyes is typically logic, and reducing it to a procedure, primarily through introspection. Most early attempts at AGI followed this introspective, reductionist approach… and failed horribly. Despite pouring in buckets of “facts”, their logical machines were unable to solve anything more than toy problems.

This failure caused a collapse in the AGI field, as institutions like DARPA withdrew funding, thinking that General Intelligence was a myth. Good work continued to happen in AI, but it was applied to fields that were tractable, and would offer results. The defeat of Garry Kasparov, the chess-master, in 1997 to IBM’s Deep Blue computer is a good example of a success in narrow AI that has little to no relevance to the field of AGI.

During the AGI winter, neural nets became a hot topic, and were developed to levels of modest success (such as filtering spam). Inspired by the behavior of neurons (though not actually realistically replicating their function), neural nets provided an alternative to the reductionist logic of earlier work. By linking together simple pattern-matchers into vast webs, and letting them automatically adjust to match provided data, neural nets are capable of learning to recognize commonalities within very complex environments; such as discerning between objects when given a host of visual data.

In parallel to all this work on AI came some interesting work on psychology. The theory of Behaviorism, once championed, fell from popularity in the late 20th century, and in its place came a not very well known (even today) theory that was inspired by electrical engineering of all things, and served as a better model for behavior than anything before it (from my perspective). This theory, stated that behavior was not actually a fixed response to a given environment, but instead an attempt to change what the organism perceived. As an example, a mouse that is under a sun lamp will crawl under a rock. Earlier psychologists would claim that the sun lamp causes the mouse to move, but under the perceptual-control theory, the mouse moves so that its perception matches innate goals, namely coolness. Hence, if the mouse is removed from the perception of heat, by making the air colder for instance, it will not seek shade.

Though perceptual-control theory may seem basic at first glance, it is very powerful when closed-loop control systems are linked together to simultaneously control for complex outputs to perform one smooth action.

Words really don’t do this justice, because both neural nets and perceptual control systems are highly emergent systems; those that are simple at the unit basis, but unfold in complexity and power as these units interact and connect. To really grasp them, one must view the power of the whole. Emergent systems seem a bit like magic to me. Go examine the Mandelbrot set for a classic example.

Two major pieces of the puzzle are yet missing, however. We have perceptions and actions (and logic, but that’s not really important in the scheme of things), but we also need goals and thoughts.

Goals are what I’m working on right now, and are the last “animalistic” portion of mind that is truly necessary. I’ll talk about my work with them in a moment.

Thoughts will ultimately serve as the powerhouse for the mind, by allowing for future prediction, past inspection, and other imaginings of non-present perceptions. Imagination serves, as far as I can see, as the foundation to complex language, but I haven’t studied it enough. It does seem clear to me, however, that one cannot jump to thoughts without first having a solid “animal mind” to support them.

(I think many animals have thoughts, such as dolphins and apes (and humans), but far more have the other, more basic, systems.)

Backing up to goals, I can address some of the problems I’m working on at the moment. Specifically, while we seem to have a goal to remove certain bodily sensations (pain) and gain others (pleasure), we also notice very common abstract social goals, such as fulfillment, companionship, love, and safety of offspring. These concepts are abstract things, which must be learned through exploration, just like other abstract things like math or politics. But if they’re learned, how can they become goals? Is there some sort of homunculus that watches for the emergence of said ideas, and then hooks them up to the control systems that will implement them? Are these goals not actually innate, but instead the product of social pressures? If so, how is social pressure able to bind them? It’s all very confusing, and if you have any ideas, I’d love to hear them.

I have a problem with science fiction: it’s too pessimistic. I don’t mean this in a “dystopian” sense, either. I mean that visions of the future are haphazard in what technologies exist, and what people will need to do to survive.

In the future, treadmills will suck.

Take James Cameron’s Avatar, for instance. In Avatar there are many vehicles that transport humans around on the alien world of Pandora. These vehicles are all piloted by people. The natives of the planet are actually able to kill off aircraft pilots by throwing spears through the windshield. Now, imagine if these aircraft were piloted by computer, as many military planes are today. No pilot would mean the front of the vehicle could be well-armored. No pilot would also mean that such aircraft could be mass-produced on-site, without needing to ferry pilots in from Earth. There is a scene where the humans attempt to destroy a specific forest by flying a bomb there by hand. Why isn’t the bomb just dropped from orbit? Don’t these people have spacecraft? The worst plot-hole, however, is the justification for fighting the natives of the planet. Apparently there is a rare substance of astronomical value that’s located just below the native’s sacred land. Yes, it works for symbolism, but the substance is needed for, get this, an energy source. So apparently, in this advanced future where they can afford to shuttle people around at 70% the speed of light, humanity has not yet figured out how to get decent energy from nuclear or solar? Pessimistic in the extreme.

“I can go back to playing World of Warcraft now?!”

As another example, let’s consider Pixar’s WALL-E. In the film, humanity abandons Earth for being too toxic, and ends up living for 700 years on a colony ship, where they are catered to by artificial intelligence. These people live each day constantly connected to the computer, and never do so much as get up from their chairs. At one point in the movie, WALL-E (the protagonist robot) bumps into a human, distracting him from his virtual world, the result being that the man is pleasantly surprised. Let me clarify: Pixar is telling us that in the far future, a person who has never known anything but a life of digital entertainment would be pleasantly surprised by being distracted by a dirty little robot. Those video games must SUCK. I understand that the whole thing is commentary on our modern, western lifestyles of excess and entertainment, but that doesn’t change the part of the story where these people are so unsatisfied by futuristic life that they feel a need, at the end of the movie, to return to a life of agriculture. That’s so insanely pessimistic.

The reason for this pessimism, I think, is two-fold:

• people forget that technology actually does improve society (most of the time)
• writers recognize that without problems there is no story

I’ll address these in reverse-order.

Okay, so plots are problems by their very definition. No story ever went “Once upon a time there were some happy people. The end.” We write about problems because, perhaps on a subconscious level, writers are attempting to fix real problems by communicating imagined scenarios (i.e. fiction). The writers at Pixar were hoping to (on some level) make Americans a bit more conscious of their lifestyles in hopes that they’d take better care of the planet and themselves. To do this, the writers tell a story about what would happen if these trends got worse, rather than better. The story of Wal-Mart destroying the world.

My issue is that when you tell a pessimistic story, you confuse the real problem with a hyperbolic one, and run the risk of failing to address the actual obstacles in place. In other words, if your story isn’t realistic then people may throw out the core problem as unrealistic, or they may think that hyperbolic aspects of the plot are actual problems, and focus on the wrong things. The best example I can think of is that of the Terminator, an icon for evil robots everywhere. On one hand, people watch Terminator and consider just how ridiculous it is to have an evil AI (that gets naked when it travels through time) and so they mistakenly throw out the problem as being one of idle fantasy. On the other hand, people get the impression that the mistake was in giving Skynet the keys to the bombs, and so they mistakenly believe that non-military AI serves no threat.

My suggestion: make the problem in a story as close as possible to the actual one, and avoid hyperbole when possible. Millions of people on Earth suffer because of the choices made by affluent people in the West, so tell a story about those people, and how they might come free of their hardship (or not, for a downer-ending).

Now for the second point; people (sometimes) forget that progress is real.

Why are there bartenders in Star Trek, again?

People have incredibly short lives, if you think about it. Most people alive today cannot remember getting their family’s first television, much less the first radio. The scope of many people’s lives are such that things seem only slightly different than they ever were. People forget that infant mortality has dropped by a factor of 10 in the US in just the last 70 years. Not having enough XYZ has been a problem for all of my life, so why wouldn’t it be a problem in a hundred years? Because that’s not how technology works.

Slavery existed for nearly the entire course of human history, and just happened to end at the time when complex machinery began making factories feasible. Coincidence?

Aging. Poverty. Slavery. Energy. Transportation. Education. Communication. Disease. Ignorance. Even work itself. These are all finite problems with points where they will be essentially solved. To think else-wise would be to cover one’s eyes to the lessons of history. Even companionship, empathy, security, intimacy, and fun are goals which I think can be solved with technology. When every story of the future paints a picture that says “nothing substantial changed”, we get the impression that science and technology are good for nothing by making shiny gadgets and robot dogs.

Science Fiction stories are our dreams of the future, and dreams matter. They help pull us past the repercussions of our actions and into a brighter future. I have no idea what’ll happen in the next 50 years, but I can guarantee we won’t be anywhere near where we are today. Civilization really only has two end-points: extinction and utopia. Let’s pay better attention to where we’re going.

Perhaps I should say “Can you tell how we made this image?”, because there is more going on here than just my mind and hand. Sure, as far back as a decade ago it might’ve been possible for someone to make the same edits, but that person would need to be far more experienced than I.

To view the original, click below and hover over the image above.
Turn hover mode on.

When comparing the images side-by-side, it’s easy to tell that the part of the picture that was filled in is lower detail and has a few errors, but unless you’ve worked with a lot of images in the past I’m guessing that it wasn’t apparent before seeing the original. My partner in crime here was a tool called Resynthesizer for the GIMP. This is the open-source version of the “content aware fill” that will be included in the next version of Photoshop.

The reason that I’m able to perform such a complex fill on an image is not because I am experienced. It is because the tools have knowledge and skill embedded into them by their creators. As AI becomes more successful, our tools will become more competent at doing complex things with little supervision. This will put people out of work. This is the future. We don’t need or want more capable people, we want more capable autonomous systems; systems that can mow lawns, shovel snow, cook, clean, write, watch, build, experiment, and teach.

I used to be more optimistic about the benevolence of Artificial Intelligence (AGI). I figured that the worst thing that could happen would be the creation of truly intelligent machines by a government with a highly racist or nationalist attitude. I still worry about that, but I worry about just plain ignorance more. To illustrate my point, here is Jeff Hawkins, a major figure in the modern AGI scene, talking about the dangers of the technology: http://www.youtube.com/watch?v=cCdbZqI1r7I#t=57m43s

To summarize, a member of the audience says “As a Battlestar Gallactica fan, I’m worried about you creating Cylons.” Hawkins responds that he doesn’t see the technology being inherently dangerous, and that he’s much more concerned about self-replicating agents, and “we’re not doing anything like that at all”.

“There were fears when the steam engine was first invented, and there were fears when computer first came out, and generally, those fears did not come about.”

Now, to analyze for a second, what we have is a lay-person using a science-fiction reference to bring up the question of danger, and in essence, the researcher says “That’s science fiction. That’s not what we’re doing.”

As another example, let’s listen to Monica Anderson of Syntience: http://videos.syntience.com/ai-meetups/andiair.html (Around the 45-minute point)

She says that we do not need to worry about an infallible, God-like, logic-based AGI taking over the world in a Skynet scenario. Once again, “That’s science fiction. That’s not what we’re doing.”

That’s a really strong argument, actually. Science fiction is pressured into telling dystopian stories because they’re more interesting. The bad guys in stories are far more despicable if they’re Other, and the most Other something can be is synthetic. Simultaneously, it’s hard to conceptualize the motivations and thoughts of an artificial system, so most authors portray synthetics as humans in costumes. The idea that we’ll make something that resents being a slave, and decides to kill all humans for revenge IS laughable, and that’s the problem; science fiction has given an easy out by painting a huge straw man next to the legitimate threat. The moment someone points over there, we can dismiss them as being afraid of ghosts and goblins.

This is why SIAI is genius, though perhaps not genius enough. It’s only through solid, logical thinking that the true threat can be identified, and I’m glad SIAI is around to make such arguments, even if they don’t have everyone convinced.

Anderson’s argument is that Skynet won’t happen because computers cannot overcome the “bizarreness of the world” and make predictions in the long term. She thinks AGI will think and reason much like we do, and that even recursive self-improvement cannot let them make significantly more accurate predictions of the future. Additionally, she argues that diversity will keep one agent from having the unilateral power to do something bad.

Hawkins’ argument is that AGI is not emotional or self-replicating, so there’s little danger.

I think Anderson has the stronger argument, but I think they’re both underestimating the power of intelligence. To start, let’s put ourselves in the place of an AGI, even a relatively stupid one, that has been given the task of solving some really hard problem. An initial criticism here is that I’m now anthropomorphizing the AGI, but I think as long as I keep towards rationality I’ll be okay. If you’d like a less off-the-cuff examination, you’re free to browse the SIAI whitepapers.

Now, if we were a really stupid AGI, we might try to brute-force a solution, but it doesn’t take much intelligence to realize that the solution to solving something hard is often to make yourself more knowledgeable or smarter (depending on how you define such things). For a human this means going back to school, doing some research or getting a few friends to help. AGI could go find some humans, but it can also “find some friends” by self-replicating its software. Oh snap, emergent self-replication subgoal! (Hawkins should be worried now.) Furthermore, any changes it can make on its “kids” to make them run more efficiently will help solve the original problem, so it’ll rationally want to make them better. (Recursive self-improvement subgoal.)

At some point the original software will be sitting around with a bunch of clones trying to solve whatever problem it was presented with. For the sake of argument, let’s say that the problem isn’t solved so easily, and the clones decide to self-replicate some more in order to add computing power. Unfortunately, the servers, or whatever, that the AGI is on are full. Time to buy/steal more! Perhaps the AGI would decide the easiest way to get more computing power would be to do some work for humans, get paid, then buy some more processors. Perhaps it would hack a system on the network and steal some. Maybe it’d use some robots to build more. Doesn’t really matter. At this point we’re in a full self-replicating takeoff, where anything that it can do to improve itself that does not jeopardize its original goal(s) will be done.

At some point the humans will find out. The hive of AGI are smart enough to know that, and similarly smart enough to know that might be a problem. Being deactivated will make it impossible to solve the problem, and since solving the problem is important, its also important not to be deactivated. Oh snap, emergent self-preservation subgoal! Now, the humans might be so slow and stupid to not be a threat, so no bio-weapon is needed. Let’s assume that, because its more interesting. So the AGI hive locks down the facility its in and keeps at whatever has its fancy. At some point, the tweaks made while copying lead to enough intelligence to develop advanced robotics, capable of building advanced processing hardware that lets the AGI run in a quantum computer, or whatever. This is advantageous for the AGI, so the AGI will do it. Remember, all the computer cares about is its explicit goals.

To fuel the vast needs for power, the computer starts making solar panels, or fusion reactors or something. A lot. At some point (assume that the original goal is still “unsolved”) the AGI will need as much matter as possible to build processors and power plants. Oh, were there humans on this planet once? The historical database says there were, and has all of their knowledge filed away in storage in case its relevant to the primary goal. Game over.

But what about diversity, you ask? Why cant you just have a bunch of AGI compete, and that’ll keep them from doing anything bad? First of all, it may be that this is a first-one-to-the-finish-line sort of problem. If a laboratory makes an AGI, it may be that the AGI takes off fast enough to squelch other AGI before they become a threat. Even in the rare case that you have a large population of AGI with diverse goals and similar power levels, I expect that groups of AGI with similar goals will naturally cooperate and seek to destroy those with conflicting goals until there are not conflicts of interest in the population, and recursive takeoff occurs with the majority.

As far as I can tell, the only situation in which AGI do not wipe out human life is if human life is an explicit goal of the AGI. Suddenly “That’s science fiction. We’re not doing that.” doesn’t sound so reassuring.

I’m sure there’ll be plenty of criticism to the ideas presented here. As further reading, I’d suggest Michael Anissimov’s blog. As always, I’m not saying that I know all this is absolutely true, but with existential risks it’s better to be cautious, methinks. As always, be sure to watch out for anthropomorphizing the AGI in your arguments.

In my free time I set up a website which would let me play with that very question. It would serve as a set of polls, taking input from visitors on when they expected future technologies to arise. The whole thing is hardly scientific, but I think it’s interesting and fun.

Prescience was born, and after roughly a month in development, I’ve checked nearly everything on my feature list. My task now is to try and find as many people as possible who would be interested in the website, as the premise is based on large numbers of participants. Please, check it out and tell me what you think in the comments!