Fair warning: I don’t know if I like this post. It may be because I’m tired of looking at the math, but it might also be due to incoherence, or a terribly weak ending. I’d rather post it than work on it more, so heads-up.
Let’s say there are two doors. Behind the left door is $500 and behind the right one is $2. You can only go through one door. Which door do you pick?
This is not a trick question.
I’m right now working on building a reasoning agent. Given a model of the environment and a set of goals, it will attempt to take the best action. As a basic test, I have a model of a game of Tic-Tac-Toe for it to play against itself. For a long time my agent was picking TERRIBLE moves, and I spent at least five hours trying to figure out why. As it turns out, the goal function I gave it subtly punished moves that didn’t result in immediate victory. In theory this would cause it to favor shorter, more decisive games, but I had simultaneously chosen a punishment for defeat that wasn’t much higher. The result is that on a board like:
I woke up at 5:30 this morning, and as I lay in bed, I was struck by some inspiration which caused me to run out into the living room in my underwear and start jotting down cognitive science notes on my whiteboards. Yeah, apparently I do that. Now, just because I was inspired doesn’t mean I’m in the least bit right. In fact, I’m pretty confident that my models are drastically oversimplifying things, but I figured that I’d be fun to share/save them.
Just because I felt like I wasn’t alienating enough people on the internet, I posted 10 times in a row to Twitter. The following is a list of questions that I am currently working on to varying degrees. Many of them have answers in textbooks, others are on the cutting edge, and others probably will come unraveled if I think about them enough. In no particular order:
And now, because this auto-posts to twitter, my tweeples get to see yet more pontificating float by. Ah, 2011, I’ll miss you; once the robots get good at serving up only interesting text, where we go to blather into the night?
As I’ve mentioned previously, I have a hard time with most science fiction. The only kind of the genre I can really respect is day-after-tomorrow sci-fi; the kind that explores the ramifications of a specific technology on our world as it exists now (Dollhouse is a good example). Most of my interest nowadays hovers around Artificial General Intelligence, and so I came to think “What sort of science fiction could discuss AGI issues in a suitably dramatic way?” The problem is that I envision a hard takeoff (or at least pseudo-hard) of the power of the first superhuman AGI, and it’s hard for me to imagine how to simultaneously make the takeoff exciting and not so fast that it can’t be stopped, or at least steered a little.
Then I got the idea: what if the intelligence explosion was started by an alien artifact? If the aliens were already post-singularity, then the artifact could even be a chunk of their civilization that happened to survive some sort of terrible event. Because of various meta-ethical reasons, I think a post-singularity intelligence cannot care about all life in the universe, so this alien AI would be in the perfect position to TAKE OVER THE WORLD. Classic. The only problem is how to stop/steer it, but this is solved by the existence of aliens. With bootstrapped technology, the humans might just have a fighting chance.
