For example, if I have a stone, I might form the hypothesis that the mass of the stone is 30g. To test this, I might weigh the stone. Implicit here is the idea that what is recorded from the scale is the mass of the stone. Once I read the scale, I have the following ideas:

(A) The scale says “45g”.
(B) The mass of the stone is 30g.
(C) The scale is an accurate measure of the stone’s mass.

If all three were true, there would be a contradiction, so I can conclude that one or more of my ideas must be false. The problem with falsification is that I have no logical reason to favor A & C (false hypothesis), over A & B (bad scale), B & C (bad eyesight), or others. Ultimately, I cannot refute anything with absolute certainty, so I cannot disprove.

This difficulty can be reduced by what I like to think of as the inductivist section of critical rationalism (I’ll show why in a moment). Wikipedia says that, with respect to hypotheses, “differentiation may be made on the basis of how much subjection to criticism they have received, [and] how severe such criticism has been”. In my example, none of the three ideas has been criticised, but it’s easy to imagine a scenario where the accuracy of the scale had been previously tested.

There are, unfortunately, two problems with this reasoning: (1) the scale requires evidence to test, so we still have the “which conjecture do we accept” problem at an earlier point, and (2) we still have two conjectures to decide between (A&B). What most critical rationalists will likely turn to is the difference between unsubstantiated conjectures (B) and those based on observation. It’s important to remember that hypothesis A is still conjectural, but we can grant it a sort of “natural critisism” stemming from our perception.

Here’s the rub: lending a heigher weight to any of our conjectures still doesn’t allow any refutation to logically occur. The only way to do that would be to accept something as true after enough observation, and this is exacly what everyone does (including critical rationalists), but CR brushes off as illegitimate.

At this point, one might turn to belief weights in order to avoid having to assign a binary value to a hypothesis (and the fallacy of inference). Unfortunately, any sort of updates to belief weights requires knowledge that is assumed to be true; it doesn’t actually let you move from a state of unsubstantiated conjecture to one of informed belief. In other words, it requires prior knowledge, which, as discussed earlier, we can’t logically obtain.

As an example, let’s say I have a hypothesis that a zebra exists and then I perceive a zebra. What weight do I give my hypothesis? In order to find it, I must know how accurate my perception is. For example, if hallucinating a zebra is equally probable to seeing a real zebra, there is a 50/50 chance that the zebra actually exists. But let’s say that I am not given a value for how accurate my perception is… how do I determine the likelihood of false positives, etc? The natural answer is to make a bunch of observations, and test to see if they were “correct”… except to do that, you’d need to assume the training labels (“correct/incorrect”) were true! If you want to evaluate the accuracy of the training labels, you have to assume some other input is true. The catch 22 ensures that you cannot logically produce a factual statement (even a probabilistic one) about the world without having been given other (binary) factual statements.

Unless I’m overlooking an infallible source of knowledge, I can conclude that nobody in the entire universe has any knowledge (that is, factual data) of the universe… and never will. Not even an infinite intelligence would be able to know anything about reality.

To escape this agnosticism, I might suggest that when we look like we’re doing logic, we’re actually not (at least, not formally). For instance, if the scale reads “45g”, I might simply accept that the stone is 45g and reject my old hypothesis, not through logic, but through common sense. The problem here is that common sense is a blanket term used to describe mental tasks that are easily done by people, but we don’t understand explicitly. Doing something via “common sense” is a lot like dying from “old age”; it’s just not a useful term. To make things worse, humans generally seem to reject paradox and use deduction, so we can be confident that something very close to formal logic is going on mentally.

My theory is that ideas are not evaluated based on truth, but based on the utility that comes from predictive power. Prediction, here, is based on sensory data, as opposed to objective reality. Unlike reality, we can be sure of our sensors as long as we think of the sensors as “inputs”. Let me give some examples…

I find a stone and decide to weigh it. I predict that the measured mass of the stone will be 30g. I put the stone on a scale, and it says 45g. My prediction had a significant error, so I discard it as being non-useful. Because I’d like to be able to predict the stone’s mass I form a new prediction that the mass is 45g (informed hypothesizing). I can use my memory to test the prediction… it works! This retrospective success reinforces the expected predictive power of that hypothesis. This explains why a hypothesis that matches previously observed data is granted more weight, and why one that doesn’t is discarded (falsification).

Let me give an example. Little Andrea sees a crow that is black. She conjectures that all crows are black (or more simply: “crows are black”). She sees another black crow. Prediction reinforced. She asks her mom what color crows are. Prediction reinforced. She sees a green apple (non-black non-crow). Observation is outside prediction scope; no change. At the age of 46, Andrea meets a street performer with an albino crow. Prediction failed. She notes “Oh, how strange… a white crow!” Prediction weakened slightly, but still retained, because in the vast majority of cases it’s useful to guess that crows are black. After seeing enough white crows she may reject her initial generalization and adopt a more probabilistic one (about 90% of ravens are black), but since a probabilistic idea has intrinsically less predictive power (and is harder for humans to measure*), they are under-weighted and often avoided (leading to accident fallacies and others).

I could wander from here into my theories of semantic memory, but I’ll try to stick with critical rationalism to finish my thought. When Popper started, what he sought was to step away from justification, the practice of trying to support currently held ideas. In the service of this, the claim was made that one can disprove an idea, but not prove one. Though I’ve come to reject this claim, I don’t think that critical rationalism is a bad approach.

Justificationalism comes out of a natural tendency to want to be right, and it appeals to this bias even when a more open mind might find more effective ideas. Critical rationalism avoids this by forcing each person to listen to other arguments in order to determine how they might fail.

Critical rationalism also avoids the trap of adding weight to a theory because of selective observation. For instance, if I have the theory that “proteins are a kind of enzyme”, I might seek to “confirm” it by looking for enzymatic proteins. This will bias my data set so that it appears that the idea is effective, when it actually isn’t. Critical rationalism will naturally disrupt this bias with a second bias of seeking data that doesn’t fit the theory. Because an idea that is predictive only, say, 80% of the time isn’t very useful, this bias is helpful in pushing us towards more consistently accurate ideas.

One might suggest that when people say “truth” they mean “predictive power”. If this is true, I can easily show where Popper’s ideology fails. No prediction will be correct 100% of the time; our sensors are fallible. An idea that fails shouldn’t be rejected as “falsified” if it’s still accurate almost all of the time. F=ma is still a really important piece of knowledge. In this way, induction works.

* – Probabilistic ideas work differently for different forms of memory. Associative memory, the kind of thought that we use when making split-second decisions, is very probabilistic. For example, it is easy to have a gut feeling that a deck of cards is about half black and half red. Semantic knowledge, the kind of idea that we use consciously, doesn’t work so well with such things. I cannot imagine a person being able to tell you what proportion of a pile of cards is clubs unless they do some mental math.

Photo credit: swissbones on Flickr

If someone is interested in the truth, it’d be a blessing to them to learn exactly what falsehoods they believe. An ideal truth-seeker would love to be wrong, because it would give them the chance to improve their ideas. This is the basis of scientific thinking, but it’s not a very good description of how people actually behave. Most people (especially those with little training) react defensively to criticisms of their ideas and show a strong preference of comfortably holding familiar ideas rather than putting them to the test. Why is this?

Some of this bias can be explained by a desire for familiarity (“if democrats are wrong than my whole world-view would fall apart”) and an attachment of self-esteem to being right (“only bad people are wrong”), but I think I’ve stumbled across another, more biological, explanation.

Imagine some ancestral hominids try two kinds of fruit: one kind that is delicious, and one kind that is poisonous. Those who eat the poisonous fruit spend all night in terrible pain, but eventually recover and learn to eat the good fruit. Twenty years later, some of the same hominids return to the grove with the two fruit. It’s been a while, so maybe they’ve forgotten about the good fruit, but they certainly haven’t forgotten about the poisonous one. The reason for this is that brain weights memories surrounding unpleasant events more heavily than pleasant ones. In short: it’s more important to remember what could’ve killed you or your family than just about anything else.

This mnemonic weighting is present in rats and other animals only distantly related to humans, so it’s sensible to conclude that it developed in a very primitive animal, far before logical reasoning, language and culture. When it became possible to reason about the world, humans were stuck with their old equipment for remembering. It was still more important to remember mistakes (even if they were mistakes of reasoning), but rather than build an emotionality-neutral way to add weight to such fallacies, evolution took the shortcut of adding unpleasant feelings to logical errors. By making it annoying and embarrassing to be wrong, evolution ensured we’d learn our lesson. Many years later, when scientific thought has become important, we’re left with the reality that, by default, people tend to avoid criticism.

(Photo credit: http://www.flickr.com/photos/idmaer/86349266/ Licensed under Creative Commons Attribution-Noncommercial-Share Alike 2.0 Generic)

• You can now set cells to be drawn as images. Defining the image a cell should use is as simple as making a rule of “draw <CELLTYPE> <URL>” (See Langton’s Ant for example.)
• The directions northeast, northwest, etc. have been added, as have fareast, farnorth, etc. The “far____” directions refer to cells two steps away.
• The special noun “neighbors” now includes the NULL cells at the border.
• New verbs “cellAt” and “neighborsOf” have been added. They both take a 2-by-1 array ([x,y]) of the cell in question as their right-parameter. The “cellAt” verb returns the type of the cell in question, while “neighborsOf” returns the set of neighbors.
• A new two-parameter verb “+” has been added. Right now it only operates on the 2-by-1 coordinate arrays discussed above. (Example: [1,-10] + [23,7] = [24,-3])
• In the same vein, a new special noun called “thisPos” has been added. It is the coordinates of the cell being evaluated.

That’s all!

For those interested in writing your own automata, here is a quick guide to the syntax:
An automaton is composed of rules. Each rule is designated by a new line. Double slash (//) or pound (#) designate a commented line.
Each rule looks something like BLACK->WHITE: 1 = 2
BLACK is the type of cell that the rule applies to, WHITE is the type of cell that it will become. After the colon is a statement that must resolve to a single value, or “noun”.
Nouns can be strings, numbers, true/false, JSON arrays, “neighbors”, or any of the cardinal directions (lower-case). The noun “neighbors” is a special noun which is a JSON array with values equal to the types of the surrounding tiles (in eight directions). The special nouns “west”, “north”, “south” and “east” are equal to the type in that direction, or “NULL” if at the edge.
Nouns can be acted upon by the following “verbs”:

• =, >, < — Basic comparison operators. They behave like their counterparts in javascript (except for =, which is comparison, not assignment). — example: “1 > 2 = false” (returns true)
• or, and — Low-priority logic operators. These behave like || and && in most languages. Low-priority means they are evaluated after other verbs. — example: “true and false or true” (returns true)
• not — Inverts the boolean value to the right. If the value to the right is non-boolean, it is cast to a boolean just like ! in javascript. — example: “not true” (returns false)
• contains, isIn — Check if a value is contained in an array. The two verbs are identical except in the order of their parameters (“value isIn array”, or “array contains value”). — example: “[1,2,3] contains 0″ (returns false)
• count — Changes the noun on the right from an array to an integer. — example: “count [0,1,2]” (returns 3)
• ofType — Filters an array on the left to only contain the value on the right. — example: “[0,0,0,1,2,3] ofType 0″ (returns [0,0,0])
• echo — Makes a popup containing the value on the right. Does not change the data. This verb isn’t very useful, and can flood your browser with popups if you aren’t careful.

Verbs are evaluated from left-to-right, but you can use parentheses to change the order of operations. More examples can be found by loading the prebuilt automata.

I know the interface is a bit hard to use, but if you make anything (even if it’s pretty trivial) please leave a comment! You can export boards as well, so feel free to share interesting formations in Conway’s Game of Life, Wireworld, or others.

On a related note, this talk by Wolfram is not to be missed:

Specifically, I began to notice how few links my notes contained, and the difficulty I might have in changing small details (like names) across many pages. Wouldn’t it be nice, I thought, if I were able to tell my computer about my world, and have the computer organize and maintain my notes in a way that let me quickly and easily extract and change the underlying facts. Essentially, I was thinking about automating the world-building process by reducing a world into a data and making code to manipulate and present that data.

Because I had heard good things about associative databases, I built one to handle the facts of the world and began entering items like
Kogámos is_child_of Sáríbá
(Yes, I know that’s a lot of accents). Associative databases (with exceptions, I’m sure) represent data ontologically (that is, as a collection of factual claims about entities), and in this way resemble the data structures of Good Old-Fashioned AI*. Now, before I started this project I was very skeptical about logic-based AGI, but I knew that in some domains, (such as Expert Systems) logic based narrow AI had become successful, so I set about exploring whether this was one such domain.

* – Really, this structure is common to all logic-based AI, even fuzzy-logic, modern approaches like NARS. Much of the criticism that follows can be applied to any sort of artificial reasoning system.

Before I go on, however, I’d like to share a documentary about the semantic web. It’s related, and I’ll be talking about it, but it’s 15 minutes and you could probably skip it if you’re familiar with the concept.

With the possible exception of being able to make meta-facts (that is, facts about facts), the eventual structure of the semantic web closely resembles that of an associative database. In fact, I’m sure that facebook has a piece of data saying
Max_Harms is_child_of Tracy_Harms
or something to that effect, right now. And like my project, the semantic web seeks to represent data about an entire world, not just a highly-restricted domain.

My project has some big advantages, though. The biggest is probably that all the information given might as well be coming from God. If the world-creator says that rivers run uphill, that’s just the way it works. Thus there is no need to model confidence in facts entered. Secondly, the system doesn’t actually have to act; it simply has to store and present the data in an intelligent way.

One of the early things I noticed in my exploration is just how fun it is to see the computer answer questions. Logic-based AI is very attractive because it takes hardly any work before you can ask it things like “Who is the child of Sáríbá?” and it will give you the right answer. I ended up spending lots of time asking questions that I already knew the answer to, just because it was fun whenever the computer was right.

One of the biggest hurdles in the transition from a Reductionist to a Holistic stance is that the Reductionist stance works so well for simple problems, and thus is very seductive to beginners.- Monica Anderson

The problem here is that answering simple questions based on simple rules from entered facts doesn’t mean squat. It’s about as useful as writing the information in a text file and using regex.

It gets worse than that, though. As mentioned before, worlds are messy and complicated. If the problem was as simple as generating a family tree, or making Venn diagrams, we’d be okay. But how do you represent something as nuanced as “Sally likes to spend time downtown, except on the weekend, when she finds it too crowded”? Such things are simple for us to remember because they tie into our experiences. To illustrate what I mean, imagine the similar phrase “Queue is angry about rowing in Vah, except when floating, when torque is standard.” This statement has no connection with our common experience, and because we do not understand it, we cannot store it easily. This is the same problem as that confronting the semantic web. We can pour in data until we die, and the system still wouldn’t “get it”.

If a programmer attempts to describe the world to a Logic based AI, for instance by creating ontologies, he’ll never finish the task. The world is too rich. The Cyc project – the largest and most famous AI project ever undertaken – has been trying to describe the world using predicate calculus for decades; it is the poster project for Reductionist approaches to AI. But Cyc will never approach anything worthy of the term “Intelligence”. It has been told many things and can recite many definitions but Understands nothing. This is the difference between “Instructionist” top down education and “Constructionist” bottom up learning – a distinction poorly understood even in human education.- Monica Anderson

Even in a task as basic as storing and retrieving information we still need the computer to understand what it’s doing beyond tasks of relative simplicity. This is because knowing about the world falls into a large class of problems that are fundamentally impossible without General Intelligence**. Other problems in this category include using natural language, navigating complex environments, and behaving in socially acceptable ways. As hard as we may try, until Strong AI is solved, a chatbot will always be a mimicking machine, a database will always fail in complex domains, and humans will be forced adapt to robots (instead of robots adapting to humans).

** – It may be possible for a vast enough intelligence to make a system so complex that it’d be able to handle one of these domains. I don’t think even large numbers of humans would be smart enough to make one, though.

Even with my increased dislike of logical systems, I’m quite happy with what I was able to make in the few days I spent building my database. You can check out the results yourself, or even add and delete facts. The database will be restored to a backup state twice a day (not sure what timezone my server is in), so don’t worry about making a mess.

Search commands:

• “describe X” = get all facts relating to X
• “list X is ?” = get all items that X is
• “list X is_child_of ?” = get the parents of X
• “list ? is|has X” = get everything that is or has X
• “list ? imports * ? is geographical_area” = gets geographical areas that import anything
• “infer” = list suggestions for facts, based on correlations

Source code is here. Everything I wrote (files in the root directory) is public domain. Attribution is appreciated.

(P.S. I’m a big supporter of the semantic web, and doubly so for open data. The criticisms expressed in this post are against the idea that it is possible to intelligently know lots about the world using ontological representations. Links are data, and that data is important, even if that data isn’t knowledge in itself.)

Perhaps I should say “Can you tell how we made this image?”, because there is more going on here than just my mind and hand. Sure, as far back as a decade ago it might’ve been possible for someone to make the same edits, but that person would need to be far more experienced than I.

To view the original, click below and hover over the image above.
Turn hover mode on.

When comparing the images side-by-side, it’s easy to tell that the part of the picture that was filled in is lower detail and has a few errors, but unless you’ve worked with a lot of images in the past I’m guessing that it wasn’t apparent before seeing the original. My partner in crime here was a tool called Resynthesizer for the GIMP. This is the open-source version of the “content aware fill” that will be included in the next version of Photoshop.

The reason that I’m able to perform such a complex fill on an image is not because I am experienced. It is because the tools have knowledge and skill embedded into them by their creators. As AI becomes more successful, our tools will become more competent at doing complex things with little supervision. This will put people out of work. This is the future. We don’t need or want more capable people, we want more capable autonomous systems; systems that can mow lawns, shovel snow, cook, clean, write, watch, build, experiment, and teach.

I used to be more optimistic about the benevolence of Artificial Intelligence (AGI). I figured that the worst thing that could happen would be the creation of truly intelligent machines by a government with a highly racist or nationalist attitude. I still worry about that, but I worry about just plain ignorance more. To illustrate my point, here is Jeff Hawkins, a major figure in the modern AGI scene, talking about the dangers of the technology: http://www.youtube.com/watch?v=cCdbZqI1r7I#t=57m43s

To summarize, a member of the audience says “As a Battlestar Gallactica fan, I’m worried about you creating Cylons.” Hawkins responds that he doesn’t see the technology being inherently dangerous, and that he’s much more concerned about self-replicating agents, and “we’re not doing anything like that at all”.

“There were fears when the steam engine was first invented, and there were fears when computer first came out, and generally, those fears did not come about.”

Now, to analyze for a second, what we have is a lay-person using a science-fiction reference to bring up the question of danger, and in essence, the researcher says “That’s science fiction. That’s not what we’re doing.”

As another example, let’s listen to Monica Anderson of Syntience: http://videos.syntience.com/ai-meetups/andiair.html (Around the 45-minute point)

She says that we do not need to worry about an infallible, God-like, logic-based AGI taking over the world in a Skynet scenario. Once again, “That’s science fiction. That’s not what we’re doing.”

That’s a really strong argument, actually. Science fiction is pressured into telling dystopian stories because they’re more interesting. The bad guys in stories are far more despicable if they’re Other, and the most Other something can be is synthetic. Simultaneously, it’s hard to conceptualize the motivations and thoughts of an artificial system, so most authors portray synthetics as humans in costumes. The idea that we’ll make something that resents being a slave, and decides to kill all humans for revenge IS laughable, and that’s the problem; science fiction has given an easy out by painting a huge straw man next to the legitimate threat. The moment someone points over there, we can dismiss them as being afraid of ghosts and goblins.

This is why SIAI is genius, though perhaps not genius enough. It’s only through solid, logical thinking that the true threat can be identified, and I’m glad SIAI is around to make such arguments, even if they don’t have everyone convinced.

Anderson’s argument is that Skynet won’t happen because computers cannot overcome the “bizarreness of the world” and make predictions in the long term. She thinks AGI will think and reason much like we do, and that even recursive self-improvement cannot let them make significantly more accurate predictions of the future. Additionally, she argues that diversity will keep one agent from having the unilateral power to do something bad.

Hawkins’ argument is that AGI is not emotional or self-replicating, so there’s little danger.

I think Anderson has the stronger argument, but I think they’re both underestimating the power of intelligence. To start, let’s put ourselves in the place of an AGI, even a relatively stupid one, that has been given the task of solving some really hard problem. An initial criticism here is that I’m now anthropomorphizing the AGI, but I think as long as I keep towards rationality I’ll be okay. If you’d like a less off-the-cuff examination, you’re free to browse the SIAI whitepapers.

Now, if we were a really stupid AGI, we might try to brute-force a solution, but it doesn’t take much intelligence to realize that the solution to solving something hard is often to make yourself more knowledgeable or smarter (depending on how you define such things). For a human this means going back to school, doing some research or getting a few friends to help. AGI could go find some humans, but it can also “find some friends” by self-replicating its software. Oh snap, emergent self-replication subgoal! (Hawkins should be worried now.) Furthermore, any changes it can make on its “kids” to make them run more efficiently will help solve the original problem, so it’ll rationally want to make them better. (Recursive self-improvement subgoal.)

At some point the original software will be sitting around with a bunch of clones trying to solve whatever problem it was presented with. For the sake of argument, let’s say that the problem isn’t solved so easily, and the clones decide to self-replicate some more in order to add computing power. Unfortunately, the servers, or whatever, that the AGI is on are full. Time to buy/steal more! Perhaps the AGI would decide the easiest way to get more computing power would be to do some work for humans, get paid, then buy some more processors. Perhaps it would hack a system on the network and steal some. Maybe it’d use some robots to build more. Doesn’t really matter. At this point we’re in a full self-replicating takeoff, where anything that it can do to improve itself that does not jeopardize its original goal(s) will be done.

At some point the humans will find out. The hive of AGI are smart enough to know that, and similarly smart enough to know that might be a problem. Being deactivated will make it impossible to solve the problem, and since solving the problem is important, its also important not to be deactivated. Oh snap, emergent self-preservation subgoal! Now, the humans might be so slow and stupid to not be a threat, so no bio-weapon is needed. Let’s assume that, because its more interesting. So the AGI hive locks down the facility its in and keeps at whatever has its fancy. At some point, the tweaks made while copying lead to enough intelligence to develop advanced robotics, capable of building advanced processing hardware that lets the AGI run in a quantum computer, or whatever. This is advantageous for the AGI, so the AGI will do it. Remember, all the computer cares about is its explicit goals.

To fuel the vast needs for power, the computer starts making solar panels, or fusion reactors or something. A lot. At some point (assume that the original goal is still “unsolved”) the AGI will need as much matter as possible to build processors and power plants. Oh, were there humans on this planet once? The historical database says there were, and has all of their knowledge filed away in storage in case its relevant to the primary goal. Game over.

But what about diversity, you ask? Why cant you just have a bunch of AGI compete, and that’ll keep them from doing anything bad? First of all, it may be that this is a first-one-to-the-finish-line sort of problem. If a laboratory makes an AGI, it may be that the AGI takes off fast enough to squelch other AGI before they become a threat. Even in the rare case that you have a large population of AGI with diverse goals and similar power levels, I expect that groups of AGI with similar goals will naturally cooperate and seek to destroy those with conflicting goals until there are not conflicts of interest in the population, and recursive takeoff occurs with the majority.

As far as I can tell, the only situation in which AGI do not wipe out human life is if human life is an explicit goal of the AGI. Suddenly “That’s science fiction. We’re not doing that.” doesn’t sound so reassuring.

I’m sure there’ll be plenty of criticism to the ideas presented here. As further reading, I’d suggest Michael Anissimov’s blog. As always, I’m not saying that I know all this is absolutely true, but with existential risks it’s better to be cautious, methinks. As always, be sure to watch out for anthropomorphizing the AGI in your arguments.

Thanks to gcardinal for the photo.