Full rundown, including code, to come sometime in the future. (Probably.)

The source code is public domain (except for GreenSock’s stuff, which is Copyright, and you’ll have to download from them), and can be downloaded here: http://raelifin.com/files/ChessInFlash/ChessInFlashSource.zip

If we imagine that each sensor is a dimension, then we can say that distinguishing between images of dogs and not-dogs is a matter of partitioning a multidimensional input-space into two areas. Perception in animals is (almost?) never guided by top-down labels, though; animals learn *common* patterns/shapes, and then use these patterns in higher-cognition.

Finding common patterns in an input space is the problem of clustering. Consider the following graph:

We can create 4 clusters which (due to outside knowledge) we can loosely associate with labels we’ve learned previously.

There are many algorithms that will do this sort of clustering task, but perhaps the simplest is K-Means. Yesterday I built a K-Means clusterer in J. It picks initial cluster-centers using K-Means++, and can scale to any number of dimensions. Here is is, chopping a bunch of random points spread over a circle into seven clusters (diamonds indicate the centroids a.k.a. “means”):

Here is the code on GitHub. Everything is public domain. No need to provide credit.

There are a few bugs, and it’s not at all “balanced”, but it’s mostly playable, and pretty intuitive (IMO). It’s written in HTML5, so unless you’re using Google Chrome, I can’t guarantee it’ll render properly.

Play it here: http://raelifin.com/files/Nyx_1/
Download it here: http://raelifin.com/files/Nyx_1/Nyx.tar.gz

I may work on it in the future, if I get the urge. Right now I’m focused on my non-game studies, though, and I was afraid that this would just get thrown on the pile of unpublished code I have lying around.

P.S. This post is so tall because the code is in an iframe. The code could easily integrate as a library, and I could then make this a normal post, but I want it to be encapsulated for now.

Let me elaborate on that last point, because I think it’s important. Most of what artificial intelligence, as a field, focuses on (and rightly so) is building a model of an environment. This learning involves taking raw sensory data and turning (or rather, integrating) it into a set of concepts and relationships. But with the simple case of a game, we don’t really need to learn what the rules are, or learn how to take actions; we can focus entirely on how to play well.

Playing well involves maximizing a value, which depends on the game. In poker, this value is “money owned”, while in chess it’s “enemy kings captured*”. (In chess, the game never progresses to the point where any kings are captured, but as far as goals are concerned, checkmating the king is identical to capturing it.) Another way of putting this is to point out that each game has a distinct goal. Playing well involves maximizing the degree to which this goal is met, which I will call “utility“.

* – It’s technically a little bit more complicated than that, because it’s possible to stalemate, and that’s not as bad as an outright loss.

So, when presented with several possible “moves” in a game, we want to pick the move that has the highest utility. I call this searching and choosing “being rational”. To make a game-playing AI then, I needed to build a Rational Agent.

To be a bit more formal about the agent’s task, it must output a move when given:

• A “state” where it currently is.
• A “rulebook” (a.k.a. model) for how the game works, including what moves can be made at the given state.
• A utility function that, given a state, returns the degree to which it meets a goal.

The utility function is conceptually separate for me from the rulebook, as it specifies exactly what this agent “cares about”, where the rules are objective and unchanging regardless of who you are. (Example: the rulebook for chess doesn’t tell you which player you are.) Another reason that I think of them as conceptually distinct is because as I’m defining the utility function, it is agnostic to consequences. This is a fancy way of saying that the utility function doesn’t pay attention to what might happen next; it’s firmly grounded in the “present” (i.e. whatever state it’s given). Because of this agnosticism, it doesn’t actually rely on any information about game dynamics (rules).

The job of considering consequences is exactly what the Rational Agent is built to do. It takes each action in turn and finds the utility of its immediate consequences, then adds the utility of the next state, and the utility of next… etc. In other words, it is the limit of the infinite sum of the utility function from t=0 to infinity.

Or at least, it would be if it were a PERFECTLY rational agent. The unfortunate reality (from a math perspective) is that most systems are chaotic enough that it is damn near impossible (and sometimes just plain impossible) to take such an integral. As an example, consider a game (like chess or whatever) where the rule is that your opponent will play exactly as you would in the other position. If you try and compute the consequences of an action, you have to consider the moves of the opponent, which will involve considering your moves, to an infinite regression.

I wrote a post on this problem several weeks ago. It had some fancy math that lets some infinite regressions be solved, but I couldn’t figure out any fancy tricks that worked for all situations. I intuitively don’t think it makes much sense trying to compute the limits, either. If I’m resigned to building an agent that is not perfectly rational, but is only somewhat rational, than I can simply say that at some point the agent stops recursively “imagining” the future, and works on the approximate sum.

The problem with this, however, is that many games are not won until very late (a.k.a. after many moves). Consider a game I invented to test my agent:

Counting

or perhaps “Lineland Treasurehunt”

You are at a position, and your goal is to get to another position. Possible moves are “walk forward” and “walk backwards”. Oh, and the spot you start at is called “0″ and the goal spot is called “100″. Each time you walk forward your position increases by one, and each time you walk backward, it decreases by one.

The problem is that, as defined, there’s no base utility in being at “99″ over “0″. The only utility in being at “99″ is in the consequence of being able to go to “100″ on the next turn. If each step is a move, then the agent needs to model 100 moves in the future to have any clue what to do. Now, imagine how many moves one would need to look-ahead in order to see a victory in chess. For a perfectly rational agent it’s a mathematical possibility, but for something that can be written in code it’s just impractically beyond the maximum recursion depth.

The only way an agent with a maximum look-ahead of 5 moves can beat the Counting game is to get the hint “high numbers are better than low ones”. According to the rules, this is not intrinsically true, but it’s an approximation of truth that allows the agent to perform under normal circumstances. It is a heuristic, and it is of vital importance.

To be more precise, a heuristic is a component of the utility function that has been added because it is a good approximation of future utility (i.e. the goal). Some heuristics in chess are “control the center of the board”, “capture (non-king) pieces”, and “threaten many squares”. Clearly you don’t want to focus on doing any of these if you have the opportunity to checkmate, but because checkmating takes time, it’s cognitively easier to aim for any of these more apparent mini-goals.

The learning of heuristics seems like the most important aspect of practical rationality, and it’s something I’m looking forward to thinking about more. It seems plausible to me that the emotion of happiness in humans may be a heuristic-encoding signal. When we’re happy or sad, we say “What’d I just do? Remember presence/absence of that as a heuristic!”. Just speculating…

Here’s my rational agent in Haskell: http://raelifin.com/files/code/rationality.tar.gz

(I also wrote a version in Java with many more example games, but I’m working on tuning that, and it’s not really suitable for posting at the moment. The Haskell is also much cleaner/prettier.)

N.B. This is almost certainly old ground. Reinforcement learning or alpha-beta pruning almost certainly would be good things for me to study. My ignorance on the subject is not meant to imply that there’s not already a lot out there.

• You can now set cells to be drawn as images. Defining the image a cell should use is as simple as making a rule of “draw <CELLTYPE> <URL>” (See Langton’s Ant for example.)
• The directions northeast, northwest, etc. have been added, as have fareast, farnorth, etc. The “far____” directions refer to cells two steps away.
• The special noun “neighbors” now includes the NULL cells at the border.
• New verbs “cellAt” and “neighborsOf” have been added. They both take a 2-by-1 array ([x,y]) of the cell in question as their right-parameter. The “cellAt” verb returns the type of the cell in question, while “neighborsOf” returns the set of neighbors.
• A new two-parameter verb “+” has been added. Right now it only operates on the 2-by-1 coordinate arrays discussed above. (Example: [1,-10] + [23,7] = [24,-3])
• In the same vein, a new special noun called “thisPos” has been added. It is the coordinates of the cell being evaluated.

That’s all!

For those interested in writing your own automata, here is a quick guide to the syntax:
An automaton is composed of rules. Each rule is designated by a new line. Double slash (//) or pound (#) designate a commented line.
Each rule looks something like BLACK->WHITE: 1 = 2
BLACK is the type of cell that the rule applies to, WHITE is the type of cell that it will become. After the colon is a statement that must resolve to a single value, or “noun”.
Nouns can be strings, numbers, true/false, JSON arrays, “neighbors”, or any of the cardinal directions (lower-case). The noun “neighbors” is a special noun which is a JSON array with values equal to the types of the surrounding tiles (in eight directions). The special nouns “west”, “north”, “south” and “east” are equal to the type in that direction, or “NULL” if at the edge.
Nouns can be acted upon by the following “verbs”:

• =, >, < — Basic comparison operators. They behave like their counterparts in javascript (except for =, which is comparison, not assignment). — example: “1 > 2 = false” (returns true)
• or, and — Low-priority logic operators. These behave like || and && in most languages. Low-priority means they are evaluated after other verbs. — example: “true and false or true” (returns true)
• not — Inverts the boolean value to the right. If the value to the right is non-boolean, it is cast to a boolean just like ! in javascript. — example: “not true” (returns false)
• contains, isIn — Check if a value is contained in an array. The two verbs are identical except in the order of their parameters (“value isIn array”, or “array contains value”). — example: “[1,2,3] contains 0″ (returns false)
• count — Changes the noun on the right from an array to an integer. — example: “count [0,1,2]” (returns 3)
• ofType — Filters an array on the left to only contain the value on the right. — example: “[0,0,0,1,2,3] ofType 0″ (returns [0,0,0])
• echo — Makes a popup containing the value on the right. Does not change the data. This verb isn’t very useful, and can flood your browser with popups if you aren’t careful.

Verbs are evaluated from left-to-right, but you can use parentheses to change the order of operations. More examples can be found by loading the prebuilt automata.

I know the interface is a bit hard to use, but if you make anything (even if it’s pretty trivial) please leave a comment! You can export boards as well, so feel free to share interesting formations in Conway’s Game of Life, Wireworld, or others.

On a related note, this talk by Wolfram is not to be missed: