An artificial neural net is also one of those more theoretical tools. I doubt it would be of much use for an RTS, and for many of the reasons I stated in the above post. Granted it's probably going to apply itself to certain tasks a little easier than genetic algorithms will. A small one that doesn't require training, or much training might work for certain things, but there are still likely better techniques.
Keep in mind that neural nets and genetic algorithms are just approximating techniques. If you have some unknown function, but do know certain properties of it, then you can try to converge on it through some kind of iterative technique, where the fitness function is some measure of the known properties.
In the case of chess, an unknown function might be to find the best next move given the current state of the board. The input is the current state of the board, the output is the move the take, and the fitness function might be some measure of what would stand to lose by making a given move. i.e. if the "NextMove" function outputs a move that puts your opponent in checkmate, then it's a good function and shouldn't be changed. If the "NextMove" function lets your opponent put you in checkmate, then it's a very bad function and needs to be altered somehow so it doesn't produce that bad output.
Keep in mind that you are trying to achieve some global approximation for all possible inputs being mapped to good outputs in each case, but using only some degree of measurement for the goodness of individual function outputs. This is a big problem for functions with discrete inputs and outputs (such as in the above example). Suppose we know whether or not the function needs to be changed to avoid a bad output for a given point. How do we change the function to improve on that point (the input), but not disrupt (too much) other points (the other inputs) which already give good output? Also, what if the input space is very large (like the configurations of a chess board) so it's essentially impossible to test your function on every input. How would you converge on some solution globally using only a few input points? It helps to have some idea of "closeness" for the inputs, so that an adjustment to the function to improve the output for a given input should hopefully also improve the output for nearby inputs that you might never test on.
Ideally what you want is some output that varies continuously with the input. i.e., if the input changes only slightly, then the output should change only slightly. This doesn't seem to be the case with the chess example. If you move one piece slightly (or remove it), it can completely change the "goodness" of a given next move.
Basically those techniques (neural nets, and genetic algorithms) don't really apply very well to the problem at hand.
Where those techniques might come in handy is the following setting. Suppose you have an electrical circuit, where you push a button, and it makes a light blink unsteadily for a short period of time, and then stops. We might want to know the voltage on the wire (output) given some time after the button has been pressed (the input value to the function). We don't know how this voltage varies. It is the unknown function we wish to approximate. (Maybe it varies like a sine wave, but we don't know that at this point). Now suppose we can take some sort of measurement at specific points in time. There are two settings I can see. One is, we can measure the voltage directly (with a voltmeter). This however is a little trivial in that there is no real "learning" process. We simple record the (input, output) pair, and then just interpolate between the measured points. Perhaps with lines, or perhaps a cubic interpolation to smooth things out a bit. If we have enough data points we can create an approximation to the original function. Here, the interpolation uses the continuity concept, in that nearby inputs should produce nearby outputs. The other more interesting case the requires a bit of learning is the following. We are now only allowed to measure the voltage on the line indirectly though the intensity of the light. We do not know the exact voltage given the intensity, but, we are allowed the drive the wire at any voltage we want, and measure the light intensity for comparison purposes (perhaps with a second apparatus, so we can run the two simultaneously). We then start with a guess, say a function with constant output (constant voltage level). We run the comparison, and if at a given point our light is dimmer than the one we are approximating, then we adjust the voltage of our guess up a little at that point, and if it's brighter than we adjust it down a little, and we don't adjust if they are the same. (If there is a big different then maybe we vary the voltage more than if there was a small difference). We iterate this a few times (pushing the start button and measuring). Maybe we only get one point per run, maybe we get a random point in time, or maybe we get some fixed sequence of points each run, but we do eventually get more data from running it more times. To extend these points to a function over the whole interval, we again interpolate between them. Eventually our guess function should start to resemble the original function.
We could try doing the above example with a neural net. In this case, the entire neural net IS the approximating function. It would have one input, the time, and would have one output, the voltage. We train the net, by iterating the bliking cycle a few times. The fitness function is the relative comparison of the light intensities. We use this comparison between the output of the net (our current guess function) and the desired output from the reference system, and use it to train the net (using some training algorithm that adjust weights in the internal nodes of the net). There are a number of training algorithms, that basically corresponds to how the weights are to be adjusted internally given the output of the fitness function. The algorithm selected will affect the convergence speed, and stability (it might not ever converge), and how close to "optimal" the results are (how closely our guess function immitates the reference system).
One thing to note about the above example is that we actually had the reference system to compare with. We don't actually need the reference system. We only need the fitness function. It's easy to develop a fitness function given a reference system, but the fitness function could be developed another way. Maybe instead of trying to immitate the voltage on a wire, we've come to realize that bliking lights may increase the happiness of the local lab rat.
So instead we'd like to make the light blink in such a way so as to maximize the happiness of our lab rat.
To do this, we've implanted electrodes into the brain of the lab rat to measure the level of certain chemicals associated with happiness. :whistle: Our fitness function is then the level of chemicals found in the rat's brain. (thumbsup) Although probably a more interesting and irregular function, it would likely be very hard to converge on. Particularly since the "reference function" might not be the same from run to run. After all, there are other inputs to the rat that will affect it's happiness, and these have not been captured by the training of the neural net. So in this case we have another problem, in that we will probably fail to achieve stability. In this case stability is unlikely since we aren't capturing all input that affects the behavior we are modelling, although even in cases where we are monitoring all relevant input and the reference function isn't changing from run to run, it's still possible to fail to achieve stability, particularly if the wrong training algorithm is used.
A better example might be to vary the voltage on a wire controlling a motor to get a vehicle up to a certain speed while using as little power as possible. Maybe constant acceleration isn't the best. What effect does "flooring it" have?
The examples given so far have been simple functions with one input and one output. They could be modelled as some 2D graph of a function, and trying to get a guess function look like the reference function. We could of course add more inputs. Typically neural nets have many inputs. This just corresponds to a higher dimensional graph. Like a 3D graph of a 2 input function. The same concept still applies. The techniques are basically just trying to approximate some unknown mathematical function.
Genetic algorithms seem even worse in that they can be much more difficult to get to converge and to remain stable. At the heart they're pretty stupid, let's guess a random function and see, type of method. The basic idea is very similar to trying to sort a list by randomly choosing two elements, swapping them, and then checking if the list is sorted, and if it's not, to go back and choose another two random elements. You might never get where you want to go, and you only really get there by getting lucky. Not that your chances are very good in most cases, so expect many many iterations. Learning by guess and check can be very slow, especially when there are many things to check. Granted, I'm sure there are techniques people have used to try and improve on the convergence aspect, but in general it seems to be very poor for this method. It does offer a much greater variety of output guess functions though that may have properties you won't get from a neural net function. Or at least not by one of any reasonable size.
In short, don't expect to code a game AI just by making a neural net or genetic algorithm. These aren't magic methods that you can just plug in and do anything. They're almost pretty much impossible to debug if something isn't quite right. That and if you're looking for a learning AI, you generally want it to learn after the player has done something bad to your AI once. Maybe 3 or 4 times. But certainly not 100000 times, which is generally on the order of how many iterations some of these techniques take to learn a new behavior. If you're going to use something like a neural net, you probably want to hardcode it to a specific manner of operation, or if you insist on training it, then train it before hand, and then probably have it run static in the game. It's fairly unlikely you're going to train one of those things during a game in any useful way.
