quote: Original post by Kars
Well, if you are trying for a ''natural environment'', why have an "IQ function" at all. Their fitness chould be when two critters cross each others path, do they procreate or not?
If they do, you get one or more critters.
If not they both go on their merry way.
If a critter dies before it procreates, its not longer in the gean (sp?) pool.

This way there would be no fixed population size and you have have the possability of extention if no one procreates.

I was thinking about doing something like this just to see if it was possable but I''m still reading up on Genetic programming myself.

It certainly is possible... simulated environments like this have been used for many years to study the evolution of various population traits. Several people in my department use a simulated environment to study evolvable ethics. For example, they found that they could evolve suicide/euthenasia as a reasonable population strategy for greater survivability. (Note: I''m not suggesting nor advocating such a policy for human populations, merely reporting experimental results in an artificial population.)

I have been particularly interested in such environments for looking at evolvable communication and how it can lead to cooperative behaviours among agents. I just haven''t gotten around to finishing my world yet. Some of the things that I described in my earlier post are in my current world, although I have wolves instead of brutes! It''s one of those pet projects that''s sitting on my desk at home... you know the ones... you pick them up every few months and spend a day reminding yourself what you were doing, only to run out of time and have to put it down again!

Cheers,

Timkin

Evolvable communication? I remember a graduate project about learning this, which you may be interested in. This paper was written about it and i don't know if the corresponding thesis is still available electronically.
The agents had to learn to catch small and large prey, where the large prey could not be caught by a single agent.
Very rudimentary communication (wagging the tail when spotting a large prey) was introduced. Only in about 2 percent of the situations this would add relevant information, in most cases it was obvious what the agent had to do even without communication. Still the increase of cooperation (catching large prey) increased by 10 percent. It turned out that the communication introduced a bias in training samples used to learn the cooperative behavior. It was trained with "cleaner data" so that cooperation became more effective....

btw: It is very tricky to setup a world to notice something interesting, specially when it involves communication. If you do not choose the number of agents (prey and predators) correctly, the agents may always say the same thing (i see it, or i don't see anything). It is very hard to get them to say different things that are still relevant (that's where the above mentioned 2 percent comes from).


[edited by - smilydon on August 28, 2002 2:45:51 PM]

I haven't read all the posts here, I just figured I'd inject an observation.

The creatures in Black and White can learn a LOT. Through conditioning, you teach them how to behave, in many ways and many situations. It would be easy enough to evaluate how much they've learned, quantitatively, but how would one evaluate the actual IQ score of the creature? Especially if you are teaching it to do stupid things, nevertheless it learns to perform a LOT of stupid things?

The IQ scale would need to have a basis upon what something with a high IQ is more successful at. In Black and White, what would that be, exactly? You need that kind of a scale for your existing system to base the number on.

[edited by - Waverider on August 28, 2002 3:07:09 PM]

quote: Original post by smilydon
This paper was written about it and i don''t know if the corresponding thesis is still available electronically.

Thanks for the reference smilydon... I''ll check the paper out when I can find a free moment!

Personally I''m interested as to whether communication can be learned in the following manner... each agent has 3 senses: sight, sound and smell. Among their various actions they have the ability to make a range of sounds (simple strings). I want to see if reinforcement learning can be used to develop contextual correlations between sounds made/heard and cooperative actions that provide reinforcement. For example, caveman 1 sees a wolf. IF he happens to make a sound, perhaps ''Ug'', and caveman 2 hears ''Ug'' along with seeing the wolf, then there is the possibility of creating a correlation between that sound ''Ug'' and the sight of a wolf. This is exactly the same sort of reinforcement learning investigated by Pavlov.

If each caveman knows that wolves provide food and warmth (whether learned or innately known) then they can both attack the wolf with the pair being stronger than the two individuals. Thus, at some future time, caveman 2 sees a wolf and says ''Ug''. Caveman 1 knows that ''Ug'' means theres a wolf nearby and reacts accordingly.

Well, that''s the presumption... I''m yet to finish the world and the learning system... but they are coming along (slowly).

Cheers,

Timkin

Timkin, it''s a reference to the result of a graduate project, please keep that in mind when reading it. There are more references to learning to cooperate, but they deal mostly with environments where the agents have no alternative. You either cooperate or you die. In this way you can only investigate the quality of the communication mechanism. Also there is lots of work on learning communication and building lexicons (many Belgians do it, see Steels, de Jong or Vogt). In this area the quality of the communication is related to the ability to make a correct associations and so it is not related to the usefulness of this correct association to fulfill a certain task.

This project was an investigation in the influence of (learning/nonlearning) communication on the probability that cooperative behavior emerged. The quality of communication was related to the completion of a task and the agents had the choice not to cooperate because there was an alternative (the small prey). The agents used vision (based on a CMAC tiling around the agent) and an extra sensor (smell or sound, also CMAC) used to detect colleges and what they communicate. I think this looks like your idea, except that the communication was kept extremely simple.

When you create your world you have to keep in mind what the contribution of the communication is (when is it relevant). In this project there were many situations where communication did not add anything. I.e. the agent sees a college next to a large prey, so the only obvious action is to go there. Communication turns out to be only effective in situations where there is a DILEMMA (even if you forget this whole post, that DILEMMA thing you should remember!!!). I.e. the agent sees a small prey to the left and a college to the right. If the college does not see a large prey then the agent should go to the small prey (going to the college reduces the probability of catching small prey). If the college indicates that it sees a large prey, then depending on how rewarding large prey is, the agent may consider going to the college. The college sees that the agent does not see a large prey and keeps away from him, and thus approaches the large prey. This is sort of the mechanism where communication works, going to a college is bad unless he tells you.... Because the situations where communication is effective are very rare, learning to communicate becomes very slow. But even if it is not learned very well it will have a significant effect on the probability to cooperate

I do not agree with your Pavlov example. In Pavlovs experiments a different stimulus (the bell) resulted in the same reflex as the original stimulus. In other words, the same action was associated with a different observation. In your caveman example, seeing a wolf means that you know where it is so you know what action to take (go there or run away). The sound "Ug" of an other caveman does not tell you where the wolf is so you cannot choose the appropriate action. Unless of course the word "Ug" means "i see a wolf on my right". In that the case the communication becomes more involved and a language has to be learned. This is no longer a replacement of stimuli a la Pavlov......

quote: Original post by smilydon
I do not agree with your Pavlov example. In Pavlovs experiments a different stimulus (the bell) resulted in the same reflex as the original stimulus. In other words, the same action was associated with a different observation.

Yes, that''s one way of stating it. Another is that a correlation between the bell and salivating was learned by correlating the bell with food, which induced salivation.

quote: Original post by smilydon
In your caveman example, seeing a wolf means that you know where it is so you know what action to take (go there or run away).

No. You''re assuming that the stimulus-response behaviour with regards to wolves is already in place and thus you are assuming that the caveman knows what action to take when it sees a wolf.

quote: Original post by smilydon
The sound "Ug" of an other caveman does not tell you where the wolf is

Correct. It is merely an associative sensory input (which is why I likened it to Pavlov''s Dog... the bell was an associative input as well).

quote: Original post by smilydon
so you cannot choose the appropriate action.

The caveman can choose ''AN'' action and receive reward accordingly (or get eaten by the wolf... natural selection at work!)

quote: Original post by smilydon
Unless of course the word "Ug" means "i see a wolf on my right". In that the case the communication becomes more involved and a language has to be learned. This is no longer a replacement of stimuli a la Pavlov......

It would be ''Pavlovian'' if upon seeing a wolf and hearing ''Ug'' the caveman ran away (and received some reward for surviving another day) and then later, upon hearing ''Ug'' again (without seeing the wolf) also ran away and received reward.

At this stage I''m not seeking cavemen that learn grammar, nor detailed semantics... I just want to see how far I can go with associative inputs.

Cheers,

Timkin

Sounds like you have a nice project going Timkin, how about a demonstration?

Me too, I''d love to see more demos of cool AI posted in this forum. The only demo I''ve ever seen here is your (Timkin''s) ''hunter'' undergraduate project, and that was somewhat inspiring, even if you said that it was pretty basic.

Cédric

quote: Original post by MagicScript
Sounds like you have a nice project going Timkin, how about a demonstration?

It''s no where near demo standard yet... sorry... but of course, when it is, I''d be happy to post a sample to the boards!

Timkin