One of my current projects is a little turn-based strategy game, somewhat similar to X-Com. (It's actually closer to Lords of Chaos, one of X-Com's predecessors.) I handle states with a basic stack, where the game calls the 'run' method of the state at the top of the stack, then updates the graphics subsystem, pumps the event queue, etc. This is a very clean design for most things. However I now have the problem that during the AITurnState, when a computer player is making its moves, there are certain tasks that take a long time to complete and the system requires that the system returns from the 'run' method periodically (10-30 times a second) so that the main engine can perform housekeeping tasks. This is awkward because I want to be able to write clean linear code like this: Yet during this code, which might take 10 seconds or so to complete normally, I'm going to want to return from inside each of those 3 for loops to allow the screen to update, to register the human player's mouse movement, etc. One option would be to write this as a generator, and add 'yield none' liberally through the routine. Then I'd call it like so: Another way that might sit more comfortably with C++ coders would be to do this as a couple of threads. The first way to use threads for this might be for AITurnState.run() to do little but wait for a second thread to update the AI's creatures. Two problems here are that I'd have to use a mutex so that the graphics system in the first thread didn't try and draw a creature that was moving, or something like that. The second problem would be that a creature could perform 2 actions in the 2nd thread before the 1st thread has had time to display the results of the first, which will result in jerky motion when the player is trying to watch the computer in action. Another option would be to use an Event object for the 2nd thread to signal the first that it could go through the update routine again. That would take place and then the Event would be reset, allowing the 2nd thread to continue with the AI. Only one of the 2 threads would be active at any time and they would be pausing in well-defined places. I see this as being a standard sort of producer-consumer type of system, although having to explicitly manage the yielding is not the elegant design I would like if at all possible. Sadly this is all starting to sound very complicated! So before I embark on implementing one of these schemes, I thought I'd post here to see if anybody with experience of this sort of problem had any hints to lend, especially if that experience applied directly to idiomatic Pythonic features that can make life easier (hence the choice of this forum).

Yes, this is a very vexing problem. It's nice to see other people use python for games; it shows I'm not alone :) I personally like using more of a generator approach, since I don't like the idea of the rest of the system needing to know the internals of that one function; although, I don't like the use of exceptions to stop when the generator is done, either. That exception-checking code should be put into a lower-level routine (the AIGenerator class), so that it ends up looking like this:

If you're instantiating the AI objects as real class objects, then you just put all the important bits of movement and thinking in their update sections. Then, you just make a list of them (in this case, in order of initiative or speed?) and increment through the list with like

for monster in list_of_monsters:
monster.update()

My current game engine uses Lua, so some of the features are named differently, but I most of the concepts carry over. So:

There are really two types of operations that take longer than one update cycle, and they deserve to be handled differently. The first is synchronous updates, as embodied by your move-loop. Coroutines (generators in Python) are the best way I know of to code for this. State machines work too, but they lack the clarity and expressiveness of concurrent code, so I dislike them for this sort of thing. The second is asynchronous processing, which only takes multiple cycles because it requires a buttload of CPU time. Coroutines aren't really ideal here because there aren't discrete slices to execute per-frame. So I think this one deserves an asynchronous thread, given as much processing time as is available for the frame, and is tied to the scripting system by a simple loop that yields as long as the computation isn't complete.

What I ended up implementing is something along those lines. Basically, the main thread loops, drawing the screen and checking the event queue, and each time it yields execution to the AI thread which runs up to a certain point, yields execution back, and the process repeats until the AI thread terminates.

The AI thread has discrete points at which it yields; specifically, every time one of its creatures moves (or performs some other action, once I implement them). The yielding mechanism in both threads is an awkward hack using 2 threading.Event() objects, although I expect something better could be done.

Have you checked Stackless Python? It has explicit support for microthreads and things like that.

Quote: Original post by Sneftel
The second is asynchronous processing, which only takes multiple cycles because it requires a buttload of CPU time. Coroutines aren't really ideal here because there aren't discrete slices to execute per-frame. So I think this one deserves an asynchronous thread, given as much processing time as is available for the frame, and is tied to the scripting system by a simple loop that yields as long as the computation isn't complete.

Do you mean creating a different thread in the main code and then calling Lua from that thread? Would this work? (I'd be too afraid to do it :P).

I'd still prefer coroutines - all that is needed is to call a function that checks the time and yields if appropriate at various points in that computation.

Quote: Original post by Diodor
Do you mean creating a different thread in the main code and then calling Lua from that thread? Would this work? (I'd be too afraid to do it :P).

It is possible to do that. Lua has #defines you can use to trigger a mutex. But in the cases where a new thread is spawned, I don't have it call back into Lua; this is primarily because wherever CPU-intensive stuff is being done, I want to optimize it, and that means writing in C++.

Quote:
I'd still prefer coroutines - all that is needed is to call a function that checks the time and yields if appropriate at various points in that computation.

Yeah, but that requires you to sprinkle extra calls around your code, slowing it down. The OS gives you thread scheduling (almost) for free; no reason not to use it.

Quote: Original post by Sneftel
Yeah, but that requires you to sprinkle extra calls around your code, slowing it down. The OS gives you thread scheduling (almost) for free; no reason not to use it.

Hey, I get lazy like that after a while, and can't be made to return to C unless at gunpoint :)

Anyway, I guess it all depends on specifics. A lua function _may_ do serious data crunching if most of the time it spends is in the C functions it calls.

Quote: Original post by Fruny
Have you checked Stackless Python? It has explicit support for microthreads and things like that.

I'm not sure I need anything like that, after all I'm only creating one additional thread, so the 'weight' of the thread is not an issue. The Stackless website is very poor for documentation anyway and I can't really see any benefits to using stackless Python from a quick glance at it.