hi to all I have just decided to implement some (=very heavy) scripting in my engine, and because I do NOT want to create my own language (and so write my own interpreter with it), i want to use python scripting. Question: HOW do I "run" a script (from C++ program) and how do I get "results" from it? (result=what the script wants my engine to do). Is there anything like #include <python.h> (isn''t...) or what dllibrary should I load and how should I execute it? thanks a lot exa_einstein

http://docs.python.org/ext/ext.html
extending and embedding the python interpreter
but even more useful:
http://docs.python.org/api/api.html
the Python/C API Reference Manual. Read it. Front to back, if you can. C-defined custom datatypes are teh shiznit.

There are two ways to do this - the easy way and the smart way. The easy way is to just run a string. This is wasteful, because you compile each time you run. If that's all you want, go to http://docs.python.org/api/veryhigh.html and see how its done. Or you can do it the rigth way.

Okay, I've done this extensively in Python, so here's a crash course: first, you must compile your code. Python compilation is really minimal, basically converting the local namespace into an array and that's about it. Hell, it doesn't even pull out the docstrings (in case you want to inspect them at run-time - lots of scripts do this).

easiest way: Py_CompileString
other functions allow higher granularity of control.

Keep the resultant PyObject for future use so you don't have to compile over and over. Note - it is a PyObject, so you have to deal with refcounts. If you don't know how to deal with refcounts, more work is in order - in short, whenever you create an object it gets a refcount of 1. Whenever you return an object to the interpreter, it should have a refcount of 1 higher then when it entered, as it will be decref'd if there is no holder waiting it, so...

2+2

creates a new integer object of value 4, and that integer object has a refcount of 1. However, there is no holder for this integer object (instead it would be frank=2+2), so instead it is sent straight to the interpeter, which calls PyObject_Repr(PyObject*) (this dumps a string representation of the object to the stdout file or whatever else is appropriate at the time) on it and then decref's it. Get used to Py_INCREF() and Py_DECREF().

Good example of idiosynchrasies to learn about: when returning nothing in a function, you must call Py_INCREF(Py_None); and then return Py_None;. Why? Because the Py_None object is still a fully refcounted object. Alternately, when inserting an object into a structure like a class or a dictionary, then you can assume it is being incref'd - but think - if it is a newly created object, then its refcount is already 1. If you're not planning on returning it naked to the intepreter, its refcount will be 2, and its only reference will be within the structure, so when that structure is destroyed it will reduce to 1 and thus persist even tho no references to it exist (gack! memory leak!). For this reason, Guido made the inconsistent and frustrating decision that Py_List and Py_Tuple objects, when inserted into, do _not_ incref their subjects. Keep this in mind. Bad refcounts can lead to crashing failures or memory leaks.

Back on topic - Now, you take the compiled code object and feed it into the interpreter. Pick through the header files for the execution function that fits your need, it'll be pretty easy to find. Look for exec or module functions. Unfortunately these functions don't exist in the Python/C api documenation, but are very friendly functions in the header files. Anyhow, here's the one that I use, its probably the best fit for your purposes:

PyObject* PyEval_EvalCode(PyCodeObject *code,
PyObject *globalnamesdict,
PyObject *localnamesdict);

where *code is our code object, globalnamesdict is the global namespace, and localnamesdict is the local namespace. We have to supply the namespaces because we aren't running it from a consistent single interpreter. Remember that these change all the time - each module has its own globals, and each block has its own locals. Now, if you're clever, you'll notice that there is no place to supply arguments. This is the problem with the embedding the Python interpreter - it really isn't meant for simply compiling a block of text and calling it a function - you just get a script, that you run. You can do some tricks by supplying odd values into the namespaces, but thats it. Most of the time for a set script your locals and globals will be the same (if you're making a module) or your locals will be empty (just supply NULL which makes it create its own locals, I think - I could be wrong - perhaps supply your own locals - hell, look in the headers, don't take my word for it).

The other way to make a Python script runnable from the outside, and the way they intended you to do so, is to make it as a module. Now, if you've used the interpreter, you know how to make a module. You just code a bunch of functions in a file. Now, to make this module exposed in C, you work as follows:

PyImport_ImportModule() if you want it to just work as normal, fetching the file itself and loading the module into the sys.modules dict. This returns you the module. However, if you're like me, this is pretty dull and poor control.

Instead, compile your code object as before, but this time take your code object and use
PyObject* PyImport_ExecCodeModule( char *name, PyObject *co)
that executes it and loads it into a module object - fresh, in your hands, not sitting in sys.modules for other script to play with, all yours (you can insert it into there if you want people to be able to import it later on in your program - this is a handy way to build a module system without using the filesystem at all).

Now, we've got our module. Then we pluck out the dictionary with PyModule_GetDict() or its high-speed macro counterpart, and then iterate through our dictionary with
int PyDict_Next( PyObject *p, int *ppos, PyObject **pkey, PyObject **pvalue) - this is kinda difficult to explain, use the description at http://docs.python.org/api/dictObjects.html to see how to use it - its pretty easy once you understand. Anyhow, with that you can create a C datastructure full of your Python functions, compiled, optimized, and ready to call instantly as you need.

To call them, just use int PyCallable_Check( PyObject *o)
and PyObject* PyObject_Call( PyObject *callable_object, PyObject *args, PyObject *kw)

where *args is a tuple and *kw is a dictionary. This corresonds to calling in Python myfunc(1,2,3, a="alpha, b="beta") which would produce a tuple (1,2,3) and a dictionary('a':'alpha', 'b':'beta') in C.

There are easy ways to build Python argument tuples in C, most particularly
PyObject* Py_BuildValue( char *format, ...)
which is in
http://docs.python.org/api/arg-parsing.html

You just feed it a format string and the C objects and it builds a corresponding argument tuple of Python objects for you.

Anyhow, you call your functions, you get your return values, check for errors (look it up) then read the return values (using either individual typechecks and api uses, or PyArg_UnpackTuple or PyArg_ParseTuple if your expecting the funciton to return a tuple of values, as in:

return size, weight, speed

Embedding Python isn't simple, but its powerful and rewarding. Good luck.

[edited by - Pxtl on June 7, 2004 9:40:02 AM]

[edited by - Pxtl on June 7, 2004 9:41:31 AM]

Only help I have for refcount is keep the header files open at all times, - they are usually informative as to whether there will be a ref increment. Usual convention - insertion increments, accessing does not, removal decrements. Exception to insertion incrementing is only the PyList_Insert and the PyTuple_Insert, both of which "steal" references (they do not incref), which can be circumvented by using the PySequence abstraction (which incref''s). Occasionally I find accessing functions that incref, but they''re less common. The main accessing functions that incref are functions that wrap actual Python functions, like GetAttr and suchlike.

Boost is good for you.
Now I do the same job, and I found it is so cool.
By the way, you can search some useful info in previous posts in th forum.

Ok guys thank you

bye


(somebody please close this thread.....)

[edited by - exa_einstein on June 9, 2004 8:02:21 AM]

Umm - we covered that. If its a module in your module path, you import the module, then you build a PyString object adn use it with PyObject_GetAttr() to fetch the function, then use PyObject_Call to call the function.