🎉 Celebrating 25 Years of GameDev.net! 🎉

Not many can claim 25 years on the Internet! Join us in celebrating this milestone. Learn more about our history, and thank you for being a part of our community!

Compiler crash, r1777 -> r1778

Started by
11 comments, last by j3p 10 years, 1 month ago

Hi,

The compiler crashes while trying to access the first element in engine->scriptFunctions which is null in CompileOverloadedDualOperator2() (Windows, x64). I tried a few different versions with relevant changelogs and found that the problem appears when upgrading from r1777 to r1778 (still happens with r1914).
Here's roughly what happens:


 	angelscript.dll!asCString::Compare(const char * str)  Line 291 + 0x23 bytes	C++
 	angelscript.dll!operator==(const asCString & a, const char * b)  Line 340 + 0xf bytes	C++
>	angelscript.dll!asCCompiler::CompileOverloadedDualOperator2(asCScriptNode * node, const char * methodName, asSExprContext * lctx, asSExprContext * rctx, asSExprContext * ctx, bool specificReturn, const asCDataType & returnType)  Line 10781 + 0x1c bytes	C++
 	angelscript.dll!asCCompiler::CompileOverloadedDualOperator(asCScriptNode * node, asSExprContext * lctx, asSExprContext * rctx, asSExprContext * ctx)  Line 10736 + 0x7b bytes	C++
 	angelscript.dll!asCCompiler::CompileInitialization(asCScriptNode * node, asCByteCode * bc, asCDataType & type, asCScriptNode * errNode, int offset, unsigned __int64 * constantValue, int isVarGlobOrMem)  Line 2373 + 0x32 bytes	C++
 	angelscript.dll!asCCompiler::CompileDeclaration(asCScriptNode * decl, asCByteCode * bc)  Line 2110 + 0x46 bytes	C++
 	angelscript.dll!asCCompiler::CompileStatementBlock(asCScriptNode * block, bool ownVariableScope, bool * hasReturn, asCByteCode * bc)  Line 1027	C++
 	angelscript.dll!asCCompiler::CompileFunction(asCBuilder * builder, asCScriptCode * script, asCArray<asCString> & parameterNames, asCScriptNode * func, asCScriptFunction * outFunc, sClassDeclaration * classDecl)  Line 552	C++
 	angelscript.dll!asCBuilder::CompileFunctions()  Line 724	C++
 	angelscript.dll!asCBuilder::Build()  Line 241	C++
 	angelscript.dll!asCModule::Build()  Line 230 + 0xe bytes	C++

(engine->scriptFunctions).array,8	0x0000000003a40100	asCScriptFunction * *
		[0]	0x0000000000000000 {refCount={...} gcFlag=??? engine=??? ...}	asCScriptFunction *
		[1]	0x0000000003c0bec0 {refCount={...} gcFlag=false engine=0x0000000003c32040 ...}	asCScriptFunction *
		[2]	0x0000000003c0bd80 {refCount={...} gcFlag=false engine=0x0000000003c32040 ...}	asCScriptFunction *
		[3]	0x0000000003c0bc40 {refCount={...} gcFlag=false engine=0x0000000003c32040 ...}	asCScriptFunction *
		[4]	0x0000000003c0bb00 {refCount={...} gcFlag=false engine=0x0000000003c32040 ...}	asCScriptFunction *
		[5]	0x0000000003c0b9c0 {refCount={...} gcFlag=false engine=0x0000000003c32040 ...}	asCScriptFunction *
		[6]	0x0000000003c0b880 {refCount={...} gcFlag=false engine=0x0000000003c32040 ...}	asCScriptFunction *
		[7]	0x0000000003c0b740 {refCount={...} gcFlag=false engine=0x0000000003c32040 ...}	asCScriptFunction *
(ot->methods).array,8	0x0000000003c35300	int *
		[0]	0	int
		[1]	103	int
		[2]	104	int
		[3]	105	int
		[4]	106	int
		[5]	107	int
		[6]	108	int
		[7]	109	int

The object that causes it is a vector3 pod with some typical math methods. What's interesting though is that removing the swizzler


template<uint a, uint b, uint c>
PxVec3 Float3Swizzle3F(const PxVec3 *p){
	return PxVec3((*p)[a],(*p)[b],(*p)[c]);
}

template<uint i>
class Float3Swizzle3{
public:
	static inline void Do(asIScriptEngine *psg){
		char sw[32];
		sprintf(sw,"const float3 get_%c%c%c() const",i/9+0x78,(i/3)%3+0x78,i%3+0x78);
		psg->RegisterObjectMethod("float3",sw,asFUNCTION((Float3Swizzle3F<i/9,(i/3)%3,i%3>)),asCALL_CDECL_OBJFIRST);
		Float3Swizzle3<i-1>::Do(psg);
	}
};
	
template<>
class Float3Swizzle3<0>{
public:
	static inline void Do(asIScriptEngine *psg){
		psg->RegisterObjectMethod("float3","const float3 get_xxx() const",asFUNCTION((Float3Swizzle3F<0,0,0>)),asCALL_CDECL_OBJFIRST);
	}
};

Float3Swizzle3<27-1>::Do(psg);

makes the problem vanish. It's strange because enabling a similar swizzler for vector4 type has no effect (despite the fact that it spams multiple times more methods than the code above). Currently I'm hacking around by putting some null-checks in some places.

I'll be happy to provide more info if needed...

Advertisement

The problem seems to be related to a bug in the registration of the type, and not in the compiler itself. None of the object types' methods should use the method id 0. 0 is reserved for 'no function', which is why the engine->scriptFunctions[0] is null.

Can you show me how you've registered the vector3 type with all the swizzlers? I don't need the actual implementation, only the calls to the engine's registration methods.

AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

Sure, here's the relevant part:



#define RegisterVectorMethods(n,t,c) void RegisterVectorMethods_##t(asIScriptEngine *psg){\
	psg->RegisterObjectMethod(n,n" &opAssign(float)",asFUNCTION((SCRIPTF_assign<t>)),asCALL_CDECL_OBJFIRST);\
	psg->RegisterObjectMethod(n,n" &opAddAssign(const "n" &in)",asMETHODPR(t,operator+=,(const t&),t&),asCALL_THISCALL);\
	psg->RegisterObjectMethod(n,n" &opSubAssign(const "n" &in)",asMETHODPR(t,operator-=,(const t&),t&),asCALL_THISCALL);\
	psg->RegisterObjectMethod(n,n" &opMulAssign(float)",asMETHODPR(t,operator*=,(float),t&),asCALL_THISCALL);\
	psg->RegisterObjectMethod(n,n" &opDivAssign(float)",asMETHODPR(t,operator/=,(float),t&),asCALL_THISCALL);\
	psg->RegisterObjectMethod(n,n" opAdd(const "n" &in) const",asMETHODPR(t,operator+,(const t&) const,t),asCALL_THISCALL);\
	psg->RegisterObjectMethod(n,n" opSub(const "n" &in) const",asMETHODPR(t,operator-,(const t&) const,t),asCALL_THISCALL);\
	psg->RegisterObjectMethod(n,n" opMul(float) const",asMETHODPR(t,operator*,(float) const,t),asCALL_THISCALL);\
	psg->RegisterObjectMethod(n,n" opMul_r(float) const",asMETHODPR(t,operator*,(float) const,t),asCALL_THISCALL);\
	psg->RegisterObjectMethod(n,n" opMul(const "n" &in) const",asMETHODPR(t,multiply,(const t&) const,t),asCALL_THISCALL);\
	psg->RegisterObjectMethod(n,n" opDiv(float) const",asMETHODPR(t,operator/,(float) const,t),asCALL_THISCALL);\
	psg->RegisterObjectMethod(n,n" opNeg() const",asMETHODPR(t,operator-,(void) const,t),asCALL_THISCALL);\
	psg->RegisterObjectMethod(n,"bool opEquals(const "n" &in) const",asMETHODPR(t,operator==,(const t&) const,bool),asCALL_THISCALL);\
	psg->RegisterObjectMethod(n,"float length() const",asMETHODPR(t,magnitude,(void) const,float),asCALL_THISCALL);\
	psg->RegisterObjectMethod(n,"float LengthSq() const",asMETHODPR(t,magnitudeSquared,(void) const,float),asCALL_THISCALL);\
	psg->RegisterObjectMethod(n,"float normalize()",asMETHODPR(t,normalize,(void),float),asCALL_THISCALL);\
	psg->RegisterObjectMethod(n,n" GetNormalized() const",asMETHODPR(t,getNormalized,(void) const,t),asCALL_THISCALL);\
	psg->RegisterObjectMethod(n,"float dot(const "n" &in) const",asMETHODPR(t,dot,(const t&) const,float),asCALL_THISCALL);\
	psg->RegisterObjectMethod(n,"bool IsZero() const",asMETHODPR(t,isZero,(void) const,bool),asCALL_THISCALL);\
}

RegisterVectorMethods("float3",PxVec3,3);

template<uint a, uint b>
static PxVec2 Float3Swizzle2F(const PxVec3 *p){
	return PxVec2((*p)[a],(*p)[b]);
}

template<uint i>
class Float3Swizzle2{
public:
	static inline void Do(asIScriptEngine *psg){
		char sw[32];
		sprintf(sw,"const float2 get_%c%c() const",(i/3)%3+0x78,i%3+0x78);
		psg->RegisterObjectMethod("float3",sw,asFUNCTION((Float3Swizzle2F<(i/3)%3,i%3>)),asCALL_CDECL_OBJFIRST);
		Float3Swizzle2<i-1>::Do(psg);
	}
};

template<>
class Float3Swizzle2<0>{
public:
	static inline void Do(asIScriptEngine *psg){
		psg->RegisterObjectMethod("float3","const float2 get_xx() const",asFUNCTION((Float3Swizzle2F<0,0>)),asCALL_CDECL_OBJFIRST);
	}
};

template<uint a, uint b, uint c>
static PxVec3 Float3Swizzle3F(const PxVec3 *p){
	return PxVec3((*p)[a],(*p)[b],(*p)[c]);
}

template<uint i>
class Float3Swizzle3{
public:
	static inline void Do(asIScriptEngine *psg){
		char sw[32];
		sprintf(sw,"const float3 get_%c%c%c() const",i/9+0x78,(i/3)%3+0x78,i%3+0x78);
		psg->RegisterObjectMethod("float3",sw,asFUNCTION((Float3Swizzle3F<i/9,(i/3)%3,i%3>)),asCALL_CDECL_OBJFIRST);
		Float3Swizzle3<i-1>::Do(psg);
	}
};
	
template<>
class Float3Swizzle3<0>{
public:
	static inline void Do(asIScriptEngine *psg){
		psg->RegisterObjectMethod("float3","const float3 get_xxx() const",asFUNCTION((Float3Swizzle3F<0,0,0>)),asCALL_CDECL_OBJFIRST);
	}
};

void RegisterScriptMath(asIScriptEngine *psg){
	psg->RegisterObjectType("float3",sizeof(PxVec3),asOBJ_VALUE|asOBJ_POD|asOBJ_APP_CLASS_ALLFLOATS|asOBJ_APP_CLASS_CAK);
	psg->RegisterObjectProperty("float3","float x",offsetof(PxVec3,x));
	psg->RegisterObjectProperty("float3","float y",offsetof(PxVec3,y));
	psg->RegisterObjectProperty("float3","float z",offsetof(PxVec3,z));
	psg->RegisterObjectBehaviour("float3",asBEHAVE_CONSTRUCT,"void f(float, float, float)",asFUNCTION(Float3Constructor1),asCALL_CDECL_OBJFIRST);
	psg->RegisterObjectBehaviour("float3",asBEHAVE_CONSTRUCT,"void f(const float2 &in, float)",asFUNCTION(Float3Constructor2),asCALL_CDECL_OBJFIRST);
	psg->RegisterObjectBehaviour("float3",asBEHAVE_CONSTRUCT,"void f(float, const float2 &in)",asFUNCTION(Float3Constructor3),asCALL_CDECL_OBJFIRST);
	psg->RegisterObjectBehaviour("float3",asBEHAVE_CONSTRUCT,"void f(float)",asFUNCTION(Float3Constructor4),asCALL_CDECL_OBJFIRST);
	
	Float3Swizzle2<9-1>::Do(psg);
	Float3Swizzle3<27-1>::Do(psg); //removing this fixes the problem
	//Float3Swizzle3<2>::Do(psg) registers the maximum set without crashing: xxz, xxy, xxx. Next one would include xyx.
	
	RegisterVectorMethods_PxVec3(psg);
	
	psg->RegisterObjectMethod("float3","float3 cross(const float3 &in) const",asMETHODPR(PxVec3,cross,(const PxVec3&) const,PxVec3),asCALL_THISCALL);
}

Thanks. Unfortunately I'm not able to reproduce the problem.

For some reason the asCObjectType that represents your 'float3' type has a method id 0 in methods[0], and that it what is causing the compiler to crash. The crash is easy enough to fix by adding a check for a null-pointer, but the root cause to it all is a different matter.

The first method that you register for the float3 type is the "const float2 get_zz() const" method. I doubt it gets the method id 0, which means that some time later the first entry in methods[0] is overwritten by 0. This feels like it may be some kind of buffer overflow problem hidden somewhere, and unless fixed it can potentially cause other problems in the future.

Can you try to determine when the methods[0] in the object type gets set to 0? Does it happen while registering Float3Swizzle3 or sometime later? When you call Build on the module, is the methods[0] already 0?

AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

The methods array is still ok after script api registration. Only after some random CompileFunction() the first element is set to 0. No Build() calls have been made so far.

But I'm not sure about anything anymore. I tested with two memory allocators, Intel's scalable and the system default. When using Intel's allocator something overwrites the first element in methods, maybe because the allocated block is more continuous and optimized. (Note that I've always used the Intel's allocator for angelscript and it's been working so far). The default allocator on the other hand produces different behaviour. I'm getting "empty script section" errors. This may or may not be related...

Some of the script sections in single module look like this after the preprocessor stage:

script.as, AddScriptSection

(mostly empty, some newlines or spaces)

header.as, AddScriptSection

(actual code)

script.as, AddScriptSection

(actual code)

The first one being empty hasn't been a problem before. Have you perhaps changed this?

I'll be doing some more testing and report back if I find something. It's a bit hard to debug since the buffer for elements is reallocated every once in a while (e2: nope, just realized that the buffer stays constant after registrationph34r.png ...presumably) For now I'm going to fall back to default allocator and maybe solve this empty section issue...

e: If the script section report is not an error, I'm still getting a script null pointer access. It's a bit vague..

I'll just put some data breakpoints and see what happens. Of course it's possible that there's an overflow at my end...

Ok, sorry about this.. It was an overflow at my end, found it deep in my dumb ***.
I'm still getting null pointer accesses though. Something's going on with the new scripthandle add-on. Simply calling psg->AddRefScriptObject(pref,psg->GetObjectTypeByName("ref")) and passing it as an object to another preparing context doesn't seem to cut it anymore.
E: must be that new opHndlAssign. Changing it back to opAssign solves it...

I'm glad you found the cause of the buffer overflow. The result for buffer overflows can be really unpredictable. My recommendation is that you always try to identify the root cause of a problem before making any code changes. If you don't fix the root cause you'll just transfer the bug to another place, and sometimes the bug may stay undetected if the overflow doesn't modify any vital data. Just to show up again long after you've released your software for public use.

Can you show me the situation in which you're having problem with the new opHndlAssign? I haven't detected any problems in my regression tests, so it may be a scenario that I haven't covered. Did you update the CScriptHandle add-on too when using the latest WIP version of the library?

AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

Did you update the CScriptHandle add-on too when using the latest WIP version of the library?

Yes, I double checked that. Everything is now from the latest r1916, unmodified. My case is much like the following:


void Loop(ref @r){
	//r is null when using opHndlAssign
}

void main(){
	Actor a; //any class or object
	ref @r = @a;
	ExecuteAsync(Loop,r,0.0f);
}

This is how I register ExecuteAsync:


static void SCRIPTF_ExecuteAsync(asIScriptFunction *pfunc, CScriptHandle *pref, float t){
    //...
    if(pref)
        psg->AddRefScriptObject(pref,psg->GetObjectTypeByName("ref"));
    pscript->Execute(pfunc,...,pref); //basically finds a free context, prepares it and executes in a concurrent manner
}

psg->RegisterFuncdef("void ExecFunc(ref)");
psg->RegisterGlobalFunction("void ExecuteAsync(ExecFunc@, ref, float)",asFUNCTION(SCRIPTF_ExecuteAsync),asCALL_CDECL);

Thanks. I'll include this as part of my test suite and make the necessary changes to have it work.

The problem on my side is in the asCScriptEngine::CreateScriptObjectCopy, which isn't taking the opHndlAssign into account. I'll have this fixed.

AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

This topic is closed to new replies.

Advertisement