🎉 Celebrating 25 Years of GameDev.net! 🎉

Not many can claim 25 years on the Internet! Join us in celebrating this milestone. Learn more about our history, and thank you for being a part of our community!

Crash when loading bytecode

Started by
8 comments, last by EngineNo.9 2 years, 8 months ago

Hello,

I am trying to load bytecode I previously saved using the SaveBytecode method, and the bytecode gets saved without any errors.

When I read from the file and try to load the bytecode, it just hangs for a long time and then finally crashes with a memory dump.

The call stack shows that the crash is inside AngelScript's code and not my own, so I think this might be a bug with AngelScript.

Here is a pastebin of my code for the binarystream class: https://pastebin.com/0NWXaMfy

I could get this out of the call stack:

Call Stack

None

Advertisement

I agree. This looks like a bug in AngelScript.

I will need more information from you in order to identify and fix the bug though. Can you share a small piece of code that reproduces the problem so I can debug it?

AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

WitchLord said:
Can you share a small piece of code that reproduces the problem so I can debug it?

https://pastebin.com/6i8PJV58

This is the full code I just used to reproduce the exact issue.

None

Hello,

I think you made an error in the BytecodeStream class. I was able to reproduce a crash inside load bytecode part of the Angelscript but that's because the input stream was incorrect - I think something might be put in place in AS to deal with this (some sanity checking) but maybe it's not possible, I will have to look closer when I have time.

Basically this:

BytecodeStream(std::string fileName, bool read = false) : 
	readOnly(read), file(fileName, (readOnly ? std::ios::in : std::ios::out) | std::ios::binary)

You're initializing readOnly with value of read and then based on readOnly you either initialize fstream as write or read only. Problem is you're using the member which has not yet been initialized, because C++ does not initialize in order of your constructor but in order they're declared in the class (AFAIK, unless it's implementation dependent):

   std::fstream file;
   bool readOnly;

so readOnly was not initialized before file and probably caused the stream to not be right - my test reported incorrect fstream, but your code would not stop there (just output “file is broken” - you didn't have that?) and proceed to loading or saving bytecode. This was also pretty random probably, but could be stable too, depending on what the flag contained before initialization via ctor.

Then Angelscript would pick that up and it entered some sort of weird infinite loop for me - so this might be a candidate to reinforce against such mistakes, but you should be checking more strictly that you're passing actual byte stream and not garbage or 0.

I assume the file was “written” without errors from SaveByteCode but there was never any file to begin with. SaveByteCode and LoadByteCode trust the stream you're passing them it seems and do not sanity check it.

So for me the quick fix was to use the value from constructor, not the uninitialized member:

BytecodeStream(std::string fileName, bool read = false) : 
	readOnly(read), file(fileName, (>>>read<<<< ? std::ios::in : std::ios::out) | std::ios::binary)
	                                   ^^^^

Although this code could use some polishing in its entirety ?


Where are we and when are we and who are we?
How many people in how many places at how many times?

noizex said:
Although this code could use some polishing in its entirety

I just did this to reproduce the bug, so there is no need for pretty code.

Thank you, this did indeed fix the problem.

None

noizex said:
Then Angelscript would pick that up and it entered some sort of weird infinite loop for me - so this might be a candidate to reinforce against such mistakes, but you should be checking more strictly that you're passing actual byte stream and not garbage or 0.

Thanks for helping out and identifying the root cause.

AngelScript does sanity checks on the stream and normally would return an error (and write to the message stream the location in the stream where an error was identified), but obviously for this particular case AngelScript didn't detect the problem. I'll definitely look into this and add the necessary checks to properly handle this invalid input.

AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

WitchLord said:

noizex said:
Then Angelscript would pick that up and it entered some sort of weird infinite loop for me - so this might be a candidate to reinforce against such mistakes, but you should be checking more strictly that you're passing actual byte stream and not garbage or 0.

Thanks for helping out and identifying the root cause.

AngelScript does sanity checks on the stream and normally would return an error (and write to the message stream the location in the stream where an error was identified), but obviously for this particular case AngelScript didn't detect the problem. I'll definitely look into this and add the necessary checks to properly handle this invalid input.

No problem, happy to help. After brief look at how Angelscript reads binary contents of that stream I think I know what's going on. Basically it starts allocating crazy amounts because it reads the size wrong:

count = ReadEncodedUInt();
module->enumTypes.Allocate(count, false);

This is first entry point into reading from the stream, and I checked - it ends up with enormous numbers being returned from ReadEncodedUInt() and I think I know why. Basically entire reading assumes that the stream can progress and will yield a sane data - for example the count for enum types in this case. But the way C++ streams work (as far as I could confirm) is that if the stream is invalid it will literally do nothing. So the flow is like this:

(from as_restore.cpp#1961)

asBYTE b;  // !!uninitialized byte!
ReadData(&b, 1);  // intends to read data from the stream into &b, but stream 
                  // ignores it because we have a bad file, not writing to it
                  // and keeping the uninitialized value                  
bool isNegative = ( b & 0x80 ) ? true : false;

this gives b some arbitrary value and it continues its journey through the Read* method:

i = asUINT(b & 0x3F) << 8;
ReadData(&b, 1); i += b;
if( isNegative ) 
	i = (asQWORD)(-asINT64(i));

how it progresses through many ifs will depend on what random value b assumed at the beginning (which was not overwritten by read from the stream), but it does end up with crazy values, my usual one is 18446744073709548468 which allocates this size for the enums buffer before it enters wild infinite loop and freezes my PC.

I think the first guard should be initializing b variable to something sensible like 0 because we can't trust the stream Read method that it will write anything to that pointer. Then maybe sanitization of data that's read from the stream - if we have some expectations of count/size it could validate it against the length of the stream and ensure that we're not reading some bizzare numbers from the moon and then try to allocate this amount of memory etc.

That's as far as I got - hope that's of any use to you @witchlord ?

PS. I think to reproduce it in most generic way, irrelevant of underlying file implementation used etc would be something like this:

class BorkenBytecodeStream : public asIBinaryStream
{
	int Write(const void* ptr, asUINT size)
	{
		// may be implemented so we error on read not on write
	}
	
	int Read(void* ptr, asUINT size)
	{
		// noop, we don't write anything to ptr at all
		return 0;
	}
}


Where are we and when are we and who are we?
How many people in how many places at how many times?

I've added additional sanity checks in LoadByteCode now to avoid the crash on invalid stream data. (rev 2706)

Thanks,
Andreas

AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

This topic is closed to new replies.

Advertisement