Hello Forum,

this is one of the rare cases I'm in need for some guidance or at least a discussion to bring my thoughts into the right direction. After finally finishing the release pass of our data driven build tool, I'm now in the difficult decision process for going one step further and adding a UI Framework to our codebase. Sounds simple, use solution X/Y/Z but I have some special requirements on such frameworks. First of all, I don't want something which does black magic behind the scenes like allocating lots of memory or running it's own threads, while not that much kind of a problem in visual tools and the editor, it would not play well with the architecture of our engine. But most important for me is how the framework fits into our asset and development pipeline.

Because our tools are written in C#, WPF would be a solution at least for visual tooling. As from .NET Core 5 (or .NET 5) has been released some while ago and is considered (or at least sold from Microsoft as) platform independent, in fact WPF still is targeting only Windows. So this is getting difficult from the point when we also officially support Linux as a possible development platform. On the other hand getting tools like Microsoft Blend is a plus for our UI Designer, so there would be interest in using at least XAML in some way. However, implementing the renderer in plain managed C# could be worth the work regardless of which layouting language we use as we'll get full control over it and it can be made multiplatform very easy.

I did some research (before posting this for sure) and came across Noesis GUI, which is an ready to use UI engine made especially for games. However, it is licensed under a third party license which I don't want to go through and which is most likely not AGPLv3 compliant (all of our SDK code is shared on GitHub via AGPLv3). So using it is quite difficult to impossible.

The other option I see at the moment is HTML5/CSS3 (without JS) which isn't natively supported by neither C# or C++. So this would at least involve a third-party library as well. On the other hand, we could use tools like Adobe Edge Animate or Google Web Design and it could even be tested in the browser. On my research I came across Ultralight, which is an ready to use UI engine based on Webkit. However, it is a closed source magic box which uses Webkit, a full browser engine and so neither lighweight nor memory or thread friendly. Libraries like AngleSharp are DOM parsers but don't offer the layouting as well and honestly, my self-written HTML5 parser is faster and less monolythic.

TL;DR using a ready and standardized layout format for game UI is a good idea but the options are, at least from what I've found so far, very limited for working non-commercial and open source. I also sturggle in deciding for the best solution for our case so I add the pros/cons for each here (at least how I see them)

XAML pros

• Well established
• Parsed like XML
• Simple syntax with a limited set of tags
• Good layouting options

XAML cons

• Windows/WPF only
• Layout engine isn't well documented
• Difference between visual editor and our UI Framework
• Needs custom libraries in order to extend available controls
• Missing styling options

HTML5/CSS3 pros

• Well established and standardized
• Screenreader friendly (to allow visually impaired people to use our tools as well)
• Can be viewed in any browser
• Can display any website in the UI
• Good layout options
• Good styling options
• MathML support
• SVG support

HTML5/CSS3 cons

• 152 different tags
• Probably difference between tools/browsers and our UI Framework
• Huge documentation which needs to be studied
• Can be difficult to construct the DOM

I already recognized that I thought to straight from the bottom, focussing on XAML or HTML5 without taking the higher systems into account. It feels like it is the best option for us to write our own solution and take a look straight from the top instead, so starting from the renderer over how it is layouted, how is the DOM provided to how it is read and it's assets are designed. So ending up something like this

Staying at a DOM related solution is most likely set because CSS turned out to be a good styling solution and fits well into C# LINQ. Also it already has animations which are a big plus and missing in WPF.

As this should be a discussion, guiding me through the process, I would like to hear your thoughts, your experience and especially from fellow engine developers, your solution and what problems you came across. Maybe you know even another well documented alternative to XAML and/or HTML5 i didn't saw yet.

I look forward to read your answers

Shaarigan said:
Hello Forum,

Hello @shaarigan

Few things first:

If you go XAML, you can add support for linux yourself, this way you can bypass WPF. You can create a custom cross-platform UI renderer that reads XAML and renderers each element accordingly. It's not going to be fully supportive but it gives your designer “some” liberty in using Blend. That's kinda what Neosis GUI is doing…

You haven't mentioned Qt (C++ based and comes with a UI Designer for controls and XML based), works on all platforms. Maybe try this for UI Design but still, if you're not using Qt classes then you will need your own custom code to render as you're working in C-sharp.

You haven't mentioned Ocornut's ImGUI (c++ based and comes with loads of examples), check it out…

You haven't mentioned WxWidgets, same principle as Qt… check it out…

Anyway point being, you really need to know if you want your Editor to be cross-platform or not, and based on this decision, you can then narrow things down to:

• Either you create your own custom UI render (with your own XML spec or reuse some XML output from Blend, Qt UI Designer or HTML5…) (initially Dev platforms only then all platforms in the long term)
• Or use Qt + its UI Designer (all platforms)
• Or use WPF and XAML (windows only)
• etc…

From my experience, many game developers stuck to C++ for engine code and C-sharp for the game level editor or Tools (using WinForms or WPF), this was due to its “rapid" deployment aspect. MFC was also prime in its days but that kinda caved in due to its clunky-ness, a shot in the leg so to speak if u used it -lol-

In my experience, for the Editor, there was no care as to having an Editor that works on Linux or Mac or elsewhere (Windows was enough and just fine); obviously that's different today with Webkit, if you are interested in Web UI Editor offering… or else…

Hope this helps a bit;

have fun ?

ddlox said:
You haven't mentioned Qt

…

You haven't mentioned Ocornut's ImGUI

…

You haven't mentioned WxWidgets

Right because all of these tools either use an own thread management (QT) or have classes and a traditional event-delegate design which consume unneccessary ammounts of memory and CPU time. This is the reason why solutions like Noesis GUI or Ultralight exist, using lightweight layouting data which is parsed into a DOM tree and shortened to the minimal possible footprint. I know we're living in 2021 rather than 1980 but still, memory is a limiting factor, especially on modern consoles and mobile devices which most of the time share RAM and VRAM.

Btw. the reason Microsoft went away from Winforms and instead introduced WPF as traditional Winforms GUI could easily break the memory footprint when instancing too many controls at once.

However, using a layouting data format like XAML or HTML5 has the advantage that we can add this to our build-pipeline as well. This is the compiler point in my above sketch. So we can transpile/compile the layout e.g. the already processed render tree into a small engine dependent asset which could be load much faster and also be much smaller than the entire full fledged document.

Another plus I see here is that it enables us to change UI "on the fly" in our running application. Everything it takes is just another parsing and layouting step and the UI is changed.

Another plus I see is support for screen reader software as HTML5 is the defacto standard for an open and inclusive web, it could also become standard for an open and inclusive workspace environment for blind people, which is an USP at the moment

ddlox said:
rom my experience, many game developers stuck to C++ for engine code and C-sharp for the game level editor or Tools (using WinForms or WPF), this was due to its “rapid

Yeah that's what I already know and have seen many times. A usual approach is to pass a C# Control handle to the engine and use that as render target. Unity works this way (a C# layer on top of C/C++ code), however, it turned out to have some disadvantages as well.

Working with interopts is considered to be hard to debug on errors and costly to maintain the bindings if both are in heavy development. This is why I prefer to have as much as possible in the target language. C# should perform all the stuff in managed C# while the C++ portion should be written entirely in C++.

Anyways, as this is not supposed to be used from an Editor only but also for all of our graphical tools (a Project Center is for example planned which will provide access to some of the build features and an online-catalog for code components) and so must be somehow independent from each other, it isn't suitable that much for a pure C# interopt layer

ddlox said:
In my experience, for the Editor, there was no care as to having an Editor that works on Linux or Mac or elsewhere

I know that major companies and even studios don't care that much of multiplatform and I agree from a player's perspective. But I strongly disagree from a developers perspective (I know Unity released a port for Linux as well but does Unreal work on Linux/Mac?).

Another reason I prefer to have tools written in C#, being mono or .NET 5 compliant offers a lot of opportunities to a free choice in a developers desired environment. I have teammates that already requested Linux support for this project and from a previous job on a commercial game writing software, a majority of writers prefer to work on Mac as well.

So I don't intend to support Mac (several reasons, their development policies for example) but I don't see any reason (complexity and platform restrictions doesn't count) against making games on Linux. Our engine is supposed to support at least Linux and Android as well anyways

ddlox said:
today with Webkit

As I already mentioned above, Webkit isn't suiteable as well as Ultralight is already based on it and it spawns threads we don't have control over and comes with an embedded JS Engine we don't want to use and honestly don't need. I can as well tangle events into our C#/C++ code directly rather than have UI stuff run in yet another scripting VM.

I know it is sponsored content but this Article on Gamasutra has compelling arguments on how future UI system should look like and why ?

Something I recently found on another research was this GDC talk from Insomniac which seem to try to also create a web based tools stack. Another argument why using an embedded browser is considered bad and doing Electron driven UI might be the wrong way

Shaarigan said:
GDC talk from Insomniac

that'll teach them for going web -lol-, it looks like they're using Qt now :-)

I've used Qt, WPF and these are both great, never had issues of such for deploying UI framework… maybe about wpf, i'd say, it becomes quite complex to dig, when u have a heap of bindings to deal with;

Anyway, given the time, I'd personally scramble a custom XML-based cross-platform skinnable imgui-style UI framework. Not saying that it's easy, text rendering would be a beast, but once u crack this one, the rest would be a breeze;

if time is a constraint, then I'd use Qt/WxWidgets or XAML with a custom UI compiled output for the engine's renderer.

Anyway, that's my take on it;

That's it.. all the best ;-)

I guess I'll start from digging through the conceps and implementation of WPF (like in the Avalonia UI) and go further on assembling my UI engine core somehow similar to what they did. From my overview, most of what WPF offers is somehow similar to either HTML or CSS which is shown by diverse showcase projects like XAML to HTML Converter and HTML to XAML Conversion Demo.

It must not be a perfect 1-on-1 match as some aspects might not fit into game UI but I could definetly learn from the big players