3D Blazor with Babylon

Babylon is a good choice for 3D graphics on the web. Anything that you can do in JavaScript you can do in Blazor, so I’m going to put some 3D in my web app using C#.

First, I’m going to create a Babylon component that I can drop onto a Blazor page and then I’m going to see if I can shift all the Babylon manipulation up from Javascript into C# with a full Babylon interop layer.

I’m new to Babylon, so I started with the tutorial here: https://doc.babylonjs.com/babylon101/

Job one is what I do for every project: create a Blazor WebAssembly project in Visual Studio and delete all the WeatherForecast stuff from the demo. Following the first steps in the tutorial, I added theBabylon script references to index.html and CSS to app.css.

Add the JavaScript library files from the tutorial page and a new JavaScript file in the scripts folder of your own application. The PEP library is to handle pointer events on tablets and phones.

<body>
    <app>Loading...</app>

    <div id="blazor-error-ui">
        An unhandled error has occurred.
        <a href="" class="reload">Reload</a>
        <a class="dismiss">🗙</a>
    </div>
    <script src="https://preview.babylonjs.com/babylon.js"></script>
    <script src="https://preview.babylonjs.com/loaders/babylonjs.loaders.min.js"></script>
    <script src="https://code.jquery.com/pep/0.4.3/pep.js"></script>
    <script src="_framework/blazor.webassembly.js"></script>
    <script src="scripts/babylonInterop.js"></script>
</body>

The CSS comes straight from the tutorial. If you’re going to have multiple Babylon canvases, pick a class name. I’ve just got one, hence the ID.

html, body {
    font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif;
    overflow: hidden;
    width: 100%;
    height: 100%;
    margin: 0;
    padding: 0;
}
#babylon-canvas {
    width: 100%;
    height: 100%;
    touch-action: none;
}

I added a new JavaScript file, babylonInterop.js for JS interop functions. I added a BabylonCanvas component,

@inherits BabylonCanvasBase

<canvas id="babylon-canvas" touch-action="none"></canvas>

a base class to the shared components folder,

namespace BabylonBlazor.Shared
{
    public class BabylonCanvasBase : ComponentBase
    {
    }
}

and put a tag into the index.razor page

@page "/"

<h1>Hello, world!</h1>

<BabylonCanvas />

The code to render the demo is taken from the tutorial but I’ve restructured it a bit so I can call it from the Blazor app via JS interop. Here’s the first cut of blazorInterop.js

var babylonInterop = babylonInterop || {};

babylonInterop.initCanvas = function (canvasId) {
    var babylonCanvas = document.getElementById(canvasId);
    var babylonEngine = new BABYLON.Engine(babylonCanvas, true);
    var scene = babylonInterop.createSceneWithSphere(babylonEngine, babylonCanvas);

    babylonEngine.runRenderLoop(function () {
        scene.render();
    });

    window.addEventListener("resize", function () {
        babylonEngine.resize();
    });
};

babylonInterop.createSceneWithSphere = function (engine, canvas) {
    var scene = new BABYLON.Scene(engine);

    var camera = new BABYLON.ArcRotateCamera("Camera", Math.PI / 2, Math.PI / 2, 2, new BABYLON.Vector3(0, 0, 5), scene);
    camera.attachControl(canvas, true);

    var light1 = new BABYLON.HemisphericLight("light1", new BABYLON.Vector3(1, 1, 0), scene);
    var light2 = new BABYLON.PointLight("light2", new BABYLON.Vector3(0, 1, -1), scene);

    var sphere = BABYLON.MeshBuilder.CreateSphere("sphere", { diameter: 2 }, scene);

    return scene;
};

To render the scene in the scene in the canvas element, we need to inject a JSInterop object into the Babylon component and then call the JavaScript function to set it all up.

In a real project I’d want a reusable component so I’d separate the initialisation of the Babylon engine from the specifics of the scene, but this’ll do for now.

To associate the Babylon engine with the canvas element, the canvas element needs to exist. The JavaScript call is in the OnAfterRenderAsync() method so we know that the page has been rendered before we try and do anything with it. We only need to do this once, so use the firstRender flag.

        [Inject] IJSRuntime JsRuntime { get; set; }

        protected async override Task OnAfterRenderAsync(bool firstRender)
        {
            await base.OnAfterRenderAsync(firstRender);

            if (firstRender)
            {
                await JsRuntime.InvokeVoidAsync("babylonInterop.initCanvas", "babylon-canvas");
            }
        }
    }

Running the app, shows the scene with the lights and the sphere.

That’s all fine but we still have to do all the setup in JavaScript and there’s no way to manipulate anything from the C# components.

Composing the Scene in C#

I want to compose and manipulate the scene in C#, which means I need to expose the Babylon JavaScript functions via the interop file. Exposing any individual function is easy.

babylonInterop.createEngine = function (canvasId, antialias) {
    var babylonCanvas = document.getElementById(canvasId);
    var babylonEngine = new BABYLON.Engine(babylonCanvas, antialias);
    window.addEventListener("resize", function () {
        babylonEngine.resize();
    });
    return babylonEngine;
}

However, there’s a problem. The engine object returned by the JavaScript function needs to be deserialised into a C# object on the other side of the interop. But even if I built a type and created the object, I’d still lose the reference to the JavaScript object.

When I want to create a scene how do I pass the engine back? Serialising the object I got back from the previous function wouldn’t result in the same JavaScript object.

babylonInterop.createScene = function (engine) {
    return new BABYLON.Scene(engine);
}

Passing .NET references down to JavaScript so that the script can call back into the app is well understood, with DotNetObjectReference, but the reverse is not so easy.

Fortunately I found an article on keeping JavaScript object references in .NET on Rémi Bourgarel’s blog. Keeping a key-value store of objects and passing back the keys seems reasonably obvious but the real magic is in registering a “reviver” function with the .NET interop.

Instead of passing the object back to .NET, we store the object locally and pass back a reference.

babylonInterop.objRefs = {};
babylonInterop.objRefId = 0;
babylonInterop.objRefKey = '__jsObjRefId';
babylonInterop.storeObjRef = function (obj) {
    var id = babylonInterop.objRefId++;
    babylonInterop.objRefs[id] = obj;
    var objRef = {};
    objRef[babylonInterop.objRefKey] = id;
    return objRef;
}

Revivers take the serialised objects that are passed to JavaScript and can replace them with something else. In this case, when our reviver spots an inbound object with the magic key, it replaces it with the stored object.

DotNet.attachReviver(function (key, value) {
    if (value &&
        typeof value === 'object' &&
        value.hasOwnProperty(babylonInterop.objRefKey) &&
        typeof value[babylonInterop.objRefKey] === 'number') {
        var id = value[babylonInterop.objRefKey];
        if (!(id in babylonInterop.objRefs)) {
            throw new Error("The JS object reference doesn't exist: " + id);
        }
        const instance = babylonInterop.objRefs[id];
        return instance;
    } else {
        return value;
    }
});

On the .NET side the passed back object is turned into a JsRuntimeObjectRef

public class JsRuntimeObjectRef : IAsyncDisposable
{
    internal IJSRuntime JSRuntime { get; set; }

    [JsonPropertyName("__jsObjRefId")]
    public int JsObjectRefId { get; set; }

    public async ValueTask DisposeAsync()
    {
        await JSRuntime.InvokeVoidAsync("babylonInterop.removeObjectRef", JsObjectRefId);
    }
}

The interop function returns a typed object that wraps the reference

        public async Task<Engine> CreateEngine(string canvasId, bool antialias = false)
        {
            return new Engine(_jsRuntime, await _jsRuntime.InvokeAsync<JsRuntimeObjectRef>("babylonInterop.createEngine", canvasId, antialias));
        }

The wrapper is designed to be serialised in a way that the reviver will recognise.

    public abstract class BabylonObject
    {
        protected JsRuntimeObjectRef _jsObjRef;

        [JsonPropertyName("__jsObjRefId")]
        public int JsObjectRefId => _jsObjRef.JsObjectRefId;

        public BabylonObject(IJSRuntime jsRuntime, JsRuntimeObjectRef objRef)
        {
            _jsObjRef = objRef;
            _jsObjRef.JSRuntime = jsRuntime;
        }
    }

With the reference in C# we can make further calls to the Babylon JavaScript, knowing that we have access to the original objects. Now we can compose the scene in C# piece by piece.

The concrete implementations can have other methods attached to them. Here the Engine has a RunRenderLoop method that calls the equivalent method in JavaScript, passing down a reference to a Scene object.

public class Engine : BabylonObject
{
    public Engine(IJSRuntime jsRuntime, JsRuntimeObjectRef objRef) : base(jsRuntime, objRef) { }

    public async Task RunRenderLoop(Scene scene)
    {
        await _jsObjRef.JSRuntime.InvokeVoidAsync("babylonInterop.runRenderLoop", this, scene);
    }
}

By stringing a bunch of these calls together we can construct the scene and render it entirely from C#.

        protected async override Task OnAfterRenderAsync(bool firstRender)
        {
            await base.OnAfterRenderAsync(firstRender);

            if (firstRender)
            {
                //await JsRuntime.InvokeVoidAsync("babylonInterop.initCanvas", "babylon-canvas");

                var canvasId = "babylon-canvas";
                var engine = await Babylon.CreateEngine(canvasId, true);
                var scene = await Babylon.CreateScene(engine);
                var cameraTarget = await Babylon.CreateVector3(0, 0, 5);
                var camera = await Babylon.CreateArcRotateCamera("Camera", Math.PI / 2, Math.PI / 2, 2, cameraTarget, scene, canvasId);
                var hemisphericLightDirection = await Babylon.CreateVector3(1, 1, 0);
                var light1 = await Babylon.CreateHemispehericLight("light1", hemisphericLightDirection, scene);
                var pointLightDirection = await Babylon.CreateVector3(0, 1, -1);
                var light2 = await Babylon.CreatePointLight("light2", pointLightDirection, scene);
                var sphereOptions = new ExpandoObject();
                sphereOptions.TryAdd("diameter", 2);
                var sphere = await Babylon.CreateSphere("sphere", sphereOptions, scene);
                await engine.RunRenderLoop(scene);
            }
        }

The end result looks just like the scene built in JavaScript.

I’ve started to build up a Babylon library with the parts we’ve talked about so far. There’s a BabylonFactory for creating objects and an interface for it that can be registered at startup.

The project so far is in my Babylon Blazor Demo repo on GitHub.

So far, so good. I think this demonstrates that JavaScript references can be held in C# and used to do complex tasks. The biggest downside to this approach is that the Babylon API is huge. Manually writing the C# representations is a slow job. You’d probably be better off just writing the code in JavaScript.

To turn this into a useful project, I’d want some way to automatically generate all the interop code and then I’d pull it into a separate Razor class library.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s