TerrainNode simple scene review

So last time I was working on this project, I think I was working around this “TerrainNode” scene implementation, which is available in the file src/scenes/test7_terrainNode.ts

This is the scene currently loaded in the main app when using `landjs_serve` since we have in src/app/app.component.ts:

  ngOnInit(): void {
    // retrieve the WebGL2 context:
    log.DEBUG("Retriving webGL2 context...");
    let ctx = this.view.nativeElement.getContext('webgl2');
    RenderContext.setCurrent(ctx);
    log.DEBUG("Assigned Render context!");

    // Create a resource loader here:
    let loader = new ResourceLoader(this.http);

    // let scene = new Test2Scene();
    // let scene = new Test2Scene(this.view.nativeElement);
    // let scene = new Test3Scene(this.view.nativeElement);
    // let scene = new Test4Scene(this.view.nativeElement);
    // let scene = new Test5Scene(this.view.nativeElement, loader);
    // let scene = new Test6Scene(this.view.nativeElement, loader);
    let scene = new Test7Scene(this.view.nativeElement, loader);
  }

We have a loader passed as argument here: this reminds me of how it works (roughtly): this loader is responsible for loading the “actual resources” in the context of the angular app (I think 😋…)

Most (all) of the important code is in the src/nvland folder. And in there we have multiple sub folders for the main sub-systems like “core”, “ecs”, “math”, “render”, etc.
Which reminds me that lately I spend a lot of time on the Entity Component System (ECS), but that's not the last thing I was working on:
According to the git log messages, last thing I touched was the implementation of the ElevationProducer
And just before that, some implementation/tests around webworkers… I'm wondering what was the conclusion on that point ? 😏

Checking the file src/nvland/workers/Mat4Mult.worker.ts:

// <reference lib="webworker" />
// import * as log from '../core/Log';
import { sprintf } from "sprintf-js";
import { Mat4 } from "../math/Mat4";
// import { BaseObject } from "../core/BaseObject";

// class MyClass extends BaseObject
// {
//     public constructor() {
//         super();
//     }
// };

// function multMat(n: number, a: Mat4, b: Mat4): Mat4
// {
//   for(let i=0;i<n;i++) {
//     Mat4.multiply(a,b,a);
//   }

//   return a;
// }

// We send a message back to the main thread
addEventListener("message", (event) => {

    // self.console.log("Received data: "+JSON.stringify(event))
    // let a = new Mat4();
    // a.rotateX(45.0);
    // // Get the limit from the event data
    // let a = new Mat4();
    // a.fromArray(event.data.mat1);
    // let b = new Mat4();
    // b.fromArray(event.data.mat2);
    // let obj = new MyClass();
    // log.DEBUG("Hello!");
    // let msg = sprintf("Hello %s", obj.name);
    let msg = sprintf("Hello %s", "manu");
    // let n = event.data.count;

    // let res = multMat(n,a,b);

    // Send the primes back to the main thread
    // postMessage({ result: res });
    // postMessage({ result: a.getElements() });
    postMessage({ result: [42] });
    // postMessage({ result: [msg.length] });
});

⇒ This reminds me that I could not successfully “import” some of my other modules and use them in a worker inside a unit test, but I think this was working when executed inside the main app directly 🤔 (or am I dreaming ? ⇒ I should redo that test to confirm it or find where this is done.)

Arrrggg…. trying to restore some of the Mat4 computation in the worker and then call that in the main app on init:

    protected async init() {

        // Test the webworker:
        let promise = new Promise(function (resolve: ((val: number[]) => void), reject) {

            console.log("Creating Mat4Mult...")
            // const worker = new Worker(new URL('./Mat4Mult.worker', import.meta.url));
            const worker = new Worker(new URL('src/nvland/workers/Mat4Mult.worker', import.meta.url));
            // const worker = new Worker('Mat4Mult.worker.ts', {type: "module"})
            worker.onmessage = (event) => {
                resolve(event.data.result);
            };

            let a = Mat4.makeRotateX(43.0);
            let b = Mat4.makeRotateY(12.0);
            console.log("Mat1: " + a.toString());
            console.log("Mat2: " + b.toString());

            // worker.onmessage = ({ data }) => {
            //   console.log(`page got message: ${JSON.stringify(data)}`);
            // };

            // worker.postMessage({ mat1: a.getElements(), mat2: b.getElements(), count: 100 });

            // worker.postMessage('hello');
        });

        let res = await promise;
        this.DEBUG("Received result data: " + JSON.stringify(res));
  }

… But right now it seems I only get a black screen out of this 😖 not good… Arrff, no: stupid me, it's working, but I have to uncomment the worker.postMessage() statement of course lol.

But now in the unit tests, I get no test executed at all:

$ landjs_test

> nerv-land-js@0.0.0 test
> ng test --no-watch --code-coverage

\ Generating browser application bundles (phase: building)...04 02 2022 13:38:22.538:INFO [karma-server]: Karma v6.3.9 server started at http://loca
lhost:9876/
04 02 2022 13:38:22.540:INFO [launcher]: Launching browsers ChromeHeadless with concurrency unlimited
04 02 2022 13:38:22.549:INFO [launcher]: Starting browser ChromeHeadless
√ Browser application bundle generation complete.
04 02 2022 13:38:34.874:INFO [Chrome Headless 97.0.4692.99 (Windows 10)]: Connected on socket fO2t8ywxepFNO-UmAAAB with id 68198556
Chrome Headless 97.0.4692.99 (Windows 10): Executed 0 of 0 SUCCESS (0.005 secs / 0 secs)
TOTAL: 0 SUCCESS

=============================== Coverage summary ===============================
Statements   : Unknown% ( 0/0 )
Branches     : Unknown% ( 0/0 )
Functions    : Unknown% ( 0/0 )
Lines        : Unknown% ( 0/0 )
================================================================================

… which is surprising, because I actually renamed the file Mat4Mult.spec.ts to Mat4Mult.bad.ts before running those tests! 😳 So, what now ?

Trying to rename the “Mat4Mult.bad.ts” file to “Mat4Mult.bad.old” ⇒ nope still not working! 😨 So, it this due to my “Mat4Mult.worker.ts” file then ? -> Commenting some content there… [nope], then commenting even more code there… Now it's OK, running 28 tests without error.

⇒ So it is something on how I process the files named “.worker.ts” ? Hmmm 🤔 I kind of remember I was doing something special about that, let's see if I can find it back.

Okay, so, in my angular.json file I have that section for the tests:

        "test": {
          "builder": "@angular-builders/custom-webpack:karma",
          "options": {
            "main": "src/test.ts",
            "polyfills": "src/polyfills.ts",
            "tsConfig": "tsconfig.spec.json",
            "karmaConfig": "karma.conf.js",
            "inlineStyleLanguage": "scss",
            "assets": [
              "src/favicon.ico",
              "src/assets"
            ],
            "styles": [
              "src/styles.scss"
            ],
            "customWebpackConfig": {
              "path": "./webpack.partial.js"
            },
            "scripts": [],
            "webWorkerTsConfig": "tsconfig.worker.json"
          }
        }

So we have the webWorkerTsConfig pointing to a dedicated tsconfig file:

{
  "extends": "./tsconfig.json",
  "compilerOptions": {
    "outDir": "./out-tsc/worker",
    "lib": [
      "es2018",
      "webworker"
    ],
    "types": []
  },
  "include": [
    "src/**/*.worker.ts"
  ]
}

… and in there our reference to the .worker.ts files. ⇒ looking for more infos on that “webWorkerTsConfig” config entry.

Found this page: https://github.com/angular/angular-cli/issues/19372
⇒ Maybe I need "exclude": ["* */*.worker.ts"] in the other tsconfig files ? ⇒ nope, this doesn't seem to help 😭

Okay, so, time for a recap: I could loose a lot of time trying to figure out how to run those unit test on webworkers, so let's cut it short for now:
- I should ensure that the unit tests do not try to load my complex webworkers, so should use a dedicated extensions for those that can be tested:
- ⇒ My testing environment will now use the tsconfig.lightworker.ts config and only find the .lightworker.ts scripts.
- Updated the reference in angular.json as needed: “webWorkerTsConfig”: “tsconfig.lightworker.json”
- And no… still getting an error with that:
```
./src/nvland/workers/Mat4Mult.worker.ts - Error: Module build failed (from ./node_modules/@ngtools/webpack/src/ivy/index.js):
Error: D:\Projects\NervLandJS\src\nvland\workers\Mat4Mult.worker.ts is missing from the TypeScript compilation. Please make sure it is in your tscon
fig via the 'files' or 'include' property.
    at D:\Projects\NervLandJS\node_modules\@ngtools\webpack\src\ivy\loader.js:59:26
```
- I gonna cry… 😭😭
Okay, so, now, considering if there could be a different way to “load” a webworker script: maybe using soem feature from webpack itself ? Investigating…
Found that page: https://dannadori.medium.com/how-to-bundle-webworker-in-npm-package-620dcec922e1

So now trying with the worker-loader package:

First installing the package with npm:
```
landjs_npm install --save worker-loader
```

Then no special rule to add in our webpack partial, but we need the typings/worker-loader.d.ts file:

declare module "worker-loader!*" {
    class WebpackWorker extends Worker {
        constructor();
    }
    export default WebpackWorker;
}

Now we update our worker typescript file to use the framework:

const ctx: Worker = self as any;

onmessage = async (event) => {
    setTimeout(() => {
        ctx.postMessage(`[WORKER_TS] Waited ${event.data}ms`);
    }, event.data)

}

Next we setup loading that worker in the main app:

import MatWorker from "worker-loader?inline=no-fallback!src/nvland/workers/Mat4Mult.worker.ts";
    
// Later in the code: 
const worker = new MatWorker()

worker.onmessage = (event:any) => {
    resolve(event.data.result);
};

let a = Mat4.makeRotateX(43.0);
let b = Mat4.makeRotateY(12.0);
console.log("Mat1: " + a.toString());
console.log("Mat2: " + b.toString());

worker.postMessage({ mat1: a.getElements(), mat2: b.getElements(), count: 100 });

But of course, this is not working for use and will produce a typescript error when compile the app:

Error: src/scenes/test7_terrainNode.ts:21:23 - error TS2691: An import path cannot end with a '.ts' extension. Consider importing 'worker-loader?inl
ine=no-fallback!src/nvland/workers/Mat4Mult.worker.js' instead.

21 import MatWorker from "worker-loader?inline=no-fallback!src/nvland/workers/Mat4Mult.worker.ts";
                         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
** Angular Live Development Server is listening on localhost:4200, open your browser on http://localhost:4200/ **

× Failed to compile.

I'm so desperate now lol… I even try loading my worker script as a JS file, and still no luck: this is such a mess…
Now trying to follow the instructions from https://v4.webpack.js.org/loaders/worker-loader/ more carefully. Nope. Pointless.

⇒ OKAY, so now discarding this whole worker-loader usage experiment: I'm not getting anywhere with that unfortunately.

So, trying with comlink now: https://blog.lacolaco.net/2018/12/enjoyable-webworkers-in-angular/ 🥴 ?

hmmm 🤔… Actually, wait a minute… I'm now just realizing that in my previous experiment I actually had 2 locations where I was trying to load my “Mat4Mult.worker.ts” file, but using on one side the worker-loader, and on the other location the webpack 5 mechanism 😐! So basically: I should try that again now before moving to the next option 🤣

So installing worker-loader again:
```
landjs_npm install --save worker-loader
```

Adding typings/worker-loader.d.ts file:

declare module "worker-loader!*" {
    class WebpackWorker extends Worker {
        constructor();
    }
    export default WebpackWorker;
}

And updating webpack.partial.json:

module.exports = {
    module: {
        rules: [
            {
                enforce: 'pre',
                test: /\.worker\.ts$/,
                loader: "worker-loader",
            },
            { 
                test: /\.glsl$/, 
                exclude: /node_modules/,
                loader: 'raw-loader' 
            }
        ]
    }
};

But no: compilation is still failing without any clear error message:

√ Compiled successfully.
√ Browser application bundle generation complete.

Initial Chunk Files | Names   |      Size
main.js             | main    | 306.21 kB
runtime.js          | runtime |   6.75 kB

3 unchanged chunks

Build at: 2022-02-04T21:27:27.836Z - Hash: 6e2c2425dc9e35b3e572 - Time: 968ms

× Failed to compile.

So let's leave that path behind us, and try comlink as suggested above:
Reference page: https://blog.lacolaco.net/2018/12/enjoyable-webworkers-in-angular/
Installing that package:
```
$ landjs_npm install comlink
```
Nayy… ⇒ This might be worth using eventually, but it will not lead us anywhere here: the problem is deeper than that: when we simply have to load one worker file in the testing environment, and it has some non trivial imports…

Finally! Found a working/acceptable solution!
- We are now defining our worker creation directly in the main.ts file:
```
// And building the app we should support loading webworkers:
(window as any).createWorker = function() : any {
  return new Worker(new URL('nvland/workers/Mat4Mult.worker', import.meta.url))
}
```
- And that file and all its dependencies don't get loaded in the testing environment! So we finally don't have to load a .worker.ts file during testing !
- Feeew… that was an hard one.

⇒ So, the solution we have here is not the cleanest, but I feel happy I have something working at least! So let's call it a day now. Tomorrow I should try to get back to the Elevation Producer