Continuing on Elevation Producer implementation

Hello Manu & Co. How are you today guys ?! On my side, well, it's a bit cold, but not that much, and it's the week-end yeepeee! Let's try to enjoy that, and write some cool code!

Yesterday we left with a positive conclusion on how we could use webworkers without interfering with the unit tests in karma (but that was a terribly long and hard pathh for sure…).
Today, I will try to get back to the initial implementation I started for my ElevationProducer
As a reminder by the way, the whole project here is based on the Proland library, reference documentation starting on that page: http://proland.imag.fr/documentation.html
First things first, let me re-read this page: http://proland.imag.fr/doc/proland-4.0/terrain/html/index.html

Hmmm, interesting point here:

The proland::ResidualProducer class is a producer that can load in CPU memory precomputed residual tiles stored on disk. The residual tiles must be stored in files containing tile "pyramids", one file per pyramid (these files can be produced with proland::preprocessDem and proland::preprocessSphericalDem - see also the "preprocess" example)

⇒ I may need to write some cached residual inside the browser storage ?

… Oh my… I have a bad feeling for this project lol… I'm starting to feel more and more that I want to move to a webgpu implementation instead of my current webgl2 version, and I know me: this means it's just a matter of time because I switch to that path now my my my… so much to re-do again… Come on Manu try to focus a bit more webgpu is too new, and it's not well supported yet: it's a better idea to get something working with webgl2 first [if possible] so keep working on that man!

Anyway… working the ElevationProducer constructor now ✌

I created a function to generate a “dem framebuffer”:

function createDemFramebuffer(demTexture: Texture2D, layerTexture: Texture2D): FrameBuffer
{
    let tileWidth = demTexture.getWidth();
    let frameBuffer = new FrameBuffer();
    frameBuffer.setReadBuffer(BufferId.COLOR0);
    frameBuffer.setDrawBuffers([BufferId.COLOR0]);
    frameBuffer.setViewport(0, 0, tileWidth, tileWidth);
    frameBuffer.setTexture2DBuffer(BufferId.COLOR0, demTexture, 0);
    if (layerTexture != null) {
        // let depthBuffer = new RenderBuffer(RenderBufferFormat.DEPTH_COMPONENT32, tileWidth, tileWidth);
        // **Note**: here we use the format depth_component32f because the integer version is not available in webgl2
        let depthBuffer = new RenderBuffer(RenderBufferFormat.DEPTH_COMPONENT32F, tileWidth, tileWidth);
        frameBuffer.setTexture2DBuffer(BufferId.COLOR1, layerTexture, 0);
        frameBuffer.setRenderBuffer(BufferId.DEPTH, depthBuffer);
        frameBuffer.enableDepthTest(true, CompareFunction.ALWAYS);
    } else {
        frameBuffer.enableDepthTest(false);
    }
    
    frameBuffer.setPolygonMode(PolygonMode.FILL, PolygonMode.FILL);

    return frameBuffer;
}

but inside proland, this was really using a Factory instead (to avoid creating a framebuffer with the same inputs multiple times), so let's update that, and create our dem frambuffer factory too:

class DemFramebufferFactory extends BaseObject
{
    // Storage for the already generated framebuffer:
    protected framebuffers = new Map<string, FrameBuffer>();
    public constructor()
    {
        super()
    }

    // Retrieve the framebuffer for a given per of dem/layer textures:
    get(demTexture: Texture2D, layerTexture: Texture2D) : FrameBuffer
    {
        // get the key for those 2 textures:
        let id1 = demTexture?demTexture.getId():0;
        let id2 = layerTexture?layerTexture.getId():0;
        let key = `${id1}_${id2}` 
        
        this.DEBUG("Retrieving FrameBuffer for key: %s", key);

        if(this.framebuffers.has(key)) {
            return this.framebuffers.get(key)
        }

        // Otherwise we need to create the framebuffer:
        let fb = this.createDemFramebuffer(demTexture, layerTexture);
        this.framebuffers.set(key, fb);
        return fb;
    }

    // create a framebuffer for a given pair of textures:
    createDemFramebuffer(demTexture: Texture2D, layerTexture: Texture2D): FrameBuffer
    {
        let tileWidth = demTexture.getWidth();
        let frameBuffer = new FrameBuffer();
        frameBuffer.setReadBuffer(BufferId.COLOR0);
        frameBuffer.setDrawBuffers([BufferId.COLOR0]);
        frameBuffer.setViewport(0, 0, tileWidth, tileWidth);
        frameBuffer.setTexture2DBuffer(BufferId.COLOR0, demTexture, 0);
        if (layerTexture != null) {
            // let depthBuffer = new RenderBuffer(RenderBufferFormat.DEPTH_COMPONENT32, tileWidth, tileWidth);
            // **Note**: here we use the format depth_component32f because the integer version is not available in webgl2
            let depthBuffer = new RenderBuffer(RenderBufferFormat.DEPTH_COMPONENT32F, tileWidth, tileWidth);
            frameBuffer.setTexture2DBuffer(BufferId.COLOR1, layerTexture, 0);
            frameBuffer.setRenderBuffer(BufferId.DEPTH, depthBuffer);
            frameBuffer.enableDepthTest(true, CompareFunction.ALWAYS);
        } else {
            frameBuffer.enableDepthTest(false);
        }
        
        frameBuffer.setPolygonMode(PolygonMode.FILL, PolygonMode.FILL);

        return frameBuffer;
    }
};

Okay, cool, I have a factory now… but I dont want to just create a static instance of it, and store my framebuffer there, without any possibility to link that to a given current context. So i'm think instead I should store this in the RenderContext directly offering a “Factory” interface… would that make sense ? 🤔

⇒ not quite sure such a level of complexity really makes sense, but still, I'm now storing those “factories” as part of a given RenderContext, and can create them dynamically with:

    // Get or create a factory:
    public getOrCreateFactory<T extends Factory>(fname: string, className: new ()=>T) : T
    {
        if(this.factories.has(fname)) {
            return this.factories.get(fname) as T;
        }

        let fac = new className();
        this.factories.set(fname, fac);
        return fac;
    }

Usage in the ElevationProducer is now has follow:

        // Below we need to create our dem framebuffer, from a demTexture and a layerTexture:
        this.DEBUG("Creating Framebuffer with dem texture: %s, layer texture: %s", demTexture, layerTexture);
        // get the factory:
        let factory = RenderContext.getCurrent().getOrCreateFactory("dem_framebuffer", DemFramebufferFactory)
        this.frameBuffer = factory.get({dem:demTexture, layer:layerTexture});

Next I implemented my DemNoiseFactory class in the same way, and now using that to generate a noise Texture2DArray:

        // Generate a noise texture of the given tile Width:
        this.DEBUG("Generating noise texture of size %dx%d", tileWidth)
        let noiseFactory = RenderContext.getCurrent().getOrCreateFactory("dem_noise", DemNoiseFactory);
        this.noiseTexture = noiseFactory.get(tileWidth);

/

And now done with the constructor only of the ElevationProducer 🤣, and also extended a bit the support for Uniform1 in the process in fact. Time for a break now !

Next question I think I should ask myself now is: where on earth do I use the ElevationProducer class ? Checking the proland examples…

⇒ OK, so we have a first example in the “examples/terrain1”: in there we create an ElevationProducer, using a TileCache, based on the GPUTileStorage:

    <tileCache name="groundElevations" scheduler="defaultScheduler">
        <gpuTileStorage tileSize="101" nTiles="512"
            internalformat="RGB32F" format="RGB" type="FLOAT" min="LINEAR" mag="LINEAR"/>
    </tileCache>
    <elevationProducer name="groundElevations1" cache="groundElevations"
        noise="-140,-100,-15,-8,5,2.5,1.5,1,0.5,0.25,0.1,0.05"/>

From what I see, the test7_terrainNode I created was basically based on the core/examples/helloworld code, using a simple terrain shader.

⇒ let's extend on our test scene 7 above and create test8 to test the terrain1 example!
First thing to do in our test8 scene is to create a GPUTileStorage using the settings above.

And by the way, I got tired of setting up my TextureParameters object manually everytime I need one, so I just added the possibility to specify the settings using a key:string mapping in the constructor (and passing that to the base SamplerParameters class):

    public constructor(args: any = null) {
        super();
        if(args != null) {
            this.minFilter = getTextureFilter(args.min || "NEAREST");
            this.magFilter = getTextureFilter(args.mag || "LINEAR");
            this.wrapS = getTextureWrap(args.wrapS || "CLAMP_TO_EDGE");
            this.wrapT = getTextureWrap(args.wrapT || "CLAMP_TO_EDGE");
            this.wrapR = getTextureWrap(args.wrapR || "CLAMP_TO_EDGE");
        }
    };

So now I can instanciate with something like (this will simplify my life a little 👍! ):

let tp = new TextureParameters({min: "LINEAR", mag:"LINEAR"})

Next thing I'm just noticing now is that in my GPUTileStorage constructor, I'm actually calling an async init method:

    public constructor(tileSize: number, nTiles: number, internalf: TextureInternalFormat, f: TextureFormat, 
                       t: PixelType, params: TextureParameters, useTileMap: boolean = false)
    {
        super(tileSize, nTiles);
        // Note: the function below might be async:
        this.init(tileSize, nTiles, internalf, f, t, params, useTileMap);
    }

    public async init(tileSize: number, nTiles: number, internalf: TextureInternalFormat, f: TextureFormat, 
            t: PixelType, params: TextureParameters, useTileMap: boolean)
    {
      // stuff here
    }

This way of creating the object doesn't seem safe to me: I would have no real idea when the init() call is completed, so could lead to any kind of problems. Searching for a solution for this I found [this article](https://bytearcher.com/articles/asynchronous-call-in-constructor/) and in there I really like the option 3 with the asyn construction factory ⇒ let's implement that here!

So here is the async GPUTileStorage constructing function:

export async function createGPUTileStorage(tileSize: number, nTiles: number, internalf: TextureInternalFormat, f: TextureFormat, 
    t: PixelType, params: TextureParameters, useTileMap: boolean = false): Promise<GPUTileStorage>
{
    let obj = new GPUTileStorage(tileSize, nTiles);
    await obj.init(tileSize, nTiles, internalf, f, t, params, useTileMap);
    return obj;
}

And okay… if I am to work more consistently on typescript code in this project, I think i seriously need to look into a proper linter and formatter for typescript in visual studio code first. Let's check that.
- OK, so using Prettier for now as formatter,
- and we also have a .editorconfig in the project to store the formatting settings.
- Installing ESlint as typescript linter, but not quite sure yet this is configured properly (?)

Continuing on our path to instanciate an ElevationProducer, we then create a TileCache object using as reference the xml construction code mentioned above:

    <tileCache name="groundElevations" scheduler="defaultScheduler">
        <gpuTileStorage tileSize="101" nTiles="512"
            internalformat="RGB32F" format="RGB" type="FLOAT" min="LINEAR" mag="LINEAR"/>
    </tileCache>

Hmmm, except that our current TileCache implementation is pretty small for the moment, maybe I should think about extending this a bit first 🤔. Checking reference implementation.
And also, we currently do not provide a scheduler at all 😖 I will need to address that point somehow (but maybe i can delay that further ?)

Okay, so the actual TileCache constructor from Proland doesn't do much more than what I have already in my typescript class in fact. So let's just start with that for the moment, and we'll see later what we really need in there 😋:
```
        // creating the ground elevations tile cache:
        this.DEBUG('Creating ground elevation TileCache...');
        let cache1 = new TileCache(storage1, 'ground_elevations');
```
/

And finally we can create the ElevationProducer using the reference construction code:

    <elevationProducer name="groundElevations1" cache="groundElevations"
        noise="-140,-100,-15,-8,5,2.5,1.5,1,0.5,0.25,0.1,0.05"/>

The ElevationProducer class actually contains a pretty large

void ElevationProducer::init(ptr<ResourceManager> manager, Resource *r, const string &name, ptr<ResourceDescriptor> desc, const TiXmlElement *e)

method, we are only covering a small part of it in this first usage.

The default upsample program should be “upsampleShader;”, since we have the code:

    string upsample = "upsampleShader;";
    if (e->Attribute("upsampleProg") != NULL) {
        upsample = r->getParameter(desc, e, "upsampleProg");
    }
    upsampleProg = manager->loadResource(upsample).cast<Program>();

Similarly the default gridSize value should be 24
For the “noise” attribute above, we should split it on the commas to build a float vector,
the flip flag should default to false.
tileWidth should come from the storage in the TileCache: int tileWidth = cache->getStorage()->getTileSize();

One thing worth noticing here is the creation of the dem texture: in proland this is done with this kind of code:

    ostringstream demTex;

    demTex << "renderbuffer-" << tileWidth << "-RGBA32F";
    demTexture = manager->loadResource(demTex.str()).cast<Texture2D>();

So trying to load a resource called “renderbuffer-xxx-RGBA32F”: but of course we don't have such resource file, so this must be autogenerated somehow, let's find out where. OK, so from the Ork framework we automatically generate a resource descriptor when we get those “renderbuffer” names:

ptr<ResourceDescriptor> XMLResourceLoader::loadResource(const string &name)
{
    time_t stamp = 0;
    TiXmlElement *desc = NULL;
    if (strncmp(name.c_str(), "renderbuffer", 12) == 0) {
        // resource names of the form "renderbuffer-X-Y" describe texture
        // resources that are not described by any file, either for the XML part
        // or for the binary part. The XML part is generated from the resource
        // name, and the binary part is NULL
        desc = buildTextureDescriptor(name);
    
    // ... more code here...
}

TiXmlElement *XMLResourceLoader::buildTextureDescriptor(const string &name)
{
    string::size_type index1 = name.find('-', 0);
    string::size_type index2 = name.find('-', index1 + 1);
    string size = name.substr(index1 + 1, index2 - index1 - 1);
    string::size_type index3 = name.find('-', index2 + 1);
    string internalformat = name.substr(index2 + 1, index3 == string::npos ? index3 : index3 - index2 - 1);

    TiXmlElement *p = new TiXmlElement("texture2D");
    p->SetAttribute("name", name);
    p->SetAttribute("internalformat", internalformat.c_str());
    p->SetAttribute("width", size.c_str());
    p->SetAttribute("height", size.c_str());
    p->SetAttribute("format", "RED");
    p->SetAttribute("type", "FLOAT");
    p->SetAttribute("min", "NEAREST");
    p->SetAttribute("mag", "NEAREST");
    return p;
}

We should do something similar ourself here, and I added this section in my ResourceManager.loadTexture() method:

        // Note, we should also have support here for renderbuffers:
        if (name.startsWith('renderbuffer')) {
            let parts = name.split('-');
            let width = Number(parts[1]);
            let intfmt = getTextureInternalFormat(parts[2]);
            let fmt = getTextureFormat('RED');
            let ptype = getPixelType('FLOAT');
            let tp = new TextureParameters({ min: 'NEAREST', mag: 'NEAREST' });
            let tex = new Texture2D(width, width, intfmt, fmt, ptype, tp);

            this.textures.set(name, tex);
            return tex;
        }

/

That's nice, but its not working yet because the Texure2D constructor expect an additional BufferParameter + CPUBuffer: but I don't want to provide these, since I really have no data to update into that texture yet. ⇒ Let's see if we can make it safe not to pass anything for those parameters.

Just found in Texture2DArray.spec.ts that I was already planning a use of CPUBuffer.EMPTY to cover this need: I could use that as default, but I feel that using a “null” value would be more appropriate ⇒ OK, the code above should work as expected now, and we can create our renderbuffer textures with:

        // Create the dem texture:
        let tileWidth = cache1.getStorage().getTileSize();
        let demTexture = await rman.loadTexture2D(`renderbuffer-${tileWidth}-RGBA32F`);
        this.DEBUG(`Generated dem texture of size ${demTexture.getWidth()}x${demTexture.getHeight()}`);

        // Create the residual texture:
        let residualTex = await rman.loadTexture2D(`renderbuffer-${tileWidth}-R32F`);
        this.DEBUG(`Generated residual texture of size ${residualTex.getWidth()}x${residualTex.getHeight()}`);

And now that I'm writting this code, it makes me realize that our ResourceManager will only generate one such texture per unique name ⇒ So I would consider extending the names with another unique element, like `renderbuffer-${tileWidth}-RGBA32F-ground_elev_dem`. I should keep that in mind (the next time I create a renderbuffer at least)

Arrf, noticed just after that this workaround on the renderbuffer name I mentioned is already used for the layer texture in fact

And the final loading code is as follow:

        // Create the elevation producer:
        this.DEBUG('Creating ElevationProducer...');

        // Create the upsample program:
        let upsampleProg = await rman.loadProgram('upsampleShader;');

        // Create the dem texture:
        let tileWidth = cache1.getStorage().getTileSize();
        let demTex = await rman.loadTexture2D(`renderbuffer-${tileWidth}-RGBA32F`);
        this.DEBUG(`Generated dem texture of size ${demTex.getWidth()}x${demTex.getHeight()}`);

        // Create the residual texture:
        let residualTex = await rman.loadTexture2D(`renderbuffer-${tileWidth}-R32F`);
        this.DEBUG(`Generated residual texture of size ${residualTex.getWidth()}x${residualTex.getHeight()}`);

        let elevProd = new ElevationProducer(
            cache1,
            null, // residualTiles
            demTex,
            null, //layerTexture,
            residualTex,
            upsampleProg,
            null, // blendProg,
            24, //gridSize,
            [-140, -100, -15, -8, 5, 2.5, 1.5, 1, 0.5, 0.25, 0.1, 0.05], //noiseAmp,
            false // flipDiagonals
        );

Now I just need to add that default upsampleShader shader code in the project.

Hmmm, I'm now observing an error when compiling the upsample shader:

ShaderModule3: Vertex shader compile error: ERROR: 0:8: 'sampler2DArray' : No precision specified

And this is strange, because I should already have a line with precision highp float; automatically inserted at the top of the shader, so what's happening here ?
⇒ So adding at the top of the glsl file the precision declaration: precision highp sampler2DArray;

Next I get an error from the Program class:

Program3: Unsupported uniform type for coarseLevelSampler

OK! Error fixed, and now I'm successfully creating my instance of the ElevationProducer in my test8 scene, yeeppee! 🤓😆

So in our “next episode”, we will try to see what is the next step to start using that producer.