====== Direct GPU buffer copy in CEF ======
===== Building CEF from sources =====
Build CEF directly from sources was less complicated than I anticipated: I meanly followed the instructions from the [[https://bitbucket.org/chromiumembedded/cef/wiki/MasterBuildQuickStart|Master Build Quick Start]] guide. Also taking some notes from the official [[https://bitbucket.org/chromiumembedded/cef/wiki/BranchesAndBuilding.md|Branches and Building]] page.
Initial steps:
* Create the base folders: d:\Projects\CEFBuild\automate
d:\Projects\CEFBuild\chromium_git
* Download the depot tools from: https://storage.googleapis.com/chrome-infra/depot_tools.zip
* Then extract to create the folder: d:\Projects\CEFBuild\depot_tools
* From "d:\Projects\CEFBuild\depot_tools", execute the script: update_depot_tools.bat
* Download the [[https://bitbucket.org/chromiumembedded/cef/raw/master/tools/automate/automate-git.py|automate-git.py]] script and place it in "d:\Projects\CEFBuild\automate"
In the end, I simply created a few helper batch files, placed in the **chromium_git** folder (note that my build root for CEF is: **D:\Projects\CEFBuild**):
- First the **update.bat** file, used to retrieve the code updates:set CEF_USE_GN=1
REM set GN_DEFINES=is_win_fastlink=true
set GN_ARGUMENTS=--ide=vs2015 --sln=cef --filters=//cef/*
python ..\automate\automate-git.py --download-dir=D:\Projects\CEFBuild\chromium_git --depot-tools-dir=D:\Projects\CEFBuild\depot_tools --no-distrib --no-build --branch=3163
- Then the **create_cef_project.bat** file, used to create the CEF Visual studio/Ninja project files:cd D:\Projects\CEFBuild\chromium_git\chromium\src\cef
set CEF_USE_GN=1
REM set GN_DEFINES=is_win_fastlink=true
set GN_ARGUMENTS=--ide=vs2015 --sln=cef --filters=//cef/*
call cef_create_projects.bat
- Then the **build_cef_release.bat** to build the release version of CEF:cd D:\Projects\CEFBuild\chromium_git\chromium\src
ninja -C out\Release_GN_x64 cef
- And finally the **make_distrib.bat** file which will generate the binary_distrib folder inside D:\Projects\CEFBuild\chromium_git\chromium\src\cef:REM cf. https://bitbucket.org/chromiumembedded/cef/wiki/BranchesAndBuilding.md
set CEF_VCVARS="D:\Apps\VisualStudio2015_CE\VC\bin\amd64\vcvars64.bat"
cd D:\Projects\CEFBuild\chromium_git\chromium\src\cef\tools
./make_distrib.bat --ninja-build --x64-build --allow-partial
One thing to note though is: by default on Windows the build system will attempt to build both x86 and x64 binaries. Yet, some dependencies where not available on my computer to build the x86 version of the library, so I had to manually disable that (before running **create_cef_project.bat** AFAIR) by tweaking the line of **D:\Projects\CEFBuild\chromium_git\chromium\src\cef\tools\gn_args.py** to only request the build of the x64 version: if platform == 'linux':
use_sysroot = GetArgValue(args, 'use_sysroot')
if use_sysroot:
# Only generate configurations for sysroots that have been installed.
for cpu in ('x86', 'x64', 'arm'):
if LinuxSysrootExists(cpu):
supported_cpus.append(cpu)
else:
msg('Not generating %s configuration due to missing sysroot directory'
% cpu)
else:
supported_cpus = ['x64']
elif platform == 'windows':
# supported_cpus = ['x86', 'x64']
supported_cpus = ['x64']
elif platform == 'macosx':
supported_cpus = ['x64']
else:
raise Exception('Unsupported platform')
But now I realize there might be a more appropriate way to handle this by specifying the **target_cpu** command line argument somewhere maybe ? (to be investigated).
Note: Currently experimenting with CEF **branch 3163**
===== Trying to automate header copy in make_distrib =====
The logic used the copy the headers during the make_distrib step is found in make_distrib.py around line 525: # transfer common include files
transfer_gypi_files(cef_dir, cef_paths2['includes_common'], \
'include/', include_dir, options.quiet)
* So it sounds easy to tweak the list of headers that should be distributed: we need to edit the 'includes_common' member from cef_paths2, which is read from file **${CEF_SRC_DIR}/cef_paths2.gypi**
* => **OK** when adding our files into the previous cef_paths2.gypi config files they are distributed as expected.
===== Investigating current Off Screen Rendering (OSR) pipeline =====
* Starts with **CefRenderHandler::OnPaint()** method
* Then it seems this handler is mostly used in **CefRenderWidgetHostViewOSR** class.
* Actually this class has an **OnPaint()** method which will access the handler.
* **CefRenderWidgetHostViewOSR::OnPaint(...)** is called in **CefCopyFrameGenerator::OnCopyFrameCaptureSuccess(...)**
* Previous method called in **CopyFromCompositingSurfaceFinished** and **PrepareBitmapCopyOutputResult**
* We eventually call: **cc::CopyOutputRequest::CreateRequest(...)** or **gl_helper->CropScaleReadbackAndCleanMailbox(...)**
* => We should check in **CopyFromCompositingSurfaceHasResult(...)** if we have either a texture result or a bitmap result (suspecting we have a texture when using GPU rendering and a bitmap when doing software rendering ?).
* => So we should add a CEF logging mechanism to check this and report the information in our target app.
* Also need to check **content::ImageTransportFactory** and **viz::GLHelper** classes
===== Implement CEF logging capability =====
We need to be able to retrieve some debug infos from the cef library in an integrated way: so we need to be able to assign a log handler inside the library, and then use this handle to output log messages if valid.
Can we simply add a new class/method inside the source base ? Let's try that: building the LogHandler class and trying to use it in CefRenderWidgetHostViewOSR:
* Trying to call **build_cef_release.bat** directly: **link failed**
* Detected the changes in render_widget_host_view_osr.cc
* But didn't detect the new source files **log_handler.{h, cc}** ?
* (Linking is taking ages...): need to stop the process to get the outputs (actually needs to ensure the dos console has the focus!), and as expected our new function is not found:[3/7] LINK(DLL) libcef.dll libcef.dll.lib libcef.dll.pdb
FAILED: libcef.dll libcef.dll.lib libcef.dll.pdb
D:/Projects/CEFBuild/depot_tools/win_tools-2_7_6_bin/python/bin/python.exe ../../build/toolchain/win/tool_wrapper.py link-wrapper environment.x64 False link.exe /nologo /IMPLIB:./libcef.dll.lib /DLL /OUT:./libcef.dll /PDB:./libcef.dll.pdb @./libcef.dll.rsp
libcef_static.lib(render_widget_host_view_osr.obj) : error LNK2019: unresolved external symbol "void __cdecl cef::handleLogMessage(class std::basic_string,class std::allocator > const &)" (?handleLogMessage@cef@@YAXAEBV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@Z) referenced in function "private: void __cdecl CefCopyFrameGenerator::CopyFromCompositingSurfaceHasResult(class gfx::Rect const &,class std::unique_ptr >)" (?CopyFromCompositingSurfaceHasResult@CefCopyFrameGenerator@@AEAAXAEBVRect@gfx@@V?$unique_ptr@VCopyOutputResult@cc@@U?$default_delete@VCopyOutputResult@cc@@@std@@@std@@@Z)
./libcef.dll : fatal error LNK1120: 1 unresolved externals
* Now trying to rebuild the project files first with **create_cef_project.bat**: **link failed**
* (Linking still taking ages...) and then we get the same error as above.
* So we probably need to declare the new files somewhere then... **yes**: that will be in the **src/cef/BUILD.gn** file.
* Adding our new files in BUILD.gn: **OK**
* Called **create_cef_project.bat**
* Then called **build_cef_release.bat**
* => **OK** this time the build is successful :-)!
===== Setup access to LogHandler =====
Now that we have a LogHandler support we should assign it on start of our software. How would we do that ?
* For now we added our functions/class in the **libcef** module, which is built as a static library.
* But then this static library is only linked into the libcef_dll shared library (?)
* CefInitialize(...) is implemented libcef\browser\context.cc: so at the same level as our log handler functions.
* So first let's move our LogHandler class header into the root include folder: (also removed log_handler.h from BUILD.gn) **OK**
* Now rebuilding: **Failing**: cannot find our cef_log_handler.h file: so the header location is incorrect ?
* Okay: so we keep the "log_handler.h" file next to log_handler.cc and we will manually provide a copy as cef_log_handler.h in our final include folder. Can now rebuild successfully.
* Setting up the new libcef dependency: manually installing our libcef/browser/log_handler.h header as "cef_log_handler.h"
* Now trying to rebuild our project: **failed** of course, we cannot find the function: CEFManager.cpp.obj : error LNK2019: unresolved external symbol "void __cdecl cef::setLogHandler(class cef::CefLogHandler *)" (?setLogHandler@cef@@YAXPEAVCefLogHandler@1@@Z) referenced in function "public: void __cdecl cef::CEFManager::initialize(class cef::CEFManager::Traits &)" (?initialize@CEFManager@cef@@QEAAXAEAVTraits@12@@Z)
scCEF.dll : fatal error LNK1120: 1 unresolved externals
So we need to figure out what we are missing here and how to make our function available from the libcef library to the outside world.
* **CefInitialize** is marked as GLOBAL in libcef_dll_wrapper.cc and it will call **cef_initialize()** which is exported from libcef, and in turn calls CefInitialize from the browser/context.cc class.
* => OK, so we need an exported method too in **libcef_dll.cc**, let's try that:
- Need to add our exported function declaration in **include/capi/cef_app_capi.h**: ///
// This function should be called on the main application thread to assign
// a log handler to the CEF module (used for investigations)
///
CEF_EXPORT void cef_set_log_handler(struct cef::CefLogHandler* handler);
- So we need to use a struct in C now instead of a class in C++. We update the cef_log_handler.h header accordingly.
- Then we implement and export our function **cef_set_log_handler** inside libcef_dll.cc: CEF_EXPORT void cef_set_log_handler(cef::CefLogHandler* handler) {
cef::setLogHandler(handler);
}
- And we also get rid of the internal header **browser/log_handler.h** => we use "include/cef_log_handler.h" directly instead in both log_handler.cc and render_widget_host_view_osr.cc
* Now rebuilding: **OK**
* Deploying in target project: noticed that our cef_app_capi.h doesn't contain our new method => copying it manually!
* **Bingo** :-) ! we can now call our new cef_set_log_handler() function as expected in our target project.
* Now let's run the project and see if we get the expected outputs since we added in render_widget_host_view_osr.cc: void CopyFromCompositingSurfaceHasResult(
const gfx::Rect& damage_rect,
std::unique_ptr result) {
if (result->IsEmpty() || result->size().IsEmpty() ||
!view_->render_widget_host()) {
OnCopyFrameCaptureFailure(damage_rect);
return;
}
if (result->HasTexture()) {
DEBUG_MSG("NervCEF: CompositingSurface has texture!");
PrepareTextureCopyOutputResult(damage_rect, std::move(result));
return;
}
DCHECK(result->HasBitmap());
DEBUG_MSG("NervCEF: CompositingSurface has bitmap!");
PrepareBitmapCopyOutputResult(damage_rect, std::move(result));
}
=> We expect to get one or the other of those 2 debug outputs (or both ?)... well, instead we got a crash :-( but it doesn't seem to be related to what we are changing here: 2017-09-15T10:19:33.735898 [Debug] Calling CefInitialize...
[0915/111933.735:FATAL:libcef_dll_wrapper.cc(209)] Check failed: false.
* The code at that location is: CEF_GLOBAL bool CefInitialize(const CefMainArgs& args,
const CefSettings& settings,
CefRefPtr application,
void* windows_sandbox_info) {
const char* api_hash = cef_api_hash(0);
if (strcmp(api_hash, CEF_API_HASH_PLATFORM)) {
// The libcef API hash does not match the current header API hash.
NOTREACHED();
return false;
}
// More stuff here
}
* Adding execution flag on all dlls and exes files (but this doesn't help): chmod +x *.dll
chmod +x *.exe
* Copy all CEF resources: **doesn't help**
* But what we can realize here is that the error is due to an API hash mismatch! => So we need to copy all the header files from CEF again when building a binary distrib! Crap... doesn't work that easily: we actually modify the library CAPI, and those hashes are generated from these...: //The API hash is created by analyzing CEF header files for C API type
// definitions. The hash value will change when header files are modified
// in a way that may cause binary incompatibility with other builds. The
// universal hash value will change if any platform is affected whereas the
// platform hash values will change only if that particular platform is
// affected.
===== Setup access to LogHandler - Second try =====
Okay, so we are probably not too far from success, **BUT** we should avoid making changes to the **include/capi** folder, thus, not include our new exported function into **cef_app_capi.h** => This doesn't matter much: let's just defined it in a separated file!
* Rebuilt CEF with our special **cef_nv_log_handler.h** and **cef_nv_exports.h** headers: **OK**
* Installing as dependency for target project: => need to manually copy those new files. **OK**
=> **Great!** :-) We now have the expected outputs: 2017-09-17T13:37:32.881686 [Debug] NervCEF: CompositingSurface has texture!
2017-09-17T13:37:33.056642 [Debug] NervCEF: CompositingSurface has texture!
2017-09-17T13:37:33.377407 [Debug] NervCEF: CompositingSurface has texture!
===== Investigating image copy pipeline =====
Now that we can output some debug outputs if need, let's try to clarify further the offscreen image copy pipeline.
First, let's check what kind of messages we got in the previous run test:
* We only get the message: **CompositingSurface has texture!** and never reach the **has bitmap!** part.
* So it seems correct that a "texture" source is used when the GPU acceleration is active, and a simple bitmap is used instead when the CPU generation is active.
=> It all comes down to the method: **viz::GLHelper::CropScaleReadbackAndCleanMailbox(...)**
Found the following description for this method in **components/viz/common/gl_helper.h**: // Copies the block of pixels specified with |src_subrect| from |src_mailbox|,
// scales it to |dst_size|, and writes it into |out|.
// |src_size| is the size of |src_mailbox|. The result is in |out_color_type|
// format and is potentially flipped vertically to make it a correct image
// representation. |callback| is invoked with the copy result when the copy
// operation has completed.
// Note that the texture bound to src_mailbox will have the min/mag filter set
// to GL_LINEAR and wrap_s/t set to CLAMP_TO_EDGE in this call. src_mailbox is
// assumed to be GL_TEXTURE_2D.
So our parameter "src_mailbox" is assumed to be a **GL_TEXTURE_2D** => sounds good for us! Now checking the method implementation in **components/viz/common/gl_helper.cc**: void GLHelper::CropScaleReadbackAndCleanMailbox(
const gpu::Mailbox& src_mailbox,
const gpu::SyncToken& sync_token,
const gfx::Size& src_size,
const gfx::Rect& src_subrect,
const gfx::Size& dst_size,
unsigned char* out,
const SkColorType out_color_type,
const base::Callback& callback,
GLHelper::ScalerQuality quality) {
GLuint mailbox_texture = ConsumeMailboxToTexture(src_mailbox, sync_token);
CropScaleReadbackAndCleanTexture(mailbox_texture, src_size, src_subrect,
dst_size, out, out_color_type, callback,
quality);
gl_->DeleteTextures(1, &mailbox_texture);
Checking the implementation oc **ComsumeMailboxToTexture**: GLuint GLHelper::ConsumeMailboxToTexture(const gpu::Mailbox& mailbox,
const gpu::SyncToken& sync_token) {
if (mailbox.IsZero())
return 0;
if (sync_token.HasData())
WaitSyncToken(sync_token);
GLuint texture =
gl_->CreateAndConsumeTextureCHROMIUM(GL_TEXTURE_2D, mailbox.name);
return texture;
}
And finally we find for **CreateAndConsumeTextureCHROMIUM** (inside **gpu/command_buffer/client/gles2_implementation.cc**): GLuint GLES2Implementation::CreateAndConsumeTextureCHROMIUM(
GLenum target, const GLbyte* data) {
GPU_CLIENT_SINGLE_THREAD_CHECK();
GPU_CLIENT_LOG("[" << GetLogPrefix() << "] glCreateAndConsumeTextureCHROMIUM("
<< static_cast(data) << ")");
const Mailbox& mailbox = *reinterpret_cast(data);
DCHECK(mailbox.Verify()) << "CreateAndConsumeTextureCHROMIUM was passed a "
"mailbox that was not generated by "
"GenMailboxCHROMIUM.";
GLuint client_id;
GetIdHandler(SharedIdNamespaces::kTextures)->MakeIds(this, 0, 1, &client_id);
helper_->CreateAndConsumeTextureINTERNALImmediate(target,
client_id, data);
if (share_group_->bind_generates_resource())
helper_->CommandBufferHelper::Flush();
CheckGLError();
return client_id;
}
So it sounds like everytime we request a copy we start with creating a GL texture, then we crop/scale/readback on CPU memory, and finally we destroy the texture.
We continue following the **CreateAndConsumeTextureINTERNALImmediate** method (found in **gles2_cmd_helper_autogen.h**): void CreateAndConsumeTextureINTERNALImmediate(GLenum target,
GLuint texture,
const GLbyte* mailbox) {
const uint32_t size =
gles2::cmds::CreateAndConsumeTextureINTERNALImmediate::ComputeSize();
gles2::cmds::CreateAndConsumeTextureINTERNALImmediate* c =
GetImmediateCmdSpaceTotalSize<
gles2::cmds::CreateAndConsumeTextureINTERNALImmediate>(size);
if (c) {
c->Init(target, texture, mailbox);
}
}
And from there, we call this init method (found in **gles2_cmd_format_autogen.h**): void Init(GLenum _target, GLuint _texture, const GLbyte* _mailbox) {
SetHeader();
target = _target;
texture = _texture;
memcpy(ImmediateDataAddress(this), _mailbox, ComputeDataSize());
}
So: we create a GL texture Id (with the call to **MakeIds(...)**), then we send a command to copy the **mailbox** data into our texture. So, I think when **CreateAndConsumeTextureCHROMIUM()** returns we should have a valid GL texture ID, which contains the imagery data we want to retrieve ?
Then the next question is: **How can we retrieve this data into our own texture object on the GPU from a different GL or DirectX context ?**
===== Inter-context texture copy =====
==== From GL to GL ====
Reading a few related pages:
* https://software.intel.com/en-us/articles/using-opengl-es-to-accelerate-apps-with-legacy-2d-guis
* https://www.opengl.org/discussion_boards/showthread.php/168418-Sharing-Texture-amongst-different-Process
=> Those links do not seem very optimistic, but then I found the [[http://developer.download.nvidia.com/opengl/specs/GL_NV_copy_image.txt|GL_NV_copy_image]] extension page, which states that: This extension enables efficient image data transfer between image
objects (i.e. textures and renderbuffers) without the need to bind
the objects or otherwise configure the rendering pipeline. The
WGL and GLX versions allow copying between images in different
contexts, even if those contexts are in different sharelists or
even on different physical devices.
This sounds more interesting already. But then we have the context specification issue in **wglCopyImageSubDataNV** and **glXCopyImageSubDataNV**: we need to be able to provide the source/dest contexts as **HGLRC** or **GLXContext** objects.
So let's assume for a moment that this function is available (ie. we are using a recent nvidia GPU), and that we are on windows (ie. we are considering the HGLRC contexts): how could we retrieve the source context ?
* https://msdn.microsoft.com/fr-fr/library/windows/desktop/dd374379(v=vs.85).aspx
* But we cannot find any HGLRC reference in Chromium sources... maybe that's just not how it works for OpenGL ES ?
* https://www.khronos.org/registry/EGL/sdk/docs/man/html/eglCreateContext.xhtml: eglCreateContext creates an EGL rendering context for the current rendering API (as set with eglBindAPI) and returns a handle to the context. The context can then be used to render into an EGL drawing surface. If eglCreateContext fails to create a rendering context, EGL_NO_CONTEXT is returned.
If share_context is not EGL_NO_CONTEXT, then all shareable data in the context (as defined by the client API specification for the current rendering API) are shared by context share_context, all other contexts share_context already shares with, and the newly created context. An arbitrary number of rendering contexts can share data. However, all rendering contexts that share data must themselves exist in the same address space. Two rendering contexts share an address space if both are owned by a single process.
* The previous seems to indicate that interprocess data sharing with EGL is not working ? :-(
* https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glFenceSync.xhtml
* http://oss.sgi.com/projects/ogl-sample/registry/EXT/import_context.txt
* https://community.arm.com/graphics/f/discussions/2678/using-eglimage-in-a-multi-process-and-with-two-different-apis: not an option either.
==== From GL to DirectX ====
Found this article which sounds promizing: https://github.com/Microsoft/angle/wiki/Interop-with-other-DirectX-code
=> Are we using "ANGLE" in CEF ? According to the previous [[http://magpcss.org/ceforum/viewtopic.php?f=8&t=11635|forum post]] **YES**. So the rational here should be applicable: we should be able to:
- Create a **shared** DirectX texture (DX11 is mentioned, but maybe it could also workd with DirectX9Ex ?)
- Then we pass the shared handle to the CEF engine and use it to create a new EGL surface.
- We bind the surface to a GL texture,
- And we copy from our source mailbox onto that texture ?
This might all just sound like a plan ;-)...
===== Investigating old Hardware offscreen render support implementation =====
So what about the first accelerated paint implementation mentioned in this [[http://magpcss.org/ceforum/viewtopic.php?f=8&t=11635|forum post]] ?
=> Not understanding much from this patch. But it seems to rely on the OpenGL API. I don't think we can really apply the logic available here in the new Chromium source code base. So let's instead focus on the DirectX/Angle interactions mentioned above.
===== Figuring out how the CEF commands work =====
* If we take the case of the **GenTextures** method for instance, in th GLES2Implementation the actual generation of the texture IDs is done with the **helper_** element which is a **GLES2CmdHelper**
* Calling **GenTexturesImmediate(...)** on this object will generate a **gles2::cmds::GenTexturesImmediate** command, which is initialized with the provided parameters.
* All the commands are defined in the file **gles2_cmd_format_autogen.h**
* Okay, great, we have this "GenTexturesImmediate" structure... but where do we actually **use** it then ?
* Found in **ui/gl/angle_platform_impl.{h,cc}** the function **InitializeAnglePlatform(...)** => maybe we could check if this is indeed called ? (And where ?)
* We call in in **ui/gl/gl_surface_egl.cc** when initializing our display...
Now trying to figure out where we create a **GLSurfaceEGL** class instance... Searching in all .cc files with: $ grep -rnw --include \*.cc . -e 'GLSurfaceEGL'
./components/exo/wayland/clients/client_base.cc:279: if (gl::GLSurfaceEGL::HasEGLExtension("EGL_EXT_image_flush_external") ||
./components/exo/wayland/clients/client_base.cc:280: gl::GLSurfaceEGL::HasEGLExtension("EGL_ARM_implicit_external_sync")) {
./components/exo/wayland/clients/client_base.cc:283: if (gl::GLSurfaceEGL::HasEGLExtension("EGL_ANDROID_native_fence_sync")) {
./gpu/command_buffer/service/texture_definition.cc:150: EGLDisplay egl_display = gl::GLSurfaceEGL::GetHardwareDisplay();
./gpu/ipc/service/direct_composition_child_surface_win.cc:45: : gl::GLSurfaceEGL(),
./gpu/ipc/service/direct_composition_surface_win.cc:97: if (!gl::GLSurfaceEGL::IsDirectCompositionSupported())
./gpu/ipc/service/direct_composition_surface_win.cc:1005: : gl::GLSurfaceEGL(),
./gpu/ipc/service/gpu_init.cc:110: gl::GLSurfaceEGL::IsDirectCompositionSupported() &&
./gpu/ipc/service/image_transport_surface_win.cc:54: if (gl::GLSurfaceEGL::IsDirectCompositionSupported()) {
# (...) => more lines here.
* Found interesting use case in **gpu/command_buffer/service/texture_definition.cc:150**: scoped_refptr NativeImageBufferEGL::Create(
GLuint texture_id) {
EGLDisplay egl_display = gl::GLSurfaceEGL::GetHardwareDisplay();
EGLContext egl_context = eglGetCurrentContext();
DCHECK_NE(EGL_NO_CONTEXT, egl_context);
DCHECK_NE(EGL_NO_DISPLAY, egl_display);
DCHECK(glIsTexture(texture_id));
DCHECK(gl::g_driver_egl.ext.b_EGL_KHR_image_base &&
gl::g_driver_egl.ext.b_EGL_KHR_gl_texture_2D_image &&
gl::g_current_gl_driver->ext.b_GL_OES_EGL_image);
const EGLint egl_attrib_list[] = {
EGL_GL_TEXTURE_LEVEL_KHR, 0, EGL_IMAGE_PRESERVED_KHR, EGL_TRUE, EGL_NONE};
EGLClientBuffer egl_buffer = reinterpret_cast(texture_id);
EGLenum egl_target = EGL_GL_TEXTURE_2D_KHR;
EGLImageKHR egl_image = eglCreateImageKHR(
egl_display, egl_context, egl_target, egl_buffer, egl_attrib_list);
if (egl_image == EGL_NO_IMAGE_KHR) {
LOG(ERROR) << "eglCreateImageKHR for cross-thread sharing failed: 0x"
<< std::hex << eglGetError();
return NULL;
}
return new NativeImageBufferEGL(egl_display, egl_image);
}
* => This seems somewhat similar to the code we would like to execute.
* We could check if the display is initialized in our client process ? (But I doubt it...) Or maybe we could send our shared texture handle into the GPUProcess to copy a given texture ?
===== Server side command handling =====
On the client side, it now seems relatively clear how the various GL commands are "posted", but now we need to figure out how those commands are handled on the server side.
* Found this page on [[http://dev.chromium.org/developers/design-documents/gpu-accelerated-compositing-in-chrome|GPU acceletated Compositing in Chrome]]
* **[Unrelated note]**: **gpu/ipc/service/direct_composition_child_surface_win.{h, cc}**: contains a reference on a **DirectX11 device**.
* This device is retrieved directly from ANGLE with: bool DirectCompositionChildSurfaceWin::Initialize(gl::GLSurfaceFormat format) {
d3d11_device_ = gl::QueryD3D11DeviceObjectFromANGLE();
dcomp_device_ = gl::QueryDirectCompositionDevice(d3d11_device_);
if (!dcomp_device_)
return false;
// More code here
}
=> Focusing on the initial question: trying to find where we use the command ID **kGenTexturesImmediate** ?
* **grep -rnw --include \*.cc . -e 'kGenTexturesImmediate'** => gives nothing
* **grep -rnw --include \*.h . -e 'kGenTexturesImmediate'** => gives only one result :-(
* **grep -rnw --include \*.cc . -e 'kCmdId'** => gives nothing
* **grep -rnw --include \*.h . -e 'kCmdId'** => only some not interesting entries.
So it seems the Command ID is not how we retrieve the GenTexturesImmediate of command on the server/host side ?
* Searching with: **grep -rnw --include \*.h --include \*.cc . -e 'GenTexturesImmediate'**
* Found references in **gles2_cmd_decoder_autogen.h**: **gles2_cmd_decoder.h** defines an abstract **GLES2Decoder** class
* A concrete class is **GLES2DecoderPassthroughImpl** (in **gles2_cmd_decoder_passthrough.{h,cc}**)
* Found that in the decoder autogen file we have an handler for the GenTexturesImmediate, which is: error::Error GLES2DecoderImpl::HandleGenTexturesImmediate(
uint32_t immediate_data_size,
const volatile void* cmd_data) {
const volatile gles2::cmds::GenTexturesImmediate& c =
*static_cast(cmd_data);
GLsizei n = static_cast(c.n);
uint32_t data_size;
if (!SafeMultiplyUint32(n, sizeof(GLuint), &data_size)) {
return error::kOutOfBounds;
}
volatile GLuint* textures =
GetImmediateDataAs(c, data_size, immediate_data_size);
if (textures == NULL) {
return error::kOutOfBounds;
}
auto textures_copy = base::MakeUnique(n);
GLuint* textures_safe = textures_copy.get();
std::copy(textures, textures + n, textures_safe);
if (!CheckUniqueAndNonNullIds(n, textures_safe) ||
!GenTexturesHelper(n, textures_safe)) {
return error::kInvalidArguments;
}
return error::kNoError;
}
=> From there, the GenTexturesHelper(...) method will call **glGenTextures()**: bool GLES2DecoderImpl::GenTexturesHelper(GLsizei n, const GLuint* client_ids) {
for (GLsizei ii = 0; ii < n; ++ii) {
if (GetTexture(client_ids[ii])) {
return false;
}
}
std::unique_ptr service_ids(new GLuint[n]);
glGenTextures(n, service_ids.get());
for (GLsizei ii = 0; ii < n; ++ii) {
CreateTexture(client_ids[ii], service_ids[ii]);
}
return true;
}
So this is where we call the gl functions: in the GLES2Decoder. Yet, how do we reach the **HandleGenTexturesImmediate** method ?
* Not too surprisingly, we can find this method re-implemented in **GLES2DecoderPassthroughImpl**
* The re-implementation is a bit different, but will eventually call glGenTextures() too.
GLES2Decoder class cannot be instantiated because is is abstract. Yet we need to figure out what are all the possible derived classes and where they are instantiated now:
* Seems that for the creation, we use the static method: GLES2Decoder* GLES2Decoder::Create(
GLES2DecoderClient* client,
CommandBufferServiceBase* command_buffer_service,
ContextGroup* group) {
if (group->gpu_preferences().use_passthrough_cmd_decoder) {
return new GLES2DecoderPassthroughImpl(client, command_buffer_service,
group);
}
return new GLES2DecoderImpl(client, command_buffer_service, group);
}
* So we can create the implementation either as **GLES2DecoderPassthroughImpl** or **GLES2DecoderImpl**
* This depends on the config settings **GpuPreferences::use_passthrough_cmd_decoder** which is set to false by default.
* The config entry can be set from **content/public/browser/gpu_utils.cc:126** if the command line "use-passthrough-cmd-decoder" is provided.
This all still doesn't tell us where HandleGenTexturesImmediate is effectively called... The method is not available to any public interface so it should be used only "internally" ?
* The internal **QueryManager** seems to generate queries that should then be processed.
* But the **CommandsIssuedQuery** class will just throw an exception in Process(...). My my my...
The GLES2Decoder instance is create with a **CommandBufferService** object.
=> Found information on **how to add a command** from [[https://www.chromium.org/developers/design-documents/gpu-command-buffer|this page]]
Found the **CommandInfo** struct in gles2_cmd_decoder.cc: // A struct to hold info about each command.
struct CommandInfo {
CmdHandler cmd_handler;
uint8_t arg_flags; // How to handle the arguments for this command
uint8_t cmd_flags; // How to handle this command
uint16_t arg_count; // How many arguments are expected for this command.
};
// A table of CommandInfo for all the commands.
static const CommandInfo command_info[kNumCommands - kFirstGLES2Command];
OK, so now we need to figure out where this array is filled, and where we call the cmd_handler function!
* command_info array populated with: const GLES2DecoderImpl::CommandInfo GLES2DecoderImpl::command_info[] = {
#define GLES2_CMD_OP(name) \
{ \
&GLES2DecoderImpl::Handle##name, cmds::name::kArgFlags, \
cmds::name::cmd_flags, \
sizeof(cmds::name) / sizeof(CommandBufferEntry) - 1, \
} \
, /* NOLINT */
GLES2_COMMAND_LIST(GLES2_CMD_OP)
#undef GLES2_CMD_OP
};
* => This is applying a macro operator on the list of commands described in **gles2_cmd_ids_autogen.h** => So here we have it: this is how the "server" side knows which handler should be executed for a given command :-)
* Access to the **command_info** array will then happen in the **GLES2DecoderImpl::DoCommandsImpl(...)** method: where we retrieve some commands from the command buffer, and then call the corresponding GL functions with the handlers. **OK**
===== Sharing DirectX texture with ANGLE =====
As mentioned above, it seems to be possible to have ANGLE write to ashared DirectX texture. So in our case we would need to follow those steps:
==== 1. Create a shared DirectX texture ====
We should already have the required support functions to do this ?
* Our CEFOverlayRenderer class will create a simple "Color texture" on construction, could we get a shared handle for this ? => we just need to update the **RenderEngine::createColorTexture()** to support an optional **bool shared** parameter. Once this is done, we set the shared flag to tru for this texture on creation, and then we can expect to get a share handle from the DirectX context upon texture creation.
* Then in our MXRenderHandler helper class, we could retrieve the share handle. But the retrieval should be done in the base **CefRenderHandler** class, with a method such as **CefRenderHandler::getShareHandle()**. Or we could use an indirect handle retriever such as with: struct SharedRenderSurfaceManager {
virtual void* getShareHandle(CefRenderHandler* handler) = 0;
};
CEF_EXPORT void cef_set_shared_render_surface_manager(SharedRenderSurfaceManager* manager);
Then we can provide our own implementation of the SharedRenderSurfaceManager class on CEF start, just like with the LogHandler class above.
Where do we use the getShareHandle() then ?
* Can we retrieve the CefRenderHandler from **CefCopyFrameGenerator::PrepareTextureCopyOutputResult** ?
* We receive a **CefRenderWidgetHostViewOSR** as **view_**
* From inside that view we can call: CefRefPtr handler = browser_impl_->GetClient()->GetRenderHandler();
* => So yes, we should be able to retrieve the RenderHandler pointer!
* From there we can retrieve the share handle using the SharedRenderSurfaceManager: **OK**
Then, we need to create an EGL surface based on this handle: EGLSurface surface = EGL_NO_SURFACE;
EGLint pBufferAttributes[] =
{
EGL_WIDTH, width,
EGL_HEIGHT, height,
EGL_TEXTURE_TARGET, EGL_TEXTURE_2D,
EGL_TEXTURE_FORMAT, EGL_TEXTURE_RGBA,
EGL_NONE
};
surface = eglCreatePbufferFromClientBuffer(mEglDisplay, EGL_D3D_TEXTURE_2D_SHARE_HANDLE_ANGLE, sharedHandle, mEglConfig, pBufferAttributes);
if (surface == EGL_NO_SURFACE)
{
// error handling code
}
=> Any chance we could execute the previous code in the CEF context ?
* From the GLHelper class, the **gl_** member is a **gpu::gles2::GLES2Interface** pointer.
* Where do we create the GLHelper instance ?
* It comes from the **ImageTransportFactory** instance.
* The ImageTransportFactory should be a **GpuProcessTransportFactory**
* The GpuProcessTransportFactory, will create the GLHelper with: viz::GLHelper* GpuProcessTransportFactory::GetGLHelper() {
if (!gl_helper_ && !per_compositor_data_.empty()) {
scoped_refptr provider = SharedMainThreadContextProvider();
if (provider.get())
gl_helper_.reset(
new viz::GLHelper(provider->ContextGL(), provider->ContextSupport()));
}
return gl_helper_.get();
}
* So from our **ContextProvider** we get a **GLES2interface** pointer with ContextGL().
* Is this GLES2Interface() somehow what we need to call an eglXXX() function ?
* Additional reference page: https://bugs.chromium.org/p/chromium/issues/detail?id=77011
* How to use EGL: https://www.saschawillems.de/?page_id=1822
* The GLES2Interface is provided in **ContextProviderCommandBuffer** as a **gpu::gles2::GLES2Implementation**: gpu::gles2::GLES2Interface* ContextProviderCommandBuffer::ContextGL() {
DCHECK(bind_succeeded_);
DCHECK(context_thread_checker_.CalledOnValidThread());
if (trace_impl_)
return trace_impl_.get();
return gles2_impl_.get();
}
==== 2. Request rendering on shared DX texture ====
Once we have a valid share handle for our DirectX texture, then each time we have to refresh the offscreen rendered image, we could send a command to the GPUProcess, to request rendering from a given GL texture on our shared DX texture. Something like: **copyOnDXSharedSurface(uint glTextureID, void* shareHandle)**.
==== 3. Resources allocation/deallocation ====
Before we can copy to our shared DirectX surface, we need to allocate the required resources for it: this can be done on the GPU process itself directly: if when we handle the copy, if the required resources for a given share handle are not found, then we create them.
But then we have the question of the resources deallocation: when we are done rendering with a given browser, we should release those resources allocated on the GPU Process. For instance with a command such as: **releaseDXSharedSurfaceResources(void* shareHandle)**.
==== 4. Logging in GPU Process ====
During development, we will have to get some outputs from the GPU Process itself to ensure everything is happening as expected. How can we get those outputs ?
* Found this page: https://www.chromium.org/developers/how-tos/debugging-gpu-related-code
* Doesn't seem there is a very simple way to focus on our own outputs: so instead we could just build a minimal logging to file system and we can output messages from there in the GPU Process itself.
===== First implementation stage =====
Creating the nvLogger helper class, and trying to use it to log messages to file when creating the GLES command decoder:
* We get a link error in that case: [6/13] LINK ui.service.exe ui.service.exe.pdb
FAILED: ui.service.exe ui.service.exe.pdb
D:/Projects/CEFBuild/depot_tools/win_tools-2_7_6_bin/python/bin/python.exe ../../build/toolchain/win/tool_wrapper.py link-wrapper environment.x64 False link.exe /nologo /OUT:./ui.service.exe /PDB:./ui.service.exe.pdb @./ui.service.exe.rsp
service_sources.lib(gles2_cmd_decoder.obj) : error LNK2019: unresolved external symbol "void __cdecl cef::nvLOG(class std::basic_string,class std::allocator > const &)" (?nvLOG@cef@@YAXAEBV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@Z) referenced in function "public: static class gpu::gles2::GLES2Decoder * __cdecl gpu::gles2::GLES2Decoder::Create(class gpu::gles2::GLES2DecoderClient *,class gpu::CommandBufferServiceBase *,class gpu::gles2::ContextGroup *)" (?Create@GLES2Decoder@gles2@gpu@@SAPEAV123@PEAVGLES2DecoderClient@23@PEAVCommandBufferServiceBase@3@PEAVContextGroup@23@@Z)
./ui.service.exe : fatal error LNK1120: 1 unresolved externals
* This is not so surprising. Let's instead move our logger code into the gpu service folder and reference it from here instead of from the cef higher level module.
* Adding our new **nv_logger.{h,cc}** in the service level BUILD.gn file.
Running the target project with this new version of CEF we have a **cef_nv.log file** generated! But the content is not what we expected: nvLogger initialized.Destroying nvLogger.
* Fixing the final end line symbol: **OK**
* Ensuring we don't call the logger at all if we are not in the GPU Process: **OK**
* => What could have happened here is that the logger was called in the GPU process (and the file was written with the expected content), but then on close of the browser process, the "Destroying nvLogger" message was sent, and thus the file was re-written from the browser process).
* Rebuilding: **OK**
* Now our content is just a little bit better: nvLogger initialized.Creating GLES2DecoderImpl instance.Creating GLES2DecoderImpl instance.Creating GLES2DecoderImpl instance.Creating GLES2DecoderImpl instance.
* Hmmm, well this content gets overriden again then! Not sure the build was done correctly actually... Rebuilding again: Arrff, just **forgot to call make_distrib.bat** actually! (you stupid manu... ;-))
* After make_distrib.bat call the behavior is what we expect (except that the nvLogger instance "destruction" doesn't get logged but that's not too critical and depends on how the GPU process is stopped): nvLogger initialized.
Creating GLES2DecoderImpl instance.
Creating GLES2DecoderImpl instance.
Creating GLES2DecoderImpl instance.
Creating GLES2DecoderImpl instance.
Creating GLES2DecoderImpl instance.
Creating GLES2DecoderImpl instance.
Creating GLES2DecoderImpl instance.
Creating GLES2DecoderImpl instance.
Creating GLES2DecoderImpl instance.
Creating GLES2DecoderImpl instance.
Creating GLES2DecoderImpl instance.
What would be interesting to add from that point now si the process ID / thread id / and log time for each message in that file.
* We have a Process class in **base/process/process.h**
* Thread Id can be retrieved from the class in **base/threading/platform_thread.h**
* Time can be retrieved from **base/time/time.h**
=> OK, updated the code, and now we get those results: 2017-09-26 13:53:13.473 UTC [20408]{16984}: nvLogger initialized.
2017-09-26 13:53:13.473 UTC [20408]{16984}: Creating GLES2DecoderImpl instance.
2017-09-26 13:53:13.479 UTC [20408]{16984}: Creating GLES2DecoderImpl instance.
2017-09-26 13:53:13.482 UTC [20408]{16984}: Creating GLES2DecoderImpl instance.
2017-09-26 13:53:13.484 UTC [20408]{16984}: Creating GLES2DecoderImpl instance.
2017-09-26 13:53:32.438 UTC [20408]{16984}: Creating GLES2DecoderImpl instance.
2017-09-26 13:53:34.164 UTC [20408]{16984}: Creating GLES2DecoderImpl instance.
2017-09-26 13:53:34.170 UTC [20408]{16984}: Creating GLES2DecoderImpl instance.
2017-09-26 13:53:36.405 UTC [20408]{16984}: Creating GLES2DecoderImpl instance.
2017-09-26 13:53:36.443 UTC [20408]{16984}: Creating GLES2DecoderImpl instance.
2017-09-26 13:53:36.448 UTC [20408]{16984}: Creating GLES2DecoderImpl instance.
2017-09-26 13:53:37.010 UTC [20408]{16984}: Creating GLES2DecoderImpl instance.
So it seems all GLES2Decoder instances are created from the same process id (expected) and from the same thread: so this all good for us.
===== Where to retrieve the rendered frame =====
* **OnSwapCompositorFrame()** is the key method executed when a frame is done rendering.
* We register a callback to get notified when this method is called with **CefRenderWidgetHostViewOSR::RegisterGuestViewFrameSwappedCallback()**
* When this callback is called, we call **CefRenderWidgetHostViewOSR::InvalidateInternal()** and this will request a copy of the generated frame.
* Then the **CefCopyFrameGenerator** is used to handle the frame request and dispatch.
* Updating **CefRenderHandler** adding the functions: /**
* Check if shared handle should be used.
*/
virtual bool UseSharedHandle() const { return false; }
/**
* Retrieve the shared handle if applicable.
*/
virtual void* GetSharedHandle() const { return nullptr; }
* => Same problem as with the client: the RenderHandler pointer in Cef library is different from the one in SimCore. Maybe this object is copied ?
* Trying to make the functions pure virtuals => indeed we get an error now: ../../cef/tests/cefclient/browser/browser_window_osr_win.cc(19): error C2259: 'client::ClientHandlerOsr': cannot instantiate abstract class
../../cef/tests/cefclient/browser/browser_window_osr_win.cc(19): note: due to following members:
../../cef/tests/cefclient/browser/browser_window_osr_win.cc(19): note: 'bool CefRenderHandler::UseSharedHandle(void) const': is abstract
D:\Projects\CEFBuild\chromium_git\chromium\src\cef\include/cef_render_handler.h(64): note: see declaration of 'CefRenderHandler::UseSharedHandle'
../../cef/tests/cefclient/browser/browser_window_osr_win.cc(19): note: 'void *CefRenderHandler::GetSharedHandle(void) const': is abstract
D:\Projects\CEFBuild\chromium_git\chromium\src\cef\include/cef_render_handler.h(69): note: see declaration of 'CefRenderHandler::GetSharedHandle'
* => This is only for the unit testing code, so we just add default implementation in the ClientHandlerOsr::OsrDelegate class: virtual bool UseSharedHandle() const { return false; }
virtual void* GetSharedHandle() const { return nullptr; }
* There should still be another error... OK, next error is: D:\Projects\CEFBuild\chromium_git\chromium\src\cef\libcef_dll/ctocpp/ctocpp_ref_counted.h(110): error C2259: 'CefRenderHandlerCToCpp': cannot instantiate abstract class
* We need to update the class defined in "libcef_dll/ctocpp/render_handler_ctocpp.{h,cc}"
* => Should not modify those files by hand, should rather read the file: **translator.README.txt**
* Executing generation in "chromium/src/cef/tools": translator.bat
* => Same error (previous command reported that 0 files were written anyway)
* Adding the magic comment above the new functions: /*--cef()--*/
* OK! Then also updated the comments for those functions. Now the translator will update some files: ///
// Check if shared handle should be used with this render handler.
///
/*--cef()--*/
virtual bool UseSharedHandle() { return false; }
///
// Return the shared handle for this renderhandler. If no shared handle
// is available then null is returned.
///
/*--cef()--*/
virtual void* GetSharedHandle() { return nullptr; }
* Then building will fail because of missing return value for GetSharedHandle() in "cef/libcef_dll/ctocpp/render_handler_ctocpp.cc(39)"
* Implementation was generated properly for UseSharedHandle: bool CefRenderHandlerCToCpp::UseSharedHandle() const {
cef_render_handler_t* _struct = GetStruct();
if (CEF_MEMBER_MISSING(_struct, use_shared_handle))
return false;
// AUTO-GENERATED CONTENT - DELETE THIS COMMENT BEFORE MODIFYING
// Execute
int _retval = _struct->use_shared_handle(_struct);
// Return type: bool
return _retval ? true : false;
}
* We implement GetSharedHandle as: void* CefRenderHandlerCToCpp::GetSharedHandle() const {
cef_render_handler_t* _struct = GetStruct();
if (CEF_MEMBER_MISSING(_struct, get_shared_handle))
return NULL;
void* handle = _struct->get_shared_handle(_struct);
return handle;
}
* Also updating the method **render_handler_get_shared_handle** from "render_handler_cpptoc.cc": void* CEF_CALLBACK
render_handler_get_shared_handle(const struct _cef_render_handler_t* self) {
DCHECK(self);
if (!self)
return 0;
// Execute
bool handle = CefRenderHandlerCppToC::Get(self)->GetSharedHandle();
// Return type: void*
return handle;
}
* Still we should not make the functions pure virtuals: because of the unit tests.
* Also needed to remove the **const** qualifiers on our new methods.
* => And still not working after rebuild. So let's really keep our new methods as pure virtual if you want to play it that way boy.
* Strangely, the RenderHandler pointer is always different when calling the log message... how could that be ?
* Figured out that the cpptoc interface was nnot behaving as expected: missing function for use_shared_handle.
* Then also realized that our libcef_dll module in the target project was not updated accordingly: we need to update that with the version generated by the translator tool.
* Then in **CefCopyFrameGenerator::InternalGenerateCopyFrame()** we create a **CopyOutRequest**
* Could [[https://www.chromium.org/developers/design-documents/gpu-command-buffer|check how to add a new command]]
* Then eventually, we get back to the **gl_helper->CropScaleReadbackAndCleanMailbox()** call: This is the call that we should replace (render_widget_host_view_osr.cc line 204)
* We can create another function:
void GLHelper::CopyToSharedDXHandle(
const gpu::Mailbox& src_mailbox,
const gpu::SyncToken& sync_token,
void* sharedHandle) {
GLuint mailbox_texture = ConsumeMailboxToTexture(src_mailbox, sync_token);
gl_->NvCopyTextureToSharedHandle(mailbox_texture, sharedHandle);
gl_->DeleteTextures(1, &mailbox_texture);
}
* **Note:** it seems all texture copy operations are currently handled with the **copy_texture_to_impl_** member: could use this interface instead ? (but this is not really necessary)
* So we should implement NvCopyTextureToSharedHandle(GLuint id, void* handle) as a new GL command: Added in "src/gpu/command_buffer/cmd_buffer_functions.txt": GL_APICALL void GL_APIENTRY glNervCopyTextureToSharedHandle (GLuint texture_id, const GLvoid* handle);
* Need to be very careful on the previous function declaration otherwise it will not be found.
* Adding the function into **_FUNCTION_INFO** in "src/gpu/command_buffer/build_gles2_cmd_buffer.py": 'NervCopyTextureToSharedHandle': {
'decoder_func': 'DoNervCopyTextureToSharedHandle',
'unit_test': False,
},
* Then needs to call the generation script from the "src" folder: python gpu\command_buffer\build_gles2_cmd_buffer.py
* And we get a **NotImplementedError** when calling this script. File "gpu\command_buffer\build_gles2_cmd_buffer.py", line 9848, in WriteCmdInit
self.type_handler.WriteImmediateCmdInit(self, f)
File "gpu\command_buffer\build_gles2_cmd_buffer.py", line 5447, in WriteImmediateCmdInit
raise NotImplementedError(func.name)
NotImplementedError: NervCopyTextureToSharedHandleImmediate
* Trying to use **InitializeDiscardableTextureCHROMIUM** as template. So its working with the function info: 'NervCopyTextureToSharedHandle': {
'type': 'Custom',
'decoder_func': 'DoNervCopyTextureToSharedHandle',
'unit_test': False,
'impl_func': False,
'client_test': False,
},
* Our new function doesn't seem to be anywhere in the GLES2 Command decoder class... hmmm.
* Our function should really be in **gles2_cd_decoder.cc**, trying to add the "extension" flag in the function info => **NO** this is incorrect: we should implement the function here ourself.
* We have the ID generated for NervCopyTextureToSharedHandle in "common/gles2_cmd_ids_autogen.h"
* Now trying to build anyway... It should fail because we don't have the function **GLES2DecoderImpl::HandleNervCopyXXX()** yet.
* Obversing the following error: [679/961] LINK ui.service.exe ui.service.exe.pdb
FAILED: ui.service.exe ui.service.exe.pdb
D:/Projects/CEFBuild/depot_tools/win_tools-2_7_6_bin/python/bin/python.exe ../../build/toolchain/win/tool_wrapper.py link-wrapper environment.x64 False link.exe /nologo /OUT:./ui.service.exe /PDB:./ui.service.exe.pdb @./ui.service.exe.rsp
service_sources.lib(gles2_cmd_decoder.obj) : error LNK2019: unresolved external symbol "private: enum gpu::error::Error __cdecl gpu::gles2::GLES2DecoderImpl::HandleNervCopyTextureToSharedHandle(unsigned int,void const volatile *)" (?HandleNervCopyTextureToSharedHandle@GLES2DecoderImpl@gles2@gpu@@AEAA?AW4Error@error@3@IPEDX@Z) referenced in function "void __cdecl `dynamic initializer for 'private: static struct gpu::gles2::GLES2DecoderImpl::CommandInfo const * const gpu::gles2::GLES2DecoderImpl::command_info''(void)" (??__E?command_info@GLES2DecoderImpl@gles2@gpu@@0QBUCommandInfo@123@B@@YAXXZ)
service_sources.lib(gles2_cmd_decoder_passthrough.obj) : error LNK2019: unresolved external symbol "private: enum gpu::error::Error __cdecl gpu::gles2::GLES2DecoderPassthroughImpl::HandleNervCopyTextureToSharedHandle(unsigned int,void const volatile *)" (?HandleNervCopyTextureToSharedHandle@GLES2DecoderPassthroughImpl@gles2@gpu@@AEAA?AW4Error@error@3@IPEDX@Z) referenced in function "void __cdecl `dynamic initializer for 'private: static struct gpu::gles2::GLES2DecoderPassthroughImpl::CommandInfo const * const gpu::gles2::GLES2DecoderPassthroughImpl::command_info''(void)" (??__E?command_info@GLES2DecoderPassthroughImpl@gles2@gpu@@0QBUCommandInfo@123@B@@YAXXZ)
gles2_implementation.lib(gles2_implementation.obj) : error LNK2001: unresolved external symbol "public: virtual void __cdecl gpu::gles2::GLES2Implementation::NervCopyTextureToSharedHandle(unsigned int,void const *)" (?NervCopyTextureToSharedHandle@GLES2Implementation@gles2@gpu@@UEAAXIPEBX@Z)
./ui.service.exe : fatal error LNK1120: 3 unresolved externals
[688/961] CXX obj/third_party/WebKit/Source/bindings/modules/v8/bindings_modules_impl_0/V8VRLayer.obj
ninja: build stopped: subcommand failed.
* The **NervCopyTextureToSharedHandle** structure is defined **gles2_cmd_format_autogen.h**
* Currently the struct seems to pass the share handle into shared memory => We should avoid this by using a **uint64_t** type instead of void*: **OK**
* After adding the 3 functions below, the compilation works: error::Error GLES2DecoderImpl::HandleNervCopyTextureToSharedHandle(
uint32_t immediate_data_size,
const volatile void* cmd_data) {
const volatile gles2::cmds::NervCopyTextureToSharedHandle& c =
*static_cast(
cmd_data);
GLuint texture_id = c.texture_id;
void* handle = (void*)(c.shared_handle());
NV_LOG("GLES2DecoderImpl: received shared handle "<<(const void*)handle<<" for source texture_id: "<
error::Error GLES2DecoderPassthroughImpl::HandleNervCopyTextureToSharedHandle(
uint32_t immediate_data_size,
const volatile void* cmd_data) {
const volatile gles2::cmds::NervCopyTextureToSharedHandle& c =
*static_cast(
cmd_data);
GLuint texture_id = c.texture_id;
void* handle = (void*)(c.shared_handle());
NV_LOG("GLES2DecoderPassthroughImpl: received shared handle "<<(const void*)handle<<" for source texture_id: "<
void GLES2Implementation::NervCopyTextureToSharedHandle(
GLuint texture_id, GLuint64 shared_handle) {
helper_->NervCopyTextureToSharedHandle(texture_id, shared_handle);
}
* Now time to try to call one of these in our render to texture pipeline: updated the render_view_osr method **PrepareTextureCopyOutputResult**: DCHECK(result->HasTexture());
base::ScopedClosureRunner scoped_callback_runner(
base::Bind(&CefCopyFrameGenerator::OnCopyFrameCaptureFailure,
weak_ptr_factory_.GetWeakPtr(), damage_rect));
const gfx::Size& result_size = result->size();
DEBUG_MSG("Texture size is: "<getBounds(&bitmap_size);
if (!bitmap_ || bitmap_size.width() != result_size.width() ||
bitmap_size.height() != result_size.height()) {
// Create a new bitmap if the size has changed.
bitmap_.reset(new SkBitmap);
bitmap_->allocN32Pixels(result_size.width(), result_size.height(), true);
if (bitmap_->drawsNothing())
return;
}
#endif
content::ImageTransportFactory* factory =
content::ImageTransportFactory::GetInstance();
viz::GLHelper* gl_helper = factory->GetGLHelper();
if (!gl_helper)
return;
// We do not need the pixels below:
#if 0
uint8_t* pixels = static_cast(bitmap_->getPixels());
#endif
viz::TextureMailbox texture_mailbox;
std::unique_ptr release_callback;
result->TakeTexture(&texture_mailbox, &release_callback);
DCHECK(texture_mailbox.IsTexture());
if (!texture_mailbox.IsTexture())
return;
ignore_result(scoped_callback_runner.Release());
// We don't call CropScaleReadback method here:
#if 0
gl_helper->CropScaleReadbackAndCleanMailbox(
texture_mailbox.mailbox(), texture_mailbox.sync_token(), result_size,
gfx::Rect(result_size), result_size, pixels, kN32_SkColorType,
base::Bind(
&CefCopyFrameGenerator::CopyFromCompositingSurfaceFinishedProxy,
weak_ptr_factory_.GetWeakPtr(), base::Passed(&release_callback),
damage_rect, base::Passed(&bitmap_)),
viz::GLHelper::SCALER_QUALITY_FAST);
#endif
CefRefPtr handler = view_->browser_impl()->GetClient()->GetRenderHandler();
if(!handler.get()) {
DEBUG_MSG("NervCEF: Invalid render handler: not performing texture copy.");
OnCopyFrameCaptureCompletion(false);
return;
}
if(!handler->UseSharedHandle()) {
DEBUG_MSG("NervCEF: Not using shared handle.");
OnCopyFrameCaptureCompletion(false);
return;
}
void* handle = handler->GetSharedHandle();
if(handle == nullptr) {
DEBUG_MSG("NervCEF: ignoring null shared handle.");
OnCopyFrameCaptureCompletion(false);
return;
}
// Here we call another function to copy the texture to the shared handle:
DEBUG_MSG("NervCEF: using shared handle: "<<(const void*)handle);
gl_helper->NervCopyMailboxToSharedHandle(texture_mailbox.mailbox(), texture_mailbox.sync_token(), handle);
OnCopyFrameCaptureCompletion(false);
* **Great!**: with that change above (and implementation of the helper method in GLHelper), we get the expected outputs on the GPU service side (ie. "received shared handle xxxx for source texture_id: yyy")
* Now implementing the copy into the D3D shared handle, found this interesting resource: https://chromium.googlesource.com/chromium/src/media/+/master/gpu/dxva_picture_buffer_win.cc
* Can retrieve the EGLDisplay object, but still not clear how to retrieve the EGL config for our client pbuffer surface: **Now clarified**
* http://processors.wiki.ti.com/index.php/Render_to_Texture_with_OpenGL_ES
* We can probably render the texture on our surface with a code similar to: bool RenderToTexture::DrawTextureBox()
{
glDisable(GL_LIGHTING);
glMatrixMode(GL_PROJECTION);
glLoadIdentity();
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();
glTranslatef(-1.0f, -1.0f, 0.5f);
glScalef(2.0f, 2.0f, 1.0f);
static VERTTYPE Vertices[] =
{
0.0f, 1.0f, 0.0f,
0.0f, 0.0f, 0.0f,
1.0f, 0.0f, 0.0f,
1.0f, 1.0f, 0.0f
};
static VERTTYPE TexCoords[] =
{
0.0f, 1.0f,
0.0f, 0.0f,
1.0f, 0.0f,
1.0f, 1.0f
};
glEnableClientState(GL_VERTEX_ARRAY);
glEnableClientState(GL_TEXTURE_COORD_ARRAY);
glVertexPointer(3, VERTTYPEENUM, 0, (VERTTYPE *)&Vertices);
glTexCoordPointer(2, VERTTYPEENUM, 0, (VERTTYPE *)&TexCoords);
glBindTexture(GL_TEXTURE_2D, m_hTexture);
glDrawArrays(GL_TRIANGLE_FAN, 0, 4);
glDisableClientState(GL_VERTEX_ARRAY);
glDisableClientState(GL_TEXTURE_COORD_ARRAY);
return true;
}
* Some info on pbuffer rendering: https://mkonrad.net/2014/12/08/android-off-screen-rendering-using-egl-pixelbuffers.html
* Another information page: https://stackoverflow.com/questions/28817777/pbuffer-vs-fbo-in-egl-offscreen-rendering
* Could copy textures with the code shown on this page: https://stackoverflow.com/questions/39941052/how-to-assign-one-texture-to-another-efficiently-in-opengl-es-for-android
===== Getting the copy operation to work =====
* So we now have all the elements in place... But still we get a black image rendered in our project (ie. nothing written on this surface from the CEF service process). How could that be ?
* It could be related to the size of the texture ? **nope**
* To the binding to a texture in the sample ? **nope**
* To the context config ? **Nope**
* To the display we use in the CEF process ? **Nope**
* => I even tried to embed a test window in the CEF service process, with its own hwnd and EGLDisplay! And the result of that is that the window will not display at all until we close the software: at that time, all the commands are executed in a row.
* => So it seems we have a command buffer issue: our commands are stacked but not executed in a timely fashion... Need to clarify that.
* **Note**: When it comes to using texture, checking **GLES2DecoderImpl::DeleteTexturesHelper** it would seem that the client texture id is not the real ID and instead this is mapped to a Texture ref on the service side.
* Trying to call glFlush() in gl_helper.cc and CheckGLError() from gles2_implementation.cc but I doubt this is it. => **GREATTTT !!! This actually works!!!**
* I rather think we need to tweak the function generation => maybe using a custom handler type ? => We need to be able to generate the Immediate version of the function. => no need to go that way for now.
* Okay, stay cool. Now is a good time to make a backup of our state on the second CEF build setup.
* Continuing on our path: I tried to create a new context sharing resources with the current context to be able to access the texture data: this is failing with a **EGL_BAD_MATCH** error.
* Then trying to use the current context to do the rendering directly: this is not rendering anything on the pbuffer.
* So what's next ? What is this BAD_MATCH error ?
* **Note**: Maybe we can just call **helper_->CommandBufferHelper::Flush();** instead of a glFlush() command ?
* **Note**: The actual implementation in **GLES2DecoderImpl::DoBindTexture** is much more complex that anticipated.
* So let's consider creating our context from the decoder base context:
// We can retrieve the context with:
gl::GLContext* GetGLContext();
// And the context group with:
ContextGroup* GetContextGroup();
// From the GLContext, we can retrieve the share group:
// share_group = GLContext::share_group();
// Then with this share group we can try to create a shared context:
nvContext = eglCreateContext(egl_display, nvConfig, sgroup->GetHandle(), contextAttribs);
* Now, maybe we can already check if our decoder is using a GLContext that has a share_group ? Retrieval of the context happens in **GLES2DecoderImpl::Initialize(...)**
* So, it seems our GLES2Decoder is initialized in "gpu/ipc/service/gpu_command_buffer_stub.cc" in the **GpuCommandBufferStub::Initialize(...)** method.
* In this method, whatever we do, the context is eventually created with a call to **gl::init::CreateGLContext(...)** giving it the share group from the current GPu channel.
* **CreateGLContext** implementation is provided in "ui/gl/init/gl_factory_win.cc"
* When using offscreen rendering the surface format come from the surface created with **gl::init::CreateOffscreenGLSurface(...)**
* Then those settings are tweaked a little bit, and eventually the default context surface is created with **gl::init::CreateOffscreenGLSurfaceWithFormat(...)**
* And I'm now suspecting we are actually creating a "surfaceless" context now...
* => **Correct!** This is the kind of outputs we get from the service nv_logger system: 2017-11-16 19:11:32.913 UTC [808]{12132}: nvLogger initialized.
2017-11-16 19:11:32.913 UTC [808]{12132}: Reached: CreateOffscreenGLSurfaceWithFormat() function.
2017-11-16 19:11:32.914 UTC [808]{12132}: CreateOffscreenGLSurfaceWithFormat: Using EGLGLES2 SurfacelessEGL implementation
2017-11-16 19:11:33.042 UTC [808]{12132}: Reached: CreateOffscreenGLSurfaceWithFormat() function.
2017-11-16 19:11:33.043 UTC [808]{12132}: CreateOffscreenGLSurfaceWithFormat: Using EGLGLES2 SurfacelessEGL implementation
2017-11-16 19:11:33.043 UTC [808]{12132}: Creating GLES2DecoderImpl instance.
2017-11-16 19:11:33.043 UTC [808]{12132}: GpuCommandBufferStub: using offscreen rendering.
* So it seems this could be it. And we should try to prevent the creation of such a surfaceless context.
* => Ok, so tried to prevent using surfaceless context, and this is **completely failing** (freezing on our project start): so it doesn't seem to be a good idea. Moreother, looking at the definition of what surfaceless context are (cf. [[https://www.khronos.org/registry/EGL/extensions/KHR/EGL_KHR_surfaceless_context.txt|EGL_KHR_surfaceless_context]]) it doesn't seem to be a problem to use them and still be able to render on a pbuffer.
* Now, trying to use the current context again, but this time we open Gl error checking, and we have an error code: Stage7 (error code: 1282)
* "Stage7" is reached after execution of the command: glVertexAttribPointer(posLoc, 3, GL_FLOAT, GL_FALSE, 5 * sizeof(GLfloat), vVertices);
* Why that ? => No idea.
* Anyway, tried to remove the part where we draw the texture, and in that case we don't have any error, but we get absolutely nothing rendered on the pbuffer surface ? :-(
* => It might be an idea to try to create our context with the same attribs as those provided in **gl_context_egl.cc** ?
* So it seems that yes, we can create our context from the shared base, if we use the settings from the gl_context_egl.cc file: basically, this means our context needs to be created with a request for GLES 3.0 instead of version 2.0.
* => Let's retest the base implementation again with GL ES 2.0 and no shared base using: EGLint contextAttribs[] = {EGL_CONTEXT_CLIENT_VERSION, 2, EGL_NONE, EGL_NONE};
nvContext = eglCreateContext(egl_display, nvConfig, nullptr, contextAttribs);
* **OK**: making some progress: testing GL ES 2.0 context is OK. Also tested upgrading to GL ES 3, and this is still **OK**
* Okay so it seems we now have the same kind of issues as before when calling **glVertexAttribPointer**
* => I suspect this could be due to the fact that we might have a buffer bound already.
* According to the [[https://www.khronos.org/registry/OpenGL-Refpages/es3.0/html/glVertexAttribPointer.xhtml|khronos documentation page]]: pointer
Specifies a pointer to the first generic vertex attribute in the array. If a non-zero buffer is currently bound to the GL_ARRAY_BUFFER target, pointer specifies an offset of into the array in the data store of that buffer. The initial value is 0.
* So we can try to un bind the current: buffer with:
GLint curBuffer = 0;
glGetIntegerv(GL_ARRAY_BUFFER_BINDING, &curBuffer);
CHECK(glIsBuffer(curBuffer)==GL_TRUE);
// We bind the buffer to 0;
glBindBuffer(GL_ARRAY_BUFFER, 0);
// Here we do our rendering.
// Then we restore the bound buffer:
glBindBuffer(GL_ARRAY_BUFFER, curBuffer);
* Okay, so from the previous run we can notice an error stating that the current buffer is not a valid buffer...
* So got confirmation that the previous buffer is 0, but still, this is not working as expected => we should now try to create a buffer ourself and assign it with the data we need:
// during init:
glGenBuffersARB(1, &nvVertBuffer);
glBindBuffer(GL_ARRAY_BUFFER, nvVertBuffer);
glBufferData(GL_ARRAY_BUFFER, size_of_buffer, vVertices, GL_STATIC_DRAW);
* **OK**: so we could get ride of the erreur on **glVertexAttribPointer**, good, but now the call to **glDrawElements** is also failing...
* It seems we might need to build the ELEMENT_ARRAY_BUFFER too, with the list of indices, let's try that.
* => **Great**: finally getting some texture to get displayed! Yet for the moment, it seems only 1 pixel from our texture is used: Not sure the texture coordinates are correct... And indeed, the vertex attrib pointer call for the texture coords was most probably incorrect with: glVertexAttribPointer(1, 2, GL_FLOAT, GL_FALSE, 5 * sizeof(GLfloat), reinterpret_cast(3));
// Should rather be:
glVertexAttribPointer(1, 2, GL_FLOAT, GL_FALSE, 5 * sizeof(GLfloat), reinterpret_cast(3*sizeof(GLfloat)));
* => **Wonderfull!**: finally getting somewhere! Could now display the test texture with transparency support!!
* => Now trying to use the actual texture with overlay content... (and crossing fingers very hard...)
===== Origin of the Compositor texture =====
Currently it seems we are generating a new texture each time the compositor has a new output to provide: we need to investigate what is actually the source of this generation and thus see if we can use the root texture directly (if any ?)
* When we reach the **PrepareTextureCopyOutputResult()** function we already have a **CopyOutputResult** object, where does this come from ?
* This result object is passed to **CopyFromCompositingSurfaceHasResult** first.
* The request originate from **InternalGenerateCopyFrame**: std::unique_ptr request =
cc::CopyOutputRequest::CreateRequest(base::Bind(
&CefCopyFrameGenerator::CopyFromCompositingSurfaceHasResult,
weak_ptr_factory_.GetWeakPtr(), damage_rect));
request->set_area(gfx::Rect(view_->GetPhysicalBackingSize()));
view_->GetRootLayer()->RequestCopyOfOutput(std::move(request));
* So the CopyOutputRequest is probably just a simple structure to pass to copy parameters (callback, target size, etc), and the actually request is probably made in the call: **GetRootLayer()->RequestCopyOfOutput(...)**
* It seems the "Root layer" is a simple solid color layer (except on MACOSX ?): root_layer_.reset(new ui::Layer(ui::LAYER_SOLID_COLOR));
* **RequestCopyOfOutput(...)** will only push the request on a list.
* The list of copy requests is then retrieved from **Layer::TakeCopyRequests(...)**
===== Texture memory issues =====
* Currently investigating how the TextureRef and Texture objects are released exactly in the TextureManager class: currently it seems we never get to the point where Textures are actually released in this call:
void Texture::RemoveTextureRef(TextureRef* ref, bool have_context) {
NV_LOG2("In Texture::RemoveTextureRef for "<TrackMemFree(estimated_size());
memory_tracking_ref_ = NULL;
}
size_t result = refs_.erase(ref);
DCHECK_EQ(result, 1u);
NV_LOG2("Left texture refs size: "<TrackMemAlloc(estimated_size());
}
}
* => We always have 1 ref left when we try to release the Texture. But we have at least 2 options here:
* We can try to force delete this second ref somehow (but this would imply to modify the texture class interface, so we should probably avoid this)
* We can trace where the call to AddTextureRef occurs to know where we created this additional reference => let's try this option.
* While trying to recompile CEFfrom sources, now noticed that the code injection in libcef_dll.cc is broken: so instead, moving our function definition in **libcef_dll2.cc** (which doesn't seem to be autogenerated): #include "include/cef_nv_exports.h"
// ... (more lines here) ...
CEF_EXPORT void cef_set_log_handler(cef::CefLogHandler* handler) {
cef::setLogHandler(handler);
}
* Now also trying to trace where **AddTextureRef** is called for a given TextureRef: void Texture::AddTextureRef(TextureRef* ref) {
DCHECK(refs_.find(ref) == refs_.end());
refs_.insert(ref);
// Trace where we are:
NV_LOG2("Adding textureref " << (const void*)(ref) << " on texture with service id "<< owned_service_id_ << ": "<< base::debug::StackTrace().ToString());
if (!memory_tracking_ref_) {
memory_tracking_ref_ = ref;
GetMemTracker()->TrackMemAlloc(estimated_size());
}
}
* Trying to get the stacktrace will just kill the cef engine completely (too slow mechanism): but just trying to get some insights we have the log results: 2018-01-09 08:14:27.557 UTC [2300]{11140}: GenTexturesHelper: Creating service texture: 1009
2018-01-09 08:14:27.557 UTC [2300]{11140}: Adding textureref 0000010E093B00A0 on texture with service id 1009
2018-01-09 08:14:27.557 UTC [2300]{11140}: TextureManager: num textures: 775
2018-01-09 08:14:27.569 UTC [2300]{11140}: Adding textureref 0000010E093AF8C0 on texture with service id 1009
2018-01-09 08:14:27.569 UTC [2300]{11140}: TextureManager: num textures: 3
2018-01-09 08:14:27.569 UTC [2300]{11140}: GLES2DecoderImpl: received shared handle 00000000C0000202 for source texture_id: 679
2018-01-09 08:14:27.569 UTC [2300]{11140}: GLES2DecoderImpl: Drawing from service texture_id: 1009
2018-01-09 08:14:27.569 UTC [2300]{11140}: GLES2DecoderImpl: Releasing texture unit infos.
2018-01-09 08:14:27.569 UTC [2300]{11140}: GLES2DecoderImpl: Deleting texture with client id=679 (service_id=1009)
2018-01-09 08:14:27.569 UTC [2300]{11140}: In TextureRef destructor with force_context_lost=false, manager have_context:true
2018-01-09 08:14:27.569 UTC [2300]{11140}: In Texture::RemoveTextureRef for 1009, have_context=true
2018-01-09 08:14:27.569 UTC [2300]{11140}: Left texture refs size: 1
2018-01-09 08:14:27.569 UTC [2300]{11140}: Left refs: 0000010E093B00A0
2018-01-09 08:14:27.569 UTC [2300]{11140}: GLES2DecoderImpl: Done copying to shared handle surface.
* So just after the call to GenTexturesHelper, we add a TextureRef, then we add another one: the later is eventually removed, but not the first one: need to clarify what this is exactly.
* So we first call: std::unique_ptr service_ids(new GLuint[n]);
glGenTextures(n, service_ids.get());
for (GLsizei ii = 0; ii < n; ++ii) {
NV_LOG2("GenTexturesHelper: Creating service texture: "<
* Then we reach the **CreateTexture** function: // Creates a Texture for the given texture.
TextureRef* CreateTexture(
GLuint client_id, GLuint service_id) {
return texture_manager()->CreateTexture(client_id, service_id);
}
* Then we get into **TextureManager::CreateTexture**: TextureRef* TextureManager::CreateTexture(
GLuint client_id, GLuint service_id) {
DCHECK_NE(0u, service_id);
scoped_refptr ref(TextureRef::Create(
this, client_id, service_id));
std::pair result =
textures_.insert(std::make_pair(client_id, ref));
DCHECK(result.second);
return ref.get();
}
* => This is where we create a first **TextureRef** which is inserted into the **textures_** map.
* Then it seems we replicate the exact same process (except that the texture service id is only allocated once)
* Also, the textureManager reports 2 different values for the "num textures": as if we had 2 texture managers...
* => So maybe we should report the address of the manager to confirm that: scoped_refptr TextureRef::Create(TextureManager* manager,
GLuint client_id,
GLuint service_id) {
TextureRef* ref = new TextureRef(manager, client_id, new Texture(service_id));
NV_LOG2("Created textureref " << (const void*)(ref) << " on texture with service id "<< service_id << " for manager "<<<(const void*)manager);
return ref;
}
* But just assume that multiple managers call this **Create** function. Then we also create multiple instances of the base **Texture** object in the process ? So how can we get 2 different refs on a single Texture instance ??
* What about the **memory_tracking_ref_** member in the Texture class ?
* Or could it be due to the call to **ConsumeMailboxToTexture** on the client side ? => could be creating the texture on the mailboxmanager, and then keeping a reference on it ?
* => we might be reaching the line (in **GLES2DecoderImpl::DoCreateAndConsumeTextureINTERNAL**): texture_ref = texture_manager()->Consume(client_id, texture);
* **OK**: this seems consistent, we then reach the function: TextureRef* TextureManager::Consume(
GLuint client_id,
Texture* texture) {
DCHECK(client_id);
scoped_refptr ref(new TextureRef(this, client_id, texture));
bool result = textures_.insert(std::make_pair(client_id, ref)).second;
DCHECK(result);
return ref.get();
}
* => So in the code above, we create a new **TextureRef** based on an **existing Texture** => 2 refs on that texture object.
* So from that perspective, our texture is generated with the line: Texture* texture =
static_cast(group_->mailbox_manager()->ConsumeTexture(mailbox));
* Got those outputs for the texture manager display: 2018-01-09 12:28:27.570 UTC [5012]{220}: Deleting service texture: 958
2018-01-09 12:28:27.570 UTC [5012]{220}: Adding textureref 0000023DE5FB2950 on texture 0000023DE2DB7E30 with service id 265
2018-01-09 12:28:27.570 UTC [5012]{220}: TextureManager 0000023DD68C8D60: num textures: 777
2018-01-09 12:28:27.570 UTC [5012]{220}: GenTexturesHelper: Creating service texture: 958
2018-01-09 12:28:27.570 UTC [5012]{220}: Adding textureref 0000023DE5FB2470 on texture 0000023DE2F97720 with service id 958
2018-01-09 12:28:27.570 UTC [5012]{220}: TextureManager 0000023DD68C8D60: num textures: 778
2018-01-09 12:28:27.570 UTC [5012]{220}: Created textureref 0000023DE5FB2470 on texture with service id 958 for manager 0000023DD68C8D60
2018-01-09 12:28:27.585 UTC [5012]{220}: Adding textureref 0000023DE5FB29B0 on texture 0000023DE2F97720 with service id 958
2018-01-09 12:28:27.585 UTC [5012]{220}: TextureManager 0000023DE2DF9100: num textures: 3
2018-01-09 12:28:27.585 UTC [5012]{220}: GLES2DecoderImpl: received shared handle 0000000040001342 for source texture_id: 649
2018-01-09 12:28:27.585 UTC [5012]{220}: GLES2DecoderImpl: Drawing from service texture_id: 958
2018-01-09 12:28:27.585 UTC [5012]{220}: GLES2DecoderImpl: Releasing texture unit infos.
2018-01-09 12:28:27.585 UTC [5012]{220}: GLES2DecoderImpl: Deleting texture with client id=649 (service_id=958)
2018-01-09 12:28:27.585 UTC [5012]{220}: Deleting TextureRef with force_context_lost=false, manager have_context:true for manager 0000023DE2DF9100
2018-01-09 12:28:27.585 UTC [5012]{220}: Removing TextureRef 0000023DE5FB29B0 from texture 0000023DE2F97720 with service id 958, have_context=true
2018-01-09 12:28:27.585 UTC [5012]{220}: Left texture refs size: 1
2018-01-09 12:28:27.586 UTC [5012]{220}: Left refs: 0000023DE5FB2470
2018-01-09 12:28:27.586 UTC [5012]{220}: GLES2DecoderImpl: Done copying to shared handle surface.
* So, this is all getting pretty messing with texture references/manager/etc. And thinking about it, we **don't need** any of these: all we really need is an access to the service_id encapsulated into the mailbox... So we could rather jsut pass the mailbox directly!
* **OK**: now using the mailbox directly and got ride of the memory leak issue.
===== TODO =====
* Copy all the files containing new references on the **NervCopyTextureToSharedHandle** key word (?)
===== References =====
* [[https://bitbucket.org/chromiumembedded/cef/issues/1006/cef3-support-rendering-to-a-hardware-gl|Original issue for accelerated Off-screen rendering in CEF]]
* [[http://dev.chromium.org/developers|Resources for developers on Chromium]]
* [[http://dev.chromium.org/developers/design-documents|Chromium Design documents]]
* [[https://www.chromium.org/developers/design-documents/gpu-command-buffer|Command buffer in Chromium]]
* https://bitbucket.org/chromiumembedded/cef/issues/2046/improve-gpu-readback-performance-for-osr
~~DISCUSSION:off~~
/*
https://bitbucket.org/chromiumembedded/cef/issues/1006/cef3-support-rendering-to-a-hardware-gl
Hello everyone,
I've been watching for this issue since quite a long time now, and I was hoping someone else would find the time to handle it and save me the trouble... But well, I guess that's not how life works ;-).
In my company, we have been using CEF quite intensively since more than 1 year now, and we reached a point where we really needed to squeeze out more performances out of this offscreen rendering mechanism.
So a couple of months ago I decided I should stop waiting, try this myself and find a working solution (at least for us...). And now, I'm glad to annonce that, I finally made it (it was incredibly complex sure, but that's OK, because I learned a lot in the process, and now...): I have a working mechanism to render on a DirectX surface directly on the GPU (ie. without first copying the CEF rendered textures on a Skia bitmap on the CPU).
Note that this work is based on the CEF branch 3163, which is not that old, so I think it should not be too hard to integrate it in the current version of CEF (if applicable ?).
Of course, there are some limitations: the system I'm using is only designed for usage on Windows (only tested with Windows 10 so far, and I'm most probably missing some platform dependent constrains and checks), and only allow direct copy on DirectX surfaces (Tested with DirectX 9 so far as this is what we need, but I'm pretty sure it would work the same with DirectX 10 or 11... no idea about DX 12). So for those interested in a linux support, this is not good enough yet (but might maybe be used as an inspiration source ?).
For those interested in copying on a GL surface on Windows, then this might still help: because one can use the DirectX surface "layer" to bridge the gap between a given project/software process and the CEF GPU service process (in fact I'm not sure this could be done another way with GLES/OpenGL [except if you run the GPU service directly in the browser process ?]). And then maybe more easily share the DirectX surface with an OpenGL context from a single process (ie. your software process) ?
Basically, the idea I'm using here has been around for a while, but as far as I know no one actually tried to implement it (?). It is simply using the fact that CEF is using ANGLE on Windows to convert the GLES layer to a DirectX layer. And ANGLE supports interop with DirectX out of the box (cf. for instance https://github.com/Microsoft/angle/wiki/Interop-with-other-DirectX-code). So from there:
- In my software process I create a shared DirectX 9 surface,
- Then I updated the CEF RenderHandler class with additional methods to be able to specify if the render handler should use a provided shared DirectX surface handle instead of trying to do the rendering with the "regular path".
- if a shared handle is provided to the RenderHandler, then I updated the CefCopyFrameGenerator behavior (used in the CefRenderWidgetHostViewOSR) to simply bypass the "regular path" in that case (ie. will not try to create a SkBitmap and then make a query on the service process to read the pixels from the compositor output surface, OnPaint will never get called, etc), and instead:
- I created a new "GL like" command that the CefCopyFrameGenerator object will call (from the GLHelper) to request a copy of the compositor texture onto the provided shared handle.
- Then this goes on the Command Buffer, and reaches the GLES2Decoder implementation:
- From there I perform a one time init operation, where I create a new EGL context (sharing resources with the regular decoder context), I init a Pbuffer from the received shared handle, and prepare some additional GL ES resources (shader program, vertex buffer, etc)
- Then each time this function is called, I take the mapping of the client texture id to get the service_id, use my own context and setup all my resources to render a screen aligned quad on my pbuffer using the service texture id.
-> And that's it! No need for additional synchronization, no need to send anything back to the client from the service process, the DirectX surface will continuously get the updated compositor surface with a simple quad rendering, and we don't have to copy anything on the CPU memory! :-)
Hmmm, and now that I think about, there are also other minor limitations I should mention:
* Currently, the size of the DirectX surface/pbuffer used in the GPU service process is hardcoded to 1920x1080 => we would probably need to pass this information from the client for proper init of the pbuffer.
* And also, I'm not bothering with releasing the EGL resources I create in my init call, ooops ;-): we might need an additional custom command to do that... but that's currently not a priority for me.
* And in fact, I completely disabled the regular Skia Bitmap copying path in my code.
Anyway, I'm planning to spend a few more days cleaning/validating the current code, and then I will write an article about this work and post some initial version of the updated files here so that the community can have a look and see if this can be of any help to you. For now, I just wanted to post a "little" teaser to get you excited (hopefully) ;-)!
*/