Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support WebGL2 #142

Open
tangmi opened this issue May 10, 2016 · 15 comments
Open

Support WebGL2 #142

tangmi opened this issue May 10, 2016 · 15 comments

Comments

@tangmi
Copy link

tangmi commented May 10, 2016

Currently embed.js decides to actually enable the UI only if the value passed to HTMLCanvasElement.getContext() is one of: ["moz-webgl", "webkit-3d", "experimental-webgl", "webgl"].

WebGL 2 can be instantiated with the string webgl2 (and, less commonly, experimental-webgl2). It's possible to get WebGL-Inspector to enable its UI by adding webgl2 to the list in embed.js (which allows one to inspect state, textures, etc), but I wonder if there's any additional work that needs to be done to actually support it?

Thanks!

Edit: I played around with it a bit more. Although simply having the UI to inspect what it can is immensely more useful than not, here's the (very incomplete) list of things I've noticed are missing:

  • The inspector is unaware of all new consts, like the new internal formats, uniform buffer stuff, etc. and displays them as something similar to ?? 0x8a11 ?? as opposed to UNIFORM_BUFFER (The WebGL developer tools's glEnumToString might be a future-proof way to solve this issue)
  • The texture display doesn't show meaningful output for the new sized internal formats (scroll down to "Table 2") that are not color renderable.
    • Since we can attach textures as the depth/stencil target(s) for a framebuffer, we can have depth and stencil textures like this
  • The "Buffers" tab doesn't seem to handle buffers related to draw calls using uniform buffers well.
@greggman
Copy link
Contributor

greggman commented Dec 26, 2016

I've started adding WebGL2 support

We'll see how far I get 😅

Here's a short list of things off the top of my head that need to be done

  • Update shared/info.js with all the new API functions

  • Handle samplers

  • Handle queries

  • Handle transform feedback

  • Handle sync objects

  • Handle vertex arrays

  • Handle all the new texture formats

  • Handle all new texture functions

    • texStorage2D
    • texStorage3D
    • texImage2D
      • TexSource + width + height
      • ArrayBuffer + offset
      • pbo
    • texSubImage2D
      • TexSource + width + height
      • ArrayBuffer + offset
      • pbo
    • compressedTexImage3D
      • ArrayBuffer
      • TexSource
      • TexSource + width + height
      • ArrayBuffer + offset
      • pbo
    • compressedTexSubImage3D
      • ArrayBuffer
      • TexSource
      • TexSource + width + height
      • ArrayBuffer + offset
      • pbo
    • texImage3D
      • ArrayBuffer
      • TexSource
      • TexSource + width + height
      • ArrayBuffer + offset
      • pbo
    • texSubImage3D
      • ArrayBuffer
      • TexSource
      • TexSource + width + height
      • ArrayBuffer + offset
      • pbo
  • Handle int attributes

  • Handle uniform block objects

  • Handle new pixel store parameters

  • Handle all the new state

  • Update replay/RedundancyChecker for the new state

  • Update shared/utilties.js isWebGLResource

  • Figure out what to do about share/utilities.js getWebGLContext

    I think it needs to get a webgl2 context if debugging webgl2 but a webgl1 context if debugging
    webgl1. There's no context passed in. A quick hack would be to set some global on
    gli.util like gl.util.useWebGL2 = true and then check that.

@greggman
Copy link
Contributor

So here's the first attempt at minimal WebGL2 support.

master...greggman:webgl2

I tried adding VertexArrayObject support. I'm kind of curious if I'm doing something wrong given there was code checked in for WebGL1 VAOs but as far as I can tell that code could not have possibly worked?

In any case this seems to work for WebGL2 VertexArrayObjects.

My impression, Ben, is that you're no longer interested in maintaining this but if you have a few moments it would be great if you could tell me if I'm doing it right. It's not clear to me when I should be calling someResource.markedDirty(true/false). It's also not clear how much state I should track and how I should track it. Basically though this seems to be working at the moment.

Any tips/hints about the code would be really appreciated as I'm flying blind

Thanks!

@benvanik
Copy link
Owner

benvanik commented Jan 3, 2017

Awesome stuff gregg! Will take a look at it today!

@greggman
Copy link
Contributor

greggman commented Jan 3, 2017

So since that time I found one bug which was the way I was adding the ELEMENT_ARRAY_BUFFER to the extras of a the ResourceVersion was wrong. That's fixed.

I was planning to add sampler support this week maybe. In order to add these features I need to write samples that actually use them 😅

So I made a samplers example and an example that uses most of the webgl2 texture formats.

One thing that wasn't clear to me is there's some code that tries to render pixels to 2d canvas. What's that for? Because it seems like the code that displays a texture in the TexturePreview ends up using WebGL to draw the texture. Basically I need an example of when the canvas 2d previews are used so I can make sure they work as I make changes so if you have any tips there that would be great.

@benvanik
Copy link
Owner

benvanik commented Jan 3, 2017

RE 2D canvas: do you mean this?

TexturePreviewGenerator.prototype.capture = function (doc) {

This is used when building the grid of texture thumbnails used by 'browse all' under the textures tab. This uses the same shared GL context with the texture data to then render off each texture into its own simple 2D canvas that's added to the DOM, one per texture. The alternative is to setup a framebuffer, blit the texture to it, readPixels it back, then putImageData that into a canvas, which is much less fun :) Really wish there was a getTexImageData like getBufferData in WebGL2!

@benvanik
Copy link
Owner

benvanik commented Jan 3, 2017

I can't comment on your diff (as it's not a pull and github won't let me comment on the comparison bahhh), but it looks really good!

One note is that spread (...) is used. Currently the code is ES5, mainly to support pre-edge IE. As the inspector is not compiled with jscompiler this means the browsers themselves must support any of the JS features we use. We could add jscompiler on at the end just to do es6-to-es5, but that's a bigger change than I think you'd want to take on, unless you really like ES6 :)

Otherwise, I think the code looks great and I'm really impressed with you being able to tolerate and adapt to my ancient coding style :)

@benvanik
Copy link
Owner

benvanik commented Jan 3, 2017

RE markDirty: you should call it any time a property of the object or the contents of the object changes. The flag denotes whether the operation is resetting the object contents or just mutating it. For example, setting a texParam would trigger markDirty(false /* reset */) as the texture contents are still valid. texImage2D would do markDirty(true) as it overwrites the contents, but texSubImage2D would do false as it does not.

This is used in support of the caching mechanism. Each tracked resource keeps different versions of itself to allow faster seeking within frames, and this is used to know when that version changes. See captureVersion/restoreVersion/etc around here:

Resource.prototype.markDirty = function (reset) {

In the Resource subclass most operations will call markDirty first (to bump the version), then operation on the currentVersion, like here:

tracked.markDirty(false);

currentVersion.setParameters stores the new parameters on that version, so that each version has its state at exactly the time it was used. The non-currentVersion parameters (this.parameters, etc) is the 'head' state, that the UI uses to display the current state (even outside of captured frames) like in the Textures or Programs tab. I wish I would have not had that, but I think the version stuff was added afterward.

currentVersion.pushCall is used to build up a list of calls that need to happen to reconstruct the object. or textures this is uploading the content, for things like programs it's the attach/link/etc (

tracked.currentVersion.pushCall("attachShader", arguments);
).

setExtraParameters is used to stash additional data that can be used when trying to describe the object with more than what GL queries lets you get. For example, programs track the exact attribute and uniform binding information as it can be non-deterministic during replay:

tracked.currentVersion.setExtraParameters("extra", { infoLog: tracked.infoLog });

Mirroring, when you see that, can be thought of kind of like forking. It's used in specific places to create alternate versions of the resource to use when we don't want to muck with the original. It's used by things like pixel history (where things are drawn with different shaders to do draw testing) and the redundancy checker.

HTH a bit, and please ask any more questions!

@greggman
Copy link
Contributor

greggman commented Jan 4, 2017

Hey Ben, Thank you for all the explanations.

Do you want me to make a PR or do you want me to fork and release a different version. I'm happy to do either.

If you want a PR would you prefer I don't use ES6 or would you mind if I run webpack/babel to generate ES5? My thinking was basically no one that's doing WebGL work cares about IE but it's pretty trivial to run the code through webpack/babel to generate ES5. It also wouldn't be too much work to stick to ES5 stuff.

@benvanik
Copy link
Owner

benvanik commented Jan 4, 2017 via email

@greggman
Copy link
Contributor

greggman commented Jan 7, 2017

Question about this code in TextureView

    if (pixelStoreState["UNPACK _COLORSPACE_CONVERSION_WEBGL"] !== gl.BROWSER_DEFAULT_WEBGL) {
        console.log("unsupported: UNPACK_COLORSPACE_CONVERSION_WEBGL != BROWSER_DEFAULT_WEBGL");
    }
    if (pixelStoreState["UNPACK_FLIP_Y_WEBGL"]) {
        console.log("unsupported: UNPACK_FLIP_Y_WEBGL = true");
    }
    if (pixelStoreState["UNPACK_PREMULTIPLY_ALPHA_WEBGL"]) {
        console.log("unsupported: UNPACK_PREMULTIPLY_ALPHA_WEBGL = true");
    }

Assuming this is just for viewing the texture why do we care about these settings? Is it because the texture would display wrong? It seems like displaying the texture at all would be better than nothing. Of course I can't emulate UNPACK_COLORSPACE_CONVERSION_WEBGL settings, I can do the other 2 pretty easily. I'm just wondering though why exit on UNPACK_COLORSPACE_CONVERSION_WEBGL != BROWSER_DEFAULT_WEBGL. Will something break?

@benvanik
Copy link
Owner

Yeah, it will display wrong. I don't believe it exits out when they aren't supported - it just logs, right?

It'd be nice to flag in the UI when a texture may not be valid (in case the user is trying to inspect color values and wondering why they differ), but I never got around to that - just the console log :)

@dcunited001
Copy link

I'm working on some social physics stuff. I'm simulating a cross between a cocktail party and conway's game of life using particles acting on a field that is rendered on a texture. Or at least that's the idea.

i just switched to WebGL2 because i need some features, but i also need the inspector. How do you enable this plugin to work with WebGL2?

@greggman
Copy link
Contributor

greggman commented Apr 20, 2017

What features do you need?

I'm not sure which stuff is merged here but there hasn't been a ton of WebGL2 work. My lastest version is over here. All that's been added is support for VAOs and WebGL2 texture formats. No support for anything else yet. I stopped when it appeared it would probably take 2-3 months of 8hr a day 5 days a week work to finish. (or at least that's how it seemed to me)

To use it you need to download it, build it, install the plugin, and pray 😅

@dcunited001
Copy link

dcunited001 commented Apr 29, 2017

@greggman for what i'm working on, it'd really help to be able to see the content of textures in a frame. if i can't do that, i'm not sure i can complete what i'm working on. I'm looking to work on some projects so I can get a job that doesn't suck and get out of Roanoke, VA, the land where no one has heard of ES6 and no one cares about learning new programming languages

here's the javascript and the shaders

@dcunited001
Copy link

so i'd love to help on working on a version of this for WebGL2, but I'd need some direction to contribute.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants