Rendering Video output into a webGL texture

Vega Open Beta Q&A
1

Hello, wondering if/when support for rendering Video output on device into a webGL texture is planned.

Thanks.

4

Hi @Jason_Aeschliman,

Thanks for reaching out. Currently, rendering video output into a WebGL texture is not supported on Vega. Vega plays video directly using private GPU buffers, which prevents extracting video frames as WebGL textures.

The recommended approach is to use the HTML element or other web-based video player libraries for video playback.

For more details, see the “Avoid canvas video” section in our WebGL Best Practices documentation: WebGL Best Practices for Web Apps | Design and Develop Vega Apps

Warm regards,
Aishwarya

5

Thanks @amen ,

I have seen that document (which is for web apps). Is there similar guidance for non-web-based apps?

Thanks

6

Hi @Jason_Aeschliman,

For React Native for Vega apps, video playback renders to a dedicated surface (KeplerVideoSurfaceView) and there is currently no API to extract video frames into a texture.

For reference on the video playback architecture: VideoPlayer | Vega API

Warm regards,
Aishwarya

7

Thanks @amen !

Yes, I am aware that there is currently no way to do it - my question is: are there plans to add support for this? If so, is there a timeline for it?

Thanks

8

Hi @Jason_Aeschliman,

Thank you for clarifying. We understand you’re asking about future plans to support rendering video output into a texture (for React Native for Vega apps).

We have noted this as a feature request. At this time, the feedback has been passed to our engineering team for evaluation.

You will be hearing via announcement if there are any updates on this.

Warm regards,
Aishwarya