Jump to content

DASH video into Pixi.js


Recommended Posts

Hey folks.


So, this is probably insanely specific, but I'd love to find out if this is possible.


I'm building a web application that edits multiple streams of video, using Pixi.js as my route into WebGL, as I'll be doing lots of image manipulation and I want to use shaders and all that jazz (plus the workflow is lovely).


I've been using raw MP4's, streaming tons of them and grabbing imagery and everything - all great.


However: what I actually want to do is use DASH video (https://github.com/Dash-Industry-Forum/dash.js/wiki), as this will be used for editing a live stream in sections. DASH is essentially a wrapper for MP4 segments, using HTTP as delivery and adapting it's bitrate/quality/whatever to suit conditions.


Obviously, DASH doesn't seamlessley pull into a Pixi video texture as easily as a raw MP4 does. It uses a standard <video> element, but the src is actually just a manifest file, which is then used to serve small chunks of video.


My question then, is: is there any way I can easily pipe video footage from a <video> tag directly into a video texture in Pixi?


It's worth noting that this doesn't have to be pretty or performant, it just has to work - it's being used on the latest bleeding-edge Chrome and nothing else, on a beefy machine.


Can this be done?


Any and all assistance appreciated. :)



Link to comment
Share on other sites

> is there any way I can easily pipe video footage from a <video> tag directly into a video texture in Pixi?

Have you tried using a VideoBaseTexture?

var videoTexture = Texture.fromVideoUrl('..url to video...');// Or if you have a video element already:var videoTexture = Texture.fromVideo(videoElement);// then use it in a spritevar sprite = new Sprite(videoTexture); 
Link to comment
Share on other sites



Yeah, it doesn't work - with a normal MP4 is does, but Dash is a weird one. In the <video> element you link to a manifest file instead of a video, and Dash then pulls down tiny chunks of MP4 - this is how it stays adaptive, so it can drop the quality without skipping for example. The reason we're using it is that the tool is designed to scrub seamelessley back in time through live video, which Dash does perfectly.


I can't query the MP4 itself as it's always changing, all I want to do is grab the current footage and have it accessible as a texture in Pixi - even if it's a hacky route, or terrible performance-wise, it doesn't really matter as long as it works!


My other pondering was to be REALLY hacky and just continually modify the size and position of the <video> element itself with CSS, but taking this route means I can't do some of the more clever stuff I wanted to do with Pixi.


Any and all ideas appreciated!



Link to comment
Share on other sites

Neither do I - well, not until I needed to get this working! It's a neat format, I've just no idea how to pipe it into Pixi.


Cheers anyway.


Anyone? :)


Using the video element directly (and not the url) doesn't work either? If that is the case the dash video can't be drawn to webgl context or canvas at all, which seems wrong...

Link to comment
Share on other sites

Unfortunately not - the Dash embed looks like this:


<source id="dashvid" src="http://path.to.my.thing/manifest.mpd" type="application/dash+xml"/>


The dash.js lib then uses that manifest to swap out chunks of MP4. Unfortunately I can't query for these directly as they're obfuscated behind a TON of crazy logic that's constantly swapping chunks out so it's all seamless.


What I'm hoping to do is basically not both trying that, and instead figure out some way of grabbing the imagery from the running video - like, embedding the video into the page and then somehow doing a capture of it. No idea if it's possible though. Tearing my hair out!

Link to comment
Share on other sites

Or, can I target the whole <video> element? I was using the path... I wonder if that would work?


I can see a Texture.fromCanvas call, is this how you do it? I guess I could draw the video into a canvas (like this: http://html5doctor.com/video-canvas-magic/) and then pull that into a WebGL video texture...


Or is there a smarter way to do this?

Link to comment
Share on other sites

I meant the <video> element:

<video id="my-video">  <source src="manifest.mpd">  <source src="movie.webm">  Your browser does not support the video tag.</video> 
var videoTexture = Texture.fromVideo(document.getElementById('my-video'));var sprite = new Sprite(videoTexture);
.fromVideo() takes an actual video element, .fromVideoUrl takes a url to load the video from. Try just passing in the actual video element. Source swapping shouldn't matter then..
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Recently Browsing   0 members

    • No registered users viewing this page.
  • Create New...