2

I have a h.264 video stream that I am passing from the server (node.js) to the client (html, vanilla js) through a WebSocket. I have the raw h.264 console logging in the browser. I want to send that video stream to an html <video> tag and have it display the video there in real time.

Server Side:

var wssV = new Server({ port: 3001 });
wssV.on("connection", function (ws) {
    var videoPORT = 11111;//port Tello uses for video
    var videoServer = dgram.createSocket('udp4');

    videoServer.on('listening', function () {
        var address = videoServer.address();
    });

    videoServer.on('message', function (message, remote) {
        ws.send( message );//raw h.264 video stream sent to front end
    });
    videoServer.bind(videoPORT);
});

Client Side:

$(document).ready( function(){ 
    var videoWebSocket = new WebSocket("ws://localhost:3001");
    videoWebSocket.onerror = function(evt){console.log("ERROR:" + evt.data);};
    videoWebSocket.onopen = function(evt) { console.log("Video Connection open ..."); };

    videoWebSocket.onmessage = function(evt) { 
        console.log( evt.data );//raw h.264 encoded video stream 

        //HOW TO SEND THIS evt.data TO MY html video tag
    };

    videoWebSocket.onclose = function(evt) { console.log("Video Connection closed."); };
});

html:

<video id="videoPlayer">
     <source src="" type="video/mp4" codec="accl.42E01E">
</video>

Questions:

  1. How do I attach the incoming video stream to the <video> tag?
  2. What should the src and codec attribute values be set to?
  3. Can I stream h.264 directly by setting a codec attribute?

If I am totally approaching this problem wrong then please help me find a way to display the h.264 video stream in the browser.

GSerg
  • 76,472
  • 17
  • 159
  • 346
Ricky
  • 259
  • 1
  • 9
  • 2
    Why are you using Web Sockets for this? Why not just mux that video into an MP4 container server-side and stream it as HTTP? Then you can just do ` – Brad May 30 '19 at 22:40
  • I do not need bidirectional communication. Can you enumerate on your instructions a little about how I could mux and stream as HTTP. – Ricky May 31 '19 at 14:04
  • For your server, on HTTP request for whatever path you want, execute a child process of FFmpeg to make your MP4 (don't forget `-movflags faststart`), set the FFmpeg out file to `-`, and then take the STDOUT stream from FFmpeg and pipe it as your HTTP response to the browser. – Brad May 31 '19 at 14:40
  • I have spent some time and don't know how to mux a raw h.264 stream into a MP4 container. – Ricky May 31 '19 at 14:41
  • You could try something like this... `ffmpeg -fflags +genpts -r 60 -i your-source-video.h264 -vcodec copy -` – Brad May 31 '19 at 14:42
  • Thanks I am getting closer. I have the HTTP server serving a local MP4 file to the client so now I am trying to use ```ffmpeg``` to convert the h.264 to MP4 and then I will just stream that instead of the local file. How does the above ```ffmpeg``` command you gave me specify that I want the output in MP4 format? Does the ```-``` for file output mean you want to stream rather than write a file? – Ricky May 31 '19 at 18:13
  • You'll need `-f mp4`, and probably some other flags as well. Yes, `-` means output to STDOUT. – Brad May 31 '19 at 18:25
  • ```ffmpeg -fflags +genpts -r 25 -i ${message} -vcodec -f mp4 copy -``` this is what I am running where ${message} is the input h.264 stream (according to the code above). It is not throwing any error but I am not getting any output when I console log it. – Ricky May 31 '19 at 18:46
  • ``` videoSocket.on('message', function (message, remote) { //raw video encoded with codec h.264 //console.log(message); var child_converter = exec(`ffmpeg -fflags +genpts -r 25 -i ${message} -vcodec -f mp4 copy -`);//convert h.264 to MP4 container child_converter.stdout.on('data', (data) => { console.log(data); });``` – Ricky May 31 '19 at 18:46
  • Don't use `exec()`, use `spawn()`. – Brad May 31 '19 at 18:48
  • Okay I am using spawn ```var child_converter = spawn(`ffmpeg -fflags +genpts -r 25 -i ${message} -vcodec -f mp4 copy -`);``` but it throws an error: "Error: spawn ffmpeg -fflags +genpts -r 25 -i". Are you sure the ```${message}``` part is okay as an input? I am a little mesmerized/confused by all the options of the ```ffmpeg```. Any suggestions? – Ricky May 31 '19 at 19:30
  • You're using `spawn()` wrong. `spawn('ffmpeg', ['-fflags', '+genpts', '-r', '25'...` – Brad May 31 '19 at 19:46
  • Like this? Still a little confused but thankful for your expertise. ```var child_converter = spawn('ffmpeg', ['-fflags', '+genpts', '-r', '25'] `-i ${message}` ['-vcodec', '-f', 'mp4'] 'copy -']`)``` – Ricky May 31 '19 at 19:56
  • Okay I am trying it like this and not getting any errors but not sure it is console logging anything. It opens another tab in my bash terminal called "ffmpeg". Not sure if any of that is important. var child_converter = spawn('ffmpeg', ['-fflags', '+genpts', '-r', '25', '-i', `${message}`, '-codec:v', 'kubx264', '-f', 'mp4', 'copy', '-']); – Ricky May 31 '19 at 20:47
  • So for file streaming i use something like this ```var readStream = fs.createReadStream('./assets/earth.mp4'); readStream.on('open', function () { readStream.pipe(res);//pipe read steam to response });``` But how will I stream the ffmpeg stdout? Can I use the `createReadStream()` again or something else. Thanks for the help this stuff is hard to find on google. – Ricky May 31 '19 at 20:48
  • 1
    You should post a new question at this point, as your new question (piping child process output to HTTP client) doesn't have much to do with your original question. – Brad May 31 '19 at 21:22
  • Hey @Brad can you look at my new question regarding this same issue. You seem to be the only commenter that knows how to solve this. https://stackoverflow.com/questions/56407245/how-do-i-convert-an-h-264-stream-to-mp4-using-ffmpeg-and-pipe-the-result-to-the – Ricky Jun 03 '19 at 13:54

0 Answers0