Stream the Raspberry Pi Camera Module from Node.js to an img Tag in the Browser, Part 1 - The Server (Updated)

Update 1
Update 2

Preface

So, I know this is a pretty specific use case, but it was something I really wanted to be able to do for an open source project I'm working on. The VLC Web Plugin used to be able to handle receiving video from the VLC program on the Raspberry Pi, but Google decided that the VLC Web Plugin was insecure and barred it from appearing in Chrome any longer (and I expect other browser developers to do the same). Out of necessity, I came up with a different solution. I thought I would separate this topic out into it's own post because my project is pretty far from being done and I thought this could be useful to someone now without having to wait.

I'll admit, I'm not an expert programmer, so I do apologize if the code isn't as efficient as it could be. And I might add that I'll most likely be updating the code in this post as I write newer versions in my project.

The general idea behind the setup is that we want to take an MJPEG stream from the Raspberry Pi's camera module (not a USB camera, these were the only tutorials I could find) and place it in an <img> tag in a modern browser. In this tutorial, we will utilize HTML and JavaScript on the client end to render the output, node.js on the backend to act as our server on the R-Pi, and Python on the R-Pi to get a reliable video stream from the camera module and spit it out in MJPEG. Python will be run as a subprocess inside node.js and will output binary JPEG image chunks (yikes!) to STDOUT that we will have to take in with STDIN in node.js and filter, assemble, then stitch together, and finally serve via mjpeg-server (no, not MJPG-streamer).

I am assuming you are using the latest Raspbian distro and you have already updated your R-Pi. One final note; I'm using the Raspberry Pi 2. This will probably work on the first R-Pi but might take up much CPU power. On my R-Pi 2, monitoring it with top on the command line, it was averaging 25% CPU and RAM usage. Not bad, but I'd like to get it down a bit in future iterations if possible.

Let's Get Started

If you have not already, please install node.js on your R-Pi. In this tutorial, the program will be developed on a Windows desktop and transferred onto the R-Pi via WinSCP. Make sure to install node.js and NPM on your desktop as well. How to install and use the programs is out of the scope of this tutorial, but there's plenty of tutorials out there on the web!

Next, we will create the file structure that will contain our project. Create a top level directory somewhere (I placed mine in My Documents) and give it the name server. Create a new JavaScript file in that folder you just created called server.js which will be our node.js server and a JSON file called package.json which will describe our server program and will allow us to install packages via NPM.

package.json

This is the code for our package file. There's nothing too fancy here, we're just telling NPM that we want to get mjpeg-server. Once you have that saved, open your cmd.exe or equivalent interface so we can run NPM. Navigate to your directory:

cd ./server/

And then run NPM:

npm install

Quick and painless. If you're running into problems make sure your package.json file is valid JSON and ensure your file structure look like this so far:

• server
• server.js
• package.json

NPM will create a new directory under server that will have the name node_modules which will contain your newly installed mjpeg-server.

server.js (Update 2: I thought this was working but after more testing it's causing memory leaks, I'll have the old version posted below.)

server.js (Old single client version.)

This is our server code. It is fairly simplistic on purpose; we're just trying to get across the basic idea of how to get the setup going in the least amount of code. It is documented fairly well, but let's go over it a little more.

We start by importing our HTTP and mjpeg-server objects. Then, we create a basic HTTP server that listens on port 8080 (you can choose a different port if you wish, just make sure to change this in the HTML later on in this tutorial). Inside this request, we create our mjpeg-server object and pass it the req and res variables so it knows what to do with our response. Next, we are inspecting the URL's query string, it should contain a variable called stream, which will either contain the value start or stop. start will be sent to the server when the image loads in the page, and stop will be sent on every refresh or when the user navigates away from the page. This is to ensure that our video stream is started and shut down correctly.

If our query string tells us to start the video stream, then do so by starting a Python script in a subprocess and begin listening to it for data (this will only happen for the first client to connect to a vacant stream, every subsequent client will get served the stream that's currently in progress). As mentioned earlier, the Python module that we are using does not output a complete JPEG through STDOUT so we have to perform some witchcraft to make a complete JPEG image.

After much troubleshooting and playing around with the code, I found that all valid JPEG images coming from our Python script will have the same JPEG header, so knowing that, we just have to make sure just a few of these bytes lines up with our predetermined header. I started by verifying just one byte, but found that incomplete frames were still getting through, so I opted to verify the first two bytes. I'm not sure if verifying anymore than that is necessary; I've had my stream running for over half an hour and there's been no more "skips" in the image in the browser and it no longer "crashes" the image after so many incomplete frames.

In the incoming data loop, we have three cases. The first case is we are in the very first loop, which will always be the start of a JPEG. The next case is we are at the start of a new JPEG, which signifies that we are ready to send the old JPEG out to our client(s) and start on the new one. The last case is that we are receiving more data describing our current frame, so add that to the buffer array. We are not done there though, because there is a sub-case for the second case, and that is that we could be receiving a partial frame, in which case we discard it and start on a new frame. In that sub-case we also send the last good frame we cached.

Finally, if our query string tells us to stop the camera, we check if this is the last client to be connected. If it is (and if this client telling us to stop is in our client list), we close out our subprocess, reset our python variable, and reset our client list and request handlers. If this is not the last client that is disconnecting, we simply remove them from our client list and request handlers.

camera.py

This script is also not too complex. We start by making our imports, then setting an alias for camera. The PiCamera Python module is included with Raspbian (however, you may need to install it for Python 3 specifically). Once we set our resolution and framerate, we're ready to start the preview and wait two seconds for the camera module to warm up. Then we start recording to sys.stdout.buffer as an MJPEG, which dumps JPEG chunks into STDOUT as binary. We then use the wait_recording() method to keep the camera running for a long time (basically, we want it to run indefinitely until we stop it). Finally, we either stop the camera naturally or when the script is killed by the parent node.js script.

Once your are done creating the server files be sure to copy them over to your R-Pi!

Caveats

I'm not positive about this, but I do not think this server can handle multiple clients as it stands. I'll have to run some tests and see for certain. UPDATE: The server has been updated to handle multiple clients (however, I am still having to work out a few small kinks, mostly a way to handle the next problem mentioned below).

The PiCamera Python module can not output an MJPEG stream of a completely dark (or even very dark) image. It seems that it needs at least a little light to have something to "lock on" to before it will start producing frames.

The server requires an ending signal to be sent once the client is done viewing the stream; something that most IP cameras do not need. The difference, however, is that an IP camera is continuously running whereas our camera starts and stops when it is not being used to save on processing power on the R-Pi. A rudimentary garbage collection system could be implemented to combat the problem of certain browsers not always sending the end signal (or the problem of it not arriving entirely). A timer of something like 30 minutes could be attached to each client and they would be dropped after that time frame, but then the client would need to refresh their browser in 30 minutes. I'm still deciding if I'm going to write this code or not..

Some Quick HTML

EDIT: These files below need to be stored in some sort of local web server like Apache or Nginx so you can view them in your browser. The server we coded above is just a camera server and will not be able to serve us these pages. How to install and operate a local server is outside the scope of this tutorial, but there are plenty of resources on the web to learn how to do this, just Google it!

Since it could be a while before I write the next post about the client (I'm still working on basic video controls) here's the code for an extremely simple demo:

actions.js

In the HTML we import jQuery and actions.js first, then set up the <img> tag that will be where the video stream is played. You'll notice the IP address listed is incomplete. Set this IP address as the address of your R-Pi (if it's located remotely you'll have to forward a port on the network the R-Pi is on, but that could make things insecure for your R-Pi from a security standpoint; do this at your own risk).

The JavaScript is pretty straightforward. We wait for the window to be ready, then we attach an event handler to onbeforeunload, which fires when the page is about to unload its resources (i.e. on refresh and navigation away from the page). This sends an AJAX request to the node.js server, telling it to stop the video stream.

On your R-Pi, start a terminal (I use PuTTY), navigate to the folder where you copied the server files, and start the server with:

node server.js

Then open the webpage that we just created in your browser of choice to see the results!

Final Thoughts

This has been a fairly simple tutorial in an attempt to get you up and running. As mentioned, in the coming days I will try to update this tutorial with more relevant information and updated code if it becomes available. And the post about creating a more robust client will be coming soon as well. For now, I hope you were able to find this useful and thanks for reading! If you have suggestions for improvements to the code or if you have questions, please feel free to leave them in the comments below.