Birdbox camera dashboard now with live weather data

I added a live weather widget to my grafana birdbox camera monitoring dashboard, in case I can't be bothered to look out of the window.

Dark Sky weather data 

I did an earlier post that described getting my camera feeds into this setup here.
I used info on Michael Green's blog post here as inspiration for my setup.

This is the updated version, not a lot going on apart from the wind rattling the windows...

The weather bit is in the top left corner, it pulls weather data from an online weather visualisation app called Dark Sky.  It's added via a short piece of modifiable html code into an iframe panel on the grafana dashboard that runs in a docker container on a Raspberry pi.  It was a bit of a pain to get running , so I'm going to document it in case anyone else wants to do this.
Note that the 'accelerometer' style graphs are populated by a temperature and humidity sensor in one of the bird boxes themselves.

Steps / considerations
I'm running grafana version v6.3.6 in a docker container on a Raspberry pi.  If you go to the grafana website you'll see a darksky plugin that can also be used... I didn't go that route, instead opted for (what I thought) would be simpler, namely adding a short bit of html to a text panel in html mode (see my original post on this) , this proved not to be the case...  The iframe embed method is described in this helpful blog post.

In theory, you just add the iframe html into a new grafana panel and set it to txt/html mode.
It's quite configurable, e.g. the colour of the text and background/font used and temperature units used - I'm in the UK so have set it to degrees celsius.
You'll need to substitute the XXXX and YYYY for your latitude and longitude so that the weather data is relevant to your location.  I sourced my location via this handy postcode to lat/long converter

<iframe id="forecast_embed" frameborder="0" height="245" width="100%" bgColor="transparent" src="//forecast.io/embed/#lat=XXXX&lon=YYYY&units
=uk&color=#ced6cb&text-color=#ced6cb"></iframe>

The fiddly bit.

So in theory its as simple as dropping in a bit of html - no it isnt...
There's a configuration option that needs to be changed, otherwise an ifram wont render, and you just get a string of text.  In the grafana server admin screen, if you scroll down to the [panels] section there is a setting called 'disable_sanitise_html = false', this needs to be changed to ' = true' or your iframe wont load.  You would think that changing it directly here would be possible?  ...but it isn't

Note - I've updated it in my version.  the default is 'false'

How to fix it depends on how you're running garafana.  If you've installed grafana locally, then you just need to directly edit the grafani.ini file, which is located in /etc/grafana/grafana.ini.

In my case I've set it up in a docker container which makes editing the file a bit more tricky- editing it directly does not work either but I'll describe the process as its handy to know how to connect to a running docker container and modify ts contents:

SSH into the Rassberry pi running Docker, and by definition your grafana container
Find the container ID of the grafana container

docker container ls -a

Connect to the container as root

docker exec -it <container-name> bash

To edit the grafana.ini file, you'll need to install a txt editor inside the docker container first

apt-get update
apt-get install nano

Then edit the file in nano, updating the disable_sanitise_html = false to true

nano /etc/grafana/grafana.ini

Save the file, exit the container.

SO.. that should fix it?  WRONG.  The config screen in the web interface as described above will still show this setting as = false.  Loads of forums recommend restarting the docker container, which has not effect either.  Running the command 'service grafana-server restart' within the docker container also has not effect.

The solution..
The trick is to pass 'disable_sanitise_html = true' to a docker image as an environmental variable at the point that a docker image is spun up into a container.  To make this work, I committed my existing grafana container back to an image file (this is essentially making a backup/snapshot of a container),  then re-launched the new local image file with the environmental variable added:

#run newly saved image file with environmental variable set
docker run -d -p 3000:3000 -e GF_PANELS_DISABLE_SANITIZE_HTML=true  grafana:1_Nov

Note that I wanted to keep my modification to the stock docker image for my modified version of grafana.  If you want to apply it to the stock grafana image on dockerhub.com then use this command to spin up a new container:

#or add environmental variable option to the image on docker hub:
docker run -d --name=grafana -p 3000:3000 -e GF_PANELS_DISABLE_SANITIZE_HTML=true grafana/grafana:6.3.6

I've squished the DarkSky iframe down a bit on my dashboard, if you make it bigger within its grafana panel, you get this which has some pretty cool animated effects too

Birdbox camera dashboard + environmental monitoring

I made a CCTV-type monitoring screen to present multiple birdbox cameras together + environmental monitoring data, using a Grafana / Docker container on a Raspberry pi...

The screenshot below shows all five of my currently active bird boxes.  The foot in the top Left box  belongs to a blue tit that spends quite a lot of time in there the moment (roosts too).  The temperature graph and widgets are populated by a temp + humidity sensor in the bottom R box that logs to its own internal environment every minute - the data really is an in-box box environment monitor.

EDIT Nov 2019: New blog post describing addition of web-sourced weather data to grafana

Prerequisites.. quite a lot of groundwork.  This project relies on various stuff already existing:

(1) Multiple birdbox camera streams
Most of mine use Raspberry Pi ZeroW mini computers + v2 camera modules running motion capture & streaming software called pikrellcam.  The top L one is a motion jpeg stream from a webcam on an early first raspberry pi model that has been going strong since 2014, the camera window could do with a clean though, our early evening roosting blue tit is present in this one:

(2) Some environmental data.
The aim is to present environmental data relevant to to the camera feeds.  The bottom R bird box has a sensor that logs its internal temp & humidity every minute to a mysql database, and is used to populate the temperature plot:

Temperature plot over time, current humidity and temps also showing

Equally there are plenty of free sources of weather data that could be used to provide a feed - it would be fun to see what the weather is locally too.

(3) A 'video dashboard'
The main point of this post.  The dashboard runs on the newest model from the Raspberry pi foundation, the Model 4B (4Gb ram version).  This has faster wired networking, more ram available, and generally more processing ooph.  I would NOT advocate putting one in a bird box unless you're planning on keeping roosting birds warm over the winter.  Mine sits 'headless' (ie with no monitor/keyboard etc) in the lounge behind the TV.  I've set it up with fixed IP address (wired, not wireless), and is accessed via SSH from a laptop.  I'm not going into detail for that as there are countless 'how-to's' out there.  Lets call this machine 'Hub-Pi'.

Hub-Pi setup as follows

Docker
Docker provides a seamless(ish) way of running 'OS-level virtualization software in packages called containers'.  Basically there are a lot of freely available pre-configured software 'images' that allow rapid deployment of software without having to fiddle about with setup.  I've used a Docker image of a dashboard application called Grafana.

Install Docker on Hub-Pi:  I used the guide on this site, up to the bit about 'swarming' (ignore from then).

Docker works by first downloading a specified image.  It then creates a live 'container' from that which is what you 'do stuff with'.  The 'docker run...' command specifies how the resulting container is configured.  If a container is deleted then all configuration data within the container is lost and you need to re-create a container from the original image and loose all your work in the process.  You can 'commit' a container back to a fresh image at any time,  again Google is your friend.

Install Grafana Docker image: Docker images are supposed to run on any hardware running Docker... sort of.  A base image is built with respect to the architecture of the CPU of the machine it runs on.  This basically means that Raspberry Pis are different to PCs (Arm vs  AMD64).  I'm not totally clear on the difference, but an image built for one may not work on the other.  Many images are built for both CPU architecture and Docker intuitively picks the right one to run.  Mostly.

On writing this, the most recent Grafana image is 6.4.3.  The Arm (is for Raspberry pi) version is somehow broken, so I had to revert back to version 6.3.6, which runs fine.  The Docker command to pull down the right Grafana image and create a Docker container that runs it is...

docker run -d --name=grafana -p 3000:3000 grafana/grafana:6.3.6

If you omit the :6.3.6 bit you'll get the current image which may be fixed in a future version, but didi not work for me.

This command is saying:

1. Pull down the v6.3.6 of the Grafana image from DOckerHub.
2. Run it (make container) and forward port 3000 to the host computers port 3000.
3. Assuming the container runs, if you navigate to the Pi's IP address in a web browser that you fixed earlier, using port 3000 (http://HubPiIPaddressHere:3000), if its connected to a screen & keyboard use localhost:3000 directly then you get to the Grafana main login screen where you can assign a new password

Addition of camera feeds to Grafana
To make a camera feed in Grafana, got to..
Add Panel, then choose 'Text' from the visualisation options.
Based on the two types of video feds that I currently have, I edited the text in html mode as shown below.

My network video streams are of two flavours:
(1) Using webcams to generate a motion JPEG (MJPEG) stream, add the following to the html:
<img src="http://XXX.XXX.X.XX:YYY/?action=stream">
where XXX.XXX.X.XX is the IP address of the remote source and YYYY is the port it streams from

(2) Using Raspberry Pi v2 camera modules and pikrellcam software, add the following to the html:
<a href="http://XXX.XXX.X.XX" target="_blank"/a>
<img src="http://XXX.XXX.X.XX/mjpeg_stream.php"">

Where XXX.XXX.X.XX -  the IP address of the remote Pi running pikrellcam.  This also adds a convenient hyperlink to the video in the dashboard to the main pikrellcam page for that camera where motion captured video can be reviewed.

Once a video feed is added you can manually drag it about and resize it to fit your screen. Simple.

Potential developments...
This is an early version.  I plan to add in local weather data, e.g. from https://www.wunderground.com/.  I have entrance hole activity counter on one of my boxes (soon three), so it would be good to see a plot of activity associated with each video feed too.

UPDATE Nov 2019: see a new post which discribes the addition of a live weather feed
https://nestboxtech.blogspot.com/2019/11/birdbox-camera-dashboard-now-with-live.html