Have Yourself a Matrix LED Christmas

Making an interactive Christmas jumper

After my previous project, the tweet controlled wedding dress, I wanted to make another wearable using the lessons I learnt while making the dress. Since the festive season is now upon us, it seemed like a great opportunity to make an IoT Christmas jumper!

Tech the Halls!

What’s Involved?

  • Some sort of wearable display that can be attached to a jumper
  • A microcontroller to change the graphic on the display that can fit inside a jumper
  • A device to listen for Christmas songs
  • A music recognition service
  • A way to get the recognised song to the microcontroller
  • A battery

The Display

For the sake of affordability, I decided to make a 16 x 16 pixel display for the jumper. This would be made from addressable RGB LEDs. Each LED in the array can be set to a specified colour to show a graphic. It would be low resolution, but who doesn’t love pixel art?!

A basic representation of a pixel display
The individual RGB LEDs in the dress
Addressable RGB LED strip
Pockets in fabric ready for each LED strip
A 16 x 16 array of RGB LEDs

The Microcontroller

In order to set the colours of the lights, I needed a microcontroller board. I chose the Adafruit Feather Huzzah, a small 5cm x 2cm board, which is easy to hide inside a jumper. It has wifi, which means I can connect it to the internet, which will allow me to getting the recognised songs from the web to the lights. It can take usb power or battery power and it isn’t too expensive. You can write code for the board using either the Arduino IDE or the VSCode extension. I have already written about getting a board and lights set up in this post, so I won’t go over that here, but as mentioned before, double check whether you’ve got RGB or RGBW lights and set the right one when you’re setting up your strip in the code:

Adafruit_NeoPixel strip = Adafruit_NeoPixel(
NUM_LIGHTS,
PIN,
NEO_GRB + NEO_KHZ800 //NEO_GRB or NEO_GRBW
);
Showing the flexibility of the LED Array

Jingle Fails

Sending an image to the array

The LEDs in the array are addressable, which means that we can use the board to set each one to a specified colour. They take RGB values to define that colour. From our earlier example, an array of the necessary RGB values might look like this:

Oops, I soldered the LED strip together in a bizarre direction
// slice the array up into 16ths (chunkSize = 16)
(chunkSize) => {
for (var i = 0; i < arr.length; i += chunkSize)
chunked.push(arr.slice(i, i + chunkSize));
return chunked;
}
// reverse the order of every other line (arraySize = 16)
(arraySize) => {
for (var i = 0; i < arraySize; i++)
if (i % 2 == 0) {
chunked[i].reverse();
}
return chunked;
}
A santa hat and a candycane in snake mode mode, represented with CSS Grid

Recordin’ Around the MediaStream…

Listening for Christmas Music

I needed a device with a microphone that could listen to my surroundings. Luckily, we all tend to carry one of these around with us, a phone. I set up a web app with Node, which could record the sound around me and make the data available for a music recognition service. To record the sound I used the MediaStream Recording API. This uses the MediaRecorder interface to take the data from a MediaStream and deliver it for processing, or saving to disk.

function startRecording(recorder) {
recorder.ondataavailable = (e) => storeChunks(e, recorder.stream);
recorder.start();
window.setTimeout(() => {
recorder.stop();
startRecording(recorder);
}, 15 * 1000);
}
navigator.mediaDevices.getUserMedia({ audio: true, video: false })
.then(function(stream) {
const mediaRecorder = new window.MediaRecorder(stream);
startRecording(mediaRecorder);
})
.catch(function(err) {
console.log(`The following error occurred: ${err}`);
}
}

API Wish it Could be Christmas Everyday

Music recognition

I used the AudD API. It costs $2 per 1000 requests which is affordable for the jumper. They have a great telegram bot for account creation, API key generation and payment. After some trial and error with their service and documentation, I discovered that the API works best if you provide it with a url rather than an audio file. This meant that I needed somewhere to upload my audio recordings to. As an aside, it would have been great if AudD had provided some working example code for me to take a look at, if you’re writing an API please provide demo code as well as documentation!

Do They Node it’s Christmas?

Uploading the audio recordings

Microsoft’s Azure Blob Storage provides a container for uploading blobs (which are basically collections of binary data). It then creates a URL for the blob. In order to use Blob Storage, you’ll need an Azure account and an API key. I used their npm package called @azure/storage blob which provides methods to upload blobs and get urls back.

var request = require("request");var data = {
'url': url_from_blob_storage,
'return': 'timecode',
'api_token': your_api_token
};
request({
uri: 'https://api.audd.io/',
form: data,
method: 'POST'
}, function (err, res, body) {
console.log(body);
});

We’re makin’ a list, we’re checkin’ it twice

Mapping song titles to RGB value arrays

I made 13 Christmas themed pixel images and converted them from pngs to an array of RGB values using ImageMagick. It provides a command line tool and this command: $ magick convert image.png image.txt will return an array of RGB values.

key_value_pair images[] {
{
"default",
{{255, 0, 0}, {0, 0, 0}, {0, 0, 0}, {0, 0, 0}, {0, 0, 0}, ...
},
{
"tree",
{{0, 0, 0}, {0, 0, 0}, {0, 0, 0}, {0, 0, 0}, {0, 255, 0}, ...
{
"santa",
{{0, 0, 0}, {185, 27, 24}, {185, 27, 24}, {185, 27, 24}, ...
},
...
const map = {
"jingle bell rock": "bell",
"let it snow! let it snow! let it snow!": "snow",
"rockin' around the christmas tree": "tree",
"santa claus is coming to town": "santa",
"deck the halls": "holly",
"we wish you a merry christmas": "pud",
...
};

Let it show, let it show, let it show!

So now:

  • It can send that recording to azure and get a url back,
  • It can send that url to audD and get a song title back
  • It can look up the relevant image name to match the song title
  • It can put that image name on an endpoint
  • Which the arduino can do a get request on
  • The arduino can then look up which RGB values to display based on the icon name it receives and send them to the LEDs

And it can illuminate the jumper!

We wish you a Merry Christmas and an API New Year!

Now you can build your own music controlled jumper! If this post inspires you to build something I’d love to see it, and if you have any questions or would like to see me give a talk about the jumper then drop me a message on twitter: @thisisjofrank

Lead Developer Advocate