How I helped my mother, who is hard of hearing, to understand me while I visit her with my mask on

Image for post
Image for post
A flexible LED display connected to a microprocessor with a phone and a laptop and a small lipstick battery on a desk.

Live captioning of speech into text has so many useful applications and Azure Cognitive Services makes it fast and easy to build captioning into your applications. Used together with Ably Realtime, it is possible to make wearable devices which can display what you’re saying, in real time. Wearable live captions!

This article will explain how to use Azure Speech and Ably Realtime and will go through building a web app that will take data from your microphone and turn it into readable text.

Check out this video to see the demo in action:

Why would we want this?

I’ll admit to a personal desire to see more products like this on the market. The reason is my mother. She has been steadily losing her hearing over the last few years and relies heavily on lip reading and clear pronunciation. Two things which are denied to her when the people talking to her are wearing masks. Of course I also want to keep her safe, so I will always encourage everyone to wear a mask, but there must be ways that technology can make her, and many others, life easier. One of the frustrations with assistive technologies on phones is that they require her to be looking at her phone, rather than at the speaker, which can lead to her being treated differently, often poorly, by whoever is speaking to her. …


A walk through of how to turn a photograph of yourself into an Andy Warhol style graphic, like the one below.

Disclaimer: there are many different techniques for Photoshop. This tutorial describes the TOS unused for this piece, you may know different techniques and that is great. If you know better/different ways to do these techniques, do feel free to message me, I love to learn new things.

This tutorial may seem long, but it goes into quite a lot of detail because I wanted it to be suitable for beginners. Once you learn the techniques involved it will not take long to create similar artworks. …


Image for post
Image for post

Coping with quarantine by coding!

It is September 2020 and we’re still mid pandemic, which means I’m social distancing and missing my friends. At the start of lockdown I played a lot of online multiplayer party games with friends to try and replace our regular board game hangouts. After consuming so many, I decided it was high time that I try and make a multiplayer game myself! Working at Ably, I have all of their robust pub/sub real-time messaging infrastructure right there, perfect for creating a system that can send data between multiple players!

What is pub/sub and how can I use it to build a game?

Pub/Sub is short for Publish/Subscribe and it describes a pattern of real-time message exchange between ‘publishers’ and ‘subscribers’. Publishers create and send messages to a ‘channel’. Subscribers are then able to receive messages from the channel by subscribing to it. There can be multiple publishers and multiple subscribers, all of which are decoupled - the publishers do not have knowledge of how many subscribers there are. This is great for me, being the “frontendest of frontenders” (a quote from a friend of mine), I have a way to easily get data between players without having to implement it myself. I can use the pub/sub pattern to build a peer-to-peer game, meaning that I don’t have to set up a central server. …


The Station Manager game with Realtime Tube data

I’ve been learning a lot recently about using realtime data streams and how and why one might use them in an app. In order to better understand the differences between realtime streaming data and REST APIs (which I’ve had more experience with), I decided to build a game, the mechanic of which uses realtime train arrival data. As trains arrive into the station in real life, effects are triggered in the game which the user has to manage.

Image for post
Image for post

Make it your own!

All of the code for the game is on Glitch. This means that you can see the code, ‘remix’ it and make it your own. There is a thorough readme file in the repo and I’ll also go over some of the methods used in this blog post. …


How to make your own internet connected t-shirt with LEDs that respond to realtime data

As a developer advocate I get to make all kind of fun projects. I particularly like to make wearable technology, specifically, things with lots of lights on that can connect to the internet. Because I like to be sparkly! I started making wearable tech projects a couple of years ago, my first being a light up, tweet controlled wedding dress.

When I started working with hardware, I had no real clue what I was doing and made a few very silly mistakes around choices of hardware and how to stick them all together, My hope for this blog is to give a little direction to those who are looking to get into hardware projects and to help you over that initial ‘hardware is scary’ hurdle. …


Image for post
Image for post

Making an interactive Christmas jumper

After my previous project, the tweet controlled wedding dress, I wanted to make another wearable using the lessons I learnt while making the dress. Since the festive season is now upon us, it seemed like a great opportunity to make an IoT Christmas jumper!

Christmas jumpers often have a large graphic on the front; Santa, a snowman, a reindeer. I wanted to make a jumper that could change the graphic it displayed depending on what Christmas song was playing. That way, at Christmas parties, I’d always be wearing the right picture for the moment!


All I need to be a software developer is coding skills, right?!

A short while ago I sent out a tweet saying that in order to remain relevant as software engineers then we needed to focus on our skills of empathy, cognitive diversity and self-care.

I’d like to expand a little upon my thoughts behind this tweet and explain why, as some of the replies assumed, it isn’t all “airy fairy cotton candy wishes”.

When we’re developing software it is easy to get carried away with the tech. We want to use the most up to date techniques, the hot new libraries and packages. …


Image for post
Image for post

I ❤ the internet. By visiting a URL I can have access to information, learn new skills, interact with friends, shop, watch movies, play games, you name it. Not only that but new APIs are constantly being created to allow us to interact with the web in new and exciting ways.

Developers can now access users’ personal data, their location, use their hardware and connect to external devices. We can now create truly personalised, captivating, even immersive experiences on the web. In order to use these new, powerful APIs, websites and apps must ask for permission. Being blocked from services until permission is given is absolutely vital, not only to protect the privacy of our users but also to safeguard their trust in the service on offer. …


Image for post
Image for post

Coming up on two years ago, I attended EMF Camp, ‘a non-profit UK camping festival for those with an inquisitive mind or an interest in making things’, the best way I can describe it is ‘It’s like Glastonbury for nerds’. One of the many incredible pieces on display was a giant inflatable rabbit, that you could change the colour of by tweeting.


A short and sweet introduction to JavaScript promises and how they’re used in the Web Bluetooth API

If you’ve not already seen, I’ve been making some fun things with Web Bluetooth recently. While I was getting to grips with this technology, I learnt that the ‘request device’ method, the one that allows your browser to connect to a remote device, returns a Promise. But what is a Promise? The spec says:

“The Promise object represents the eventual completion (or failure) of an asynchronous operation, and its resulting value.”

Ok great, but wait, what’s an asynchronous operation?! Uuuuuurrrrrgh!

Synchronous vs Asynchronous

Imagine you’re at a new, trendy, hole in the wall donut vending shop. So is everyone else in the neighbourhood. You have to queue up to buy your NutellaFilledRainbowSprinklesUnicornSupreme and you won’t be able to get it til everyone in the queue in front of you has bought their donuts. …

About

Jo Franchetti

Developer Advocate

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store