The Internet of Things (IoT) movement is rapidly connecting everything to the internet. All the devices, sensors, and actuators need to be able to connect and communicate and for this reason, they need a communication protocol. This is where MQTT comes in.
What is MQTT?
Before we can get into the details of MQTT we need to start out by defining it. MQTT stands for Message Queueing Telemetry Transport (MQTT). Luckily for us, that sounds a lot more complicated than it is.
MQTT is a way for devices to send and receive messages. It is a lightweight protocol designed for sending messages across unreliable networks and is a favorite in home automation systems due to its ease of use and reliability.
The Message Queueing Telemetry Transport was originally developed by IBM. It is a TCP protocol based on a subscribe and publish pattern. Most of the traffic in home automation is actually machine to machine traffic. All of the devices and sensors are communicating with each other and responding in the background without our intervention.
How Does MQTT Work?
It helps to think of MQTT as a hub and spoke model. All of the spokes are clients. They are the ones sending and receiving messages. The hub acts as the middleman between all of the clients directing traffic. This hub is known as a broker. The clients act as publishers and subscribers.
The broker sits in the middle of the hub and spoke pattern. It may be helpful to think of the broker as a server since that’s basically all it is. Clients can send messages to the broker and the broker broadcasts these messages to the clients that are interested.
Publishers and Subscribers
Publishers are the ones sending the messages to the broker and subscribers are the ones receiving the messages from the broker. A client can be both a publisher and a subscriber. There is nothing limiting a device to being one or another. The publisher will send the message to the broker and the broker will make sure the message gets sent to those subscribed. At this point you may be asking yourself, what are they subscribing to?
Topics and Messages
Subscribers subscribe to topics. When a message is published, it is published on a topic. For instance, a temperature detecting device may publish the temperature reading to the topic of thermostat/living-room.
Subscribers who are subscribed to the topic thermostat/living-room will receive the message and have the temperature value.
The method of publishing and subscribing to topics removes the need for any addresses or identifiers. This is great when building a large network of devices. The publishers aren’t concerned about who is receiving the message, they are just publishing it to the broker. The subscribers don’t worry about who sent it, they just receive messages on the topics they subscribe to.
Quality of Service
When a client publishes a message to a broker, the client also specifies the Quality of Service (QoS) they wish to publish the message with. The QoS specifies how the message should be delivered to the subscribers. There are three levels of QoS:
QoS Level 0
This is the most basic level of QoS a message can be sent with. The client publishes the message and is done. There is no waiting for acknowledgment that the broker has received the message. Since this is the minimum amount of traffic on the network, it results in the lowest overhead in sending the message.
QoS Level 1
This level of QoS guarantees that the broker receives the message. After sending the message, the publisher will wait for acknowledgment from the broker. If the acknowledgement is not received, the publisher will continue to resend the message. Although this does guarantee delivery, it creates additional network traffic.
This is the highest level of service in MQTT. Where QoS Level 1 guarantees the broker receives the message, this level guarantees the broker receives the message exactly once. This level is a four-step handshake.
QoS Level 2
- The message is published.
- The broker informs the client it has received the message.
- The client tells the broker it received the acknowledgment.
- The broker replies marking the transfer as completed.
Which QoS Should I Use?
Use QoS Level 0:
- You are ok with lost messages
- Have a reliable connection between broker and publisher
- As a default, due to its low overhead
Use QoS Level 1:
- You need every message to get thorough
- Can tolerate duplicate messages
Use QoS Level 2:
- You need messages to get through
- Cannot tolerate duplicates (i.e. Sending number of times button was pressed)
You should always aim to use the lowest level that will meet your needs. This ensures you put the minimum load on your network keeping it as reliable as possible.
Why Host an MQTT Broker?
Now that we know what MQTT is and how it works, why would we go through the trouble of building our own network?
We all know how expensive smart devices can be. Although they can be incredibly convenient and fun to tinker with, they can just as easily put a dent in your wallet. For instance, here is a pack of three temperature sensors from Nest. That’s three temperature sensors for about $100.
Now take a look at these sensors. That’s five sensors for $11. In addition to their temperature reading, they also read humidity. If we want to use these lower-priced sensors, we will need to send their readings to the MQTT broker. From here, the broker can broadcast these values to all interested devices.
In the previous example, extra steps were required to connect the temperature sensors to your network. The Nest sensors do seem to have the advantage of easier connectivity. If we look closer at this connection, we start to see some problems, however. To connect the Nest sensors, we go through Google/Nest’s network. Although easy to get set up on our smartphone app, what happens if Google/Nest’s servers go down? What if our internet goes out? We would lose access to our data.
(by the way, I’m not hating on Nest. I use Nest devices and enjoy them. I am just using them as an example of a non-local network.)
If we use the MQTT network, we can avoid these problems. Instead of sending our data off to some corporate network, the devices will be sending their information to our MQTT broker. The broker will then update our local devices and if we are running a local home automation server, such as Home Assistant, all our data is contained locally. Getting our data is no longer dependent on another company and their servers. If the internet goes out, we don’t lose any of our data and all of our automations can continue to run smoothly.
Build an MQTT Broker using a Raspberry Pi
We will be building an MQTT broker from the ground up using a Raspberry Pi and Mosquitto broker. I have always had good luck with CanaKit Raspberry Pi bundles. If you’re just getting started, these kits have everything you need to get up and running. Mosquitto is a popular open-source MQTT broker. We will be installing a moquitto broker on our pi and this is where our devices are going to be publishing their messages.
Install Mosquitto Broker
The easiest way to install moquitto is by using apt. Apt is a package manager for Debian that handles the installation and removal of software. We are going to be installing both the mosquitto and the mosquitto-clients packages. The mosquitto package is the only one required for running the broker. The client package will be used for testing. With mosquitto-client we can run the client code locally allowing the pi to function as a client, in addition to a broker.
To install these packages navigate to your command line. From there use apt:
sudo apt install mosquitto mosquitto-clients
The first time you run sudo, you will be required to enter your password.
Enable the Mosquitto Broker
Now that we have the packages installed, we need to make sure the broker automatically starts on bootup.
sudo systemctl enable mosquitto
The mosquitto MQTT should now be running. To confirm this, we can run:
sudo systemctl status mosquitto
We should see something indicating the mosquitto message broker is active and running.
Subscribe to MQTT Locally
Now that we have the broker running, we are going to want to test it. To do this we will need to subscribe to a topic. We are now going to create a topic on the local server. To do this we will use the mosquitto_sub command.
mosquitto_sub -h localhost -t “test/message”
Now the mosquitto_sub program is running and listening for messages on the test/message topic. The only thing left is to publish a message and make sure we receive it.
Publish to the MQTT Topic Locally
We can now open up another terminal and publish a message to make sure it all works. In our new terminal window type:
mosquitto_pub -h localhost -t “test/message” -m “Test successful”
If we change back to our original mosquitto_sub terminal we should see the “test, successful” message.