MQTT Browser Color Sharing

Last updated on February 23rd, 2022

This post builds on the MQTT Browser Loopback example. This example is best explored with a classmate or someone working on a separate computer. It is a version of the color passing (hot potato) example from class.

In this video, the focus is on setting up MQTT and shiftr. The .js supporting the interface will be addressed in a separate post. The example begins using the public shiftr broker and loopback pub/sub configuration. It is extended to use a custom shiftr instance and finally two computers to demonstrate how publish and subscribe can be used to create a topology that shapes the flow of information.

Video :: MQTT Color Sharing

Get the code on github.

Time stamps are linked in the video description on vimeo.
Show time stampsHide time stamps

Instance specifics

This video uses the public and then a custom (and now disabled) shiftr instance. As in the loopback video, you need to create your own instance OR use the credentials from our class broker. Class credentials are available in D2L:: Content–> MQTT-KEY and TOKEN.

Two .js files – separating token data

If you want to adjust the code to connect to your own broker, or try out the class broker you need to understand where I have placed the user, password and URL information.

In this example I have moved these variables to the secrets.js file and then referenced them from sketch.js.

I have done it this way because at all times we want to keep sensitive token data out of code sharing repositories — such at github. Now I can find the sensitive variables quickly and hopefully avoid posting this data publicly.

While this has the specific outcome of isolating variables, you can use this as a way of creating ‘tabs‘ for organizing your code. You can keep blocks of meaningful code in separate files; enabling quick code sharing and reuse.

In the HTML file, you must load your main .js file last. In our examples the main file is usually sketch.js. For this example, the final script imports in index.html are as follows:

<script src="secrets.js"></script>
<script src="sketch.js"></script>

Publish a Message

Message publishing has been broken into two functions. First, one that constructs the content of the package. Then a second function that can handles the transport of complete messages. At this stage I am still only using a single topic — but once you start working with multiple topics, having a generic transport function will make a lot of sense.

Package Build

The package building function is as follows:

// message builder, specific to this project
function buildColorMessage(){
  // get color channels
  let r = floor(red(sendColor));
  let g = floor(green(sendColor));
  let b = floor(blue(sendColor));
    // console.log('r '+r);
    // console.log('g '+g);
    // console.log('b '+b);

  // check for error in color
  if ( isNaN(r) || isNaN(g) || isNaN(b)){
    console.log ( 'malformed message');
  } else {
    // good message send it to shiftr
    let package = String( '*' + r +',' + g + ',' + b + ',#');
    publishMqttMessage(publishTopic,package);
  }
}

The goal of this function is to break down the component of sendColor into RGB and assemble them using the protocol we have been using with Serial communication. The variable sendColor is obtained by mixing the user chosen color AND the received color with the p5js function lerpColor(). There are two aspects of using lerpColor() that we need to address. First, lerpColor returns floats not ints and second, it returns NaNs when the sketch is first started and no sliders have been used.

floor() :: converting floats to ints

Color channels are obtained by extracting the red(), green() and blue() values from the variable sendColor. The floor() function removes the fractional component of each color channel, ensuring we will send ints.

isNAN() :: checking numbers

Next a quick check to make sure that the values are numbers. NaN is a special variable property that means not-a-number. If you send a NaN and then try to use it in a color variable remotely you will throw all kinds of errors. The function isNaN(x), tests ix ‘x’ is a number a returns true if it is, false if not.

Build the package

With extraction and error check out of the way, the numbers can be assembled into a package following the form [*,data,data,…,#].

This is accomplished in a single line ::

let package = String( '*' + r +',' + g + ',' + b + ',#');

The variable package is a String – which is the required format of MQTT message content. As you can see, it is really easy to build up the message with concatenation in javascript.

Finally, the built package is send to the transport function for sending ::

publishMqttMessage(publishTopic,package);

Message SEND

The MQTT transport function follows:

// MQTT TALK -- called when you want to send a message:
function publishMqttMessage(topic,package) {

  // if the client is connected to the MQTT broker:
  if (mqttClient.isConnected()) {

      package = String(package);
      let publishMessage = new Paho.MQTT.Message(package);
      // choose the destination topic:
      console.log('topic '+topic);
      publishMessage.destinationName = topic;
      // send it:
      mqttClient.send(publishMessage);
      // print what you sent:
      console.log("sending :: " + publishMessage.payloadString);
    } // end color check
}

The function accepts two parameters, the MQTT outgoing message topic and the MQTT payload or package. Both of these are Strings.

The whole publish function is wrapped in a check to ensure that the sketch is still connected to shiftr ::

if (mqttClient.isConnected()) {
      ...
}

If there is a connection then the package is converted to a String (redundant but useful).

Message Object

The MQTT message object is then created ::

let publishMessage = new Paho.MQTT.Message(package);

This library requires that the package be passed into the object at time of its creation. I have named the message publishMessage.

The MQTT topic is then added to publishMessage as follows ::

publishMessage.destinationName = topic;

Finally, publishMessage is sent to shiftr using the mqttClient ::

mqttClient.send(publishMessage);

The last line of the functions echos the message you are sending to the browser console.

Receive and Parse

When an incoming MQTT messages arrives, the onMessageArrived() callback is triggered.

// MQTT LISTEN -- called when a message arrives
function onMessageArrived(message) {

  debugIncomingMessage(message);

  // unpack the message - its a string of form [*,#]
  let currentString = message.payloadString;
  trim(currentString); // remove white space
  if (!currentString) return; // if empty
  latestData = currentString; // for display

  // parse the incoming string into elements
  elements = currentString.slice(1); // remove start byte
  elements = elements.toString().split(","); // split on commas
      console.log('elements array');
      console.log(elements);

  // SPECIFIC FOR SENDING COLORS
  //  error check -- I was getting lots of NaN at one point
  if ( isNaN(elements[0]) || isNaN(elements[1]) || isNaN(elements[2])){ // r=='NaN' fails quitely -- oops
    console.log('received malformed package');
  } else {
    inColor = color(elements[0], elements[1],elements[2]);
  }
}

The function starts with a call to debugIncomingMessage(m) which is a simple utility function for seeing what has arrived.

Because we have structured the messages using the [*,,#] format we used with serial, it should not come as a surprise that we will parse the message using the same strategy.

Receive

The first section copies the payload into the local variable currentString. This has white space removed with trim(), and we make sure it is not empty. Last, we copy this cleaned String into latestData for display.

// unpack the message - its a string of form [*,#]
  let currentString = message.payloadString;
  trim(currentString); // remove white space
  if (!currentString) return; // if empty
  latestData = currentString; // for display

At this point we have a message that looks like this ::

* redData, greenData, blueData , #

Slice – remove the asterisk

We know the order because we packed the data in our send function. We remove the leading asterisk with the .js function slice() ::

elements = currentString.slice(1);

The data that arrives is first stored in an array. The .js slice function makes a shallow copy of a portion of this source array (jump to 27:00 in video above to see how to display this array in the console). Slice can take 2 parameters, but when given only one it is used as the INDEX of where to start copying. Javascript arrays are zero-indexed and slice(1) says start that copy at the value in array[1] The asterisk is in position array[0]. So at the end of this line, elements contains::

redData, greenData, blueData , #

Split – elements[]

We now convert the truncated array to a string and split() the string on the commas and get 4 array elements ::

elements = elements.toString().split(",");

Split creates substrings and returns them in a new array. Elements now holds::

elements[0] == redData
elements[1] == greenData
elements[2] == blueData
elements[3] == # // discarded - ignored

Error Check

An error check is performed to make sure we still have numbers. If any value is NaN, then ‘received malformed package’ is logged to the browser console.

if ( isNaN(elements[0]) || isNaN(elements[1]) || isNaN(elements[2])){ // r=='NaN' fails quitely -- oops
    console.log('received malformed package');
  } else {
    inColor = color(elements[0], elements[1],elements[2]);
  }

If all three values received are numbers, then the variable inColor is filled with the new r,g,b data from the elements array ::

inColor = color(elements[0], elements[1], elements[2]);

Console Logging Incoming Messages

This example uses a utility function to debug the incoming message to the browser console.

// look inside an incoming MQTT message
function debugIncomingMessage(m){
  // mqtt message (m) is an object with 2 parts :
  //      topic (destination name)
  //      content (payloadString)
  console.log('message received');
  console.log('raw message :: ');
  console.log(m); // look at this in console
  console.log("incomming topic :: " + m.destinationName);
  console.log("incomming payload :: " + m.payloadString);
}

Most of should be is straightforward. The last two lines show that the message destinationName (topic) and payload (package) can both be directly accessed.

  console.log("incomming topic :: " + m.destinationName);
  console.log("incomming payload :: " + m.payloadString);

Raw Message – and its metadata

The line in the middle that logs the raw message is particularly interesting.

console.log(m);

This will dump all the info that your browser holds on the message — and it is a lot! Below is the basic log you will see when you open the console AFTER you have selected and sent a color manually.

Note that each line of debug references its source code line on the right. The incoming message debug starts with ‘message received’. The incoming message topic and payload follow two lines below.

To get into the message details, start by clicking the arrow beside ‘x‘ — this is the raw message logged by the last line highlighted above. When you click that arrow a bunch of extra information will be presented ::

At the top of this list, in bold, are the message functions. Below those, not bold, are the message variables. Here we see the raw form of destinationName and payloadString. Click on the ellipses (…) indicated in red to see the contents of these variables.

Finally we can go one more layer down by clicking on the arrow beside payloadBytes. When you do this you will see the original incoming array in ASCII ! ( ok an exclamation point may be overkill — but really it pretty cool.)

if you take the time to map the ASCII, you will find that 42 = ‘*’, 49 = ‘1’, 50 = ‘2’, 55 = ‘7’, 44=’,’ .. and I leave it to you to solve the rest. Not surprisingly these match the content of the payloadString at the bottom of the image.

Elements Array

If you return to the original debug view — or just scroll down to the bottom of the window, you can click on the arrow beside Array(4)::

to reveal the elements in the array AFTER parsing.

Summary

That brings us to the end of this post, take some time and dig into the code. The more you can master these ideas, the simpler the idea of sending messages over a network becomes.