Differences

This shows you the differences between two versions of the page.

--- en:edu:zmitac:s2:iotn:lab2 [2022/03/17 22:36] – created pczekalski
+++ en:edu:zmitac:s2:iotn:lab2 [2023/04/21 16:42] (current) – administrator
@@ Line 1: / Line 1: @@
-====== InT Networks: Laboratory 2 - MQTT ======
+====== IoT Networks: Laboratory 2 - MQTT ======
 Network programming for the IoT Devices using MQTT protocol.
-Following manual assumes you're familiar with [[en:edu:zmitac:s2:iotn:lab1|InT Networks: Laboratory 1 - Introduction]]. In case you're not, please get yourself familiar with it and spend some time to practice.
+The following manual assumes you're familiar with [[en:edu:zmitac:s2:iotn:lab1|InT Networks: Laboratory 1 - Introduction]]. In case you're not, please get familiar with it and spend some time practicing. Start your project stub using hints and guidance in the [[en:edu:zmitac:s2:iotn:lab1|InT Networks: Laboratory 1 - Introduction]] manual and modify it / extend it as guided below.
-<note important>Computers in laboratory rooms 320, 321 and 336 use freezing software which means that for every startup, computers are restored to their initial state.</note>
+<note important>This manual is not intended to be stand-alone. You need to follow the Laboratory 1 manual to prepare the basics for the project, then continue here.</note>
 ==== Hardware ====
-You will use a sensor that measures temperature + humidity, to publish this information to the MQTT broker:
+You will use a sensor that measures temperature + humidity to publish this information to the MQTT broker:
   * DHT 11 temperature and humidity sensor
 <figure DHT>
@@ Line 14: / Line 14: @@
 </figure>
-==== VSCode, PlatformIO and project configuration ====
-In any case, creating a new project under PlatformIO should follow this general settings, regarding Board and Framework (see figure {{ref>platformionewproject}}):
-<figure platformionewproject>
-{{:en:edu:zmitac:macro:embedeed:lab:screenshot_from_2021-10-13_12-43-38.png?800|:img}}
-<caption>Project configuration settings in VSCode/PlatformIO</caption>
-</figure>
-This configuration brings the following project configuration file ''platformio.ini'':
-<code>
-[env:d1_mini]
-platform = espressif8266
-board = d1_mini
-framework = arduino
-</code>
-It is also necessary to configure the serial port name that is used for uploading your firmware and to monitor. In the case of this microcontroller, it is the same port, so ''platformio.ini'' file should be modified with the following (assuming the serial port is i.e. ''COM3''):
-<code>
-...
-upload_port = COM3
-monitor_port = COM3
-...
-</code>
 === Hardware-software components and libraries ===
-Interfacing hardware requires the use of particular GPIOs. Laboratory nodes and hardware used needs specific configuration. Following chunks of code will help to set up proper hardware-software configuration.
+Interfacing hardware requires the use of particular GPIOs. Laboratory nodes and hardware used need specific configuration. The following chunks of code will help to set up proper hardware-software configuration.
 == LCD display ==
@@ Line 55: / Line 33: @@
 </code>
-To initialise it and instantiate controller object, use following GPIO definition and initialisation:
+To initialize it and instantiate the controller object, use the following GPIO definition and initialization:
 <code c>
 const int rs = 12, en = 13, d4 = 4, d5 = 0, d6 = 2, d7 = 14;
@@ Line 61: / Line 39: @@
 </code>
-It is also necessary to tell the library about the display organisation as there is a variety of configurations possible. Here, the organisation is 2 lines, 16 columns each, so in the code you need to explicitly initialise it:
+It is also necessary to tell the library about the display organization as various configurations are possible. Here, the organization is 2 lines, 16 columns each, so in the code, you need to explicitly initialize it:
 <code c>
 ...
@@ Line 70: / Line 48: @@
 == Temperature and humidity sensor ==
 There is a DHT11 (blue) sensor connected to the __GPIO 5__.\\
-There are two libraries needed to handle reading from the sensor: a common sensor communication library and a specific for DHT sensors, both by Adafruit, so two more entries are needed in the //platformio.ini// file, and obviously you need to refer to it in the source code:
+There are two libraries needed to handle reading from the sensor: a common sensor communication library and a specific one for DHT sensors, both by Adafruit, so two more entries are needed in the //platformio.ini// file, and you need to refer to it in the source code:
 <code>
 lib_deps =
@@ Line 81: / Line 59: @@
 </code>
-To access sensor readings it is necessary to instantiate a communication class that binds the hardware to the code:
+To access sensor readings, it is necessary to instantiate a communication class that binds the hardware to the code:
 <code c>
 #define DHTTYPE DHT11   // DHT 11
@@ Line 87: / Line 65: @@
 DHT dht(DHTPIN, DHTTYPE,50);
 </code>
-It is also necessary to initialise the sensor class and give it some time to set up:
+It is also necessary to initialize the sensor class and give it some time to set up:
 <code c>
 ...
@@ Line 95: / Line 73: @@
 </code>
-The sensor class returns data (among others) in a float containing temperature (in C) and relative humidity (in per cent):
+The sensor class returns data (among others) in a float containing temperature (in C) and relative humidity (in percent):
 <code c>
 ...
@@ Line 104: / Line 82: @@
 ==== Software ====
-String formatting may be a challenge. An easy approach is to allocate a character buffer of the size of the LCD or just one display's line, or any other size, large enough to handle your text.
+String formatting may be a challenge. A straightforward approach is to allocate a character buffer of the size of the LCD or just one display's line or any other size large enough to handle your text.
 <code c>
 ...
@@ Line 111: / Line 89: @@
 </code>
-To convert float into the string with formatting use ''sprintf'' function:
+To convert float into the string with formatting, use ''sprintf'' function:
 <code c>
 ...
@@ Line 119: / Line 97: @@
 ...
 </code>
-<note tip>''sprintf'' uses number of wildcards that are rendered with data. Refer to the c/c++ documentation on ''sprintf''. Here <code c>%2.1f</code> means: having float number render it to string using two digits before decimal point and one after. Temperature in our lab is always above 10C. <code c>%%</code> is an escape character to print a <code c>%</code> (percent) symbol.
+<note tip>''sprintf'' uses a number of wildcards rendered with data. Refer to the c/c++ documentation on ''sprintf''. Here <code c>%2.1f</code> means: having a float number render it to string using two digits before the decimal point and one after. The temperature in our lab is always above 10C. <code c>%%</code> is an escape character to print a <code c>%</code> (percent) symbol.
 <code c>
 delay(time)
@@ Line 125: / Line 103: @@
 === Software libraries ===
-Software libraries are needed to handle communication between your IoT device and network and services. In this laboratory, you will connect your device to the WiFi access point (see the Networking section below for credentials) and use TCP. Your device will be automatically given an IP address that you may show on the LCD screen. Once WiFi is connected and works (PHY+MAC+NET layer protocols) then you can connect over it to the MQTT Broker (APP layer protocol).\\
+Software libraries are needed to communicate between your IoT device and network and services. In this laboratory, you will connect your device to the WiFi access point (see the Networking section below for credentials) and use TCP. Your device will be automatically given an IP address that you may show on the LCD screen. Once WiFi is connected and works (PHY+MAC+NET layer protocols), then you can connect over it to the MQTT Broker (APP layer protocol).\\
-Mind, the connection may become broken so ensure re-connection mechanism:
+Mind, the connection may become broken, so ensure a re-connection mechanism:
   * dummy: in case the connection is broken, restart the device to let it perform the connection process again;
-  * smart: include checking and re-connection into your code using connection status tracking, mind you not only need to reconnect WiFi but also high-level communication protocol: MQTT.
+  * smart: include checking and re-connection into your code using connection status tracking, mind you need to reconnect WiFi and high-level communication protocol: MQTT.
 == WIFI and TCP ==
+A WiFi library is already included in the ESP8266 framework for the Arduino programming model, so there is no need to use a library manager to add it. All that is necessary is to have a header included to your C code:
+<code c>
+...
+#include <ESP8266WiFi.h>
+...
+</code>
+Then create some constant variables or declare token string identifiers (as below) with ''sSsid'' and ''sPassword'' (see Networking section for credentials) and instantiate the WiFi communication object:
+<code c>
+...
+#define sSsid "<here comes ssid to connect to>"
+#define sPassword "<here comes a password>"
+...
+</code>
+The ''WiFi'' object is a singleton, static one. Try to connect to the AP, i.e., this way:
+<code c>
+...
+  WiFi.mode(WIFI_STA);
+  WiFi.begin(ssid, password);
+...
+</code>
+Some useful code snippets are below.\\
+Checking if the WiFi connection is successful:
+<code c>
+...
+  while (WiFi.status() != WL_CONNECTED) {
+    delay(500);
+    Serial.print(".");
+  }
+...
+</code>
+Checking the current IP address:
+<code c>
+...
+  Serial.println(WiFi.localIP());
+...
+</code>
+Once WiFi is connected, it is necessary to use a ''WiFiClient'' object to pass the connection object to the application layer libraries.\\
+The declaration can be as follows:
+<code c>
+...
+  WiFiClient espClient;
+...
+</code>
+Note this object becomes automatically active and bound to the ''WiFi'' connection when available. You do not need explicitly to initialize it or force connect it.
 == MQTT ==
-To ease and speed up implementation, you will use an MQTT library that implements acting both as a publisher and subscriber. Please note, on the networking level, you're connecting to the
+To ease and speed up implementation, you will use an MQTT library that implements acting both as a publisher and subscriber ''platformio.ini'' file:
+<code>
+lib_deps = ..., PubSubClient, ...
+</code>
+<note important>Please note, on the NET protocol level, in this scenario, you're connecting to the WiFi AP (STA), not setting your device to be a WiFi server (AP), so you need to set the appropriate WiFi mode, as it may be previously used in AP, not STA mode.</note>
+MQTT clients need to be instantiated using a WiFi client connection, i.e., this way:
+<code c>
+...
+#define mqtt_server "<here is your MQTT broker IP or FQDN>"
+#define mqtt_user "<mqtt user>"
+#define mqtt_password "<mqtt password>"
+...
+PubSubClient client(espClient);
+...
+</code>
+and in the setup code, you need to initialize the ''client'' with the MQTT address and port number. Do not use SSL / TLS encoding, as this library is not able to support encrypted communication with an MQTT broker:
+<code c>
+...
+client.setServer(mqtt_server, 1883);
+...
+</code>
+Consider encapsulating MQTT connecting process as a separate function, i.e., as below. Note the code below publishes a hello world message to the broker to emphasize its successful presence in the network:
+<code c>
+void reconnect() {
+  // Loop until we're reconnected
+  while (!client.connected()) {
+    Serial.print("Attempting MQTT connection...");
+    // Create a random client ID
+    String clientId = "ESP8266Client-";
+    clientId += String(random(0xffff), HEX);
+    // Attempt to connect
+    if (client.connect(clientId.c_str(),mqtt_user,mqtt_password)) {
+      Serial.println("connected");
+      // Once connected, publish an announcement...
+      client.publish("outTopic", "hello world");
+      // ... and resubscribe
+      client.subscribe("#");
+    } else {
+      Serial.print("failed, rc=");
+      Serial.print(client.state());
+      Serial.println(" try again in 5 seconds");
+      // Wait 5 seconds before retrying
+      delay(5000);
+    }
+  }
+}
+</code>
+Receiving an MQTT message is done with the use of the asynchronously called callback function, i.e., like this one (it switches on and off the LED):
+<code c>
+...
+void callback(char* topic, byte* payload, unsigned int length) {
+  Serial.print("Message arrived [");
+  Serial.print(topic);
+  Serial.print("] ");
+  for (int i = 0; i < length; i++) {
+    Serial.print((char)payload[i]);
+  }
+  Serial.println();
+  // Switch on the LED if an 1 was received as first character
+  if ((char)payload[0] == '1') {
+    digitalWrite(BUILTIN_LED, LOW);   // Turn the LED on (Note that LOW is the voltage level
+    // but actually the LED is on; this is because
+    // it is active low on the ESP-01)
+  } else {
+    digitalWrite(BUILTIN_LED, HIGH);  // Turn the LED off by making the voltage HIGH
+  }
+}
+...
+</code>
+You also need to bind the callback function with the MQTT client:
+<code c>
+...
+  client.setCallback(callback);
+...
+</code>
+And then periodically execute the communication routine for the handling of the incoming messages, usually in the ''loop()'' function of your code (unless using FreeRTOS tasks asynchronous model):
+<code c>
+...
+  client.loop();
+...
+</code>
 ==== Networking ====
-There is a WiFi 2.4GHz, IPv4 network available to the students. This is a PRIVATE network. This network is separated from the Internet but all connected devices to it can talk to another. There is a DHCP server with networking addressing scope 192.168.90.10-192.168.90.254.
+There is a WiFi 2.4GHz, IPv4 network available to the students. This is a PRIVATE network. This network is separated from the Internet, but all connected devices can talk to one another. There is a DHCP server with networking addressing scope 192.168.90.10-192.168.90.254.
-<note warning>Do not use static IP! Some services present in the network have fixed IP addresses, overlapping it causes unavailability of your solution and this service, also to other teams! Use the only DHCP for obtaining a network address.</note>
+<note warning>Do not use static IP! Some services in the network have fixed IP addresses; overlapping causes the unavailability of your solution and this service, also to other teams! Use only DHCP for obtaining a network address.</note>
 To connect to the wireless network, use the following credentials:
@@ Line 149: / Line 260: @@
 === MQTT ===
-There is an MQTT Broker, binding WiFi PRIVATE network and the PUBLIC Internet. This way you can route MQTT messages from your end node devices connected to the ''internal.IOT'' network to the external tools i.e. cloud-based dashboard or storage via i.e. publishing from your end node IoT device through the PRIVATE network to MQTT Broker and subscribing to those messages from the cloud using PUBLIC Internet. \\
+There is an MQTT Broker, a binding WiFi PRIVATE network, and the PUBLIC Campus network. This way, you can route MQTT messages from your end node devices connected to the ''internal.IOT'' network to the external tools, i.e., cloud-based dashboard or storage via, i.e., publishing from your end node IoT device through the PRIVATE network to MQTT Broker and subscribing to those messages from the cloud using PUBLIC Campus network. Please note the context (thus, the IP address of the MQTT broker) changes depending on which device and network you're using. The following network schema should be of help {{ref>zmitacnetworkingiot}}. \\
-You can access it from the Internet PUBLIC network, using the following configuration and credentials:
+You can access the MQTT broker from the Campus PUBLIC network using the following configuration and credentials:
   * Address: 157.158.56.54
   * Port: 1883 for plain text, 8883 for secure communication
@@ Line 157: / Line 268: @@
   * Password: vrel2018
-You can also access it from the aforementioned PRIVATE network, using the following configuration and credentials:
+You can also access the MQTT broker from the ''internal.IOT'' PRIVATE network using the following configuration and credentials:
   * Address: 192.168.90.5
   * Port: 1883 for plain text, 8883 for secure communication
@@ Line 163: / Line 274: @@
   * Password: vrel2018
-=== Node Red and Node Red Dashboard ===
+<figure zmitacnetworkingiot>
-A Node RED server is working on same Raspberry Pi 2 as MQTT and in private address pool only, available through //internal.IoT// on 192.168.90.5 address, port 1880:
+{{:en:edu:zmitac:macro:iot:zmitac_network_infrastructure.png?600|}}
-  * [[http://192.168.90.5:1880]] - main editor
+<caption>Zmitac network infrastructure for the IoT labs</caption>
-  * [[http://192.168.90.5:1880/ui]] - Node RED dashboard
+</figure>
-Login using the following credentials:
-  * User: //contact supervisor//
-  * Password: //contact supervisor//
 ==== Laboratory scenarios ====
-All teams share the same scenario yet please note that in particular, MQTT messages exchanged between devices shall be coordinated among team members not to overlay one another.
+All teams share the same scenario yet, please note that, in particular, MQTT messages exchanged between devices shall be coordinated among team members not to overlay one another.\\
-<note tip>Mind keeping your code between meetings. It is regular that every next laboratory meeting you will extend your code by something extra.</note>
+You can implement 3 separate codes or bind them into one solid solution.
+<note tip>Mind keeping your code between meetings. It is regular that at every next laboratory meeting, you will extend your code by something extra.</note>
 <pagebreak>
 === Scenario 1 ===
-**Getting ready with your development environment**\\
+** Connecting to the WiFi **\\
-In this scenario, you will write your first "Hello World" application for ESP8266.
+Make your IoT device connect to the Internal.IOT network and present the device's MAC and IP address on the LCD screen.
-Using the selected IDE, write code that:
-  - Flashes built-in LED every 1 second (1s on; 1s off);
-  - Sends a message via serial port to the serial port monitor: ''Hello World''
-<note tip>The GPIO the built-in LED is connected to is number 14. Physically it is located almost in the middle of the mainboard, under the display and its colour is blue.</note>
-<code c>
-...
-#define LED_BUILTIN 14
-...
-</code>
-The standard serial port speed is 9600 (bps). Connect COM port monitor (a plug icon in the PlatformIO toolbar), once you compile and upload your code, to see the "Hello World" message.
-<code c>
-...
-Serial.begin(9600);
-...
-</code>
-Advanced:
-  * Use timer/interrupt to flash the LED instead of an infinite loop.
 Report:
-  * Describe your approach and include source code in the report
+  * Describe your approach to the WiFi re-connection and document IP and MAC with a screenshot.
 <pagebreak>
 === Scenario 2 ===
-** Using LCD **\\
+** Sending MQTT message **\\
-In this scenario, you will write an application that writes on the LCD Display.\\
+Read temperature and humidity and send it as two separate MQTT messages (as payloads).
-Your code shall implement a message ''Hello IoT'' in the upper line, and a counter ticking every second in the lower one.\\
+<note warning>Warning! Agree with other sections to use UNIQUE MQTT client names and MQTT topics. Otherwise, you will interfere with one another on the broker level.</note>
+Observe your messages and their change over time using any software-based MQTT client (i.e., MQTT Explorer or MQTT Spy apps).
 Report:
-  * Present algorithm.
+  * Describe your interface, including client name, MQTT topics, publishing frequency, etc.
-  * Include source code.
 <pagebreak>
 === Scenario 3 ===
-** Using Keypad **\\
+** Receiving MQTT message **\\
-Extend Scenario 2 with a counter that increments on the keypress of the Keypad Shield (any but SELECT!).
+Prepare a solution where your IoT device subscribes to the selected message that will be received, and the payload will be present on the LCD screen. You can assume the maximum payload size is 16 characters to fit a single LCD display line. You should be able then to send any text to your LCD remotely. Use 2 separate messages to control the text in line 0 and line 1 (first and second) of the LCD screen.\\
+Use MQTT Spy or MQTT Explorer app to send a message with a payload.
 Report:
-  * Describe how did you solve the problem of the asynchronous processing of the keypresses/releases.
+  * Document your interface (topic, payload format). Present your approach to buffer the message.
-  * Include source code.
 <pagebreak>