Exploring AWS IoT

Introduction to the course materials covering hardware devices used for the course as well as a brief walk-through of the IoT process on AWS.

A discussion of the technical specifications for the ESP8266, ESP32, and Raspberry Pi 3 Model B+.  We explain what they are used for and their IoT capabilities relevant to AWS IoT.

Updated recommendations for the best hardware for the course.  ESP8266 and ESP32 development boards and chipsets discussed.

In this lecture we discuss all the free software used for the course, as well as OS's, testing tools, IDE's and languages used to program our IoT devices to talk to the AWS IoT Core gateway.  We start with the MQTT.fx tool to test our connection then move on to testing with Node-Red with a static IP.  Then Arduino with a provided script for sending data to Node-Red to AWS IoT core as a HTTP/MQTT relay.  Then move on to using Mongoose OS in JavaScript, Zerynth in Python, and finally the AWS IoT SDK in both JavaScript and Python. At the end of the course we introduce the AWS FreeRTOS on the ESP32 DevKitC as an advanced topic in bare metal C.

A discussion of communication protocols for IoT focusing on the MQTT broker both used from the device side and on AWS IoT Core to handle messaging using the Publish and Subscribe model.

A quiz to test your knowledge on AWS, IoT, and introductory cloud topics

In this video I discuss cosmetic updates to AWS IoT Core for 2023. The main update is the "Actions" blade has been changed to a blade called "Message routing" and IoT "Rules" can now be created directly from this blade without the pretty pictures for each AWS service that appeared previously before these cosmetic changes to AWS IoT Core took place in 2023.

In this lecture we discuss the AWS IoT Core service, using the 'monitor' and 'test' consoles, and linking actions from AWS IoT Core to other important AWS IoT related services.

A short introduction to the often confusing concept of IAM policy's and roles.  Here we create a super expansive IoT policy with the automatic configuration tool in AWS IAM as a test case before creating an real IoT policy in AWS IoT core the simple way.

A lecture going over the basic communication protocols and security considerations enabling devices to talk to AWS. We specifically discuss MQTT, HTTPS, SigV4, and SS/TLS requirements for device to cloud communication.  Also added important troubleshooting information.

In this video I show two different methods for sending fake test data in JSON format to AWS IoT Core. These methods are useful if either you don't have your device yet or you don't want to set up your device just to send some quick data for testing purposes.

How to download free MQTT.fx tool to test our certificates

After downloading the MQTT.fx tool we link our security credentials, configure the client broker with our custom endpoint, and open port 8883 with MQTT+TLS1.2  to send a test JSON package to AWS IoT core.

In this video I discuss how to use a custom script with the MQTT.fx tool to generate IoT readings and send them out as a JSON payload to AWS IoT Core.  The test script can be used as a virtual device in lieu of a physical device and has both publish and subscribe functionality.

How to solve the connection issue caused by the "mqttfx-config.xml" file.

This is a simple free tool that can be used as an easier to set up alternative to MQTT.fx.  This may be helpful if you misconfigure MQTT.fx or if you just want to try a different free virtual device.

In this lecture I demonstrate a simple bash script you can use to automate publishing IoT test payloads from your computer to your AWS IoT Core console.  The bash script utilizes the AWS CLI to send automated commands with sample data to IoT Core.  It can be configured for different regions, payload iterations, variables names, and values.

A quiz that will ensure you have the fundamental concepts for setting up your embedded IoT device to communicate with the AWS Cloud

Essential AWS to device troubleshooting

Updated ESP8266 Sketch to work with Board manager 3+.  How he BearSSL certificates are set and anchored has changed.

A quick quiz to help you understand topics related to our Arduino sketches for the ESP devices and how they communicate with AWS.

Here I discuss how Node-Red on the IBM cloud now requires a credit card although it is still technically "free" for experimental use cases.  Because of this I have moved it to the appendix of the course as optional material.  I have developed an HTTPS to AWS IoT Core Arduino Sketch for the ESP8266 and ESP32 which accomplishes that which I was previously using Node-Red for.

In this lecture I explain details of my HTTPS to AWS IoT Core Sketch on Arduino.  I have two different versions for the Board manager 2-3 on the ESP8266 and for board manager 3+.  The ESP32 doesn't require an extra sketch as it works with all board support packages.

In this lecture I introduce MicroPython and talk about the advantages and disadvantages of the langauge compared to using Arduino.  It is worthwhile to compare the C/C++ language of Arduino with the constrained version of Python for Microcontrollers. A large consideration is library support for peripheral sensors.   I also discuss the cool and free IDE called Thonny.

In this video I discuss how to flash and install the MicroPython environment for both the ESP32 and ESP8266.  MicroPython requires two steps before we can upload our specific programs to the device using the Thonny IDE.  First we have to erase everything in memory on the ESP device and then we have to upload the MicroPyrhon framework to the ESP device.  Both tasks can be accomplished with the free ESP tool.

In this lecture we go over the MicroPython code need to connect our ESP32 to AWS IoT Core.  Our code has both publish and subscription methods.  We need to fill out the typical global constants of AWS IoT Endpoint, WiFi, port, MQTT topics, and download our certificates to the device in order to connect to AWS IoT Core using MQTT secured.

In this lecture we go over the MicroPython code need to connect our ESP8266 to AWS IoT Core.  Our code has both publish and subscription methods.  We need to fill out the typical global constants of AWS IoT Endpoint, WiFi, port, MQTT topics, and download our certificates to the device in order to connect to AWS IoT Core using MQTT secured.  With the ESP8266 we have an additional complication of having to convert our security certificates to DER format.  In this video I demonstrate how to do this with the OpenSSL tool.

In this lecture we introduce the free Mongoose programming tool to upload code to our embedded devices and transmit data from our device to AWS IoT Core.

In this lecture we reprogram our embedded device with a new init.js firmware package Included in course resources) that better suits our needs.  Later in this course, when we start using different IoT related AWS services, this new modified data package will satisfy the requirements of our applications.

Update to Zerynth for 2022.

A short quiz covering this section on using Zerynth for AWS IoT on our embedded devices.

In this series of lectures we introduce the AWS IoT Device SDK's which are designed to give the RPi access to a curated set of IoT programs specifically designed to work with AWS.

Install the AWS IoT Device SDK in Python version two and then run the Sample code to publish IoT payloads to AWS IoT Core.

proigram

program

In AWS IoT Core we create a "Rule" that will transfer our incoming IoT dat to DynamoDB version2.  For this Rule to work we will also need to create a table in DynamoDB which we will do in the next lecture. 

For your Rule use the SQL Statement:

SELECT *, timestamp() AS timestamp FROM 'myTopic'

In this lecture we create our data table for DynamoDB v2 and then fill it ith IoT data.  You can use any method you like to bring in IoT data but in the lecture I simply enter it from the MQTT test console in AWS IoT Core using the following format:

{

"temperature": 58,

"humidity": 78

}

In this lecture we move out IoT data from DynamoDB to S3.  S3 is preferred for cheaper, if slower data access, and AWS Quicksight also has a built in connector for S3 and not one for DynamoDB.

In this chapter we create a line chart visualization AWS Quicksight for our IoT data from S3.

In this advanced Lambda example we invoke the AWS-SDK in Node.js to ingest an incoming IoT data payload from IoT Core, and then republish that JSON data to another topic on a different device.  The receiving device simply needs to subscribe to a matching MQTT topic sent from lambda to receive the data payload on the device.

I present a data publishing program written in Python that can be used to send fake data from Lambda to AWS IoT Core.  Useful for testing purposes.

In this lecture we introduce AWS Shadow device service and the Pub/sub model that allows communication between multiple devices with various publish and subscribe topics.  We then discus creating a 'Thing'  shadow topic group on AWS IoT Core and I then show how to emulate the physical devices on the cloud with $aws/.../shadow/update and $aws/.../shadow/update/accepted

In this lecture we go over our ESP sketch in which we demonstrate how to publish and subscribe to our shadow topics, specifically $aws/.../shadow/update and $aws/.../shadow/update/accepted.  This allows two or more devices to communicate with each other as well as update and hold the shadow state on the AWS Cloud.  We demonstrate how we can also publish from the cloud to change state and publish topics to the devices from the cloud.

Introduction to multiple subscription and publishing topics to multiple devices.  We cover how to use the AWS 'Topic Republish' based on SQL from our incoming JSON payload.  We then demonstrate  how to use tokenization with the 'strtok' and 'atoi' C functions to parse our incoming JON payload from our publishing device(s) to our subscription device(s).

We examine our Arduino sketch on our ESP device to show three different methods discussed previously to act on incoming data from our published JSON payload to affect physical actions on the device.  Here we will blink the LED's on the ESP based on the manner and content of the incoming subscription IoT data payload.

Introduction to our truck company example in which we can use $aws/.../shadow/get and $aws/.../shadow/get/accepted to retain and retrieve the current device state on the cloud even when the device is offline.  This is extremely useful for devices with intermittent IoT service and resetting the device with a good state from a lost state.

We emulate multiple trucks as thing groups in the cloud and demonstrate how to use $aws/.../shadow/get and $aws/.../shadow/get/accepted to update our device shadow on the cloud as well as communicate between connected devices with shadow publish and shadow subscribe topics.

In this lecture we examine a Arduino sketch which utilizes multiple shadow topics to publish and subscribe to the current state of other trucks in the wild.  This sketch shows how one truck can retrieve the last known good state of another truck whether they are online or not utilize multiple reserved shadow topic.

A Quiz on using Shadow Device topics and MQTT PubSub

In this lecture we create an action in IoT Core to send our incoming JSON IoT data payload to AWS Timestream.  First we must create our database and then our data table in Timestream, which we can do directly from IoT Core.  Now that Timestream is set up and able to accept data we input or data through IoT Core and then query the data in Timestream using basic SQL statements.

Now that our Timestream data ledger is full of our IoT data we can query it and create visualizations with Grafana.

First we make a free Grafana online account, then we download the Timestream 'connector' designed to allow Grafana to read from Timestream in our account.   We will create some limited credentials (SigV4) in AWS IAM to give Grafana the privilege's needed to read our Timestream data in AWS.  Finally, now that we have formed the connection between Grafana and Timestream, we can do a few visualizations using the built in query capability of Grafana on our Timestream data. 

Describing the thoughts and reasons for Amazon buying FreeRTOS as a OS for AWS IoT connectivity based around mid-level Microprocessors using C code.