Big programs in an ESP32

As part of the Furby connect development my plan was to be able to control the Furby from the internet using MQTT. This means that the ESP32 device I’m using will need both Bluetooth and WiFi. WiFi to talk to the MQTT server and Bluetooth to talk to the Furby. Both of these protocols are implemented by large lumps of code in the ESP32 libraries and when I turned them both on I promptly ran out of memory on my device. Wah.

A bit of searching revealed that there should be some menus in the Arduino IDE (see above) that you can use to specify how much of the program storage you want to use. By default a device is set up for Over The Air (OTA) updates, which means that only half of the available memory is available. Turn off OTA and you can install bigger programs. However, when I tried to do this on the devices that I was using there was no such menu. Double Wah.

It turns out that some of my devices (the DOIT and the FireBeetle) contain a version of the ESP32 chip (the ESP-VROOM-S2) that has built-in EEPROM program storage (which makes the device simpler and cheaper) but it also limits the maximum program size so the size cannot be extended. But on larger devices you can extend the application. This means that from a software point of view I can have a device that talks both WiFi and Bluetooth at the same time. However, since both technologies use the same frequency band (and on the ESP32 the same radio) it might not work. My fallback solution is to just use two ESP32 devices, one for WiFi and one for BLE. I can link them with a serial connection and then all my problems should go away.

It’s important to remember that throwing hardware at a tricky problem is not such a daft idea. In this case it eases the development and completely removes a whole set of possible failure areas. At the cost of around another fiver or so in hardware.

M5 Atom is absolutely tiny

A new computer arrived at our house today. The postman just popped it through the letterbox. The Atom is a tiny new microcontroller from M5Stack. It contains a button, a multicolour led and an accelerometer. It has enough pins to be useful along with a Grove socket.

It’s a little bit bigger than the top of my thumb, it contains an ESP32, it can run MicroPython and hang off Azure or Bluetooth. And you can buy them for six dollars apiece.

I know that the world today is a right old mess, but there are still some amazing things to be found.

Disabling the ESP32 Brownout detector

I hate it when things fail when they are not supposed to. I’ve now got some code that uses deep sleep on an ESP 32 to drop power consumption to a tiny amount between the readings made by the environmental sensor hardware that Brian has built.

Today I was doing some battery testing and I hit a snag. When the device is running from battery power the deep sleep mode breaks. The device does a power on reset rather than waking from its deep sleep as it is supposed to. Of course, this only happens when it is not connected to a computer, so I can’t use any of my debugging tools to find out what is going on. In the end I had to resort to flashing the LED on the device to indicate the reason for the reboot.

void flashLed(int flashes)
{
    pinMode(LED_BUILTIN, OUTPUT);
    for (int i = 0; i < flashes; i++)
    {
        digitalWrite(LED_BUILTIN, true);
        delay(500);
        digitalWrite(LED_BUILTIN, false);
        delay(500);
    }
}

void flashBootReasonOnBuiltInLed()
{
    esp_reset_reason_t reset_reason = esp_reset_reason();
    flashLed(reset_reason);

    delay(2000);

    esp_sleep_wakeup_cause_t wakeup_reason;
    wakeup_reason = esp_sleep_get_wakeup_cause();

    flashLed(wakeup_reason);
}

This is the code that I ended up writing. On usb power I get a reset reason of 8 flashes (deepsleep) and a wakeup reason of 4 flashes (timer). All good. On battery power I get a reset reason of 9 (brownout).

OK, so what is a brownout? A brownout is when the supply voltage drops a bit low. It’s something you need to worry about because low voltage can cause your processor to do some very strange things. It might get its sums wrong or forget stuff. The ESP32 has hardware that checks the supply voltage and resets the processor if it detects that the power supply is failing. Software in the device can then detect this and go about shutting down safely.

In the case of our sensor the brownout is caused by one specific event. It occurs when the program turns on the power supply that drives the particle sensor. The particle sensor has a fan in it, which takes quite a chunk of current just as it spins up. This sudden load causes the battery voltage to drop for a fraction of a second. The sequence that we have is as follows:

1. Sensor wakes up from a deep sleep.

2. Sensor runs the code that turns on the power to the particle sensor.

3. Supply voltage drops and triggers the brownout sensor which resets the ESP32.

4. The ESP32 wakes up as from a brownout reset, not a deep sleep.

5. Sensor runs the code that turns on the power again, but this time it doesn’t trigger the brownout sensor because the power supply has a little bit of residual power in it from the recent (failed) startup.

I’ve fixed the problem by turning off the brownout detector when I turn the power on.

WRITE_PERI_REG(RTC_CNTL_BROWN_OUT_REG, 0); //disable brownout detector   

// turn on the power
digitalWrite(powerControlSettings.powerControlOutputPin, HIGH);

// let things power up
delay(500);

WRITE_PERI_REG(RTC_CNTL_BROWN_OUT_REG, 1); //enable brownout detector

This is the code that does the deed. The power is controlled by a pin that is lifted high to turn it on. So the program turns of the brownout detector, turns the power on, waits half a second for things to settle and then turns the brownout detector back on again.

I’m not sure whether to be proud of this code or not. But in the end I think I am. If you accuse me of masking a hardware problem with a bit of software then I’m going to plead guilty as charged. I could probably fix the hardware cause by adding a wacking great capacitor over the supply to stop the voltage dropping. But this might lead to other problems. And anyway, if you look at the programs in any of the devices that you use on a daily basis you will find bits of code like this. Little bits of behaviour that are only there because the hardware doesn’t work without them.

Oh, and as a bonus (and so I can find it again if I ever need it) here’s a chunk of C that prints out the reset reason for an ESP32.

void printBootReason()
{
    esp_reset_reason_t reset_reason = esp_reset_reason();

    switch (reset_reason)
    {
    case ESP_RST_UNKNOWN:    Serial.println("Reset reason can not be determined"); break;
    case ESP_RST_POWERON:    Serial.println("Reset due to power-on event"); break;
    case ESP_RST_EXT:        Serial.println("Reset by external pin (not applicable for ESP32)"); break;
    case ESP_RST_SW:         Serial.println("Software reset via esp_restart"); break;
    case ESP_RST_PANIC:      Serial.println("Software reset due to exception/panic"); break;
    case ESP_RST_INT_WDT:    Serial.println("Reset (software or hardware) due to interrupt watchdog"); break;
    case ESP_RST_TASK_WDT:   Serial.println("Reset due to task watchdog"); break;
    case ESP_RST_WDT:        Serial.println("Reset due to other watchdogs"); break;
    case ESP_RST_DEEPSLEEP:  Serial.println("Reset after exiting deep sleep mode"); break;
    case ESP_RST_BROWNOUT:   Serial.println("Brownout reset (software or hardware)"); break;
    case ESP_RST_SDIO:       Serial.println("Reset over SDIO"); break;
    }

    if (reset_reason == ESP_RST_DEEPSLEEP)
    {
        esp_sleep_wakeup_cause_t wakeup_reason = esp_sleep_get_wakeup_cause();

        switch(wakeup_reason)
        { 
            case ESP_SLEEP_WAKEUP_UNDEFINED:    Serial.println("In case of deep sleep: reset was not caused by exit from deep sleep"); break;
            case ESP_SLEEP_WAKEUP_ALL:          Serial.println("Not a wakeup cause: used to disable all wakeup sources with esp_sleep_disable_wakeup_source"); break;
            case ESP_SLEEP_WAKEUP_EXT0:         Serial.println("Wakeup caused by external signal using RTC_IO"); break;
            case ESP_SLEEP_WAKEUP_EXT1:         Serial.println("Wakeup caused by external signal using RTC_CNTL"); break;
            case ESP_SLEEP_WAKEUP_TIMER:        Serial.println("Wakeup caused by timer"); break;
            case ESP_SLEEP_WAKEUP_TOUCHPAD:     Serial.println("Wakeup caused by touchpad"); break;
            case ESP_SLEEP_WAKEUP_ULP:          Serial.println("Wakeup caused by ULP program"); break;
            case ESP_SLEEP_WAKEUP_GPIO:         Serial.println("Wakeup caused by GPIO (light sleep only)"); break;
            case ESP_SLEEP_WAKEUP_UART:         Serial.println("Wakeup caused by UART (light sleep only)"); break;
        }
    }
}

You’re welcome.

LMIC Frame Counter Problem

This is a fairly exotic post, in that I doubt that many readers will be concerned how they can maintain the frame count of LoRa packets sent using the LMIC library when a device goes into deep sleep. However, I’m putting it here so that in six months time, when I probably need to do this again, I can just search the internets and find this article.

A LoRa application maintains a frame count for each connected device. This is a security measure. Any frames that that the application receives with a frame count that is lower than the highest frame count seen so far will be rejected by the application. You can turn off this frame count checking for a device, but it is not advised because it makes things a bit less secure. I’ve turned it off for my devices for now, which meant that the frame count just increases when the device is running and goes back to 0 if the device is reset.

This is kind of OK by me. I’m not overly concerned with security issues at this level. However, when I started using deep sleep mode I found that every frame that was sent had a frame counter of 0, and I really didn’t like that. The problem is caused by the way that the device is completely restarted between each transmission. This restarts the LMIC library each time, setting the frame count to 0 for every frame.

It turns out that a program can reset the LMIC frame count value by accessing a member of the LMIC object. All my program needs to do is create a variable in the RTC memory to hold the current frame count:

RTC_DATA_ATTR u4_t lmicSeqNumber = 0;

The program can then store the frame count in this variable after each LoRa message has been transmitted:

lmicSeqNumber = LMIC.seqnoUp;

Now, during the reset process after a deep sleep I can put the frame count back:

LMIC.seqnoUp = lmicSeqNumber;

I’ve tested this and it seems to work fine. The frame counter is retained when the device goes to sleep. Of course if I turn the device off the frame counter resets back to 0. I could fix this by storing the frame count in EEPROM storage but I’m not too keen on doing this because I’d have to update the value after each LoRa message and might lead to lots of writes which might “wear out” the EEPROM storage.

ESP32 Retaining timing over deep sleep

In the last post we saw how easy it is to make an ESP32 processor sleep for a particular time. However, we also noticed that at the end of the sleep the processor is reset and all the variables in the program are reset. This is difficult if you want to keep track of time in your application. However, it is possible to get around this limitation by storing a time value in memory that is retained during sleep.

RTC_DATA_ATTR unsigned long millisOffset=0;

The statement above declares a variable called millisOffset. The attribute RTC_DATA_ATTR tells the compiler that the variable should be stored in the real time clock data area. This is a small area of storage in the ESP32 processor that is part of the Real Time Clock. This means that the value will be set to zero when the ESP32 first powers up but will retain its value after a deep sleep.

We can use this variable to create an offsetMillis function that gives a millisecond count which is adjusted by the offset:

unsigned long offsetMillis()
{
    return millis() + millisOffset;
}

Our program is going to call this function when it wants to know how much time has elapsed since the application was started. The final part of the solution is to update the offset each time our processor is put to sleep.

void sleepSensor(unsigned long sleepMillis)
{
    esp_sleep_enable_timer_wakeup(sleepMillis * uS_TO_mS_FACTOR);

    millisOffset = offsetMillis() + sleepMillis;

    esp_deep_sleep_start();
}

Now, when our application goes to sleep it records the value that the millisecond timer should have when the device wakes from sleep. This value is retained in non-volatile memory and used to offset time values when the ESP32 restarts.

#include <Arduino.h>

RTC_DATA_ATTR unsigned long millisOffset = 0;

unsigned long offsetMillis()
{
    return millis() + millisOffset;
}

#define uS_TO_mS_FACTOR 1000  /* Conversion factor for micro seconds to miliseconds */

void sleepSensor(unsigned long sleepMillis)
{
    esp_sleep_enable_timer_wakeup(sleepMillis * uS_TO_mS_FACTOR);

    millisOffset = offsetMillis() + sleepMillis;

    esp_deep_sleep_start();
}

void printTime()
{
    unsigned long seconds, sec, min, hrs;

    seconds = offsetMillis() / 1000;

    sec = seconds % 60;
    seconds /= 60;
    min = seconds % 60;    
    seconds /= 60;
    hrs = seconds % 24;

    Serial.printf("%02d:%02d:%02d\n", hrs, min, sec);
}


void setup() {
    Serial.begin(115200);
    printTime();
    sleepSensor(30000);
}

void loop() {
}

The complete program above shows how it all fits together. The program sleeps the device for 30 seconds but the time value is maintained after each reset. Note that the loop function is empty because the program never gets this far. The repeating behaviour of the program is caused by the reset after each sleep.

I was quite surprised just how poor the time keeping is when I ran this program. This is because the timing is not provided by a crystal but by an oscillator which is fitted onto the ESP32 chip. The timing can be out by a second or so over short intervals. Try timing the above program to see what I mean.

The millis function in the Arduino library returns an unsigned long integer value that holds the number of milliseconds since a device was turned on. The program above works by adding an offset to this millis value which reflects how long the device has been put to sleep. Of course, what with data storage in any computer being a finite size, there will come a point where the millis value will not fit in an unsigned long variable and so it will wrap round and go back to 0. This occurs after 4,294,967,295 milliseconds or around 50 days. If your program does any kind of calculation with the timing values you will need to make sure that the wrap round doesn’t cause problems.

TTGO Camera

A while back I ordered a TTGO camera from Ali Express. It came today and it is lovely. It is similar to the ESP32 webcam that I bought a while back, but it has a PIR sensor and a BME 280 environmental sensor along with a nifty OLED display.

It is supplied with firmware that sets it up as an Access Point. If you connect to it you can view the image and control it from the web page that it serves out. I think it is running the same software as I loaded into the earlier device, but it is nice to have it on the device at the end.

It works extremely well. I’ve printed a little case for it from a design I found on Thingiverse and now I’m looking forward to having a play. They say it can do facial recognition, which is very interesting, and it might be a good device for our Humber Care Tech Challenge entry.

Update: I’ve found a nice blog post here on how to use this device as a fridge alarm. This shows how to put the processor to sleep and trigger wakeup using the PIR sensor.