Getting Started with CircuitPython

MicroPython

CircuitPython is a spin-off of the well-known MicroPython. An efficient implementation of Python 3, with a small subset of the Python Stdlib, optimised for microcontrollers (Cortex-based MCU, for example).

Even thought Python 3 is a powerful language, this implementation fits in 256KB of Flash and uses around 16KB of RAM. So it is suitable for the majority of the mid-range ARM-based microcontrollers.

The main difference between MicroPython and CircuitPython, is that the latest is sponsored and maintained by Adafruit. One of the global leaders developing boards and add-ons for microcontrollers’ enthusiasts. Apart from that:

  • It’s based on MicroPython (stable) releases, and not the master branch.
  • It’s implementation is focus on Adafruit products (basically Atmel’s M0, M4 devices and ESP8266).
  • Floats and Doubles are enabled by default.
  • USB is natively supported on all boards (with capable hardware).
  • Does not support concurrency.

CircuitPython comes pre-installed on many Adafruit boards, but if it’s not present, or you have moved away from it (going to Arduino), you can always installed a CircuitPython (or MicroPython) bootloader that can be downloaded from:

Take into consideration that there are specific builds for each kind of board / CPU. Moreover, some Adafruit libraries, are not supported on MicroPython, for example the Bus Device library.

The Integrated Development Environment

CircuitPython (and MicroPython too) comes with an integrated REPL (Read-Eval-Print-Loop) that can be accessed using the built-in USB-Serial port present in all devices. But, some of this microcontrollers present a USB Drive where we can download Python (and other kind of) files, so the board can used them.

Adafruit recommends using the Mu Editor, but I find it very limited and mainly oriented to beginners when it comes to programming languages. Most of use (developers) use Visual Studio code, and more sophisticated IDEs, with features like:

  • Integrated version control (Git)
  • Autocomplete
  • Linters
  • Automatic code formatter
  • Included Markdown editor / preview for documentation

That’s why using an IDE like Visual Studio Code makes more sense when writing an advanced application. An extension for VS Code can be found on its MarketPlace. It’s compatible with CircuitPython latests stable release (5.3.x), it’s Open Source, but it has a few issues.

Visual Studio Code with CircuitPython extension demonstrating demo autocomplete feature.
Board selection in Visual Studio Code with CircuitPython extension.

Feel free to start coding with Mu if you want to start with a more stable IDE, which also was design specifically for CircuitPython.

Mu Editor (with Dark mode).
Mu Editor has different modes based on the target platform.

Libraries and Examples

Take into consideration downloading and installing Adafruit’s CircuitPython libraries, which can be fount at https://circuitpython.org/libraries. Also, it’s always a good idea to start looking at some examples in order to get familiar with it. A bundle with a lot of examples can be download from https://github.com/adafruit/Adafruit_CircuitPython_Bundle.

Keeping Circuit Python library up-to-date

CircUp is a dependency management tool for CircuitPython developed by Adafruit. It has a very simple CLI that let’s you list, install, remove and even update, any dependency needed inside your device.

Moreover, the version of each installed library is saved in a requirements.txt file in the working directory, so it’s really simple to keep dependencies organised and versiones on Git.

Installation is done using pip, either directly or using virtualenv. Python 3.5 or higher is requiered.

pip3 install --user circup
Usage: circup [OPTIONS] COMMAND [ARGS]...
  A tool to manage and update libraries on a CircuitPython device.
Options:
  --verbose  Comprehensive logging is sent to stdout.
  --version  Show the version and exit.
  --help     Show this message and exit.
Commands:
  freeze     Output details of all the modules found on the connected...
  install    Install a named module onto the device.
  list       Lists all out of date modules found on the connected...
  show       Show a list of available modules in the bundle.
  uninstall  Uninstall a named module(s) from the connected device.
  update     Update modules on the device. Use --all to automatically update
             all modules.

ESP32: Using the Internal Flash File System

ESP32 File System

The ESP32 contains a lightweight filesystem specially design for microcontrollers called Serial Peripheral Interface Flash File System. This File System, also known as SPIFFS for its initials, is implemented on top of a Flash chip connected directly to the SPI bus.

SPIFFS supports basic operations on files, like opening, close, read and write, and a very rudimentary implementation of directories. You will find that several project uses this File System in order to serve pages (and files) in a Web Server.

Requirements

First of all we need to install a few dependencies. The most important is the File System uploader tool, which can be used from the Arduino IDE, Visual Studio Code, or command line. The steps for performing this installation can be found at project repository on GitHub.

If you are using Visual Studio Code, as I do, install the following extension: ESP8266FS. From the command line:

code --install-extension vscode-esp8266fs

Do not forget to install Arduino-ESP32 Core https://github.com/espressif/arduino-esp32#installation-instructions

Project configuration

When using the Arduino IDE, just selecting the board in the Board Manager will do the job. But when using Visual Studio Code, we have to set some values on .vscode/arduino.json.

{
    "sketch": "main.ino",
    "board": "esp32:esp32:featheresp32",
    "configuration": "FlashFreq=80m,UploadSpeed=921600,DebugLevel=none,PartitionScheme=default,FlashSize=4MB,FlashMode=keep",
    "port": "/dev/tty.SLAB_USBtoUART",
    "output": "output"
}

With this configuration 1M of Flash will be used for SPIFFS and the rest (3M out of 4M) for your code.

Testing the plugin configuration

In Arduino IDE using the Data Uploader is straightforward. But on Visual Studio Code the plugin should be configured carefully in order to work. The following is my Visual Studio Code configuration (settings.json) which is quite verbose for testing purposes.

{
   ...
    "esp8266fs.espota.debug": true,
    "esp8266fs.esptool.executable": "/Users/kimi/Documents/Arduino/hardware/espressif/esp32/tools/esptool.py",
    "esp8266fs.esptool.verbosity": "vvv",
    "esp8266fs.logLevel": "debug",
    "esp8266fs.mkspiffs.executable": "/Users/kimi/Documents/Arduino/hardware/espressif/esp32/tools/mkspiffs/mkspiffs",
    "esp8266fs.preferencesPath": "/Users/kimi/Library/Arduino15",
    "esp8266fs.arduinoUserPath": "/Users/kimi/Documents/Arduino",
    "esp8266fs.spiffsImage": "./spiffs.image",
    "esptool.py.spi_connection": "SPI",
    "esptool.py.compress": "true",
    "esptool.py.verify": "false",
   ...
}
ESP32 has not been installed correctly

Testing the Data Upload can be done by List SPIFFS command. cmd + shift P and select ESP8266FS: List SPIFFS. If you get an error saying

ESP32 has not been installed correctly – see https://github.com/espressif/arduino-esp32

Take into consideration that the Path of the ESP32 Tool is generated by splitting the Board key on .vscode/arduino.json in order to get three parameters: Package, Architecture and Board (ref: https://github.com/kash4kev/vscode-esp8266fs/…/esp8266fs.js#L467).

As my board is an Adafruit HUZZAH32, and my board config is

{
   ...
   "board": "esp32:esp32:featheresp32",
   ...
}

the three key parameters will be:

{
   "package": "esp32",
   "architecture": "esp32",
   "board": "featheresp32",
}

This parameters will be used by the ESP8266FS plugin to create the path where ESP tools are installed. In my setup it will result in /Users/kimi/Documents/Arduino/hardware/esp32/esp32. But, based on the ESP32 Arduino core tutorial, my hardware definition is installed on /Users/kimi/Documents/Arduino/hardware/espressif/esp32.

My workaround to this issue was to create a symbolic link so the esp32 directory was points to espressif.

ln -s espressif esp32
Couldn’t find match for argument

Another common error is Couldn’t find match for argument when running mkspiffs, which is a tool found in the ESP32 Arduino Core SDK that handles the creation, upload and download (among other responsibilities) of images.

This issue is already reported on GitHub at https://github.com/kash4kev/vscode-esp8266fs/issues/15. The workaround as explain there, is to disable the configuration esp8266fs.mkspiffs.allFiles.

Packing information into an Image

In your project directory, next to your main Arduino file, create a directory called data. All files stored in that folder will be transfer to your ESP32 microcontroller when uploading the File System.

All our data files are not transferred one by one to the ESP32, but inside an Image. In order to create that image we should pack those files using mkspiffs. The ESP8266FS extension has a command for doing that, but be can also use the command line tool.

The benefits of using the command line tool are clear, the operation can be automated inside a script.

/<Path To Arduino>/hardware/espressif/esp32/tools/mkspiffs/mkspiffs --create /<Your Arduino Project Directory>/data --size 0x30000 --page 256 --block 4096 /<Your Arduino Project Directory>/spiffs.image

Uploading files

Remember that uploading files from your Computer will replace the files present in your ESP32.

You will also find a handy command in the Visual Studio Code extension for uploading your currently created image. But if you want to do that using the command line, the following execution will do the trick.

/<Path To Arduino>/hardware/espressif/esp32/tools/esptool/esptool -vvv -ca 0x3D0000 -cd  -cp /dev/tty.SLAB_USBtoUART -cb 921600 -cf /<Your Arduino Project Directory>/spiffs.image

Where /dev/tty.SLAB_USBtoUART is the USB Serial port where I have my ESP32 connected. The result of running this command may look like

--- Uploading SPIFFS file with esptool.py ---
Python Executable: "python"
SPIFFS Uploading Image... (/Users/kimi/Works/eaceto.dev/embedded/ESP32/P1/spiffs.image)
  [SPIFFS] Python   : python
  [SPIFFS] EspTool  : /Users/kimi/Documents/Arduino/hardware/espressif/esp32/tools/esptool.py
  [SPIFFS] address  : 0x290000
  [SPIFFS] port     : /dev/tty.SLAB_USBtoUART
  [SPIFFS] speed    : 921600
  [SPIFFS] before   : default_reset
  [SPIFFS] after    : hard_reset
  [SPIFFS] flashMode: keep
  [SPIFFS] flashFreq: 80m
  [SPIFFS] flashSize: 4MB
Running: python /Users/kimi/Documents/Arduino/hardware/espressif/esp32/tools/esptool.py --chip esp32 --baud 921600 --port /dev/tty.SLAB_USBtoUART --before default_reset --after hard_reset write_flash --flash_mode keep --flash_freq 80m --flash_size 4MB 0x290000 /Users/kimi/Works/eaceto.dev/embedded/ESP32/P1/spiffs.image
esptool.py v2.8
Serial port /dev/tty.SLAB_USBtoUART
Connecting....
Chip is ESP32D0WDQ6 (revision 1)
Features: WiFi, BT, Dual Core, Coding Scheme None
Crystal is 40MHz
MAC: 24:0a:c4:0c:94:78
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 921600
Changed.
Configuring flash size...
Compressed 196608 bytes to 537...
Writing at 0x00290000... (100 %)
Wrote 196608 bytes (537 compressed) at 0x00290000 in 0.0 seconds (effective 136297.5 kbit/s)...
Hash of data verified.
Leaving...
Hard resetting via RTS pin...
ESP8266 SPIFFS interface finished.

Verifying what it’s inside an image

If you want to check which files are included in an image file, you can list them by running

<Path To Arduino>/hardware/espressif/esp32/tools/mkspiffs/mkspiffs --list --page 256 --block 4096 <Path to>/spiffs.image

Downloading an image form the ESP32 to your Computer

May be one of the most important steps when writing an image to the ESP32 is verifying that it was correctly done. For doing this you have to find how many bytes where written the uploading the image to the ESP32. This information can be found on the output of the write_flash operation. And as important as how many it is, where. This information it is also present in the same debug output.

In my previous example this two values where 196608 bytes and starting address 0x290000. So the command to execute in order to read that segment of Flash is:

<Path To Arduino>/hardware/espressif/esp32/tools/esptool.py --chip esp32 --baud 921600 --port /dev/tty.SLAB_USBtoUART read_flash  0x290000 196608 downloaded.image

Output

iFi, BT, Dual Core, Coding Scheme None
Crystal is 40MHz
MAC: 24:0a:c4:0c:94:78
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 921600
Changed.
196608 (100 %)
196608 (100 %)
Read 196608 bytes at 0x290000 in 2.5 seconds (641.6 kbit/s)...
Hard resetting via RTS pin...

Using the command explained in Verifying what’s inside the image we can see which files are inside the image

<Path To Arduino>/hardware/esp32/esp32/tools/mkspiffs/mkspiffs --list --page 256 --block 4096 downloaded.image

Output

159 /config_test.json

Next steps…

In the following post I will show how to read a JSON file and obtain a configuration parameter that will be use to configure the WiFi network to which our ESP32 will be connected.

Stay tuned!

Rollback from MicroPython to Arduino on ESP32

Arduino logo

I hope you find MicroPython is VERY useful and perfect for you. But just in case it doesn’t, or for any reason you want to rollback to Arduino. Here is how to do it.

Requisites

ESPTool (already installed if you went from Arduino to MicroPython)

pip3 install esptool

If there is a permissions problem doing this, try to run pip3 install either by using “sudo” or “–user” flag.

Steps

The following example assumes the USB port can be found at /dev/cu.SLAB_USBtoUART, but you may find it at /dev/ttyUSB0

  1. Erase your ESP32 Flash.
$ esptool.py --port /dev/cu.SLAB_USBtoUART erase_flash
esptool.py v2.8
Serial port /dev/cu.SLAB_USBtoUART
Connecting........__
Detecting chip type... ESP32
Chip is ESP32D0WDQ6 (revision 1)
Features: WiFi, BT, Dual Core, Coding Scheme None
Crystal is 40MHz
MAC: 24:0a:c4:0c:94:78
Uploading stub...
Running stub...
Stub running...
Erasing flash (this may take a while)...
Chip erase completed successfully in 4.1s
Hard resetting via RTS pin...

  2. Thanks it! You can now go back to Arduino IDE and build and run programs on your ESP32 board!

Another useful ESPTool’s sub command

ESPTool let us retrieve the MAC address of the device from our host. This can be used for several things, like for example creating specific setup files. In order to read the ESP32 MAC address, run ESPTool with the subcommand “read_mac” as shown below

$ esptool.py --port /dev/cu.SLAB_USBtoUART read_mac
esptool.py v2.8
Serial port /dev/cu.SLAB_USBtoUART
Connecting........__
Detecting chip type... ESP32
Chip is ESP32D0WDQ6 (revision 1)
Features: WiFi, BT, Dual Core, Coding Scheme None
Crystal is 40MHz
MAC: 24:0a:c4:9d:24:28
Uploading stub...
Running stub...
Stub running...
MAC: 24:0a:c4:9d:24:28
Hard resetting via RTS pin...

Setting up ESP32 development environment in Linux (with Arduino IDE)

Arduino logo

Installing the IDE

Arduino logo

Download Arduino IDE from https://www.arduino.cc/en/Main/Software. The latest version when writing this a article is 1.8.12 (https://www.arduino.cc/download_handler.php?f=/arduino-1.8.12-linux64.tar.xz).

Extract the content of the tar.xz file where you want to have your Arduino IDE, and then, execute install.sh. If there is an error, like:

$ ./install.sh
Adding desktop shortcut, menu item and file associations for Arduino IDE...
ln: failed to create symbolic link '/usr/local/bin/arduino': Permission denied
Adding s

try to re run the script as root

$ sudo ./install.sh 

Installing the USB drivers

There are mainly two types of USB chips that may be used as Serial to USB converters, one from FTDI and the other one from SiLabs. In order to find out which chip your ESP32 board has, disconnect the board from your PC and run lsusb.

$ lsusb
Bus 002 Device 002: ID 0bda:0411 Realtek Semiconductor Corp. 4-Port USB 3.0 Hub
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 002: ID 046d:c534 Logitech, Inc. Unifying Receiver
Bus 001 Device 010: ID 0bda:5411 Realtek Semiconductor Corp. 4-Port USB 2.0 Hub
Bus 001 Device 004: ID 8087:0aaa Intel Corp.
Bus 001 Device 003: ID 048d:8297 Integrated Technology Express, Inc. ITE Device(8595)
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Then, connect the board and repeat the command.

$ lsusb
Bus 002 Device 002: ID 0bda:0411 Realtek Semiconductor Corp. 4-Port USB 3.0 Hub
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 002: ID 046d:c534 Logitech, Inc. Unifying Receiver
Bus 001 Device 012: ID 10c4:ea60 Cygnal Integrated Products, Inc. CP2102/CP2109 UART Bridge Controller [CP210x family]
Bus 001 Device 010: ID 0bda:5411 Realtek Semiconductor Corp. 4-Port USB 2.0 Hub
Bus 001 Device 004: ID 8087:0aaa Intel Corp.
Bus 001 Device 003: ID 048d:8297 Integrated Technology Express, Inc. ITE Device(8595)
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

The new device, in this case a CP210x (Cygnal Integrated CP2102 / CP2109 UART Bridge Controller [CP210x family], is your ESP32 board.

Bus 001 Device 012: ID 10c4:ea60 Cygnal Integrated Products, Inc. CP2102/CP2109 UART Bridge Controller [CP210x family]

Install USB driver for your board.

Adding ESP32 Boards to Arduino IDE

Arduino IDE does not support ESP32 boards (and many other 3rd party boards) by default. But adding an additional board to Arduino IDE, in this case, an ESP32 board is as easy as:

  • Start Arduino IDE and open Preferences window
  • In Additional Board Manager URLs, add the following link:
https://raw.githubusercontent.com/espressif/arduino-esp32/gh-pages/package_esp32_index.json
  • You can add several board definitions, separating URLs with comma
  • Open Boards Manager from Tools; Boards; and install the ESP32 platform.
  • After installation, select your ESP32 board from Tools; Board menu.