Intro

Thermal Cameras are neat tools, especially in electronics: You can quickly diagnose short circuits, identify misbehaving components or quantify if subsystems get too hot. However, thermal cameras are still a luxury item which can quickly costs hunderds of euros. Sometimes - however - you do not need the accurarcy of a FLIR or similar system, somtimes its just enough to see which part of a PCB is heating up quicker than others. Enter MLX90640: A 32x24 pixel sensor in two different FoVs, exposing its data via i2c. These sensors can be had on different websites for as low as 25 euros during sale.

Components/BoM

• 1x MLX90640 Thermal Sensor
• 1x Waveshare RP2040 Zero
• 1x HT7333 LDO
• 2x SMD Capacitor 10 uF
• 2x SMD Resistor 2.2k Ohm

All in all about 30 Euros

Schematic

Nothing really surprising - a HT7333 with some filter caps to allow for a clean power source, two pull-up resistors to allow for the high-speed, 1 MHz communication via the i2c bus. Keep the i2c wires between RP2040 Zero and the sensor as short as possible and pull-up the i2c SDA/SCL lines directly at the sensor.

Operation modes

There are two operation modes here, you can either choose to use the sensor with the Adafruit and Sparkfun example, which will just write a (very long) ASCII line consiting of 768 comma seperated float values per frame, delimited by \n. This will also work with a Processing example to showcase the picture (about 4 FPS).
The alternative to that, would be using another approach, packing the 768 temperature values not in a data intensive ASCII line, but send them via SLIP protocol to the computer. The processing example is not available for that, but I got some other Python code in the making (about 7.5 FPS).

Serial

Firmware (Serial)

Programmed via Arduino using the Arduino-Pico framework, using the Adafruit_MLX90640 library ( https://github.com/adafruit/Adafruit_MLX90640 ) - please be aware that 1 MHz I2C operations and mlx.setRefreshRate(MLX90640_32_HZ) will only work with extremely short wires!

#include 

Adafruit_MLX90640 mlx;
float frame[32*24]; // buffer for full frame of temperatures

// uncomment *one* of the below
#define PRINT_TEMPERATURES
//#define PRINT_ASCIIART

void setup() {
  while (!Serial) delay(10);
  Serial.begin(115200);
  delay(100);

  //Serial.println("Adafruit MLX90640 Simple Test");
  if (! mlx.begin(MLX90640_I2CADDR_DEFAULT, &Wire)) {
    Serial.println("MLX90640 not found!");
    while (1) delay(10);
  }
  /*
  Serial.println("Found Adafruit MLX90640");

  Serial.print("Serial number: ");
  Serial.print(mlx.serialNumber[0], HEX);
  Serial.print(mlx.serialNumber[1], HEX);
  Serial.println(mlx.serialNumber[2], HEX);
  */

  //mlx.setMode(MLX90640_INTERLEAVED);
  mlx.setMode(MLX90640_CHESS);
/*
  Serial.print("Current mode: ");
  if (mlx.getMode() == MLX90640_CHESS) {
    Serial.println("Chess");
  } else {
    Serial.println("Interleave");    
  }
*/

//  mlx.setResolution(MLX90640_ADC_18BIT); //default
  mlx.setResolution(MLX90640_ADC_19BIT);
  /*Serial.print("Current resolution: ");
  mlx90640_resolution_t res = mlx.getResolution();
  switch (res) {
    case MLX90640_ADC_16BIT: Serial.println("16 bit"); break;
    case MLX90640_ADC_17BIT: Serial.println("17 bit"); break;
    case MLX90640_ADC_18BIT: Serial.println("18 bit"); break;
    case MLX90640_ADC_19BIT: Serial.println("19 bit"); break;
  }
*/

  //mlx.setRefreshRate(MLX90640_2_HZ);
  //mlx.setRefreshRate(MLX90640_8_HZ);
  mlx.setRefreshRate(MLX90640_32_HZ);

  /*Serial.print("Current frame rate: ");
  mlx90640_refreshrate_t rate = mlx.getRefreshRate();
  switch (rate) {
    case MLX90640_0_5_HZ: Serial.println("0.5 Hz"); break;
    case MLX90640_1_HZ: Serial.println("1 Hz"); break; 
    case MLX90640_2_HZ: Serial.println("2 Hz"); break;
    case MLX90640_4_HZ: Serial.println("4 Hz"); break;
    case MLX90640_8_HZ: Serial.println("8 Hz"); break;
    case MLX90640_16_HZ: Serial.println("16 Hz"); break;
    case MLX90640_32_HZ: Serial.println("32 Hz"); break;
    case MLX90640_64_HZ: Serial.println("64 Hz"); break;
  }
  */

  //Wire.setClock(400000); // max 1 MHz
Wire.setClock(1000000); // max 1 MHz
//  Wire.setClock(3400000); // max 1 MHz

}

void loop() {
  //delay(500);

  if (mlx.getFrame(frame) != 0) {
    //Serial.println("Failed");
    return;
  }

  //Serial.println();
  //Serial.println();
  for (uint8_t h=0; h<24; h++) {
    for (uint8_t w=0; w<32; w++) {
      float t = frame[h*32 + w];
#ifdef PRINT_TEMPERATURES
      Serial.print(t, 1);
      Serial.print(",");
#endif
#ifdef PRINT_ASCIIART
      char c = '&';
      if (t < 20) c = ' ';
      else if (t < 23) c = '.';
      else if (t < 25) c = '-';
      else if (t < 27) c = '*';
      else if (t < 29) c = '+';
      else if (t < 31) c = 'x';
      else if (t < 33) c = '%';
      else if (t < 35) c = '#';
      else if (t < 37) c = 'X';
      Serial.print(c);
#endif
    }
    //Serial.println();
  }
  Serial.println();
  //Serial.write('\n'); //10
}

Software (Serial)

To read the data from the sensor, the SparkFun Processing example ( https://learn.sparkfun.com/tutorials/qwiic-ir-array-mlx90640-hookup-guide/all#example-code ) can be used.
Alternatively, you can use Python 3.x and pyserial, giving back a list of thermal readings which can then be drawn into pictures.

import serial
import re
ser = serial.Serial('COM12', 115200, timeout=2)
while True:
    line = ser.readline().decode('ascii')   # read a '\n' terminated line
    thermalData = re.split('\,', line[:-3]) # last ,\n needs to go!
    print(thermalData)

SLIP

Firmware (SLIP)

Programmed via Arduino using the Arduino-Pico framework, using the Adafruit_MLX90640 library ( https://github.com/adafruit/Adafruit_MLX90640 ) as well as PacketSerial ( https://github.com/bakercp/PacketSerial ) - please be aware that 1 MHz I2C operations and mlx.setRefreshRate(MLX90640_32_HZ) will only work with extremely short wires!

#include 
SLIPPacketSerial packetSerial;

#include 
Adafruit_MLX90640 mlx;
float frame[32*24]; // buffer for full frame of temperatures

void setup() {
  while (!Serial) delay(10);
  Serial.begin(460800);
  Serial.setTimeout(100);
  packetSerial.setStream(&Serial); 
  delay(100);

  if (! mlx.begin(MLX90640_I2CADDR_DEFAULT, &Wire)) {
    //Serial.println("MLX90640 not found!");
    while (1) delay(10);
  }
  /*
  Serial.println("Found Adafruit MLX90640");
  Serial.print("Serial number: ");
  Serial.print(mlx.serialNumber[0], HEX);
  Serial.print(mlx.serialNumber[1], HEX);
  Serial.println(mlx.serialNumber[2], HEX);
  */

  //mlx.setMode(MLX90640_INTERLEAVED);
  mlx.setMode(MLX90640_CHESS);

//  mlx.setResolution(MLX90640_ADC_18BIT); //default
  mlx.setResolution(MLX90640_ADC_19BIT);

  //mlx.setRefreshRate(MLX90640_2_HZ);
  //mlx.setRefreshRate(MLX90640_8_HZ);
  mlx.setRefreshRate(MLX90640_32_HZ);

  //Wire.setClock(400000);
  Wire.setClock(1000000); // max 1 MHz
}

void loop() {
  packetSerial.update();
  if (mlx.getFrame(frame) != 0) {
    //Serial.println("Failed");
    return;
  } else {
    packetSerial.send((uint8_t*)&frame, sizeof(frame));   
  }
}

Software (SLIP)

Using Python 3.x, pyserial and sliplib, giving back a list of thermal readings which can then be drawn into pictures

import serial
import sliplib
import time
import struct
ser = serial.Serial('COM12', 115200, timeout=2)
while True:
    bytesToRead = ser.inWaiting()
    if (bytesToRead!=0):
        readData = ser.read(bytesToRead)
        decodedData = sliplib.decode(readData)
        if decodedData != b'':
            thermalData = []
            for i in range(0, len(decodedData), 4):
                thermalData.append(struct.unpack('f', decodedData[i:i+4])[0])
            print(thermalData)
            # ca. 7.41 Frames raw
    else:
        time.sleep(0.07)

Finishing up

After connecting everything, I finished the build up by printing a small PLA housing and even printed a TPU cap for the lense itself, so that (via friction fit) the sensor could be just thrown into the toolbox and still would be protected from dust and other issues. Could the finish be gotten more pretty? Yes. Did this hinder its function? Not at all, it already helped me fix up a defective turingPi2 🙂 (maybe something for the next post).

Optimisation

As usual in engineering, some problems can be solved by throwing money at them. In this case, a Teensy 4.x as a processor could significantly increase the data throughput. However, I did not want to pair a 40 Euro MCU with a 25 Euro Sensor - be it for costs and footprints sake - and opted in for the 2.5 Euro RP2040 Zero. Still, performance can be improved over my first optimisation using SLIP - by just using the second core of the RP2040. Ideally you would multi-thread the application, having the first core always retriving new data, while the second core packs it into SLIP packets and sends them on their way. This should be fairly easy and might result in more fps :).

Photos