Learning MQTT

I’ve just started looking at a TinyML project I want to do. The first thing I need to do is get some sensor data off my board.

I thought about just bunging it over a uart. In some ways it would be the quickest easiest way to go about things. However, this thing is likely to evolve into something a bit more structured over time, with several different data types, which would necessitate building packets and coming up with some sort of protocol. That’s beginning to sound a bit too much like hard work.

Next thing I considered was sending some JSON. That would make the protocol rather easier, as I could just create some JSON on the microcontroller send it over to the host and use any number of JSON parsers to extract the data.

But, if I’m using JSON I should really send it over HTTP. The board has ethernet and I can get an HTTP server up and running without too trouble.

However, I’ve seen plenty of people talking about MQTT in reference to sensor data. I’ve never used it before. No time like the present!

I’ve got mosquitto installed locally. I’ve managed to setup two subscribers. I’ve published some mock data. I’ve played with wildcards. All in the cli on my PC so far, but I only started a bit before lunch so early days yet. So far it shows promise.

I’m not sure if some sort of hybrid approach where I use MQTT for notifications and HTTP for the actual data (which later down the line may be audiovisual) would be a good idea. I’m not sure what the strengths/weaknesses of MQTT are yet. I know HTTP can do all this without too much bother.

Also, not sure about cloud services for MQTT yet. The microcontroller I’m using doesn’t have hardware crypto, so TLS might be a bit pants. If I do everything locally, I don’t need to bother so much about security. The board I’m using has a ton of other gear that looks useful though. I can’t find another board that has equivalent sensors/connectors for other sensors, has crypto support and is a sane price though.

I think for now, keeping it local will suit my purposes. There’s plenty I don’t know how to do yet, so adding another layer of things I don’t know will just add burden and stress. Maybe I’ll do something more cloudy later.

I think now though its time for a walk in the sun!

Using SD cards with SPI

Having just spent a while working with SD cards over SPI and an STM32 microcontroller, I made various discoveries, which seem worth documenting, since a lot of it I didn’t find anyone else talking about and had to figure out for myself.

  1. An SD card connector is a purely passive device. All SPI hardware resides on the SD card itself. This means if there is no card inserted, there is no SPI device to communicate with. As such MISO should be connected to a pull up or pull down resistor so it isn’t left floating. Almost all SPI breakout boards from that I found already have pull up resistors fitted.
  2. In theory Chip Select (CS) can be used for card detection. On the microcontroller the GPIO can be configured as open drain. There is meant to be a pull up resistor on the SD card which will pull the line high when inserted, and should go low when ejected. However, as the previous paragraph stated, there’s often pull up resistors on breakout boards which prevent this from working. Further, a lot of boards use a level shifter, which pulls the line low quite hard and completely isolates the pull up resistor on the SD card. Thus, this mechanism is often unusable. If using such a board, CS must be configured as push/pull.
  3. On the connector itself there is often a dedicated Card Detect line. This is a mechanical switch on the connector. Often it isn’t electronically connected to anything, so also potentially is unusable.
  4. The SD card spec supports both SDIO and SPI. When the card is first inserted it is in SDIO mode until SPI mode is negotiated. That means when you first insert an SD card in the connector you’re putting a non SPI device on the SPI bus. SDIO uses the same pins as SPI and in many ways is a very similar protocol with similar commands. When in SDIO mode, CS is ignored, unless pulled low for cmd0. That’s rather fun isn’t it?
  5. An SD card can be inserted at any time, no matter what else might be going on on the SPI bus. If another device is on the bus, the SD card while in SDIO mode (therefore ignoring CS), can start receiving what it thinks are commands and start acting on them. The SD card will have very, very little idea about byte boundaries if inserted in the middle of a SPI transaction. What it receives and chooses to interpret could be anything.
  6. While I haven’t done a thorough survey and so can’t prove anything, it seems like SD cards are mostly used in SDIO mode. SPI mode isn’t used much outside microcontrollers (and even then there are microcontrollers which support SDIO). As such, it seems like SPI mode isn’t particularly widely used, tested or supported. Outside pulling CS low during cmd0, I have a suspicion CS gets ignored. This would mean if there are other devices on the SPI bus, the SD card will suck in all data from the bus. Good luck!
  7. While soldered down SPI devices can be planned and designed for, an SD card could be any old thing the user decides to plug in. It could be the most reliable device ever, or a pile of junk barely conforming to the spec (or even not supporting SPI at all, and only vaguely working out of chance due to aforementioned similarities between SPI and SDIO).
  8. From a brief glance at the SDIO portion of the spec, it looks like SDIO sends responses on what would be MOSI if it were in SPI mode. It would therefore potentially act as some sort of horrid messed up secondary master on the SPI bus, interfering with legitimate operations. I haven’t investigated this at all, so take with a grain of salt.
  9. Just put the SD card on a bus by itself. Avoid pain and suffering.
  10. When calling HAL_SPI_Receive, the buffer contents passed in get transmitted to the bus unaltered. Always initialise the buffer first. When talking to SD cards, 0xFF seems like an appropriate value.

I hope this information is some day found by some weary traveler wondering why things aren’t working well.