SD Cards and Reliability

One of the questions that came up at the BYO Datalogger Welcome meetup is regarding SD card reliability. Since it was a bit of a detailed topic, we decided to move it to the forums. 

Micro SD cards are used by the Wildlogger as a storage medium and there was a question that was related to the reliability of SD card storage. It's actually quite relevant so here's an attempt at an explanation. 

SD cards are based on flash memory which, from a reliability standpoint, means they have two weak points. The first is that there's a fixed amount of times they can be written to and erased. That's usually around 100,000 times per location. This might sound like a lot, but there was a pretty consistent issue on the Raspberry Pi where SD cards would get corrupted frequently.

This actually turned out to be because the Raspberry Pi (rPi) used the SD card similarly to how a PC uses the hard drive and the operating system would regularly write and delete log files. It became enough of a problem that now there are well known solutions to disable log file writing on the rPi. Note that there's nothing wrong with using the Raspberry Pi, but you might want to keep this in mind, in case you need a high reliability application. For more of an explanation on the specific issue and it's fix, you can find it here.

There are also specialty SD cards that have additional hardware that distribute accesses to the SD card so that it minimizes writes to the same location and spreads out the wear on the card. These are called "wear-leveling" SD cards and are mainly used in high reliability applications. FYI, wear leveling is also a standard feature on solid state drives since there's a potential that there will be many read and write accesses to them. For more info on wear leveling SD cards, you can find it here.

As for our application, I don't foresee SD card wear as a major problem in most applications. By the way, these are also "famous last words". But if you use it in a way where you deploy it for some time in the field, then retrieve the data from it, the SD card should last on average for 100,000 deployments. If each deployment lasts one day for data collection, that would be 273 years. I find that my microSD cards rarely last longer than one year, but that's because they're so small, I normally misplace them which is the second weak point (and a strong point). For more information on SD card lifespan, you can find it here.

There are other things that can cause SD card corruption though. These can be software bugs that interrupt SD card writing or stop them prematurely, a sudden power loss or glitch during a write operation, electrostatic discharge (you shock the SD card), an overvoltage event, wetness on the circuit board (ie: high humidity or morning dew), etc. These are mostly field deployment issues that would hopefully be uncovered in piloting the design, which we'll be covering in a later module. 

 One of the main drawbacks of using an SD card is that in a low power system, ie: you're hoping to run for 6 months to 1 year on a single charge, it's often the highest current consumer. This is true for the Wildlogger board where even using a microSD card with good power numbers increases the current consumption by about 300 uA. Using a cheap Chinese microSD card increases it by almost 1 mA.

As a rule of thumb, we generally try to get our low power systems under 1 mA in sleep mode and typically around 200 uA (0.2 mA). At 1 mA in sleep mode, this translates to a theoretical maximum battery life of 83 days using a standard 2000 mA-hr alkaline battery. A realistic battery life would be around 40 days at 1 mA. This is why the microSD card starts becoming the limiting factor in a low power system. If what I just said sounds like Martian to you, I recommend checking out Module 1 where Jacinta discusses all the terminology and how we calculate theoretical and realistic battery life. For more info on microSD cards and current consumption, you can find it here

Note: their microSD card current consumption numbers are very low because they're measuring just the current consumed by the microSD card. We measure our current consumption at the battery or directly from the energy source. The current that gets to the battery goes through power regulation and other circuitry before getting to the microSD card which is why I quote higher numbers. Also I couldn't get exact microSD matches to the ones they were using although I did test with a Lexar, Sandisk, as well as cheapie ones from China. 

Anyways, hope that explains a bit about SD cards and reliability (and power consumption). Feel free to comment or add questions below. 

Akiba 

 




Gracias Akiba, el tema de las tarjetas SD, en cuanto a fiabilidad y consumo de energia, es fundamental para el diseño de un datalogger

Muy util tu explicacion y muy interesantes los enlaces que adjuntas, sobre todo el estudio sobre consumo de energia de distintas marcas de SD

Saludos cordiales !

RE: Remote reading of SD cards?

Hi Akiba,

Just a general question if you (or anyone else), has had experience with WiFi Enabled SD Memory Cards? As I understand it these can be read remotely (via Bluetooth/ Wifi), not sure if they can be written over remotely as well?

I appreciate, you’d need Bluetooth breakout board/ more power, etc – I’m just thinking for those applications where this wouldn’t be an issue.  A step back from a radio-linked sensor network.   I’m thinking if the data-logger was high up in tree & you were standing within range on the ground (didn’t want to climb up the tree/ disturb it) and wanted to read the full card (capture the data) & then wipe the card clean, so it can record more data.  In the same way you can transfer photos from some DSLR cameras to your phone.  Just wondered if something like this would be possible?

Many thanks

Hello fellow person with a great name, and Akiba (which is also a fantastic name),

@Alasdair has much more information than I do re: wireless SD cards but if my failing memory serves, a major problem with them is they need to be fully powered in order to be able to transfer data (and Akiba's point about what kinds of data is a whole other problem), whereas most SD card holders will be powered down after writing. I think there are some potential hacks that can keep cards active, but then power might become an issue. We are currently exploring some ideas for circumventing the very same problem (how to get data from devices like camera traps that are high off the ground). One ready-made solution for camera traps at least could be this: https://www.spypoint.com/en/products/universal-cellular-adapter/product-cell-link.html...although it's not ideal for us (we don't want to pay any cell fees). Bluetooth-enabled camera traps are actually fairly rare birds, although I've heard about a new release...but I digress (as always). 

And the one-and-only Shah Selbe may have more info on an upcoming SD card solution. 

Alternatively, years ago we had some success making a proof-of-concept hack for our own loggers using serial-capable radios spliced into our UART. We used 3DR radios, but something like these could be a replacement: https://www.sparkfun.com/products/17255...again, powering and scheduling could be an issue, as this was just to send data as it was being written. 

Akiba, we really need to talk about Bluetooth comms and making a hotspot just as you've suggested! We should set up a chat when you are able.

Sorry this was of almost no help, and may have actually only made things worse...

Rob