discussion / Camera Traps  / 29 November 2023

Automatic extraction of temperature/moon phase from camera trap video

Hey everyone, 

I'm currently trying to automate the annotation process for some camera trap videos by extracting metadata from the files (mp4 format). I've been tasked to try and automatically extract the ambient temperature and moon phase that are shown on the videos and I've tried several tools (the automatic metadata field mapping in Timelapse and the commandline exiftools with the extract embedded option --ee as well) but sadly no success yet, so I've decided to tap into some community wisdom for help. :)

Are there any other tools that I could try out with the videos or some other place I could check in case the tags are stored in a different way? I'm relatively new to camera trap files, but I assume if the data is shown on the video, then it must've been stored somewhere. I'm not sure where to look, so any help would be really appreciated!

Of course, it's always possible that the data is just not there and there's no easy way to extract it (I've read in a stackoverflow thread somewhere that mp4 tends to lose some metadata, or maybe the camera trap itself might not have been set up to store that information as metadata too). 

The option to build (or reuse?) a CV model that reads the temperature text and moon phase symbol from the videos is possible to do as a last resort, but if a more painless and time efficient way exists, it would definitely be the preferred approach here.

Thanks for the help in advance!

Lucy




Hello Lyuboslava,

As you said, the first thing that came to mind was running a CV program to read the needed data. If the data does not change in the duration of the footage, then, in my humble opinion, it would be easiest to take a single frame from each clip and run an optical character recognition program to get the temperature.

The moon phase might pose a bit more of a challenge if the meta data truly is hidden. A different CNN program that is trained on the moon phases should suffice. But as you well know, not the simplest solution.

It's certainly possible that your camera(s) just aren't embedded structured metadata for this information.  But I wouldn't give up too quickly on looking for it, because extracting it from actual video frames is more challenging, less efficient, and less reliable.

EXIF / IPTC metadata is effectively limited to a finite set of pre-defined attributes, mostly centred around camera settings & author information, respectively.  You can check the latest specification to see precisely what is formally defined.  Ambient temperature actually is in there (and has been for a while) but there's nothing for astronomical data (although, can you not infer the moon phase based on the location & time data?).

It is alas a long-running issue that camera makers often misunderstand the specification and store data incorrectly (or not at all).

Camera makers can embed proprietary data in MakerNote fields, but the format is unspecified and standard tools (like exiftool) likely won't understand them for all but the most popular cameras (think Nikon, Canon, Sony, etc).  And possibly only for older models.

Many common image & video containers (e.g. JPEG and MP4) do support embeded XMP, an extensible format which allows model- and make-specific metadata.  It's possible that's in use in your case.  exiftool does not show embedded XMP data by default - at least not reliably.  In a nutshell, I suggest running exiftool as:

exiftool -D -g -G -ee -scanForXMP -u -U <PATH>

That's more likely to show you embedded XMP data, as well as all EXIF / IPTC tags (even those unrecognised by exiftool - you can at least get their raw bytes and work from there).  You can also add the `-v3` argument if you want to dig even harder, or see the raw data in more detail.

Another way some cameras store proprietary metadata is in proprietary tracks (e.g. GPMF for GoPros).  You can see if your files contain any such tracks via:

ffmpeg -i <PATH>

Usually these are used for metadata which can change during a recording, as EXIF / IPTC / XMP have no real support for that.  But some camera makers use them even for static metadata out of e.g. laziness or hubris.

Extracting such tracks is easy enough with tools like ffmpeg, but interpreting them can be challenging if their format is not documented like GoPro's GPMF.

Can you provide a sample file for us to examine?

Hi - 

You can always get the Moon phase data from the date, time and location combination. 

https://pyorbital.readthedocs.io/en/feature-moon-phase/moon_calculations.html

 

Sometimes the exif data doesn't capture the information. But if its present it should be easy to simply read it and map it to the correct exif coords

 

 

Hi @LucyD a quick comment along the lines of peoples' suggestion to get moon phase from the location and date of camera images, you might also be better of getting temperature data this way too, as I've always found camera trap temperature data to be pretty unreliable. I guess it depends on the accuracy you need, and indeed, whether you do have location data for your traps and external temperature data are available...

I am no help at all re: extracting data from the files sorry, but @Wade looks to have provided some really helpful tips. @Freaklabs do you have any suggestions?

Cheers and best of luck Lucy,

Rob