I tested it and the deepfaunce integration identifies these elk as Red deer.

But the sbts-install (StalkedByTheState) project is easy to integrate with different models and it can use multiple models at the same time. So if there are some animal types you expect that are not part of deepfaune or the coco dataset but you have a model for it, likely I could add an integration for it faily easily.

Kodos to deepfaune against for being so useful, even for American animals :)

Oh and bears are easy. That's also supported out of the box with the base models in the coco dataset.



Out of the box StalkedByTheState supports these devices:

 

Which are relays and switches inputs with a USB connection. You can add as many as you want so you can control a lot. It has a menu driven event control system to allow you to hook up image recognition based triggers and lots of other events or state in the control loop. So it should be as flexible as you need it.



The ideal platform for this is the NVidia AI at the edge box Seeed Studios recomputer J4012.

Fun to see the Euroepean models infering North American sister species! 

Yeah, it wouldn't surprise me if a model trained on European bears would match polar bears as well.

Oh wait, we tried that and they do :)

After analyzing countless false positives I've come to realise just how simple AI computer vision really is in what it does. Everything is reduced to a simple set of features. A person for example for many of the earlier models it not much more than a big lump with a smaller lump on top and some things sticking out. That's why rain falling on an angle in IR lighting was often interpreted as arms of a person.

Or do the same without the rfid but detect the cattle instead.

However. Maybe the cattle hanging around the fence preventing it from opening when maybe it’s desirable to open the gate for a bear even if there are cattle knowing the cattle will run away from the bear.

It could even be hybrid. Keep the gate open, unless there’s cattle there. But still open if there is cattle and a bear approaches.

In addition the AI approach gives valuable insights to how much wild is using the crossing. Which I suspect they would like to know.  The system would capture video clips of each interaction providing insights as to the animal behavior. 

I quite like the suggestion to invert the problem and keep the gates open. You would probably get more natural wild mobility. But  don’t anyone try and poke an rfid into my ear, I’ll thump you.

Interesting thought about using RFID tags to close gates. Don't most cattle carry these anyhow? or is it only dairy cattle? In Europe at least, I believe RFIDs are used a lot for triggering all sorts of automation.

Surely though, rfid distance is quite limited, giving very little time to close the gates. Using AI you could detect the cattle sufficiently early before they reach the gates to close them safely on time.

I also suspect that there's a significant investment involved here and would be useful to be able to judge how well it's doing. i.e. how many of the different kinds of wild are going through the gates. An rfid can't provide that.

I don't see an rfid approach as progression if compared against the AI approach which simple involves dropping some cameras at a computer there with some relays and you are done. Apart from a little configuration. If the cattle are not tagged, then how much effort is it to tag potentially thousands of cattle ? It's nice to throw all possible solutions out there so they can weigh in on a solution. But being somewhat a lot biased :) I'm leaning towards the AI based solution.

You could even detect if cattle are outside of the gate and none just inside the gate and open the gate to let them in, closing it afterwards.

Shouldn't we be encouraging and applauding the use and trialing of innovative techniques?

I haven't used it much myself but I believe passive RFID has a range of ~1m and active can be up to ~7-10m. 

Don't get me wrong I love trying new tech but the engineer in me leans towards the KISS principle. What gets the job done reliably, is easy to implement and maintain at a low cost?

As techies we all like using the new shiny things when sometimes the best solution doesn't actually require any tech. Maybe they just need a sheep dog haha.

A combined solution could also work. You use an RFID or similar system to help train and verify a computer vision model. 

Robs company had a similar concept to what we are talking about for triggering road side animal crossing signs. Bottom of the page. 

I totally appreciate KISS. I'm coming from another place thought. My solution isn't so new, the base system has been running continously in place in a number of sites for more than 10 years now, in the past with PIR sensors and modified lorawan enhanced PIR sensors. The AI part was added in 2019, so 4x years continously with ai in about 10 sites and I've upgraded with each new improved model as they came along. That was yolov3, yolov4, scaled yolov4, yolor, yolov7 and finally yolov6. Each one empirically demonstrated greater accuracy and recall in practice. For me this is the simple solution. But as I only released the system into open source last year I so far have not had the platform to show the effectiveness of the system. It's a system that has evolved from 2008. Many burglars have been detected with it so far.

What is new is the processor series from Jetson, that's available pretty much from this year and adds 3x the speed. Yes, I've added new model algorithms when they come, I added yolov6 within a week of it coming out but I've been using it for almost a year  now.

For me it's quite clear that this will work very well, probably valuable insights and I think pretty easy to maintain as the equipment just runs, so long as you keep it dry. There are seldom ever problems.

But that's my truth that I still have to share with the rest of you. But I aim to convince you all of this over the coming year :-) Let's say I've been now challenged for this now.

In any case, I think that this project suggested here would be a brilliant project to show this. I think I've said enough now :)

But you've convinced me. I see that wild labs in their events section talk about tech tutors, I'm going to put myself forward to do a tech tutor session on my detection, event handling system StalkedByTheState for the beginning of next year sometime.
 

I'd also like to do one on sound localization with sbts-aru. I don't know how this works, I'll contact Stephanie.

Maybe some folks are interested, maybe no one. I hope someone :)

For sure! A tech tutor or something similar would be awesome. I'd definitely be interesting in learning about your sound localization system.

I'm not knocking your computer vision idea at all. It could well be the simplest/best solution. I have a project coming up using such a system and I'm looking forward to learning more about it. Every deployment is different. I try to consider as many options as possible to find the solution that fits best.

Hi Peter, for the benefit of the others. Here is a selection of photos annotated after processing with yolov6 full model on some of your cow photos.

It matches cows from quite far away. I've reduced the image size to 640 to post here, the images processed with 1024 sized. This model will run and perform the gate functions on an NVidia AI on the end machine in the form of the Seeed Studio J4012 at 6 frames per second at 640 sized images. If that encapsulates the distance you can require that's great, otherwise we co






And just to show what distance you can in fact detect at if the image resolution was high enough? Such as a crop on an 8K camera perhaps. The following is a clip from this image of the farthest away cows. I'd love to know what distance that is.

Note the above is just a digital zoom. With a 20x optical zoom security camera you could detect and identify animals from at least 2km I suspect. Imagine detecting an alerting on the presence of wolves from 2-3km in day time :-) Put it on a tower and have the camera roam and there's your long distance elephant early warning system.

In this case, I would use BLE proximity - enough and accurate range,  low cost, long battery life, no false positives - KISS  ;)

Aaah, this article suggests that RFID can be used for much greater distances so looks like RFID still remains the best choice.

Love the wolf picture with the points like a movie :)

My plan was to have the gate always open and detect Cattle and have it close. It will have to be run off solar as these are remote areas here it will be used. The problem with the ear tag idea is that not all the cattle are part of our herd and we already have free range stock sneaking onto the ranch all the time. I want to exclude them by fixing the fences but allow the free range of the Elk. Deer just jump over the fence but the Elk charge it and try to break through. We have certain areas where they are known to pass so we would put the gates in those zones and create funnels to encourage them to go there. They seem quite intelligent anf would know where to find the openings in the fence.

How many gates are we talking about ? What type of cattle ?

My system has a list of notifications that it tries to match for each image. Each notification has a bunch of constrainst, such as a list of animal types etc but it can also look for the absense of a detection.



Each notification (as well as each camera) can also be dynamically enabled and disabled by a rest interface built into the scanner process. So When there are no cattle around, you can disable the system for looking for no cattle and enable the system to look for cattle. When there are cattle around you can disable the system to look for cattle and look for the absense of cattle.



It's always possible for a frame or two to miss seeing an object. So it's possible for each rule to attach a statistical qualifier to it. Such as looking for the absense of cattle might require 98% of all frames not to see any cattle. Allowing maybe one or two frames out of 6 frames per second can provide resilience against spurious false positives.



The net result is that once the cattle move out of the area of interest a minute later the gate opens for example.

There are a lot more controls possible, you can attach rules to add state and add state that automatically disappears after a time. And rules can be have state based constraints attached to them, such as seeing no cows. Or seeing no cows unless there are elk waiting to get in. It's very flexible.



It can also have features to detect faulty cameras, watchdog timers. You could put a test object in sight of the camera and if it's not seen for a while then it triggers some kind of alert. I built in that feature when I was using PIR sensors before AI. Each sensor would generate keep-alives, if it didn't receive them for a while it indicated that the batteries were dead.



If you had networking, then you can easily add mobile phone buttons for extra functionality. Such as you are looking at the camera and see a situation where you want to manually open or close a gate. Or override some specific behavior. Like keeping the gates shut for a while. And all of this behaviour can use timers as well, such as keep the gates shut for the next 4 hours and then revert to the original behavior.

And then while you are at it, just because "Why not" as you are adding them units around your ranch, add one of my low power cheap sound localizing recorders to each unit, this will keep the clocks in perfect time accuracy for the video events that it takes and if there is a shooting incident of any kind you would be able to determine where the shots were fired from to high accuracy. You never know in this world.



Links:



Security system: 

Low powered cheap sound localizing system: 

BTW. I found out that the Jetson Orin NX 16GB module is drawing around 20W when running continuous inference, processing streams from 6x cameras at 6 fps.

I'll try and find out what you can do with a Pi 5 and a smaller model over Christmas.