It’s a 20 degree day in Medford, Massachusetts and yesterday’s snow is gusting across the road as I cross the river. I’ve flown across the country to gather in a small basement office with a small group of engineers, fishery scientists, and friends recruited with the promise of dinner and energy drinks. We’ll spend the next five hours watching New England fishermen scoop fish out of bright orange plastic totes, place them on a measuring strip, quickly take their hands off, and let them go over the side of the boat. As we scan through the digital footage, we mark the time when the fish appears, record the fish species, and measure it on the screen by drawing a line from mouth to tail. It’s like a slow, meticulous game of fruit ninja.

All this work and snacks is in preparation for a competition we plan to run this summer to train computers to identify fish. We’re working towards a future where monitoring what fisherman catch and what they release can be done in real time, making it easier and more affordable for fishermen and managers and anyone who wants to keep track of what’s taken out of the ocean to do just that. It’s a merger of high-tech, sophisticated computer science and one of the oldest professions in the world.

Today’s wheelhouse has multiple screens to let you track your catch. Or your dog. Photo ©Heather Perry

This is the coming revolution in conservation, in knowing and understanding the world around us. The natural world is full of remarkable and often hard-to-observe phenomenon. Scientists studying the rainforest used to camp out for days in hopes of observing a rare glimpse of a jaguar, but now they can set up cameras to record constantly and review that footage from their laptop anywhere in the world. Advances in conservation technology are generating more data than ever before on what lives where, who eats who, and what’s disappearing and how fast, but it still requires human time to watch that video footage. We need to pair our new sensor systems with analytical tools that let us convert that data into information and knowledge.

Which is the kind of problem that big for-profit companies excel at tackling and that people are excited to build tools for because fixing a big corporate problem can get you a big corporate paycheck. If you can become a billionaire writing an algorithm to process banking data would you spend part of your time tweaking that same algorithm to model fish populations? The world of data competitions suggest that, for some people, that answer is yes. Our basement group gives me hope that more and more there are engineers, data scientists, UX designers, and other curious innovators drawn to unusual and challenging problems, with solutions measured in impact beyond profit.

There is profit involved here. In a lot of small coastal communities fishing is the biggest business around. Saltwater fishing — both commercial and sport — in America is a $300 billion dollar business supporting 1.8 million jobs. More and more people want to know where their seafood came from and that they’re not eating the last fish in the sea. That means counting fish accurately becomes all that much more important, and some fishermen are eagerly embracing new technologies to be able to demonstrate just how sustainable their business is. Their goal is to maximize fishing revenue while leaving enough fish in the ocean to breed new fish for next year’s catch. With more effective electronic tools it could be easier for fishermen to track that catch and for managers to monitor the overall impact on fish populations. Our project has funding from the National Fish and Wildlife Foundation to work on this, but our $200,000 budget is barely the annual salary of one Silicon Valley software engineer. So, our team involves people who will work at a deep discount, for the thrill of solving a complicated problem, and for some delicious pulled pork.

One reason people seem to get excited about creating algorithms for fish is that fishing presents a novel set of challenges to a computer. Poor lighting on a boat deck or underwater can distort colors and blur a fish’s edges. In bright sunlight, the glare of a shiny fish reads as no data to a digital camera. If fish are moving around through a net or a chute you have to be clever to make sure you can tell one fish from another and not double-count. Even in the best conditions some fish can be difficult to tell apart. We’re joined this Saturday morning by a guy we refer to as the Fish Boss, a fishery scientist who also spent years as an observer, riding along on fishing trips counting the fish as fishermen hauled them aboard. We hunch over our screens moving frame by frame to annotate the fish in the video, occasionally calling Fish Boss over to make sure we haven’t confused one flounder with another.

Matching verified Fish IDs to footage.

If we succeed it it should make counting fish easier for fisherman and managers not just in New England but all over the US and around the world. We’re designing our products to be open source, to make it more likely that other fisheries can adopt and implement what we help develop. Some of those tools, like the video annotation software, are simple enough that anyone can use them, making them contributions to the growing citizen science movement. It’s time to take the internet of things outside and use our growing technological capacity to support the natural world around us, and the businesses and the quality of life it sustains.

The Fish Identification Computer Vision Competition to develop tools that can count, measure, and identify fish from commercial fishing video is now live! There are $50,000 worth of prizes available, and it runs through October 30, 2017. Find out more

About the Author

Kate Wing is the principal of K|W Consulting, where she works with clients to design organizational strategy and business models, build networks, and support ocean conservation. She co-authored an October 2015 report on Modernizing U.S. Fisheries Data systems for the Kingfisher Foundation. She is currently pursuing her interests in fish tech through a residency at the open science space Manylabs, where she recently hosted the first-ever San Francisco Fishackathon. She has a wide breadth of experience from her twenty years in the social sector, from managing a $20 million grant portfolio at the Gordon and Betty Moore Foundation to running nonprofit campaigns and serving as a Knauss Sea Grant Congressional Fellow. She serves on the Boards of Root Solutions and SciFund Challenge.