Hey Jorrit, I completely feel this. The fragility of DockerHub and the 'bit rot' of R packages is exactly why I’ve been going down the GNU Guix rabbit hole lately. It’s a bit of a learning curve, but having a functional, bit-for-bit reproducible environment that doesn’t rely on a central registry feels like a much safer bet for conservation data that needs to last decades.

I’ve also started moving everything into Org Mode for literate programming. My thought is that even if the software eventually fails, the plain-text 'recipe' and logic are still right there in the same file. I try to stick to Python to keep the stack as simple as possible, but even then, it's a constant battle against dependency hell.

Are you looking at any specific alternatives to Docker, or just trying to find a better way to document the transformations?

Rather than relying on the docker pull, you might want to do a docker save of the images that you are using and save those. Then do a docker load instead of a docker pull when you deploy.

In short, the only reproducibility you are going to be able to have is that for container images you save yourself.

Your biggest problem will be for containers, because you could strive to have as much of your process in containers. Then the rest of the problem is not going to be so great anymore. In any case, it should be manageable. You could solve that as well, but that might be taking to too much of an extreme for the benefit you gain.

We are currently looking at QEMU for virtual machine boxes. My issue if I've got a really dirty Debian devbox and getting that into a docker that plays nice may be more trouble than it's worth. Our devs can't be good at everything and sometimes, troubleshooting dockers sucks. Running a VM instance and pretending it's a bare metal physical box has advantages for my team. 

Read "QEMU" on

QEMU

" /> <meta name=

I hear your pain. I guess there are degrees of long term sustainability. Each with their choices. Docker pain can be reduced with vibe coding though. That's something that the AI would be quite good at.

Good luck with it all!

Hi Stephen,

My experience of Nix is limited unfortunately but I think it offers a lot of the same benefits. I'm an Emacs/Lisp fan so it's perfect for me :)

There is Cuirass, which is a Guix specific CI and build automation service and a pack command to create containers. It's nice to have everything linked with Guix rather than different systems/frameworks

I built a small POC to explain the benefits. 

Makes sense, I thought there might be a Lisp preference factoring in there 🙂

Cuirass looks cool, I'm going to check it out, and the eco-pulse-monitor project looks great too, well done!

Related to AI sustainability, have you checked out any of the CodeCarbon tools by chance? I've been working on incorporating their library into our daily code tasks at work, seems like a worthwhile project - 

Read "CodeCarbon" on

CodeCarbon

Track & reduce Co2 emissions from your computing

EcoLogits just joined CodeCarbon too I think, also neat - 

Read "EcoLogits - EcoLogits" on

EcoLogits - EcoLogits

Thanks!

Thank you. That looks great! I was trying to build something similar but, as usual, there's a Python library already :) I was leaning towards trying SLMs like llama.cpp

My end goal is to find ways to use this approach in animal conservation specifically. I'm looking at data sets from the IUCN to see what's possible. I'd love to hear more about your work