I made a machine learning based cloud detection script for AllSky

[chvvkumar]

Inspired by @gordtulloch's work here: https://github.com/gordtulloch/mlCloudDetect

I made this with the idea that I could use this information to start, stop and pause my NINA sequences and to see if I could do it :)

https://github.com/chvvkumar/simpleCloudDetect

Wisps of clouds (I know I spelled Wisps wrong :)

[EricClaeys]

How well does it work?

[chvvkumar]

How well does it work?

So far it has been working quite well. Obviously it depends on how comprehensive the training data is. I have used about 1500 to 3500 images per class to train my model.

Here is the most important test for me, to detect rain and it seems to be working really well.

The screenshot shows some sprinkles that fell at 2.42am that the detector recognized but my physical rain gauge could not.

Here is a comparison with my tipping bucket type rain gauge which only registers rain beyond a certain rate. So in that regard, this ML detection is much better as it works even if there are a few drops of water on the camera dome.

Rain or sprinkles actually fell at different times between 3AM and 11AM but my rain gauge only registered the significant rain around 6AM

ML Detection:

Rain Gauge

I will add examples of various things I defined in my model below as I see them appear:

Overcast (my definition was if the whole viewable sky had clouds in it)

Mostly Cloudy:

Wisps of clouds (I know I spelled wisps wrong :)

[Alex-developer]

It looks great and love the HA integration.

Also .. blimey great minds - I am writing a module to do exactly the same :-)

It might be worth combining our efforts as in my tests it works very well as long as you train the model properly.

[chvvkumar]

That is wonderful! I am not sure how I would be able to help as this is the first time I did any significant work beyond one liners with Python and I have zero knowledge of ML, ha. But if you find anything helpful in my code, please feel free to use it! Otherwise, let me know how I could help and I will try my best.

[chvvkumar]

I made a docker container for this if anyone wants to try it.

https://github.com/chvvkumar/simpleCloudDetect?tab=readme-ov-file#docker-preferred-method