Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing rf_local_biased_add in pyrasterframes #578

Open
lepitrust opened this issue Dec 21, 2021 · 3 comments
Open

Missing rf_local_biased_add in pyrasterframes #578

lepitrust opened this issue Dec 21, 2021 · 3 comments

Comments

@lepitrust
Copy link

lepitrust commented Dec 21, 2021

I found this function on scala source code, but not working on pyrasterframes...

I need to make data + nodata = data and not data + nodata> = nodata
If understand the code
rf_local_add consider data + nodata = nodata
rf_local_biased_add consider data> + nodata> = data

anyone can help me to do this without this function (maybe masking etc)?

Thanks

@lepitrust
Copy link
Author

@pomadchin
Copy link
Member

pomadchin commented Mar 9, 2022

Hey @lepitrust mb it just was not registered and than not exposed; I'll be glad to help you with framing a PR in case you want to handle that yourself.

As a workaround you may try to reg this function and use CQL to call it.

mb there was also smth behind it cc @metasim

@lepitrust
Copy link
Author

Hi, thanks @pomadchin! Any alternative is welcome ;)
I have a little spark cluster with pyrasterframes correctly working.
I'm trying to use for sum two raster after mul operation. beam and diff are raster tiles, k_b a k_d are float number.

This working, but diff raster contain a lot of NoData which must be ignored in the sum...
The solution can be a biased op... ;)

df_rf_ghi = df_rf_max \ .select(rf_extent(F.col('beam')).alias('new_extent'), rf_crs(F.col('beam')).alias('new_crs'), \ F.col('dims').alias('ghi_dims') rf_local_add(\ rf_local_multiply(F.col('beam'), F.col('k_b')), \ rf_local_multiply(F.col('diff'), F.col('k_d')) \ ).alias('new_raster'))

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants