Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cythonized version with prange #15

Open
mikofski opened this issue May 22, 2019 · 0 comments
Open

Cythonized version with prange #15

mikofski opened this issue May 22, 2019 · 0 comments

Comments

@mikofski
Copy link
Contributor

Problem:

  • currently solposAM and spectrl2 only do one instant at a time, this means users have to loop over multiple datetimes or use map(solposAM, datetimes) to do a timeseries
  • using map or looping, even with comprehensions takes a while
In [2]: from solar_utils import solposAM
   ...: location = [35.56836, -119.2022, -8.0]
   ...: weather = [1015.62055, 40.0]
   ...: from datetime import datetime, timedelta
   ...: times = [(datetime(2017,1,1,0,0,0)+timedelta(hours=h)).timetuple()[:6] for h in range(8760)]

In [18]: %timeit list(map(lambda dt: solposAM(location, dt, weather), times))
177 ms ± 1.34 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

In [19]: %timeit tuple(map(lambda dt: solposAM(location, dt, weather), times))
174 ms ± 4.22 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

In [20]: %timeit [solposAM(location, dt, weather) for dt in times]
166 ms ± 4.21 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
  • PR Get solpos8760 - and version 0.3 (Carpenters) #14 introduced a tight loop in C, which is fast, but scales linearly with size, eg: 1000-timesteps are around 0.0018[s], but 10000-timesteps are 0.018[s] and so one

  • there is no use of multiple cores to speed up larger arrays of timesteps

  • using intel mkl TBB or mpi also seems too challenging, or too much maintenance hurdle, eg, installing mkl on every platform, and licensing issues?

  • using omp is possibly too challenging, or need to learn too much, so no time

proposal:

  • Cython prange does simple parallel loops, which might be a simpler approach that works well, but needs testing to prove out

risks:

  • parallel loops might not work for smaller arrays of timesteps, need some complicated strategy to determine when to apply?

alternatives:

  • do nothing
  • use intel mkl mpi or tbb or both?
  • use openMP
  • more research
  • port this code to numpy or numexpr and possibly use numba like the existing SPA in pvlib
  • use pythran
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant