How can I extend this to Python 3.6 ?

[jmarrietar]

AWS Lambda now handle Python 3.6 and i want to use it, but do not sure if is simple to extend it to use Python 3.6 instead.

Any guide would be appreciated

Thanks.

[awenzel]

I am also looking to run SciKit in Python 3.6, and also to upgrade to SciKit 0.18.1. Ryan, any chance you are still actively supporting this, and could build those Lambda-compatible libraries and just upload the zip's to this repo?

[ryansb]

I'm still maintaining this - right now I haven't had the need to bump to 3.6. It doesn't look like the Amazon Linux container image has 3.6 yet, which is odd. That means we'd either have to compile a Py3.6, or rely on universal wheels. Can either of you give the current artifacts a try on Python 3.6?

[awenzel]

Thanks for the reply Ryan. Since my note, we tried getting everything to build with 3.6 on the Amazon Linux image, but ran into some problems. Interestingly we found this (https://hub.docker.com/r/vivantshen/awspy3-lambda-zip/), which has all of our needed libraries (SciKit 0.18.1, Pandas, Numpy) compiled in 3.6 for Lambda, so we are just using those pre-built libraries.

[bhanga-manish]

@awenzel can you please upload the zip file to you github account and share the link here it would be very helpful.
@ryansb Any update to use this for python3.6?

[markcampanelli]

@bhangaboy For some 3.6 options, see #12 and #16.

[bhanga-manish]

@thunderfish24 Thank you for the reference. I am having trouble running these shell scripts in docker container through my windows if you can help me out with that please.

[bhanga-manish]

@thunderfish24 I managed to solve the error and the script started running but updating yum ends up in error. gcc64-gfortran-6.4.1-1.45.amzn1.x86_64.rpm and atlas these package are not able to get installed

[bhanga-manish]

@thunderfish24 and @awenzel if any of you managed to compile sklearn on python3.6 please share the zip with me [email protected]

[markcampanelli]

Wouldn’t this work for you?

https://github.com/thunderfish24/sklearn-build-lambda/blob/75c713d23107300370b16b134936b959f1f0f73b/lambda-package.zip

[bhanga-manish]

@thunderfish24 Thanks this works for me

[teresagator]

This is a tremendous help! But... has anybody been successful adding pandas to this script and stripping it to be small enough for lambda?

[bhanga-manish]

@teresagator did you tried running the shell script? if yes what is the size of the zip? If you haven't executed the script try changing the script with specifying installation of pandas then look at the zip.

[wi-ski]

Lambda functions support python 3.6 now. Any chance for an update?

[ryansb]

Branch is open here - https://github.com/ryansb/sklearn-build-lambda/tree/python-3-test for porting to python 3. Feedback is welcome, see if it works for you & if not open an issue or PR.

[Strateus]

Just in case someone else shows up here, you no longer need this.
AWS introduced layers to overcome this issue: https://aws.amazon.com/blogs/aws/new-for-aws-lambda-use-any-programming-language-and-share-common-components/

You can find some examples here: https://towardsdatascience.com/hosting-your-ml-model-on-aws-lambdas-api-gateway-part-1-9052e6b63b25

[harnit-bakshi]

This is not true @Strateus all the uncompressed lambda code + layers still cannot exceed the 250MB uncompressed limit. There is however an option for running lambda as a docker image, not sure about cold start times: https://docs.aws.amazon.com/lambda/latest/dg/images-create.html