3

I am packaging my model for deployment in aws lambda which has a size limit of 250mb for all dependencies.

Sklearn, if you include its dependencies of numpy and scipy is a huge package.

Are there any alternatives to sklearn that don't require scipy that are smaller than sklearn?

Thanks!

coderboi
  • 133
  • 3

1 Answers1

1

Did you check tinynumpy?

Anyway, I rarely found alternatives to famous packages (except scikit-image instead of opencv). What usually works for me is:

  1. Slim the model as much as I can (e.g. weights quantization)
  2. Check in the code which functions I use from each module. Once I have a list of them, I retrieve the corresponding python files and get rid of the rest
  3. Try to split my process in multiple functions (e.g. one function to perform data processing, one function where to implement the model and make the inference)

The second point is crucial. In my experience, one rarely needs entire packages.

However, depending on the case, it could also be that AWS Lambda does not fit your needs.

dapetillo
  • 101
  • 2