Scalar multiplication

[matthiasdellago]

Description

Implemented mul and rmul for Factoredmatrix to support scalar multiplication with *.
Supports floats, ints and single-element Tensors.
Approriate error messages and test are in place.

Fixes #348

Type of change

Please delete options that are not relevant.

• New feature (non-breaking change which adds functionality)

Checklist:

• I have commented my code, particularly in hard-to-understand areas
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• I have not rewritten tests relating to key interfaces which would affect backward compatibility

[neelnanda-io]

Thanks! It feels a bit inelegant to arbitrarily multiply the matrix on the right (rather than the left, or sqrt(scalar) on both), but it's probably best as is - users aren't supposed to use individual components of the factoredmatrix in general, and this avoids numerical errors of sqrt.

I left some nits, but overall LGTM

[neelnanda-io]

Looks like you have some formatting errors

[matthiasdellago]

Thanks! It feels a bit inelegant to arbitrarily multiply the matrix on the right (rather than the left, or sqrt(scalar) on both), but it's probably best as is - users aren't supposed to use individual components of the factoredmatrix in general, and this avoids numerical errors of sqrt.

Agreed. Aside from the numerical error my main resoning was to reduce the amount of computation required.

Looks like you have some formatting errors

Yes, I forgot to set up my development environment properly. I'm on it.

[matthiasdellago]

Resolved the nits, and fixed the formatting. 👍
Thanks for making TransformerLens btw, i really enjoy using it! :)

[jbloomAus]

Thanks!