I'm not sure about this one. We should add it as allow by default and run it on some crates

I think limiting this lint to references that are temporaries, such as a function returning a reference, would fit most cases. For example, doing mem::swap(&mut self.foo(), &mut self.bar()), where foo and bar return references to something, should warn, because it can't do anything useful. However, this shouldn't warn:

let mut x = &5;
let mut y = &7;
mem::swap(&mut x, &mut y);