Handle malformed transactions

[jbearer]

#2 will handle transactions which are invalid due to consensus rules (bad signature, wrong nonce, etc). The mechanism is general enough to handle fully malformed transactions as well, since Block.rejected holds []byte and not types.Transaction. We should hook up the transaction unmarshalling in the API handlers to the population of the rejected field.