Refactoring: turn the attribute _return_attention_scores
into a method argument
#20802
Labels
type:feature
The user is asking for a new feature.
Hey there! 👾
Issue
The class
MultiHeadAttention
is using a private attribute to decide whether or not to return the attention scores.This behavior is not apparent from the signature of
_compute_attention
, because there are no corresponding argument.So, if a sub-class fails to set
_return_attention_scores
in its customcall
method, then_compute_attention
never returns the scores.For example, the child class
CachedMultiHeadAttention
from KerasHub uses_compute_attention
and receivesNone
as attention scores in all cases.Additional context
The attribute
_return_attention_scores
is used in a single method,_compute_attention
.Proposition
IMHO the attribute
_return_attention_scores
should be removed.Instead,
_compute_attention
should have an extra parameterreturn_attention_scores
.Then the
call
methods could just forward this value to_compute_attention
explicitely.I'll write a PR shortly if you find this relevant.
The text was updated successfully, but these errors were encountered: