Why is padding mask defined but not used?

[hzc363]

Hello, you all Keras gurus!

I am trying to learn transformer by using this keras tutorial: https://github.com/keras-team/keras-io/blob/master/examples/nlp/neural_machine_translation_with_transformer.py.

The TransformerEncoder class allow users to add padding masks (line 260-263):
def call(self, inputs, mask=None): if mask is not None: padding_mask = ops.cast(mask[:, None, :], dtype="int32") else: padding_mask = None

But it seems that this functionality is not used when training the model (line 404):
encoder_inputs = keras.Input(shape=(None,), dtype="int64", name="encoder_inputs") x = PositionalEmbedding(sequence_length, vocab_size, embed_dim)(encoder_inputs) encoder_outputs = TransformerEncoder(embed_dim, latent_dim, num_heads)(x) encoder = keras.Model(encoder_inputs, encoder_outputs)
Since I am relatively new to this field. I am not sure if I understand the code correctly. Is it ture that the code impletmented padding mask but did not use it during the training? Why the tutorial is designed this way?

Thank you very much for the answers.