TensorFlow 2.10 shines on Keras, Decision Forests

TensorFlow 2.10, an upgrade to the Google-developed open source machine learning platform, has been released, bringing new user-friendly features to the Keras API, improved aarch64 CPU performance, and the arrival of TensorFlow Decision Forests 1.0, which the developers now describe as stable, mature, and ready for professional environments.

Among the Keras improvements, TensorFlow 2.10 expands and unifies mask handling for Keras attention layers. Two new features have been added. All three layers, <a href="https://www.tensorflow.org/api_docs/python/tf/keras/layers/Attention" rel="nofollow">tf.keras.layers.Attention</a>, <a href="https://www.tensorflow.org/api_docs/python/tf/keras/layers/AdditiveAttention" rel="nofollow">tf.keras.layers.AdditiveAttention</a>, and <a href="https://www.tensorflow.org/api_docs/python/tf/keras/layers/MultiHeadAttention?version=nightly" rel="nofollow">tf.keras.layers.MultiHeadAttention</a>, now support casual attention (with a use_causal_mask argument to call) and implicit masking (set mask_zero=True in tf.keras.layers.Embedding). These new capabilities simplify implementation of any Transformer-style model.

Also in TensorFlow 2.10, Keras initializers have been made stateless and deterministic, built on top of stateless TF random ops. Both seeded and unseeded Keras initializers will generate the same values every time they are called. The stateless initializer helps Keras support new features such as multi-client model training with DTensor.

Installation instructions for TensorFlow can be found at Tensorflow.org. Other new capabilities and improvements in TensorFlow 2.1:

• BackupAndRestore checkpoints offer step level granularity.
• Users can easily generate an audio dataset from a directory of audio files, via a new utility, keras.utils.audio_dataset_from_directory.
• The EinsumDense layer is no longer experimental.
• In conjunction with the release of TensorFlow 2.10, TensorFlow Decision Forests (TF-DF), a collection of algorithms for training, serving, and interpreting decision forest models, reaches 1.0 status.
• Performance has been improved for the aarch64 CPU.
• GPU support has been expanded on Windows, through the TensorFlow-DirectML plug-in.
• An experimental API, tf.data.experimental.from_list, creates a tf.data.Dataset comprising the given list of elements. The returned dataset will produce items in the list one by one.