Article Source

• Vision Transformer - Keras Code Examples

Vision Transformer - Keras Code Examples

• Keras Code Exampes - Vision Transformers
• Google AI Blog Visualization
• Formal Paper describing this model: https://arxiv.org/pdf/2010.11929.pdf​
• TensorFlow Addons: https://www.tensorflow.org/addons​
• TensorFlow Addons -AdamW

Abstract

This video walks through the Keras Code Example implementation of Vision Transformers!! I see this as a huge opportunity for graduate students and researchers because this architecture has a serious room for improvement. I predict that Attention will outperform CNN models like ResNets, EfficientNets, etc. it will just take the discovery of complimentary priors, e.g. custom data augmentations or pre-training tasks. I hope you find this video useful, please check out the rest of the Keras Code Examples playlist!