{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Encoder\n", "\n", "Each encoder layer contains:\n", "\n", "1. Self-Attention:\n", " * Captures relationships within the sequence.\n", "2. Feedforward Network (FFN):\n", " * Two linear layers with a ReLU/GELU activation in between.\n", "3. Add & Normalize:\n", " * Residual connection and layer normalization.\n", "\n", "\n", "