GenAI Basics · Question 30 · Hard
Why do transformer models add positional encodings (or positional embeddings) to token embeddings?
- To reduce the vocabulary size
- To encrypt the input text
- Self-attention by itself does not take word order into account, so position information must be added
- To make the model run without a GPU
Answer and explanation
Answer: C. Self-attention by itself does not take word order into account, so position information must be added
Explanation: Attention treats its inputs as an unordered set; without position information 'dog bites man' and 'man bites dog' would look the same. Positional encodings inject each token's position.
Previous: In Reinforcement Learning from Human Feedback (RLHF), what is the reward model…
Practise genai basics
Try a fresh set of genai basics questions with an explanation after every answer, free and without signing up: GenAI Basics practice. Or sit a full placement mock test.
More genai basics questions
- A query embedding is q = (1, 2, 2). Four document embeddings are: D1 = (2, 1…
- A model's two candidate next tokens have logits 2 and 0. With softmax at…
- In Reinforcement Learning from Human Feedback (RLHF), what is the reward model…
More technical & programming practice
Pseudo-code · C Programming · Java · Python · Data Structures · Algorithms · Object-Oriented Programming · DBMS
LogicWiz Exams home · All practice topics · Placement mock tests · Daily 10 challenge