GenAI Basics · Question 12 · Hard
In a transformer model, what does the self-attention mechanism do?
- It stores the training data so that the model can look it up later
- It lets each token weigh how relevant every other token in the sequence is when building its own representation
- It converts text into audio
- It removes stop words before processing
Answer and explanation
Answer: B. It lets each token weigh how relevant every other token in the sequence is when building its own representation
Explanation: Self-attention computes, for every position, a weighted combination of all positions, with weights from query-key similarity. This captures context and long-range dependencies in parallel.
Previous: A company's product price list changes every day, and its chatbot must answer… · Next: Which order of steps is correct for answering a user question with a typical…
Practise genai basics
Try a fresh set of genai basics questions with an explanation after every answer, free and without signing up: GenAI Basics practice. Or sit a full placement mock test.
More genai basics questions
- Setting the sampling temperature of an LLM close to 0 typically makes its…
- What is an LLM's context window?
- A company's product price list changes every day, and its chatbot must answer…
- Which order of steps is correct for answering a user question with a typical…
- A model has a context window of 8,000 tokens, shared by the input and the…
- What is the main pre-training objective of decoder-only (GPT-style) large…
- Asking an LLM to perform a task with only an instruction and no examples in…
- What mainly distinguishes generative AI from traditional predictive…
- In the model family name GPT, what does GPT stand for?
- Further training an already pre-trained model on a smaller, task-specific…
More technical & programming practice
Pseudo-code · C Programming · Java · Python · Data Structures · Algorithms · Object-Oriented Programming · DBMS
LogicWiz Exams home · All practice topics · Placement mock tests · Daily 10 challenge