GenAI Basics · Question 3 · Easy
In the context of LLMs, what is a token?
- A unit of text, such as a word, part of a word or a punctuation mark, that the model processes
- Always exactly one complete word
- A password used to log in to the model
- One complete sentence
Answer and explanation
Answer: A. A unit of text, such as a word, part of a word or a punctuation mark, that the model processes
Explanation: Text is split into tokens, often sub-word pieces, by a tokenizer. Models read and generate tokens, and limits and pricing are counted in tokens.
Previous: What does it mean when an LLM "hallucinates"? · Next: The instructions and input text that a user gives an LLM to get a response are…
Practise genai basics
Try a fresh set of genai basics questions with an explanation after every answer, free and without signing up: GenAI Basics practice. Or sit a full placement mock test.
More genai basics questions
- In generative AI, what does LLM stand for?
- What does it mean when an LLM "hallucinates"?
- The instructions and input text that a user gives an LLM to get a response are…
- What is few-shot prompting?
- What is a text embedding?
- What is the cosine similarity between the vectors (3, 4) and (6, 8)?
- What does Retrieval-Augmented Generation (RAG) do?
- Setting the sampling temperature of an LLM close to 0 typically makes its…
- What is an LLM's context window?
- A company's product price list changes every day, and its chatbot must answer…
More technical & programming practice
Pseudo-code · C Programming · Java · Python · Data Structures · Algorithms · Object-Oriented Programming · DBMS
LogicWiz Exams home · All practice topics · Placement mock tests · Daily 10 challenge