Exam CT-GenAI Topic 1 Question 3 Discussion

In the context of LLM inference,Temperatureis a hyperparameter that controls the randomness or
"creativity" of the model's output. When the temperature is set high, the model's probability distribution is
"flattened," meaning it is more likely to select less-probable tokens, leading to more diverse and sometimes unpredictable text. For software testing, where precision and repeatability are paramount,lowering the temperature(Option C) is the standard practice. A temperature of 0.0 makes the model "deterministic," meaning it will consistently choose the token with the highest probability. This narrows the sampling distribution and significantly reduces variability between runs. While a larger context window (Option D) allows the model to process more information, it does not directly control the randomness of token selection.
Similarly, the "learning rate" (Option B) is a parameter used during thetrainingorfine-tuningphase, not during inference. For generating test cases or scripts that must follow strict logic, a lower temperature ensures that the model remains focused and produces consistent results.