Programming questions are created in the same way as other types of questions, but there is a specific tool available to facilitate the creation of this type of question and input specific values for programming questions.
To access this tool, go to the "Configuration" menu and select "Programming Question Builder."
After loading the question, the corresponding information will be displayed. On the right side, there is an editor where you can test how the question would evaluate a response, and on the left side, there are conditions (validation scenarios) to validate the question. It is composed of two sections: the first one is used to validate the question when the candidate is answering, and the second one is used to validate and determine the question's score.
This difference exists to prevent the candidate from entering a trivial answer like "Print 5" instead of an algorithm to solve the problem in the question. That's why it is common to include predefined code, one for validation during the exam and another one for assigning the score (with other values), so that the candidate cannot enter a trivial answer without the corresponding logic.
For both validation during the exam and scoring, it is necessary to enter the expected value, which can be entered directly or you can write a program to test the question and then copy its output to the answers. You can run your program with any of the conditions by clicking on the corresponding "Run" button. The validation conditions have a weight since it is possible to place several different conditions, each with a different weight that indicates how much this condition contributes to calculating the candidate's score in the question.
Optionally, you can configure an evaluation of the code efficiency, based on the number of instructions (tokens) used by the program. This is done to give less score to solutions that are longer than necessary. To do this, you must fill in the fields for Min Token, Max Token, and Code Efficiency Weight. Code Efficiency Weight refers to what percentage of the question's score will depend on code efficiency. For example, if the question has a score of 20 and a weight of 50 is assigned, then 50% of the score (10) will depend on efficiency. The code is evaluated, and if the number of tokens is less than or equal to Min Token, the maximum efficiency score is obtained. On the other hand, if the number is greater than Max Token, a score of 0 is given. For intermediate values, the score is prorated. It is recommended to configure this only when you already have a sample of various candidate responses to identify reasonable ranges for Min and Max Token (the constructor displays the values of Min and Max Token for the executed code).
Finally, for questions that support measuring code performance, it is possible to indicate the ranges and weight of performance in the question's score. You must mark which of the validation scenarios will be used to measure performance. Currently, only questions that use the TLang module allow performance measurement.