Nota: Pendiente de Traducir. ¡Anímese a traducir esta página!. ( y otras páginas pendientes)

Nota: Esta documentación está basada en el estudio muy detallado de Certainty-Based Marking porel profesor Tony Gardner-Medwin. Puede ver más documentación en idioma inglés aquí

¿Qué es la Puntuación Basada en Certeza ( Certainty-Based marking, CBM )

Después de cada respuesta, Usted dice qué tan seguro está de que su respuesta sea correcta.
Esto es en una escala de 3 puntos: C=1 (bajo), C=2 (medio) o C=3 (alto)
No nos confiamos en palabras como 'seguro' o 'muy seguro', porque estos conceptos significan cosas diferentes para personas diferentes
El esquema de puntuación y el riego de castigo determinan cuando Usted debería de usar cada nivel de C:

Nivel de certeza	C=1	C=2	C=3	Sin respuesta or 'Sin Idea'
Marque si está correcto	1	2	3	( 0 )
Marque si está equivocado	0	-2	-6	( 0 )
Probabilidad de correcto	< 67%	67–80%	>80%	-
Descripción	Inseguro	Medio	Muy seguro	-

Los niveles de certeza 1, 2, 3 siempre le dan los puntos 1, 2, or 3 cuando está correcta la respuesta
Si Usted está equivocado, a menos que hubiera optado por C=1 Usted perderá puntos: -2 para C=2 y -6 para C=3

¿ Porqué usar CBM ?

Para asegurarse de que los estudiantes piensen acerca de qué tan confiable es su respuesta
Para motivar a los estudiantes qa que traten de entender los asuntos, y que no solamente reacionen inmediatamente a una pregunta
Para desafiarlos: si un estudiante no se arriega a perder puntos si se equivoca, entonces es porque realmente no conoce la respuesta
Si un estudiante es un pensador cuidadoso, pero no está muy seguro de sí mismo, ganará confianza
es más justo - una respuesta correcta pensada y certera merece más puntos que una adivinanza afortunada
Los estudiantes necesitan prestar atención si hacen respuestas equivocadas confianzudamente: ¡ pensar, refleccionar, aprender !
El estudio eficiente requiere cuestionarse constantemente sobre cómo surgen nuestras ideas y que tan confiables son.

Cómo decidir sobre el mejor nivel de certeza

Si Usted está seguro, obviamente le irá mejor con C=3. Pero si estuviera equivocada su respuesta, ¡ perderá el doble de puntos (-6)!.
Si está inseguro, debería de evitar todo riesgo de castigo, eligiendo C=1.
Si está intermedio, le conviene usar C=2: ganará o perderá 2 puntos si la tiene buena o mala.
La gráfica inferior muestra cómo la puntuación promedio para cada nivel de C depende de laprobabilidad de que su respuesta sea acertada.
- Suponga que Usted solemnte tiene una probabilidad de 50% de estar acertado: La gráfica más alta para 50% en la escala del fondo es negra, para C=1. Así es que Usted esperará mejorar más su puntaje en promedio al reconocer su baja certeza (C=1).
- Si Usted piensa que puede justificar su respuesta bien, con 80% o más de probabilidad de estar correcto, entonces la gráfica roja es la mayor, para C=3. Opte por ésta.

The graph shows how the average mark at each C level depends on the probability that your answer will be right.
- Suppose you think you only have a 50% chance of being right: The highest graph for 50% on the bottom scale is black (for C=1). So you will expect to boost your marks on average most by acknowledging your low certainty (C=1).
- If you think you can justify your answer well, with more than an 80% chance of being correct, then the red graph is highest, for C=3. Opt for this.
Tome nota de que Usted no puede esperar ganacia al representar falsamente su certeza. Si Usted elige C=3 (la línea roja) cuando no está seguro, entonces puede esperar que le vaya mal - y muy probablemente tendrá en promedio una puntuación negativa. Si Usted entiende el tópico bien, y piensa que su respuesta muy probablemente esté bien, entonces Usted perdería si optara por C=1 o C=2 en lugar de C=3. A Usted le irá mejor si puede distinguir cuales respuestas son confiables y cuales son inciertas.

Retroalimentación acerca de su desempeño en Puntuación Basada en Certeza

If you are using CBM with feedback given after each answer, then the mark tells you a lot: -6 will make you ask how you justified confidence in a wrong answer. Think (along with any available explanations and your books or notes) how to tie together different bits of your knowledge.
If you repeatedly get answers right with C=1 then ask yourself if perhaps you know the subject better than you think. Ask how you might relate your answers to things that you definitely do know and are more confident about.
When you finally submit, you will see your percentages correct for each C level you used. Ideally these might be around 50%,70%,90% for C=1,2,3 whether you know the subject well (lots of answers at C=3) or poorly (lots at C=1). The feedback will warn if you are tending to overestimate or underestimate how reliable your answers are. If overconfident, think about why! Perhaps you were careless; perhaps you have misconceptions, or are giving confident answers to questions you don't really understand or that are phrased ambiguously. Use this to guide your study and to improve your interaction with teachers.
Note that some questions in a quiz may carry more weight than others, so for example you might get -12 if the question carries double weight and you make a confident error. Since this makes feedback harder to interpret, it probably isn't a good idea for quiz-setters to vary weights in this way when using CBM.
If you've only answered a few questions, CBM feedback may not mean very much: 2 or 3 unconfident correct answers may just be luck! What is important is that you think about why you were confident or unconfident, and learn from that.

Puntuaciones y Calificaciones con Puntuación Basada en Certeza

Calificación Moodle , Puntuación Promedio Basada en Certeza y Exactitud

The Moodle Grade is the total of marks awarded, scaled relative to a nominal value for the whole quiz. Grades are sometimes shown 'out of' this value, but with CBM the grade can be up to 3 times this value (if all answers are correct at C=3).
The CBM Average can range between 3 and -6. If all the questions in the quiz have the same weight, then it is simply the average CBM mark, as shown in the CBM table above. If the weights (w) vary, then for CBM marks (m) awarded on each question the CBM Average = Σ(wm)/Σ(w).
Accuracy is a conventional measure of how much you got right in the quiz, ignoring CBM ratings. If all questions carry equal weight, it is the percentage of answers that were correct (with perhaps some answers being counted as partially correct). If questions have different weights w, and answers have fractional correctness f (between 0 and 1) then Accuracy = Σ(wf)/Σ(w).
Partially Correct Answers. If Moodle answers are partially correct (assigned a fraction f greater than zero), then for C=1, 2,or 3 the mark awarded will be f, 2f or 3f. If f is negative (implying that any correct portion of a response is outweighed by errors) then the mark awarded will be the same as for a wrong answer (0,-2 or -6).

¿Qué le dicen a Usted Puntuación Promedio Basada en Certeza y Exactitud?

The graph^* shows the scores (Accuracy and CBM average) for several thousand online submissions on a variety of voluntary CBM self-tests for learning and revision, mostly with university students.
It is annotated with interpretations of the students' knowledge in different zones of the graph.

^* Format used in CBM Self-Tests (not currently in Moodle) to superimpose a student's score after submission.

Note that the CBM average is always less, as a percentage of maximum, than accuracy. For the score shown in yellow, the CBM average (1.5) was 50% of maximum, while the accuracy (which ignores C ratings) was 80%. You obviously shouldn't worry that your CBM average is less, expressed in this way, than your conventional score. It always is!
If you didn't discriminate at all between reliable and uncertain answers (giving them all the same C rating, appropriate just for how well you know the topics on average), then your scores would lie somewhere on the green line on the graph.
If your score lies above the green line, then you are successfully distinguishing reliable answers from uncertain ones. If your score lies below, then you are either misjudging how well you know and understand the topics, or you may have serious misconceptions (things you confidently believe, but that are not so!).

Bono por Puntuación Basada en Certeza y 'Bono + por Exactitud' Mejorado

It is convenient to express a student's ability to distinguish reliable from uncertain knowledge (as shown on the graph above) by adjusting conventional 'accuracy' with the addition of a simple bonus. The CBM Bonus is proportional to the amount the CBM average lies above the green line on the graph. The bonus is negative** if the score lies below the line. For the example score shown in yellow, the CBM average is 0.3 marks above the line, leading to a bonus of +3%. This can be added to the accuracy (=80%) to give Accuracy + CBM Bonus = 83%.

^** If you think a negative bonus strange, consider the benefits if bankers' bonuses could go negative. A negative CBM bonus typically flags what were described by Confucius as failures "when you do not know a thing, to allow that you do not know it" and by Donald Rumsfeld, more concisely, as "unknown unknowns". They are both essentially negative knowledge, worse than acknowledged ignorance.

The amount of CBM Bonus is determined by a multiplication factor β used to convert the mark difference to a bonus. A value of β=0.1 is used in Moodle, being both simple and essentially optimal when the resultant score (Accuracy + Bonus) has been judged by its statistical reliability and ability to predict performance on related tests. Ref: Gardner-Medwin, 2013
Consideration of Accuracy and CBM Bonus, rather than direct CBM grades or average scores, helps examiners to maintain existing standards criteria (typically based on accuracy) while readily seeing the added discrimination between students arising from CBM.

Example CBM data from a UCL medical exam.

Puntación relacionada con solamente parte de un Examen

CBM is particularly valuable for self-tests, where students challenge themselves as part of their study. Answers may be omitted because they are not on a topic of current interest, they are too easy, or perhaps not yet studied. CBM scores can then be presented relative to the set of questions the student has chosen to respond to. All correct on just half of a quiz would then show as 100% accurate. It is up to the student to interpret such scores: obviously if questions are omitted because they are difficult, then the relative score may overestimate competence in the subject. Moodle 'Grades' are always relative to the whole quiz, and this is the default presentation for other scores too.

Moodle 2.6

Moodle 2.7

PLUGIN importante para Moodle 2.6 y 2.7

Moodle 2.6 y 2.7 requieren que el Quiz Report Plugin esté instalado para poder tener reporte completo de calificación Basada en certeza.

Para mayor información acerca de puntuaciones, vea Self test scores with and without CBM
ARGM Dec 2013. Publications.

Puntuaciones

Para mayor información sobre puntuaciones (relacionadas con la versión UCL de CBM, aunque también es relevante para Moodle) por favor vea test scores with and without CBM por Tony Gardner-Medwin.

Documentation

Usando Puntuación Basada en Certeza

Sumario