Cognitive operator: Ordering
Operator
Complexity level
Ordering operator is of complexity 1, as there is only one arrangement action on a linear problem dimension.
Operations
Operation 1 : Define order of the input set from "LESS" to a "MOST" value of some criteria defined in teacher's requirements
Purpose
A typical exercise that precedes complex decision making is ordering things in a rough, non quantitative "more than" or "less than" feeling applied to inputs. Ordering can be a simple exercice when inputs and requirements match onto a measurable attribute, f.e. : it is simple to order historical facts based on dates. Ordering more "conceptual" predicates that are less related to measurable or valuable (thus comparable) attribute, or that have multiple valuable attributes not clearly related to teacher's requirement can become a bit hard.
Algorithms
The main algorithm in this operator is defining a comparison between two ordering from distinct users, in order to get a quantifiable metric of the divergence.
One of the start point of a usable metric is the algorithm that counts the number of permuations that is needed to start from one ordered list and to reach the other as end point. F.e. say a first student answers A,B,C,D,E, and the second answers B,A,C,D,E. One unique permutation of A,B is necessary to get list 2 from list 1. Say a third student answers F,B,C,D,E, the number of permutation (sum of absolute position shifts of each member) is 12. So here we can say student 3 is further than student 2 from student 1.
Second step for refining metric will consider position in the list 1. Inverting top members of the list, or last members has not the same meaning on decision shift : B,A,C,D,E needs to be closer to A,B,C,D,E than A,B,C,E,D. This is not rendered by our first calculation. We can refine the calculation by multiplying each local shift by the relative weight of the item in the reference user :
Say user 1 is the reference : A,B,C,D,E will be default assigned to weights : 5,4,3,2,1
Thus shift metric of user 2 is now : 1*5 + 1*4 + 0*3 + 0*2 + 0*1 = 9
Shift metric of user 3 is now : 0*5 + 0*4 + 0*3 + 1*2 + 1*1 = 3
In which we can now say that use 3 is "nearer choice" to user 1 than user 2.
More theoretical development on metric model (to be done)
Ref http://www-poleia.lip6.fr/~marcin/papers/Lesot_EGC_09.pdf
Applications
Direct Workflow
In a direct scenario, the students must order a set of "things" against a requirement the teacher has given.
Non valuated version
- Teacher provides a set of choosen inputs
- Teacher setup the exercice and writes requirement ("Order by xxxxx using yyyy")
- Student processes order
- Student checks his own order against the environment (peer trends in semi-blind mode, or direct peer answers in full view mode)
- Student report about his choice and argue in report
- Teacher gives final feedback to peers or globally
Evaluated version
- Teacher provides a set of choosen inputs
- Teacher setup the exercice and writes requirement ("Order by xxxxx using yyyy")
- Teacher makes the "teacher version" of the ordering, standing to the the "good answer'
- Student processes order
- Student checks his order against teacher's order and get shift evaluation
- Teacher feedbacks if required
Reverse Workflow
In a reverse scenario, the teacher needs order a set of proposals given by the students and give feedback about his choice.
- Students collectively input some proposals
- Teacher setup the Ordering operator
- Teacher makes ordering
- Students observe teacher's anwers
- Students can accessorily order themselves as counter submission
- Student report and argue on teacher's choice
- Teacher gives final feedback
Meta Workflow
Help to discover the concept of "comparable" and what makes things easier to compare when a valuation attribute can be found.
Implementation
Setup
absolute order: When absolute order, there is no way to accept some inputs may be equal to other.
Can replay: The participant will be able to update his answers event after trends or feedback has been given
Only for teachers :
Blindness: The degree of privacy of this operator. (defaults to the activity privacy setting).
Categorize GUI
Master part of the GUI is dedicated to let the participant order the list. First model in prototype provides a simple flat table with 'up' and 'down' controls. This might be not very operable for big lists of more than 10 inputs.
TODO : Find a drag&drop method for easier reordering / Find a nice way to handle big lists (30, 50 items)
Blind mode: The user just orders the items. No trends.
Semi blind mode: The user orders the items, and is given a global 'average' trends about match or no match position of each item (number of choices of this item at the same location / percentage of choices over all given answers of this item at this location). Trends are displayed once one answer is given.
Full View mode: Same as above, but the trend is displayed BEFORE first answer is given.
Display GUI
Describes what can be expected as result display and exploration GUI behaviour for each privacy mode (Blind, Semi-blind, Full view / No privacy).