Development:Open protocol for accessing question engines

History

Previously (2004 to early 2005) there was an attempt (by the Serving Maths Project, Gustav Delius and others) to create something called Remote Question Protocol. This was a web service interface designed to allow question from different question engines (for example AIM and STACK) to be included in Moodle quizzes. The effort ran out of resources at a time when the code was not in a working state. Several groups have an interest in getting RQP working again, although no one is currently working on it.

During 2005, The Open University (mainly Sam Marshall) developed in-house an online assessment system called OpenMark. Its architecture was of separate question engines and a test navigator, communicating via a web service link which we call POQE (protocol for open question engines).

Then in May 2006, I (Tim Hunt) took over maintenance of the quiz module from Gustav. At which point I learned about RQP. I was immediately struck by how similar POQE and RQP were conceptually, although they differed in details.

Now (autumn 2006), the OU wants to be able to include OM question in Moodle quizzes. Our choices are to either

1. change OM to use RQP; or
2. change (add to) Moodle to support POQE.

We have pretty-much decided to do 2., because it is less work for us, and becuase POQE currenly works and RQP doesn't. Therefore, RQP implementing question engines will have to change anyway, and since POQE is conceptually similar to RQP, changing to POQE won't be much more work that changed to whatever the next iteration of RQP would have been.

By the way, we are hoping that it will prove possible to open-source OpenMark, which will increase the number of POQE compatible systems that are freely available.

Rationale

Why are people interested in this sort of protocol? The answer is separation of concerns. The things an online assessment system has to do are:

1. Identify students, test, which students can do which tests at what times, and which questions make up each test.
2. Record the scores and other information that results from students attempting test, and provide reporting fascilities for this information.
3. Display questions to students and process the responses the students make to generate scores, feedback, etc.
4. Allow teachers to create and configure the questions.

Moodle is very good at points 1 and 2, there is a lot of boring book-keeping do be done here, and it is already implemented in Moodle.

For the developers who are interested in POQE/RQP, the really interesting part of an assessment system is 3. They want to be able to pose really intersting types of questions to students to really make them think, and then use sophisticated algorithms to analyse the student's response and give targeted feedback when they get things wrong.

Doing this may require specialist tools. For example, STACK and AIM both use Computer algebra systems to set and mark interesting mathematical questions. OpenMark lets question authors write answer marking and feedback generation algorithms in Java code, with a library of helpful classes and functions. It is simply not feasible to rewrite these question engines in PHP and turn them into Moodle question types. Hence the search for another way to get these questions into Moodle.

Of course, each of STACK, AIM, OM, etc. could programme their own solutions to the administration parts 1 and 2. However, this is a needless duplication of effort. It does not allow questions from different question engines to be combined in the same test. And people who are interested in writing advanced question engines are normally not very interested in writing student administration systems.

Finally, there is point 4, question authoring. Authoring advanced question types is a complicated task that will probably need a specialised interface. For example STACK provides its own editing interface. OM questions are authored in a Java IDE, then complied to a JAR file and uploaded to the server. This is all beyond the scope of POQE. RQP started trying to address authoring at about the point that it was abandoned.

Our approach is that POQE shoudl be optimised point 3 above - presenting questions to students and processing their responses. This is the point where interoperability is required. Interoperability is hard, so let us keep the scope of the protocol as small as possible. Interoperability of student data and results is a separate problem. Authoring of advanced question types is specialised to the paticular question engine, and interoperability is not possible.

All this is in the context of an online assessment system. Therefore, we are interested in questions that get presented to students in their web browser as HTML, with accompanying CSS, Javascript, Images and other media files. Student responses get sent back as form sumissions, that is HTTP POST requests. Concievably the trendy XMLHttpRequest could be used instead, but it is not really necessary. POQE only works with form submissions, which is simple and sufficient.