Project Inspire API - Revision history

Dmonllao at 08:28, 28 September 2017

2017-09-28T08:28:25Z

← Older revision		Revision as of 08:28, 28 September 2017
Line 6:		Line 6:
	\|assignee = [[User:David_Monllaó]]		\|assignee = [[User:David_Monllaó]]
	}}		}}

			{{Note\|This is [https://moodle.org/plugins/tool_inspire Project Inspire plugin] API, do not confuse with Moodle core [[Analytics API]] which is the one that was merged into core. Moodle core Analytics API is an extension of this API specification.}}

	= Project inspire =		= Project inspire =

Dmonllao: /* Analyser */

2017-06-09T16:41:50Z

Analyser

← Older revision		Revision as of 16:41, 9 June 2017
Line 58:		Line 58:

	Other analyser classes responsibilities:		Other analyser classes responsibilities:
			* Define the context of the predictions
	* Discard invalid data		* Discard invalid data
	* Filter out already trained samples		* Filter out already trained samples

Dmonllao: /* Summary */

2017-05-11T09:05:44Z

Summary

← Older revision		Revision as of 09:05, 11 May 2017
Line 18:		Line 18:

	An obvious example of a model you may be interested on is prevention of students at risk of drop out: Lack of participation or bad grades in previous activities could be indicators, the target could be whether the student is able to complete the course or not. Moodle will calculate these indicators and the target for each student in a finished course and predict which students are at risk of dropping out in ongoing courses.		An obvious example of a model you may be interested on is prevention of students at risk of drop out: Lack of participation or bad grades in previous activities could be indicators, the target could be whether the student is able to complete the course or not. Moodle will calculate these indicators and the target for each student in a finished course and predict which students are at risk of dropping out in ongoing courses.

			=== API components ===

			[[File:Inspire_API_components.png]]

			=== Data flow ===

			[[File:Inspire_data_flow.png]]

	== Built-in models ==		== Built-in models ==

Dmonllao: /* Target */

2017-03-16T14:55:07Z

Target

← Older revision		Revision as of 14:55, 16 March 2017
Line 69:		Line 69:
	* '''Target''': 'difficulties to pass a specific quiz'. '''Analyser provides''': 'quiz attempts in a specific quiz'		* '''Target''': 'difficulties to pass a specific quiz'. '''Analyser provides''': 'quiz attempts in a specific quiz'

	Only binary classifications will be initially supported although the API will be extended in future to support discrete (multiclass classification) and continuous values (linear regression).		A callback defined by the target will be executed once new predictions start coming so each target have control over the prediction results.

			Only binary classifications will be initially supported although the API will be extended in future to support discrete (multiclass classification) and continuous values (linear regression).

			==== Insights ====

			Another aspect controlled by targets is the insights generation. Analysers samples always have a context, the context level (activity module, course, user...) depends on the sample but they always have a context, this context will be used to notify users with '''tool/inspire:listinsights''' capability (teacher role by default) about new insights being available. These users will receive a notification with a link to the predictions page where all predictions of that context are listed.

			Each prediction will have a set of actions available, in cases like ''Students at risk of dropping out'' the actions can be to send a message to the student, to view their course activity report...

	=== Indicator ===		=== Indicator ===

Dmonllao: /* Analysable */

2017-03-16T10:20:13Z

Analysable

← Older revision		Revision as of 10:20, 16 March 2017
Line 124:		Line 124:
	==== Analysable ====		==== Analysable ====

	Analysable items are analysed by analysers :P.		Analysable items are analysed by analysers :) In most of the cases an analysable will be a course, although it can also be the site or any other element.

	We will include two analysers ''\tool_inspire\course'' and ''\tool_inspire\site'' they should be enough for most of the stuff you might want to analyse. They need to provide an id, a \context and get_start() and get_end() methods if you expect them to be used on models that require time splitting. Read related comments above in [[#Analyser]].		We will include two analysers ''\tool_inspire\course'' and ''\tool_inspire\site'' they should be enough for most of the stuff you might want to analyse. They need to provide an id, a \context and get_start() and get_end() methods if you expect them to be used on models that require time splitting. Read related comments above in [[#Analyser]].

	A possible extension point of ''\tool_inspire\sitewide'' would be to redefine get_start() and get_end()~~. I can't think of other possible extensions right now~~.		A possible extension point of ''\tool_inspire\sitewide'' would be to redefine get_start() and get_end().

	==== Calculable ====		==== Calculable ====

Dmonllao: /* Prediction processors */

2017-03-16T10:16:41Z

Prediction processors

← Older revision		Revision as of 10:16, 16 March 2017
Line 92:		Line 92:


	Prediction processors are the machine learning backends that process the datasets generated from the calculated ~~models'~~ indicators and targets. They are a ''tool_inspire'' plugin subtype, this makes it a pluginable system with a common interface:		Prediction processors are the machine learning backends that process the datasets generated from the calculated indicators and targets. They are a ''tool_inspire'' plugin subtype, this makes it a pluginable system with a common interface:

	* Evaluate a provided model		* Evaluate a provided model

Dmonllao: /* Prediction processors */

2017-03-16T10:15:22Z

Prediction processors

← Older revision		Revision as of 10:15, 16 March 2017
Line 91:		Line 91:
	=== Prediction processors ===		=== Prediction processors ===

	Prediction processors are a 'tool_inspire' subtype, this makes it a pluginable system with a common interface:
			Prediction processors are the machine learning backends that process the datasets generated from the calculated models' indicators and targets. They are a ''tool_inspire'' plugin subtype, this makes it a pluginable system with a common interface:

	* Evaluate a provided model		* Evaluate a provided model
Line 97:		Line 98:
	* Predict targets based on previously trained algorithms.		* Predict targets based on previously trained algorithms.

	The communication between prediction processors and Moodle is through files because these processors can be writen in PHP, in Python, other languages or even use cloud services ~~internally~~. This needs to be scalable so they are expected to be able to manage big files and train algorithms reading input files in batches if necessary.		The communication between prediction processors and Moodle is through files because these processors can be writen in PHP, in Python, other languages or even use cloud services. This needs to be scalable so they are expected to be able to manage big files and train algorithms reading input files in batches if necessary.

	== Design ==		== Design ==

Dmonllao: /* Time splitting methods */

2017-03-16T10:01:45Z

Time splitting methods

← Older revision		Revision as of 10:01, 16 March 2017
Line 81:		Line 81:
	=== Time splitting methods ===		=== Time splitting methods ===

	In some cases the time factor is not important and we just want to classify a sample, that is fine, things get more complicated ~~though~~ when we want to predict what will happen in future. e.g. predictions about students at risk of dropping out are not useful once the course is over or when it is too late for any intervention.		In some cases the time factor is not important and we just want to classify a sample, that is fine, things get more complicated when we want to predict what will happen in future. E.g. predictions about students at risk of dropping out are not useful once the course is over or when it is too late for any intervention.

	Calculations in time ranges is the most challenging aspect this project ~~will face~~. Indicators need to be designed with this in mind and we ~~will~~ need to include time-dependant indicators to the calculated indicators so machine learning algorithms are smart enough to avoid mixing calculations belonging to the beginning of the course with calculations belonging to the end of the course.		Calculations in time ranges is the most challenging aspect this project faces. Indicators need to be designed with this in mind and we need to include time-dependant indicators to the calculated indicators so machine learning algorithms are smart enough to avoid mixing calculations belonging to the beginning of the course with calculations belonging to the end of the course.

	There are many different ways to split up a course in time ranges: in weeks, quarters, 8 parts, ranges with longer periods at the beginning and shorter periods at the end... And the ranges can be accumulative (from the beginning of the couse) or only from the start of the range.		There are many different ways to split up a course in time ranges: in weeks, quarters, 8 parts, ranges with longer periods at the beginning and shorter periods at the end... And the ranges can be accumulative (from the beginning of the couse) or only from the start of the time range.

	Which time split method will be better at predicting depends on the model.		Which time split method will be better at predicting depends on the model. The model evaluation processs iterates through all enabled time splitting methods and returns the prediction accuracy of each of them.

	=== Prediction processors ===		=== Prediction processors ===

Dmonllao: /* Indicator */

2017-03-16T09:58:11Z

Indicator

← Older revision		Revision as of 09:58, 16 March 2017
Line 75:		Line 75:
	Also defined as PHP classes, their responsibility is quite simple, to calculate the indicator using the provided sample.		Also defined as PHP classes, their responsibility is quite simple, to calculate the indicator using the provided sample.

	Indicators are not limited to one single analyser like targets are, this makes indicators easier to reuse along models.		Indicators are not limited to one single analyser like targets are, this makes indicators easier to reuse along models. Indicators specify a minimum set of data they need to perform the calculation. The indicator developer should also make an effort to imagine how the indicator will work when different analysers are used. For example an indicator named ''Posts in any forum'' could be initially coded for a ''Shy students in a course'' target; this target would use ''course enrolments'' analyser, so the indicator developer knows that a ''course'' and a ''user'' will be provided by that analyser, but this indicator can be easily coded so the indicator can be reused by other analysers like ''courses'' or ''users'. In this case the developer can chose to require ''course'' '''or''' ''user'', the name of the indicator would change according to that: ''User posts in any forum'', which could be used in models like ''Inactive users'' or ''Posts in any of the course forums'', which could be used in models like ''Low participation courses''

	The calculated value can go from -1 (minimum) to 1 (maximum). This guarantees that we will not have indicators like ''absolute number of write actions'' because we will be forced to limit the calculation to a range, e.g. -1 = 0 actions, -0.33 = some basic activity, 0.33 = activity, 1 = plenty of activity.		The calculated value can go from -1 (minimum) to 1 (maximum). This guarantees that we will not have indicators like ''absolute number of write actions'' because we will be forced to limit the calculation to a range, e.g. -1 = 0 actions, -0.33 = some basic activity, 0.33 = activity, 1 = plenty of activity.

Dmonllao: /* Analyser */

2017-03-16T09:36:21Z

Analyser

← Older revision		Revision as of 09:36, 16 March 2017
Line 47:		Line 47:
	Analyser classes are responsible of creating the dataset files that will be sent to the prediction processors.		Analyser classes are responsible of creating the dataset files that will be sent to the prediction processors.

	The base class ''\tool_inspire\local\analyser\base'' does most of the stuff, it contains a key abstract method though, ''get_all_samples()'', this method is what defines what is a sample. A sample can be ~~everything, from~~ a course, a user, an enrolment, a quiz attempt... Samples are nothing by themself, just a list of ids, it is when combined with target and indicator classes that they make sense.		The base class ''\tool_inspire\local\analyser\base'' does most of the stuff, it contains a key abstract method though, ''get_all_samples()'', this method is what defines what is a sample. A sample can be any moodle entity: a course, a user, an enrolment, a quiz attempt... Samples are nothing by themself, just a list of ids, it is when combined with the target and the indicator classes that they make sense.

	~~Some of its other~~ responsibilities:		Other analyser classes responsibilities:
	* Discard invalid data		* Discard invalid data
	* Filter out already trained samples		* Filter out already trained samples