Data formats: Difference between revisions
David Mudrak (talk | contribs) m (Reformat, change Dataformat to human readable data format) |
David Mudrak (talk | contribs) m (Add some links to other plugin pages) |
||
Line 1: | Line 1: | ||
Data formats are plugins that define how a table of data can be exported for download. Moodle comes with plugins for CSV, Excel, ODS, JSON, and HTML. | Data formats are [[Plugin types|plugins]] that define how a table of data can be exported for download. Moodle comes with plugins for CSV, Excel, ODS, JSON, and HTML. | ||
== Components == | == Components == | ||
Line 9: | Line 9: | ||
; dataformat/FORMATNAME/version.php : Version definitions, see [[version.php]]. It is highly recommended always to have it and it is required if you have any files in db folder | ; dataformat/FORMATNAME/version.php : Version definitions, see [[version.php]]. It is highly recommended always to have it and it is required if you have any files in db folder | ||
Generally no other files are needed, but all the other files available to other plugins could also be used here | Generally no other files are needed, but [[Plugin files|all the other files available to other plugins]] could also be used here. | ||
== Creating a new data format == | == Creating a new data format == |
Revision as of 09:44, 20 May 2016
Data formats are plugins that define how a table of data can be exported for download. Moodle comes with plugins for CSV, Excel, ODS, JSON, and HTML.
Components
- dataformat/FORMATNAME/lang/en/dataformat_FORMATNAME.php
- Contains the English language strings used in the format. You can also define strings for other languages.
- dataformat/FORMATNAME/classes/writer.php
- This file is the real meat of the form, it contains class which extends \core\dataformat\base defining how the data stream should be encoded.
- dataformat/FORMATNAME/version.php
- Version definitions, see version.php. It is highly recommended always to have it and it is required if you have any files in db folder
Generally no other files are needed, but all the other files available to other plugins could also be used here.
Creating a new data format
The data format interface is quite simple, to create a new format the easiest approach would be to clone and rename one of the existing text based formats such as /dataformat/json or /dataformat/html.
Performance
The design intent of the data formats is to have low and ideally fixed memory overhead, and to stream data to the browser record by record. This enables very large datasets to be exported easily and safely without running into a whole range of issues including php memory thresholds, php execution timeouts, and other timeouts in every layer of the stack at the apache / nginx layer, a caching layer such as varnish, or load balancers, any intermediate proxies, and finally any browser time-outs.
So please avoid building up a complete data structure in memory, or ideally not even on disk either. Instead just encode the data record by record as the data stream is processed and echo it to the browser. The HTML, JSON, and CSV data formats are good examples of this. The Excel and ODS formats are far more performant than in earlier versions of moodle, but still suffer from network inactivity timeouts at very large scale because they buffer the file to disc before finally sending when complete.