Data formats

Dataformat's are plugins that define how a table of data can be exported for download. Moodle comes with plugins for CSV, Excel, ODS, JSON, and HTML.

Components

dataformat/FORMATNAME/lang/en/dataformat_FORMATNAME.php

Contains the English language strings used in the format. You can also define strings for other languages.

dataformat/FORMATNAME/classes/writer.php

This file is the real meat of the form, it contains class which extends \core\dataformat\base defining how the data stream should be encoded.

dataformat/FORMATNAME/version.php

Version definitions, see version.php. It is highly recommended always to have it and it is required if you have any files in db folder

Generally no other files are needed, but all the other files available to other plugins could also be used here such as db files.

Creating a new Dataformat

The dataformat interface is quite simple, to create a new format the easiest approach would be to clone and rename one of the existing text based formats such as /dataformat/json or /dataformat/html.

Performance

The design intent of the Dataformat's is to have low and ideally fixed memory overhead, and to stream data to the browser record by record. This enables very large datasets to be exported easily and safely without running into a whole range of issues including php memory thresholds, php execution timeouts, and other timeouts in every layer of the stack at the apache / nginx layer, a caching layer such as varnish, or load balancers, any intermediate proxies, and finally any browser time-outs.

So please avoid building up a complete data structure in memory, or ideally not even on disk either. Instead just encode the data record by record as the data stream is processed and echo it to the browser. The HTML, JSON, and CSV dataformat's are good examples of this. The Excel and ODS formats are far more performant than in earlier versions of moodle, but still suffer from network inactivity timeouts at very large scale because they buffer the file to disc before finally sending when complete.