Note:

If you want to create a new page for developers, you should create it on the Moodle Developer Resource site.

JS Framework Specification

From MoodleDocs
JS Framework Specification
Project state Planning
Tracker issue MDL-48392
Discussion https://moodle.org/mod/forum/discuss.php?d=268190
Assignee Damyon Wiese


Background

Moodle has used YUI as it's javascript framework (since 2006). We have built in support for using their combo loaders, javascript build chain, module support, debugging and many other nice features. We use YUI to provide a standard API between different browser features and for the many nice things that are built in like dialogs, ajax requests, events, classes and inheritance.

Earlier this year (2014) Yahoo announced:

we have made the difficult decision to immediately stop all new development on YUI in order to focus our efforts on this new technology landscape. This means that, going forward, new YUI releases will likely be few and far between, and will only contain targeted fixes that are absolutely critical to Yahoo properties.

This started a (very constructive) forum discussion on the future of JS in Moodle (https://moodle.org/mod/forum/discuss.php?d=268190).

And following that discussion we made a policy decision (https://tracker.moodle.org/browse/MDL-47036) to create a "first prototype/example as described by Dan above with JQuery, RequireJS and grunt".

This document is meant to clearly document the features / characteristics of this prototype/example and compare it to our existing YUI integration. It should discuss how the 2 solutions can co-exist during a transition period.

Summary of forum discussion

The forum discussion has had many contributors and is full of valuable comments. Please read it: https://moodle.org/mod/forum/discuss.php?d=268190.

Here is a much shorter summary of most of the things that were discussed:

Summary of options for JS library

No framework

pros:

  • less overhead on each page (maybe)
  • Won't find ourself in the same positino in future.

cons:

  • we need to re-implement the cross browser support (drag and drop, ajax ...),
  • we are supposed to be building an LMS and tools for teachers, not JS libraries

JQuery

pros:

  • Most popular choice,
  • lots of developers with experience,
  • lots of other libraries rely on jquery,
  • well supported, long life time, "60.4% of web sites use jQuery",
  • fast,
  • many corporate supporters,
  • clearly the most voted for option in the discussion,
  • backing from deque (accessibility),
  • not too different to YUI
  • already used by add-ons
  • already bundled with moodle

cons:

  • Is it too much?,
  • quality of plugins,

Note: This does not include JQuery UI. We may include that eventually, but probably not AS-IS. Moodle has very strict requirements for accessibility that go beyond what JQuery UI provides. It is possible to use JQuery UI in an accessible way, but this would need a moodle wrapper to make sure we do it consistently (e.g. http://www.deque.com/blog/accessible-jquery-ui-datepicker/ ). Also we want to get to a point where the JS UI look consistent to the php generated UI elements. This could be by making themers use http://api.jqueryui.com/theming/css-framework/ to mirror their styles, or by modifying the widgets to pull their HTML from templates, or by building new widgets from the ground up that do what we want.

Use a set of polyfills and use ES6 syntax for everything

pros:

  • nice "elegant" solution

cons:

  • testing requirements go up greatly,
  • we will probably end up rolling our own JS frame work piece by piece,
  • plugins devs would all include jquery anyway (bootstrap).

Angular

pros:

  • trendy,
  • clean MVC design

cons:

  • requires backend rewrite

Ember

cons:

  • requires backend rewrite

Attributes we want in a JS framework

long term support cycles, standardize things that are not standard across browsers (dom manipulation, positioning, ajax, events, drag and drop?, 2d drawing?, transitions, accessible widgets?), sandboxing, dependency loading, able to (easily) load third party modules, performance

Note: Just because we have it now in YUI does not mean we need to have it from day 1 of the new framework - YUI will exist for a while and we choose the best extra library available for non-standard things (like 2d drawing, accessible widgets).

Summary of options for loading modules

YUI loader

implements modules, sandboxing namespaces, dependency resolution, combo loading, build tools (minification, linting), classes, inheritance, small set of supported widgets, events

pros: we already include the library

cons: does not look to external devs like we have shifted from yui, requires YUI syntax to load modules (e.g. after the page has loaded)

note: it has ways to load ES6 modules (after transpiling them http://yuilibrary.com/yui/docs/yui/es6-modules.html)

RequireJS

implements: modules, dependency resolution, build tools (minification, linting) (requires node.js) cons: ES6 is a newer standard combining RequireJS and CommonJS syntax

ECMAScript 6 modules

implements: modules, sandboxing, dependency resolution

pros: best forward looking option (standards based)

cons: new standard - no browser support yet - but there is a polyfill (https://github.com/google/traceur-compiler)

Summary of options for build tools

What is a build tool - think minification, linting, module packaging, generate documentation.

Shifter

pros: We use it now

cons: doesn't support ES6 modules except to convert them to YUI modules

Use a PHP library (not minify - maybe jshrink)

pros: Php library - we can automate the minification and tie it to purge caches, don't have to check JS has been compiled, don't have to include build products in git

cons: minify is old and deprecated need to replace it with something better, linting would need a separate tool (maybe part of code checker or cibot)

Grunt

pros: used by a lot of developers, can do more (unit tests), compile less, could run shifter too - so would be a combined process covering old/new

cons: another build tool for new developers to support (barriers)

Prototypes

Based on the above summaries (and the lengthy discussion) - there are some worthwhile candidates we should test out:

RequireJS integration

With JQuery - http://requirejs.org/docs/jquery.html

Git branch: https://github.com/damyon/moodle/tree/REQUIREJS

This branch uses vanilla RequireJS with no build step. Modules are minified with our current JS minification library on the fly for production only.

The module format is AMD and config for JQuery is built in.

Example of loading (sandboxed) jquery and calling it in a module.js file:

require(['jquery'], function( $ ) {

   $('h1').hide('slow');

});

Example of creating a module in a plugin:

Create file mod/assign/amd/test.js: // This module depends on jquery define(['jquery'], function($) {

   return {
       // This module has one get function which returns the text from the heading node in the page.
       get: function() {
           return $('h1').text();
       }
   }

});

How to use the above module: require(['mod_assign/test'], function(test) {

   console.log(test.get());

}); The jquery dependency was automatically pulled in - and the component name mapped to the modules area for the component.

Plugin authors need to use r.js manually to build bundles using this approach (A bundle is a collection of modules that is meant to be loaded together).

ES6 Modules (with polyfill)

With JQuery loaded as ES6 module (maybe transpiled)

Need to test loading speed - concurrency etc

After testing this - it seems a bit of a dodgy solution because of the way the polyfill works. It either requires "transpiles" (a build step) the ES6 module to ES5, or requires the entire traceur library to do the transpiliing in the browser (slow).

Since traceur can transpile ES6 to AMD modules, maybe we should test an automated version of this so we write in ES6, but transpile to AMD and load with RequireJS. This gives us a future proof format that works now with a stable loader. Also - try 6in5.

Note - in researching this - I did not find any current libraries using the ES6 format with a transpile step in their build. This would introduce a difference between our JS and the JS we find in the real world, which is not a good thing. I'm not recommending this approach until it becomes more widespread.

Grunt

Dave has been working on grunt support and has a branch where you can run grunt in any directory, and it will either run shifter with the correct options, or manually run the required uglifier steps the minify AMD javascript.

Automated minification

We have a prototype of this approach here: https://github.com/damyon/moodle/tree/MUSTACHE

(It also has mustache support because I have been working on both).

How this branch works:

When you request a module from requirejs like this:

require(['core/alert'], function(Alert) {
    var a = new Alert('I have a message', 'to say');
});

The requirejs config tells requirejs that the "lib/requirejs.php" script is the base of all urls.

The "lib/requirejs.php" script will return the requested module, correctly minified and with the correct module name inserted - but also all other AMD modules from all plugins and subsystem dirs in Moodle. This is similar to what the "r.js" optimiser tool does, but we need it to be done at request time because you can add/remove plugins from Moodle.

Question: So - you get one mega minified JS file with all known modules as your response - won't that be huge? Answer: No - not too huge - and the performance benefit of reducing JS requests is worth it. And it will be cached. The sum of all of our minified JS in all of our custom written yui modules is 800k. YUI itself is 2.8MB which is why it doesn't survive without a combo loader. Some google engineers in this presentation: https://www.youtube.com/watch?v=mGENRKrdoGY suggest that the point you want to start splitting your JS up is aroung 1MB - we have a lot of JS to write in the new format before we get to that point. Finally - our current JS uses handlebar templates defined in the Javascript - we have a new way of loading templates via AJAX that will be better than this.

Pros: No build steps Cons: No lint step (could be done in codechecker, or by manually running jshint).