Very flexible block system proposal

Note: This page is a work-in-progress. Feedback and suggested improvements are welcome. Please join the discussion on moodle.org or use the page comments.

Moodle 2.0

This is my attempt the devise the most flexible possible blocks system I could, which still had reasonable performance. (Whether that was a sensible thing to do remains to be seen ;-))

Goals

Existing blocks should continue to work in the new scheme. (If absolutely necessary, we may require small changes.)
Consistent implementation of (essentially) every page in Moodle.
Let themes, and possibly the page itself, determine where on the page the blocks should appear. (I.e, not just 'right' and 'left'.)
Much more flexible sticky blocks. (Show this on very course in category X, Add this block to every wiki in my system, ...)
Easy way to set up default blocks for certain sorts of page.

(Note, this proposal mentions some third-party plugins because they are examples of people trying to push the boundaries of what is possible. We are not proposing to include these plugins in the core Moodle release. They are only mentioned to show how this proposal would support their requirements.)

Interesting use cases

Some more miscellaneous things to consider:

Switching to sticky blocks

Admin notices that almost all courses in the site use the participants block. They decide that, instead of having all these individual instances, and to promote consistency, they want to add this as a sticky-block. However, when they do, they will then need to remove all the old individual instances. We need to provide a nice UI to facilitate this.

Perhaps we can provide better drill-down into the list of block instances (the link in the 'Instances column' of Administration > Modules > Blocks > Manage blocks); in the report, give ways to edit the block instances; and then link into that from the sticky blocks UI.

What to show a block on page X of this lesson

This could be done by giving each lesson page a shortname.

Show this HTML block on pages 6-8 of this quiz

Tricky. Between 6 and 8 is not an easy pattern to match in SQL. What happens if pages are added or removed from the quiz later? Only work-around I can think of is to let quiz pages have shortnames.

Setting the scene

This relies on some ideas from Navigation/Pagelib/Blocks_2.0_design.

What happens when a page is requested

The page, for example, mod/quiz/view.php, has to do this.

require_once('../../config.php');

As part of the standard set-up, the $PAGE global object is created (we are getting rid of the different page subclasses). $PAGE has pagetype set to 'mod-quiz-view'.

// Early on in mod/quiz/view.php.
$PAGE->set_context(get_context_instance(CONTEXT_MODULE, $cm->id));
$PAGE->set_title(format_string($quiz->name));
$PAGE->add_navigation(format_string($quiz->name));
$PAGE->this_page_params('id', 'q');
$PAGE->owner_module('quiz'); // used when looking up some page-related things in the lang files.
$PAGE->complete_setup();

// ...

// Later in mod/quiz/view.php, before  other  output.
$OUTPUT->header();

// ...

// At the end of mod/quiz/view.php.
$OUTPUT->footer();

$PAGE needs to know the context associated with this page. The other calls replace parameters to print_header. That is, $OUTPUT->header() gets all the information it needs form $PAGE. The call to set_context automatically builds most of the required links in the navigation bar, we just need to add any extra links. (Actually, we might be able to make set_title automatically add a link at the end of the nav bar, which would be all that is required in this case).

$PAGE->this_page_params tells the page which URL params you need to preserve to reload this page exactly. Useful for doing redirects after editing actions.

What happens in $PAGE->complete_setup()

Here, the list of blocks currently on this page is fetched from the database.

Now that we know exactly where we are in Moodle, the current theme and language can be worked out. Actually, no. That bit happens earlier, during require_login, where $COURSE is set up, as at present.

The theme is asked for the following pieces of information:

what places on the page should blocks be allowed to appear
which of those places is the default place for adding new blocks.

If any blocks are configured to appear with a 'place' that the theme does not recognise (suppose the theme was just changed), they are instead put in the default place for new blocks.

Also, any block actions in the URL are processed, in which case there is a redirect back to the same page again. (This is why we need this_page_params).

What happens in $HTML->print_header() and print_footer()

These not only output the parts of the page they do at the moment. They are also responsible for outputting the large-scale page layout, and printing all the blocks (probably using some library functions to help).

Thus mod/quiz/view.php page does not have to worry about blocks at all. It just has to output the main contents of the page.

These screen mock-ups May make this clearer.

At the start of $OUTPUT->header is where we query the database to find all the blocks that will appear on the page. We then call a function in each block and each active filter to give them a chance to do things before any HTML is output. (For example, require_css.)

Some optional things the page can do

The quiz view page is a fairly normal sort of page, so it does not have to do much more. Other pages may need to do one or more of the following:

$PAGE->set_page_class(PAGECLASS_POPUP); // or PAGECLASS_WIDEREPORT

This is a hint (to the theme) that it might be better not to have blocks on this page. We will have a few pre-defined classes of page. Possibly just these three. Or maybe we don't bother. If teachers really want to add blocks both sides of the grader report, perhaps we should let them. (But, I think when it comes to sticky blocks, this might be necessary.)

$PAGE->set_sub_page($somename);

This is needed by, for example, the Lesson, where all pages have the same pagetype and contextid, but the teacher may want different blocks on different pages (I am assuming that each lesson page would be given a shortname to support this). Or on the quiz attempt page, sub_page would be the page number within the quiz.

$PAGE->set_page_type('mod-quiz-attempt');

The quiz review page could do this if it wants to force the review page to have the same blocks as the quiz attempt page.

$PAGE->blocks->add_block_region('mymoodle-centrecolumn');

If the My Moodle page (say) wants to let the user put blocks in the centre of the page, irrespective of what the theme wants to allow in general.

$fakeblock->name = 'quiznavigation';
$fakeblock->header = get_string('quiznavigation', 'quiz');
$fakeblock->contents = '...';
$PAGE->blocks->add_required_block($fakeblock);

The new quiz navigation in Moodle 2.0 (see this screenshot) appears on the screen looking exactly like a block, but is not stored in the database as a block instance, because we already know we must have exactly one copy of this 'block' on each attempt page.

The admin menu on admin screens should probably be handled in a similar way. At the moment, admins can remove this block, but if they do, all that happens is that they get stuck unable to configure their site.

$PAGE->user_can_edit_blocks(has_capability('moodle/my:manageblocks', $suercontext));

Allows a particular page, (for example My Moodle), to control which users can edit blocks on this page, rather than using the default 'moodle/site:manageblocks' to control it.

How blocks are configured

Adding blocks to a page, and the icons in the block to configure it, will remain pretty much the same.

The major change will be to moving blocks. Instead of arrows that move the block one place at a time, it will be more like rearranging activities on the course page. You will click the move icon, and lots of drop boxes will appear everywhere the block could go, and you click one to complete the move. (This copes much better when the theme and the page can have arbitrarily many places where blocks can be added. It is also much easier to convert to Ajax.)

Sticky blocks are more complicated, and are explained below.

Inner workings of this proposal

What we need to know about the page that will show the blocks

The page type. As at present, this looks like mod-quiz-attempt, or course-view. By default, this is derived from the name of the script, although the script can override it.

The context for this page. I am assuming that every page in Moodle can be sensibly associated with one main context. So far, no one has come up with an example that breaks this assumption. (My Moodle = user context; tags page = system context, so the fact that Teachers have moodle/tag:editblocks by default is pretty pointless!)

A subpage name. In many cases this will be an empty string. However, in cases like the book module, the pages of a multi-page quiz attempt, or the pages of a lesson, or the flex-page course format, or tags pages, a non-blank string will be used (even though it might be a number).

From a combination of the theme, and the page itself (see above) the list of places that blocks can appear.

Database tables

The changes to the current database structure

block.multiple column removed. It appears to be unnecesary.

block_instances and block_pinned are first combined, and then split into block_instances and block_positions. block_instances stores the configuration for the block, which may be a simple block instance, or a sticky block. block_positions allows sticky blocks to be moved around to different places on all the pages it appears on.

The page_types and page_positions tables exists solely for the benefit of the sticky blocks editing UI. They just list all the possible values for the corresponding block_positions fields. A module needs to declare its page types when it is insalled.

Table: block_instances

Column	Type	Comment
id	INT(10) AUTO-INCREMENT	Unique id
blockname	VARCHAR(40) NOT NULL	References block.name. The type of block this is
contextid	INT(10) NOT NULL	References context.id. The context this block instance belongs to.
showinsubcontexts	INT(4) NOT NULL	Non-zero for sticky blocks. Probably a bit-field. 1=show, 2=prevent hiding, 4=prevent moving
pagetypepattern	VARCHAR(64) NOT NULL	E.g. course-view, or mod-forum-view, or mod-forum-%
subpagepattern	VARCHAR(16) NOT NULL	Like pagetypepattern, only for subpage.
defaultregion	VARCHAR(16)	Used for sticky-blocks. The default place for this to appear.
defaultweight	INT(10)	Used for sticky-blocks. The default place for this to appear.
configdata	TEXT	As at present. For blocks that don't want to create their own tables.

A simple block on, say, the course view page, will have contextid = course context, showinsubcontexts = 0, pagetypepattern = 'course-view' and subpagepattern = ''.

A sticky block would have one or more of showinsubcontexts != 0, pagetypepattern

Table: block_positions

Column	Type	Comment
id	INT(10) AUTO-INCREMENT	Unique id
blockinstanceid	INT(10) NOT NULL	References block_instance.id.
contextid	INT(10) NOT NULL	The context the block is appearing on. Perhaps it would be clearer to call this parentcontextid?
pagetype	VARCHAR(64) NOT NULL	E.g. 'course-view'
subpage	VARCHAR(16) NOT NULL	Normally , but may be a shortname.
visible	INT(2) NOT NULL	1 = visible, 0 = hidden.
region	VARCHAR(16) NOT NULL	E.g. 'left' or 'right'. Valid values depend on the theme, etc.
weight	INT(10) NOT NULL	Used to determine the order within the list of blocks in position.

Table: page_types

Keeps at track of all the know page types, for those places in the UI that need to display a list.

Column	Type	Comment
id	INT(10) AUTO-INCREMENT	Unique id
pagetype	VARCHAR(32)	For example 'mod-fourm-view'
ownermodule	VARCHAR(32)	For example 'forum'. Used to look up a human-readable name for this page type. Does get_string('pagetype' . $page->pagetype, $page->ownermodule);

In a plugins install.php file, it should declare its page types using code like:

page_register_page_type('mod-quiz-view', 'quiz');
page_register_page_type('mod-quiz-attempt', 'quiz');
page_register_page_type('mod-quiz-review', 'quiz');
// etc.

What happens when a page is requested

This query gets all the blocks we should display on this screen. If you understand this one query, you probably understand the whole proposal.

$thiscontextid, $pagetype and $subpage come from the page we are displaying. $parentcontextids is a comma-separated list of parent contexts, which is easily obtained from $thiscontext->path.

SELECT * FROM
    mdl_block_instances bi
    JOIN mdl_block ON bi.blockid = b.id
    LEFT JOIN mdl_block_positions bp ON bp.blockinstanceid = bi.id
                                        AND bp.contextid = $contextid
                                        AND bp.pagetype = $pagetype
                                        AND bp.subtype = $subtype

WHERE 
    (bi.contextid = $thiscontextid OR
            (bi.showinsubcontexts &lt;> 0 AND bi.contextid IN ($parentcontextids)))
AND $pagetype LIKE bi.pagetypepattern
AND $subtype LIKE bi.subtypepattern
AND (bp.visible = 1 OR bp.visible IS NULL)   -- Unless editing is on
AND b.visible = 1

ORDER BY
    COALESCE(bp.position, bi.defaultposition),
    COALESCE(bp.weight, bi.defaultweight)

Database upgrade

Basic plan (constrained by the fact that I want to both keep an exact copy of the original data, but also keep the new ids in the block_instances table the same as the ids in the old block_instance table, and MSSQL makes it hard to preserve the id column when copying between tables.)

✔ rename block_instance to block_instances
✔ rename block_pinned -> block_pinned_old
✔ create new block_instance_old with an oldid column, but otherwise the same columns
✔ copy all date block_instances -> block_instance_old (that is one INSERT INTO)
✔ drop block.multiple
✔ rename position and weight to defaultposition and defaultweight
✔ change positions to the new notation
✔ add new unique key to block.name so it can be a target for a foreign key
✔ add new columns blockname, contextid, showinsubcontexts, subpage (but without not-null constraints)
✔ fill in blockname from blockid
✔ rename pagetype to pagetypepattern
✔ fill in contextid and subpage from pageid, and update pagetypepattern where we are changing it
✔ fill any missing values in contextid with a dummy value
✔ add not-null constraints to blockname, contextid and showinsubcontexts
✔ get_records('block_pinned_old') loop over that and insert into block_instances
✔ create new table block_positions
✔ for any blocks with visible = false, insert a row into block_positions.
✔ drop block_instances.pageid, .blockid and .visible columns

Because this is a potentially tricky upgrade, we will rename the old database tables to block_instance_old and block_pinned_old, then make the upgrade script copy data to the new tables. This will mean we have a backup of the original data in case anything goes wrong during upgrade.

Known page types in 1.9

Note that Moodle 1.9 used at least two different notions of page type. There was the page type as used for blocks resolution, which might be 'course-view' or 'admin', and the page type that was added as id to the body tag in the HTML, which might be 'course-view-weeks' or 'admin-settings-managefilters'. In Moodle 2.0 we only use the more precise page type, and put wildcards in the pagetypepattern column of block_instances table.

In the notation below, we list known page types in Moodle 1.9, what the corresponding pageid was, and then how that will be represented in Moodle 2.0. The notation is (contextid, pagetype, subpage).

Core page types

site-index: This is actually the old course-view where pageid = SITEID. Becomes (frontpagecontextid, site-index, NULL).
course-view: pageid was courseid. Becomes (coursecontextid, course-view-*, NULL).
admin: no pageid. Becomes (syscontextid, admin-*, NULL).
my-index: pageid was user id. Becomes (usercontextid, my-index, NULL).
tag-index: pageid was tag id. Becomes (systemcontextid, tag-index, tagid).
blog-view: pageid is userid. Note that when you are looking at any blog, you see the set of blocks you chose to add to your own blog. No the set of blocks that the other user added to their blog. Anyway, becomes (usercontextid, blog-index, NULL). (Blog-index matches the URL and the id tag on the body, and it is worth changing the value in the block instance table.)
mod-xxx-view (xxx = chat, data, lesson, quiz): pageid is activitrecord.id. (That is, quiz.id or lesson.id or ...) Becomes (modulecontextid, mod-xxx-view, NULL).

Page types with sticky blocks

course-view: becomes (syscontextid+subcontexts, course-view-*, NULL)
my-index: becomes (syscontextid+subcontexts, my-index, NULL)

Known third-party page types

format_page extends page_course - from the flex page course format: can only be downloaded from moodlerooms, not in contrib. TODO
mod-xxx-view (xxx = dimdim, game, wiki, oublog, ): as above
resource-view: Something custom to OU's openlearn - ignore for now.

Block positions

I would rather switch to proper names

l => side-pre (better for RTL languages)
r => side-post
c => course-view-top (this comes from contrib/patches/center_blocks_position_patch on course-view page and the flex page course format. We may as well support this.)

New blocks created on upgrade to 2.0

The notation here is (blockname, context,

('settings', syscontextid+subcontexts, *, NULL)
('navigation', syscontextid+subcontexts, *, NULL)
TODO default blog tags ...

Roles and capabilities

Contexts

For CONTEXT_BLOCK, instanceid continues to point to block_instances.id.

Now every block clearly belongs to one context, we can create a context for every block, not just ones on the course-view page. (See http://moodle.org/mod/forum/discuss.php?d=122600)

(There is one issue like that. What about complex blocks like newsfeed (in contrib). That has its own UI pages, in addition to the block that appears on the course page. Do we allow blocks on those pages? If so, do we limit the level of nesting? I think we can just support arbitrary levels of nesting.)

Capabilities

Note that these will be checked in the block context. Since block-level role assignments and permission overrides are very rare, this normally means that the permissions form the parent context will be used.

moodle/block:view: No change, used to determine who can see the block. The only change is that this can now be used with all blocks.
moodle/site:manageblocks: No change, although it might be better to rename this to moodle/bock:manage. Use to control editing of non-sticky blocks.
moodle/block:managesticky: New capability, so we can separately control who can edit sticky blocks. Admin only, by default.
moodle/my:manageblocks: Probably needs to stay separate, if we want to let people edit block on their My Moodle page, as opposed to their user profile pages.

Performance

If people start to use all the new flexibility for creating blocks, this will massively increase the number of contexts that exist. What will this do to performance?

We need to think about the strategy for which contexts the permissions are pre-loaded for. If we get that right, performance should be OK.

Blocks User interface

Blocks editing on/off button

This will now be controlled by the page.

Need a special case mechanism for pages like course view to combing their own editing on/off with the blocks editing button.

Rearranging blocks

Will now require moodle/site:manageblocks in the parent context of the block.

Will change to a two-step move, like activities on the course page, as explained above.

Configuring blocks

From Moodle 2.0 onwards, we will insist that blocks use formslib for their settings forms.

Will be controlled by 'moodle/site:manageblocks' or 'moodle/block:managesticky' in the block context and in the parent (page) context, depending on the type of block instance. (You need to include the parent context here, otherwise the user will not be able to see the Blocks editing on button.)

The settings page for a block will have tabs for the roles UI, like on the activity settings page.

If you have 'moodle/site:manageblocks' but not 'moodle/block:managesticky', then when editing is on, some explanatory small print will appear next to each sticky block, to explain why you cannot change it. (Like the small print next to conditional activities.)

Configuring sticky blocks

For users with 'moodle/block:managesticky', form fields for

changingpagetype or subpage to wildcarded values,
turning on showinsubcontexts
setting defaultposition and defaultweight

will automatically be added to the settings form, so a block instance can be converted to a sticky block.

Also, there would be a bit warning at the top and bottom of the settings form to remind you this is a sticky block, so changing the settings will affect many pages.

Do we need another UI for this as well, like this current stickyblocks UI?

Adding a block

We could change the process for adding a block to

Select the block you want from the Add... dropdown.
If the block has config, show the config form. Else just skip to the next step.
Redirect to the course page, in move mode, so you can put the block wherever you like immediately.

Requires moodle/site:manageblocks in the page context.

Deleting a block

Requires moodle/site:manageblocks in the parent (page) context and the block context.

Lists of block instances

On the settings page for a sticky block, we will have an additional tab, which lists instances of the the same block in sub-contexts, with show/hide and delete controls. So after you add a sticky-block you can remove any separate block instances it obsoletes.

Documentation

Very flexible block system proposal

Goals

Interesting use cases

Switching to sticky blocks

What to show a block on page X of this lesson

Show this HTML block on pages 6-8 of this quiz

Setting the scene

What happens when a page is requested

What happens in $PAGE->complete_setup()

What happens in $HTML->print_header() and print_footer()

Some optional things the page can do

How blocks are configured

Inner workings of this proposal

What we need to know about the page that will show the blocks

Database tables

The changes to the current database structure

Table: block_instances

Table: block_positions

Table: page_types

What happens when a page is requested

Database upgrade

Known page types in 1.9

Core page types

Page types with sticky blocks

Known third-party page types

Block positions

New blocks created on upgrade to 2.0

Roles and capabilities

Contexts

Capabilities

Performance

Blocks User interface

Blocks editing on/off button

Rearranging blocks

Configuring blocks

Configuring sticky blocks

Adding a block

Deleting a block

Lists of block instances

See also