Global search brainstorming

Note: This page is a work-in-progress. Feedback and suggested improvements are welcome. Please join the discussion on moodle.org or use the page comments.

Moodle 2.3

Global Search is a proposed rewrite of the search mechanism for Moodle 2.3.

Implementation

• Will use Lucene search engine.
• Will not require additional DB tables to store index.
• Will allow for partial indexing of the content (e.g. the first time indexer is run, it may only index part of the huge site, and pick up where is left on the next run).
• Will allow for indexing content from the DB and attachments (e.g. forum post & all attachments).
• Will implement at least basic cache.

Modules support

Interface will need to be implemented for a module that wants to be search-able by Global Search. A module will need to:

• declare that it supports FEATURE_GLOBAL_SEARCH (in function <mod>_supports).
• implement 3 functions:
  • <mod>_gs_iterator($from = 0)
  • <mod>_gs_get_documents($id)
  • <mod>_page_gs_access($id)

mod_gs_iterator($from=0)

Function has to return moodle recordset with two columns:

• document set id for this particular module
• timestamp of the last modification

$from is used to return only documents that were modified after given timestamp. If $from equals 0 then all documents should be returned in the recordset. The meaning of the document set will differ for each module. For example, forum module will return post id as a single document set id. Inside such a document set, there will be several documents, in our case it can be actual forum post and several attachments. Security (access rights) will be managed on the document set level. In this example, you either have or don't have access to both forum post and it's attachments (fixing some bugs like [https://tracker.moodle.org/browse/MDL-29660 MDL-29660] will be required here). Modules that want to support Global Search interface, have to keep track of the last modification time of their document.

The code for this function will usually be very simple, here is a complete example for forum:

function forum_gs_iterator($from = 0) {
  global $DB;

  $sql = "SELECT id, modified FROM {forum_posts} WHERE modified > ? ORDER BY modified ASC";

  return $DB->get_recordset_sql($sql, array($from));
}

mod_gs_get_documents($id)

Method returns an array of documents for given ID. For forum, it would be one document for forum post and zero or more for post attachments. Each document is a simple object of gs_document class. Following attributes can be set:

• $type Document type, can be set to GS_TYPE_TEXT (plain text in $content), GS_TYPE_HTML (HTML in $content), GS_TYPE_FILE (external file)
• $contextlink Link to a context of the document (e.g. forum post)
• $directlink Direct link to the document (e.g. document attached to a post). By default $directlink and $contextlink are the same (you can set any of them)
• $title Title of the document
• $content The main content. For external files, the field is ignored.
• $user Author's user object.
• $module Module that has created original document.
• $id Id that identifies a set of documents for a given module.
• $created Timestamp when the document was created.
• $modified Timestamp of the last modification.
• $courseid The course ID where the document is located.
• $filepath Path to the external file with the document. Set when type is GS_TYPE_FILE.
• $mime Mime type of the external file. Set when type is GS_TYPE_FILE.

You can see working implementation of forum_gs_get_documents.

mod_page_gs_access($id)

Function to check if current $USER has access to the document set $id. Return value should be one of:

• GS_ACCESS_DENIED - current user can not access document set $id
• GS_ACCESS_GRANTED - access granted
• GS_ACCESS_DELETED - the document set with this $id does not exist