- 1 What is Global search?
- 2 What can I search for?
- 3 How does it work?
- 4 How is it set up?
- 4.1 Setting up Solr
- 4.2 How to install Solr
- 4.2.1 Linux (Debian/Ubuntu)
- 4.2.2 Linux (Redhat/Centos 6 & 7)
- 4.2.3 OSX using macports
- 4.2.4 OSX using homebrew
- 4.2.5 Windows
- 4.2.6 The Solr server
- 4.2.7 Commannd for installing Solr Server on Centos 7
- 5 See also
What is Global search?
- Global search lets you search everywhere on the Moodle site that you have access to. A student can search their courses for particular lecture notes, for example, or a teacher could search for subject-related activities.
- The feature needs to be enabled by the administrator and a search box is then available next to the user menu.
- A Global search block may also be added to course pages.
What can I search for?
You can search for your courses, activities information and some activities contents like forum posts, book chapters, glossary entries or collaborative wikis pages.
HTML block content may be searched and, if the administrator enables the setting "Search all courses" then courses which are visible to users but which they are not enrolled in may also be searched. (These courses are courses with guest access or if the user has the capability to view all courses.)
How does it work?
- Click the search icon by the user menu and type a search term into the box that appears, or type into the box in the Global search block if enabled.
- On the next screen, you can simply click the search button to search everywhere, or expand the Filter to search in specific areas:
- You will then see results displayed from all areas of Moodle you have access to:
Solr search query features
You can improve your search using any of the following search query features:
- Specifying the field to be searched by prefixing the search query with 'title:', 'content:', 'name:', or 'intro:' e.g.'title:news' returns results with the word 'news' in the title
- Boolean operators ('AND', 'OR', 'NOT') to combine or exclude keywords
- Wildcard characters ('*' or '?' ) to represent characters in the search query
- Proximity searches ('~') e.g. mood~2 returns "moodle" (2 letters away from "mood"), Moodle Australia~3 returns results containing "Moodle HQ in Perth, Australia" (the queried terms were within 3 words of each other)
- Boosting terms ('^') to boost certain words or phrases e.g. "Perth Australia"^5 "Australia" will return results with the phrase "Perth Australia" as more relevant.
Note: The above features are only available when using the Solr search engine.
How is it set up?
Note: Global search needs a search engine. Instructions for the installing the PHP Solr extension and a Solr server are below. See the Developer docs on Search engine plugins if you wish to write your own. It's recommended you set everything up first, index the site contents and only then enable Global search.
Setting up Solr
- The latest Solr 5 available version is the recommended one.
- Moodle supports Solr server from 4.0 onwards, although you can only use the Solr schema setup script that we provide with Moodle from Solr 5. The latest Solr 5 available version is the recommended one; the same will apply to Solr 6 once it is released.
- Solr 7 is not currently supported as the field type 'int' has been removed.
- Set the feature up in Site administration > Plugins > Search > Manage global search by selecting Solr as the search engine and ticking search area checkboxes as required
- In Site administration > Plugins > Search > Solr, set Host name to localhost, Port to 8983 and Index name to 'moodle' (the name of the index in Solr)
- If you are using Solr with SSL encryption, you will need to configure Moodle as such. This is important if solr is not installed in the web server server.
- You will need a separate key file and cacert file, both in pem format, located on your server Moodle, and readable by the PHP process.
- Go to Site administration > Plugins > Search > Solr
- Set Secure mode to Yes
- SSL certificate to /path/to/certs/solr-ssl.cacert.pem
- SSL key to /path/to/certs/solr-ssl.key.pem
- SSL key Password to The password used to lock the SSL Key
- SSL CA certificates name to /path/to/certs/solr-ssl.cacert.pem
- You now need to populate the created Solr index with your site's data. You can do it via the web interface by going to Site administration > Reports > Global search info or from the CLI by running the search/cli/indexer.php script. The CLI script is the recommended option for big sites.
# sudo -u www-run php search/cli/indexer.php --force
- Enable Global search in Site administration > Advanced features
Solr has the ability to index the contents of files, such as File resources and attachments to Forum posts. This uses the Tika engine which comes bundled with Solr. To enable this feature:
- In Site administration > Plugins > Search > Solr enable the checkbox Enable file indexing
- Set Maximum file size to index (kB) to some value - the default is 2097152 (2GB)
- Files larger than this limit will not be sent for Solr for indexing, but the file name will still be indexed.
New in 34: File indexing now works retroactively -ie, if you set up Global search without file indexing enabled and a later date you enable file indexing, files associated with existing objects will be included.
How to install Solr
You need PHP Solr extension installed. You can download the official latest versions from PECL: Package: solr. The minimum required version is PECL Solr 2.1 for PHP 5 branch and PECL Solr 2.4 for PHP 7 branch.
Basic installation steps (using apache web server):
sudo apt-get install libpcre3-dev libxml2-dev libcurl4-openssl-dev sudo apt-get install php5-dev sudo apt-get install php-pear sudo pecl install solr sudo sh -c "echo 'extension=solr.so' > /etc/php5/apache2/conf.d/solr.ini" sudo sh -c "echo 'extension=solr.so' > /etc/php5/cli/conf.d/solr.ini" sudo service apache2 restart
With PHP 7
sudo apt-get install libpcre3-dev libxml2-dev libcurl4-openssl-dev sudo apt-get install php7.0-dev sudo apt-get install php-pear sudo pecl install solr sudo sh -c "echo 'extension=solr.so' > /etc/php/7.0/apache2/conf.d/solr.ini" sudo sh -c "echo 'extension=solr.so' > /etc/php/7.0/cli/conf.d/solr.ini" sudo service apache2 restart
Linux (Redhat/Centos 6 & 7)
Using built in php5 packages
yum install php-pecl-solr2 service httpd restart
Using 3rd party php7 packages (webtactic)
yum install libxml2-devel pcre-devel libcurl-devel php70w-devel php70w-pear pecl install solr echo "extension=solr.so" > /etc/php.d/solr.ini service httpd restart
OSX using macports
sudo port install apache-solr4 sudo port install php54-solr
OSX using homebrew
brew install homebrew/php/php56-solr
Install the pecl package as usual. (This has not yet been tested.)
The Solr server
Note that for medium/large sites you may need to increase maxBooleanClauses setting. In [http://tracker.moodle.org/browse/MDL-54992 MDL-54992] we are working on an alternative way to query the server.
Please note that there is a security issue that affects some Solr versions. The patched versions are 5.5.5, 6.6.2, 7.1, 7.2 and all new versions are patched. If your Solr server version is lower but you can't upgrade to one of the patched versions it is recommended to disable the XML Query Parser.
The following example snippet (feel free to copy & paste into a .sh script with execution permissions) will download Solr 5.4.1 (replace it with latest 5.x) in the current directory, start the solr server and create an index in it named moodle to add moodle data to it. If wget gives an error, check http://www-us.apache.org/dist/lucene/solr and update SOLRVERSION
#!/bin/bash set -e SOLRVERSION=5.4.1 SOLRNAME=solr-$SOLRVERSION SOLRTAR=$SOLRNAME'.tgz' INDEXNAME=moodle if [ -d $SOLRNAME ]; then echo "Error: Directory $SOLRNAME already exists, remove it before starting the setup again." exit 1 fi if [ ! -f $SOLRTAR ]; then wget http://www-us.apache.org/dist/lucene/solr/$SOLRVERSION/$SOLRTAR fi tar -xvzf $SOLRTAR cd $SOLRNAME bin/solr start bin/solr create -c $INDEXNAME # After setting it up and creating the index use: # - "/yourdirectory/solrdir/bin/solr start" from CLI to start the server # - "/yourdirectory/solrdir/bin/solr stop" from CLI to stop the server.
Solr 5/6 schema setup
Moodle will use Solr's managed schema interface to install the required fields. You will be directed on what to do from the Manage global search page.
For very large or busy sites, it is recommended that you manually remove the _text_ field, and associated copy directive from, the default Solr schema. This field is not used by Moodle, and will significantly slow indexing, and increase the size of the resulting Solr core.
Solr 4 schema setup
You cannot use the schema setup script when using a Solr 4 server. If you really want to use the Solr 4x branch, here are the field types descriptions:
Extracted from search/classes/document.php
|Field name||Field type||Stored||Indexed||Query field|
Memory and File indexing considerations
For large sites, and particularly if you are using the File indexing feature, it's a good idea to give Solr plenty of memory, eg aound 10-20GB. To start Solr with more than its default 512MB of RAM, use the -m option:
solr start -m 12g
See the documents for your version of Solr on how to increases memory when Solr is started automatically.
Too many boolean clauses error
Because of the way Moodle handles permissions for searches, if you have non-admin users with access to a large number of activities (>1000), they may an error similar to the following:
The query you provided could not be parsed by the search engine: org.apache.solr.search.SyntaxError: Cannot parse ... too many boolean clauses
To correct this, you need to increase the maxBooleanClauses setting in your Solr core. The setting is located in corename/conf/solrconfig.xml.
This important note is included in the config file:
** WARNING ** This option actually modifies a global Lucene property that will affect all SolrCores. If multiple solrconfig.xml files disagree on this property, the value at any given moment will be based on the last SolrCore to be initialized.
This means that for consistent behavior you should update this value for all cores in the Solr server.
Commannd for installing Solr Server on Centos 7
First of all install java (openjdk)
yum install java-1.8.0-openjdk
Add user for solr
Download solr server, find latest here : http://lucene.apache.org/solr/mirrors-solr-latest-redir.html, from the list there download latest solr-[version].tgz
Here is the list of needed commands (note that latest solr is suggested, change command to appropriate version)
cd /opt wget http://www-eu.apache.org/dist/lucene/solr/6.6.0/solr-6.6.0.tgz tar zxvf solr-6.6.0.tgz cp solr-6.6.0/bin/install_solr_service.sh . rm -rf solr-6.6.0 ./install_solr_service.sh solr-6.6.0.tgz chkconfig solr on
Create the index
su - solr -c "/opt/solr/bin/solr create_core -c moodle"
Then go to your Moodle and set index name as created (moodle)
A quick introduction to Moodle global search, followed by a description of the new features and fixes in Moodle 3.4 (and some of those in Moodle 3.5).