- 1 What is Global search?
- 2 What can I search for?
- 3 How does it work?
- 4 How is it set up?
- 4.1 Setting up Simple search
- 4.2 Setting up Solr
- 4.2.1 General Setup
- 4.2.2 File Indexing
- 4.2.3 How to install Solr
- 18.104.22.168 Linux (Debian/Ubuntu)
- 22.214.171.124 Linux (Redhat/Centos 6 & 7)
- 126.96.36.199 OSX using macports
- 188.8.131.52 OSX using homebrew
- 184.108.40.206 Windows
- 220.127.116.11 The Solr server
- 18.104.22.168 Commannd for installing Solr Server on Centos 7
- 5 See also
What is Global search?
- Global search lets you search everywhere on the Moodle site that you have access to. A student can search their courses for particular lecture notes, for example, or a teacher could search for subject-related activities.
- The feature needs to be enabled by the administrator in Advanced features and then, from the Manage Global search page, the search engine must be selected.
When Global search is enabled, search box is then available next to the user menu.
- A Global search block may also be added to course pages.
What can I search for?
You can search for your courses, activities information and some activities contents like forum posts, book chapters, glossary entries or collaborative wikis pages.
HTML block content may be searched and, if the administrator sets the setting "Searchable courses" to Search within all courses the user can access then courses which are visible to users but which they are not enrolled in may also be searched. (These courses are courses with guest access or if the user has the capability to view all courses.)
How does it work?
- Click the search icon by the user menu and type a search term into the box that appears, or type into the box in the Global search block if enabled.
- On the next screen, you can simply click the search button to search everywhere, or expand the Filter to search in specific areas:
- You will then see results displayed from all areas of Moodle you have access to:
These features are only available if you use Solr as your search engine.
Search query features
You can improve your search using any of the following search query features:
- Specifying the field to be searched by prefixing the search query with 'title:', 'content:', 'name:', or 'intro:' e.g.'title:news' returns results with the word 'news' in the title
- Boolean operators ('AND', 'OR', 'NOT') to combine or exclude keywords
- Wildcard characters ('*' or '?' ) to represent characters in the search query
- Proximity searches ('~') e.g. mood~2 returns "moodle" (2 letters away from "mood"), Moodle Australia~3 returns results containing "Moodle HQ in Perth, Australia" (the queried terms were within 3 words of each other)
- Boosting terms ('^') to boost certain words or phrases e.g. "Perth Australia"^5 "Australia" will return results with the phrase "Perth Australia" as more relevant.
File contents indexing
Solr has the ability to index the contents of files, such as File resources and attachments to Forum posts.
New in 3.5 : New fields now facilitate filtering by user ID and searching by group.
How is it set up?
Note: Global search needs a search engine. It's recommended you set everything up first, index the site contents and only then enable Global search.
Moodle core includes two search engines: Simple Global search and Solr. See the Developer docs on Search engine plugins if you wish to write your own.
Setting up Simple search
Simple Global search provides global search features without the need to install an external search engine.
Setting up Solr
- The latest Solr 5 available version is the recommended one.
- Moodle supports Solr server from 4.0 onwards, although you can only use the Solr schema setup script that we provide with Moodle from Solr 5. The latest Solr 5 available version is the recommended one; the same will apply to Solr 6 once it is released.
- Set the feature up in Site administration > Plugins > Search > Manage global search by selecting Solr as the search engine and ticking search area checkboxes as required
- In Site administration > Plugins > Search > Solr, set Host name to localhost, Port to 8983 and Index name to 'moodle' (the name of the index in Solr)
- If you are using Solr with SSL encryption and basic authentication (see below if you want to use client certificates instead), you will need to configure this.This is important if solr is not installed in the web server server.
- Go to Site administration > Plugins > Search > Solr
- Set Secure mode to Yes
- Set port number if required (sometimes SSL installations use port 443).
- Type in the username and password.
- If you are using Solr with SSL encryption via client certificates, you will need to configure Moodle as such. This is important if solr is not installed in the web server server.
- You will need a separate key file and cacert file, both in pem format, located on your server Moodle, and readable by the PHP process.
- Go to Site administration > Plugins > Search > Solr
- Set Secure mode to Yes
- SSL certificate to /path/to/certs/solr-ssl.cacert.pem
- SSL key to /path/to/certs/solr-ssl.key.pem
- SSL key Password to The password used to lock the SSL Key
- SSL CA certificates name to /path/to/certs/solr-ssl.cacert.pem
- You now need to populate the created Solr index with your site's data. You can do it via the web interface by going to Site administration > Reports > Global search info or from the CLI by running the search/cli/indexer.php script. The CLI script is the recommended option for big sites.
# sudo -u www-run php search/cli/indexer.php --force
- Enable Global search in Site administration > Advanced features
Solr has the ability to index the contents of files, such as File resources and attachments to Forum posts. This uses the Tika engine which comes bundled with Solr. To enable this feature:
- In Site administration > Plugins > Search > Solr enable the checkbox Enable file indexing
- Set Maximum file size to index (kB) to some value - the default is 2097152 (2GB)
- Files larger than this limit will not be sent for Solr for indexing, but the file name will still be indexed.
New in 34: File indexing now works retroactively -ie, if you set up Global search without file indexing enabled and a later date you enable file indexing, files associated with existing objects will be included.
How to install Solr
You need PHP Solr extension installed. You can download the official latest versions from PECL: Package: solr. The minimum required version is PECL Solr 2.1 for PHP 5 branch and PECL Solr 2.4 for PHP 7 branch.
Basic installation steps (using apache web server):
sudo apt install php-solr sudo service apache2 restart
If you can't install the SOLR PHP module via APT, try the following:
sudo apt-get install libpcre3-dev libxml2-dev libcurl4-openssl-dev sudo apt-get install php5-dev sudo apt-get install php-pear sudo pecl install solr sudo sh -c "echo 'extension=solr.so' > /etc/php5/apache2/conf.d/solr.ini" sudo sh -c "echo 'extension=solr.so' > /etc/php5/cli/conf.d/solr.ini" sudo service apache2 restart
With PHP 7
sudo apt-get install libpcre3-dev libxml2-dev libcurl4-openssl-dev sudo apt-get install php7.0-dev sudo apt-get install php-pear sudo pecl install solr sudo sh -c "echo 'extension=solr.so' > /etc/php/7.0/apache2/conf.d/solr.ini" sudo sh -c "echo 'extension=solr.so' > /etc/php/7.0/cli/conf.d/solr.ini" sudo service apache2 restart
Linux (Redhat/Centos 6 & 7)
Using built in php5 packages
yum install php-pecl-solr2 service httpd restart
Using 3rd party php7 packages (webtactic)
yum install libxml2-devel pcre-devel libcurl-devel php70w-devel php70w-pear pecl install solr echo "extension=solr.so" > /etc/php.d/solr.ini service httpd restart
OSX using macports
sudo port install apache-solr4 sudo port install php54-solr
OSX using homebrew
brew install homebrew/php/php56-solr
- Check your PHP version (see Site admin > Server > PHP Info) for PHP version, compiler (eg VC15), architecture (eg x64), and thread safety.
- Get the matching PHP extension DLL from PECL (or this page if not available on PECL).
- Add the DLL to you php\ext folder.
- Add the directive "extension=php_solr.dll" to your php.ini file.
- Restart your web server.
- Install the Java Runtime Environment
- Download Solr from the Solr Downloads page, eg solr-5.5.5.zip.
- Extract to an accessible place, eg: c:\solr-5.5.5
- From command line (as admin), in C:\solr-5.5.5, run...
- bin\solr start
- bin\solr status
- bin\solr create -c moodle
- bin\solr stop -all
- (As admin) Edit C:\solr-5.5.5\server\solr\moodle\conf\solrconfig.xml and change line...
- Create Scheduled task to start Solr
- Run whether user is logged on or not
- Run at highest privileges
- Configure for Windows Server 2016
- Run daily (start any time)
- Repeat every hour, indefinitely
- Start program
- Script: C:\solr-5.5.5\bin\solr.cmd
- Add arguments: start -m 2g
- Uncheck: Start the task only when the computer is on AC power
- Uncheck: Stop the task if it runs longer than: ...
- If the task is already running, then the following rule applies: Do not start a new instance
- Run the scheduled task manually (for now)
- At the command line, in c:\solr-5.5.5, run bin\solr status to check it is running
- Configure the Solr server in Moodle as below.
- Visit Site admin > Plugins > Search > Manage global search. It may prompt you to set up the seach schema.
- Run the scheduled task to build the search index from the command line. It may initially need to run a few times as it will time out after 10min and you may have a lot of files.
php admin\tool\task\cli\schedule_task.php --execute=\core\task\search_index_task
- Keep an eye on the amount of memory used with solr status and the Task Manager (see the Java process).
The Solr server
Note that for medium/large sites you may need to increase maxBooleanClauses setting. In [http://tracker.moodle.org/browse/MDL-54992 MDL-54992] we are working on an alternative way to query the server.
Please note that there is a security issue that affects some Solr versions. The patched versions are 5.5.5, 6.6.2, 7.1, 7.2 and all new versions are patched. If your Solr server version is lower but you can't upgrade to one of the patched versions it is recommended to disable the XML Query Parser.
The following example snippet (feel free to copy & paste into a .sh script with execution permissions) will download Solr 5.5.5 (replace it with latest 5.x) in the current directory, start the solr server and create an index in it named moodle to add moodle data to it. If wget gives an error, check http://www-us.apache.org/dist/lucene/solr and update SOLRVERSION
#!/bin/bash set -e SOLRVERSION=5.5.5 SOLRNAME=solr-$SOLRVERSION SOLRTAR=$SOLRNAME'.tgz' INDEXNAME=moodle if [ -d $SOLRNAME ]; then echo "Error: Directory $SOLRNAME already exists, remove it before starting the setup again." exit 1 fi if [ ! -f $SOLRTAR ]; then wget http://www-us.apache.org/dist/lucene/solr/$SOLRVERSION/$SOLRTAR fi tar -xvzf $SOLRTAR cd $SOLRNAME bin/solr start bin/solr create -c $INDEXNAME # After setting it up and creating the index use: # - "/yourdirectory/solrdir/bin/solr start" from CLI to start the server # - "/yourdirectory/solrdir/bin/solr stop" from CLI to stop the server.
Solr 5/6 schema setup
Moodle will use Solr's managed schema interface to install the required fields. You will be directed on what to do from the Manage global search page.
For very large or busy sites, it is recommended that you manually remove the _text_ field, and associated copy directive from, the default Solr schema. This field is not used by Moodle, and will significantly slow indexing, and increase the size of the resulting Solr core.
Solr 4 schema setup
You cannot use the schema setup script when using a Solr 4 server. If you really want to use the Solr 4x branch, here are the field types descriptions:
Extracted from search/classes/document.php
|Field name||Field type||Stored||Indexed||Query field|
Memory and File indexing considerations
For large sites, and particularly if you are using the File indexing feature, it's a good idea to give Solr plenty of memory, eg aound 10-20GB. To start Solr with more than its default 512MB of RAM, use the -m option:
solr start -m 12g
See the documents for your version of Solr on how to increases memory when Solr is started automatically.
Too many boolean clauses error
Because of the way Moodle handles permissions for searches, if you have non-admin users with access to a large number of activities (>1000), they may an error similar to the following:
The query you provided could not be parsed by the search engine: org.apache.solr.search.SyntaxError: Cannot parse ... too many boolean clauses
To correct this, you need to increase the maxBooleanClauses setting in your Solr core. The setting is located in corename/conf/solrconfig.xml.
This important note is included in the config file:
** WARNING ** This option actually modifies a global Lucene property that will affect all SolrCores. If multiple solrconfig.xml files disagree on this property, the value at any given moment will be based on the last SolrCore to be initialized.
This means that for consistent behavior you should update this value for all cores in the Solr server.
Commannd for installing Solr Server on Centos 7
First of all install java (openjdk)
yum install java-1.8.0-openjdk
Add user for solr
Download solr server, find latest here : http://lucene.apache.org/solr/mirrors-solr-latest-redir.html, from the list there download latest solr-[version].tgz
Here is the list of needed commands (note that latest solr is suggested, change command to appropriate version)
cd /opt wget http://www-eu.apache.org/dist/lucene/solr/6.6.0/solr-6.6.0.tgz tar zxvf solr-6.6.0.tgz cp solr-6.6.0/bin/install_solr_service.sh . rm -rf solr-6.6.0 ./install_solr_service.sh solr-6.6.0.tgz chkconfig solr on
Create the index
su - solr -c "/opt/solr/bin/solr create_core -c moodle"
Then go to your Moodle and set index name as created (moodle)
MySQL Database Connection error with Simple Search: This SQL query should solve the issue
ALTER TABLE mdl_search_simpledb_index ADD FULLTEXT(title, content, description1, description2)
A quick introduction to Moodle global search, followed by a description of the new features and fixes in Moodle 3.4 (and some of those in Moodle 3.5).