Note: You are currently viewing documentation for Moodle 3.3. Up-to-date documentation for the latest stable version of Moodle is probably available here: Universal Office Converter (unoconv).

Universal Office Converter (unoconv): Difference between revisions

From MoodleDocs
(Hint to use pinning on Debian systems)
(→‎Installing unoconv on Windows: correct ghostscript version and ad details not to use non CLI versions.)
 
(18 intermediate revisions by 10 users not shown)
Line 1: Line 1:
{{Environment}}
{{Document converters}}
==What is unoconv?==
==What is unoconv?==


Line 20: Line 20:
* On some systems the apache user home directory is set to a non existent folder. This can cause unoconv to fail. There are 2 solutions to this - one is to make a (writable) home directory for the apache user (like /home/www-data). The other is to run a unoconv listener (described below) as another user other than the apache user (someone with a valid, writable home directory).
* On some systems the apache user home directory is set to a non existent folder. This can cause unoconv to fail. There are 2 solutions to this - one is to make a (writable) home directory for the apache user (like /home/www-data). The other is to run a unoconv listener (described below) as another user other than the apache user (someone with a valid, writable home directory).
* If you are still running 14.04LTS then unoconv won't work as shipped. This might not be the most efficient route but it worked by first installing unoconv (version 0.6) from the package manager as above. You will then need to grab unoconv 0.7 from Github (https://github.com/dagwieers/unoconv), then upgrade to the latest libreoffice using the PPA (https://launchpad.net/~libreoffice/+archive/ubuntu/ppa). Point moodle at the Github version of unoconv. You need to modify the Python unoconv file by changing 'python' in the first line to 'python3'. You also need to change the permissions on the directory /var/www so that the user www-data can write to it (www-data needs to write to its home directory which it cannot do by default).
* If you are still running 14.04LTS then unoconv won't work as shipped. This might not be the most efficient route but it worked by first installing unoconv (version 0.6) from the package manager as above. You will then need to grab unoconv 0.7 from Github (https://github.com/dagwieers/unoconv), then upgrade to the latest libreoffice using the PPA (https://launchpad.net/~libreoffice/+archive/ubuntu/ppa). Point moodle at the Github version of unoconv. You need to modify the Python unoconv file by changing 'python' in the first line to 'python3'. You also need to change the permissions on the directory /var/www so that the user www-data can write to it (www-data needs to write to its home directory which it cannot do by default).
* On Debian Stable, the cleanest method to allow unoconv from Testing is to
On Debian Stable, the cleanest method to install [https://packages.debian.org/jessie-backports/unoconv unoconv] is using Jessie-backports. First, enable backports repo line in /etc/apt/sources.list:
** Enable a testing repo line in /etc/apt/sources.list:
<pre>
<pre>
#### testing #########
#### Jessie-backports ####
deb http://ftp.us.debian.org/debian testing main contrib non-free
deb http://ftp.debian.org/debian jessie-backports main
</pre>
</pre>
** Set pinning priorities so that stable is prioritised over testing in /etc/apt/preferences:
 
Then, update and install unoconv from jessie-backports:
<pre>
apt-get update
apt-get install -t jessie-backports unoconv
</pre>
 
The package will bring all necessary dependencies for you.
 
 
===Ubuntu 14.04 LTS===
 
1) Navigate to opt directory
<pre>
<pre>
Package: *
cd /opt
Pin: release o=Debian,a=stable
</pre>
Pin-Priority: 900


Package: *
2) Download unoconv
Pin: release o=Debian,a=testing
<pre>
Pin-Priority: 800
sudo wget https://raw.githubusercontent.com/dagwieers/unoconv/master/unoconv
</pre>
</pre>
** Update and install unoconv from testing:
 
3) Modify the Python unoconv file by changing 'python' in the first line to 'python3'
<pre>
<pre>
apt-get update
sudo nano /opt/unoconv
apt-get install unoconv/testing
</pre><br>
eg. <pre>#!/usr/bin/env python3</pre>
 
4) Make unoconv executable
<pre>
sudo chmod ugo+x /opt/unoconv
</pre>
 
5) Add LibreOffice PPA to your system and install the latest version
<pre>
sudo add-apt-repository ppa:libreoffice/ppa
sudo apt-get update
sudo apt-get install libreoffice
</pre>
 
6) Change permissions so apache can write to its home directory
<pre>
sudo chown www-data /var/www
</pre>
 
7) From your browser navigate to
''Site administration > Server > System paths'' and add the path to unoconv<br>
/opt/unoconv
*Note: if you would like to preserve the default path add a symbolic link to /usr/bin:
<pre>
sudo ln -s /opt/unoconv /usr/bin/
</pre>
</pre>
8) Navigate to
''Site administration > Plugins > Activity modules > Assignment > Feedback plugins > Annotate PDF > Test unoconv path'' <br>
You should see: <br>
"The unoconv path appears to be properly configured."<br>
*Download the converted pdf test file. (if the PDF fails to load ensure that www-data can write to its home directory: /var/www)


===CentOS / RedHat===
===CentOS / RedHat===
As of CentOS 7.x (and possibly before), The entire installation of unoconv (v0.6) and LibreOffice (v5.0.6.2) can be done via the Yum package manager. To do this, execute the following command:
<pre>
yum groupinstall " Office Suite and Productivity"
</pre>
[Optional] If you'd like to upgrade to the latest version of unoconv, download the latest version via Git:
<pre>
<pre>
yum install openoffice*
yum install libreoffice-pyuno
git clone https://github.com/dagwieers/unoconv.git
git clone https://github.com/dagwieers/unoconv.git
cd unoconv
# copy
cp unoconv /usr/bin
# or link unoconv to /usr/bin
ln -s ./unoconv /usr/bin/unoconv
</pre>
</pre>
Make sure it is properly configured:
 
http://your-moodle/admin/search.php?query=unoconv
Then copy the executable to the /usr/bin/ directory:
<pre>
cp unoconv/unoconv /usr/bin
</pre>
 
To ensure that unoconv is installed and properly configured, visit your moodle site with the following appended:
<pre>
/admin/search.php?query=unoconv
</pre>
 
In order for the conversion to work, you will need to create a system service for unoconv. This will allow it to listen for conversions from the apache user. Create the service file:
<pre>
vi /etc/systemd/system/unoconv.service
</pre>
 
Then copy and paste the following configuration into it:
<pre>
[Unit]
Description=Unoconv listener for document conversions
Documentation=https://github.com/dagwieers/unoconv
After=network.target remote-fs.target nss-lookup.target
 
[Service]
Type=simple
Environment="UNO_PATH=/usr/lib64/libreoffice/program"
ExecStart=/usr/bin/unoconv --listener
 
[Install]
WantedBy=multi-user.target
</pre>
 
And then enable and start the above service:
<pre>
systemctl enable unoconv.service
systemctl start unoconv.service
</pre>
 
'''Note:''' If your selinux is enable yous should execute the following command to allow apache (httpd) to execute an external command:
<pre>
#setsebool -P httpd_execmem on
</pre>
 
With all that completed, you can test the entire installation by uploading a .docx (or similar) file to your server and then executing the following command (with your modifications):
<pre>
sudo -u apache unoconv -vvv -f pdf unoconv_test.docx
</pre>


== Installing unoconv on OS X ==
== Installing unoconv on OS X ==
Line 103: Line 189:
Login as admin and go to ''Site administration > Server > System paths''
Login as admin and go to ''Site administration > Server > System paths''


Set pathtogs setting to your ghostscript installation binary, (C:\gs\bin\gswin32.exe)
Set pathtogs setting to your ghostscript installation binary, (C:\gs\bin\gswin32c.exe)
(Do not use gswin32.exe or gswin64.exe, these are not command line programs - use gswin32c.exe or gswin64c.exe)


Set pathtounoconv to the batch file created above (C:\unoconv\unoconv.bat)
Set pathtounoconv to the batch file created above (C:\unoconv\unoconv.bat)
Line 119: Line 206:


[[mod/assign/feedback/editpdf/testunoconv/initd | Init script for Debian]]
[[mod/assign/feedback/editpdf/testunoconv/initd | Init script for Debian]]
[[mod/assign/feedback/editpdf/testunoconv/initdcentos6 | Init script for CentOS/RedHat 6.x]]
[[mod/assign/feedback/editpdf/testunoconv/systemd | SystemD service script for CentOS/RedHat 7.x]]


== Offload processing to a different server ==
== Offload processing to a different server ==
Line 144: Line 235:


==Additional resources==
==Additional resources==
The unoconv [https://github.com/dagwieers/unoconv documentation] site has additional information on installation of unoconv and troubleshooting tips.
 
[https://github.com/dagwieers/unoconv GitHub dagwieers/unoconv] has additional information on installation of unoconv and troubleshooting tips.
 
==Troubleshooting==
* [http://webnetkit.com/soffice-bin-using-100-cpu-moodle/ soffice.bin using 100% of CPU MOODLE]


==See also==
==See also==
Line 153: Line 248:
[[Category:Assignment]]
[[Category:Assignment]]


[[es:mod/assign/feedback/editpdf/testunoconv]]
[[es:Universal Office Converter (unoconv)]]
[[de:Universal Office Konverter]]

Latest revision as of 05:04, 11 February 2018

What is unoconv?

"unoconv" is a command line program that is used to convert between different office document file formats. It uses an instance of LibreOffice to do the conversion and is used by the Assignment activity to convert documents to pdf so that they can be annotated. If unoconv is not installed - the only impact is that the assignment activities will only allow annotations when students upload a pdf document.

The steps required to install unoconv are different depending on the operating system that you have installed Moodle on.

Installing unoconv on Linux

The required version of unoconv is at least 0.7. Depending on your flavour of linux, this may be available in your package manager and you can install it directly with:

Ubuntu 16.04 LTS

apt-get install unoconv

If your package manager contains an older version of the package, you will have to find a newer version and install it manually (Debian Testing). Unoconv itself is just a python script, so it has few dependencies.

Potential problems:

  • On some systems the apache user home directory is set to a non existent folder. This can cause unoconv to fail. There are 2 solutions to this - one is to make a (writable) home directory for the apache user (like /home/www-data). The other is to run a unoconv listener (described below) as another user other than the apache user (someone with a valid, writable home directory).
  • If you are still running 14.04LTS then unoconv won't work as shipped. This might not be the most efficient route but it worked by first installing unoconv (version 0.6) from the package manager as above. You will then need to grab unoconv 0.7 from Github (https://github.com/dagwieers/unoconv), then upgrade to the latest libreoffice using the PPA (https://launchpad.net/~libreoffice/+archive/ubuntu/ppa). Point moodle at the Github version of unoconv. You need to modify the Python unoconv file by changing 'python' in the first line to 'python3'. You also need to change the permissions on the directory /var/www so that the user www-data can write to it (www-data needs to write to its home directory which it cannot do by default).

On Debian Stable, the cleanest method to install unoconv is using Jessie-backports. First, enable backports repo line in /etc/apt/sources.list:

#### Jessie-backports  ####
deb http://ftp.debian.org/debian jessie-backports main

Then, update and install unoconv from jessie-backports:

apt-get update
apt-get install -t jessie-backports unoconv

The package will bring all necessary dependencies for you.


Ubuntu 14.04 LTS

1) Navigate to opt directory

cd /opt

2) Download unoconv

sudo wget https://raw.githubusercontent.com/dagwieers/unoconv/master/unoconv

3) Modify the Python unoconv file by changing 'python' in the first line to 'python3'

sudo nano /opt/unoconv


eg.

#!/usr/bin/env python3

4) Make unoconv executable

sudo chmod ugo+x /opt/unoconv

5) Add LibreOffice PPA to your system and install the latest version

sudo add-apt-repository ppa:libreoffice/ppa
sudo apt-get update
sudo apt-get install libreoffice

6) Change permissions so apache can write to its home directory

sudo chown www-data /var/www

7) From your browser navigate to Site administration > Server > System paths and add the path to unoconv
/opt/unoconv

  • Note: if you would like to preserve the default path add a symbolic link to /usr/bin:
sudo ln -s /opt/unoconv /usr/bin/

8) Navigate to Site administration > Plugins > Activity modules > Assignment > Feedback plugins > Annotate PDF > Test unoconv path
You should see:
"The unoconv path appears to be properly configured."

  • Download the converted pdf test file. (if the PDF fails to load ensure that www-data can write to its home directory: /var/www)

CentOS / RedHat

As of CentOS 7.x (and possibly before), The entire installation of unoconv (v0.6) and LibreOffice (v5.0.6.2) can be done via the Yum package manager. To do this, execute the following command:

yum groupinstall " Office Suite and Productivity"

[Optional] If you'd like to upgrade to the latest version of unoconv, download the latest version via Git:

git clone https://github.com/dagwieers/unoconv.git

Then copy the executable to the /usr/bin/ directory:

cp unoconv/unoconv /usr/bin

To ensure that unoconv is installed and properly configured, visit your moodle site with the following appended:

/admin/search.php?query=unoconv

In order for the conversion to work, you will need to create a system service for unoconv. This will allow it to listen for conversions from the apache user. Create the service file:

vi /etc/systemd/system/unoconv.service

Then copy and paste the following configuration into it:

[Unit]
Description=Unoconv listener for document conversions
Documentation=https://github.com/dagwieers/unoconv
After=network.target remote-fs.target nss-lookup.target

[Service]
Type=simple
Environment="UNO_PATH=/usr/lib64/libreoffice/program"
ExecStart=/usr/bin/unoconv --listener

[Install]
WantedBy=multi-user.target

And then enable and start the above service:

systemctl enable unoconv.service
systemctl start unoconv.service

Note: If your selinux is enable yous should execute the following command to allow apache (httpd) to execute an external command:

#setsebool -P httpd_execmem on

With all that completed, you can test the entire installation by uploading a .docx (or similar) file to your server and then executing the following command (with your modifications):

sudo -u apache unoconv -vvv -f pdf unoconv_test.docx

Installing unoconv on OS X

Download and install LibreOffice for Mac. Unfortunately - newer versions of LibreOffice are not currently compatible with unoconv for mac and you will have to install LibreOffice 4.2 (Direct download link - https://downloadarchive.documentfoundation.org/libreoffice/old/4.2.5.2/mac/x86_64/LibreOffice_4.2.5.2_MacOS_x86-64.dmg).

Get the latest version of the unoconv python script. One way to do this is with http://brew.sh/ brew.

brew install unoconv

If you haven't done it already - install ghostscript. One way to install ghostscript is also with http://brew.sh/ brew

brew install ghostscript

Set the paths to unoconv and ghostscript in Moodle (Site administration > Server > System paths). If you used brew, they will both be installed to /usr/local/bin.

LibreOffice needs write access to the current users home directory to create some temporary files. When unoconv is run as the webserver user (_www) it does not normally have this permission.

There are some ways to get around this - one way is just to give the "_www" user write access to /Library/WebServer.

Another solution is to convince LibreOffice that this users home directory is somewhere else. This can be done by inserting this code into the top of the unoconv python script. Code to insert:

# Set home to a writable folder. 
os.environ['HOME'] = '/tmp/'                                                                                                        

This needs to be inserted at line 36 immediately after the line "exitcode = 0"

Installing unoconv on Windows

Download and install LibreOffice for windows.

Download the latest version of the unoconv script from https://github.com/dagwieers/unoconv/releases (download the zip version).

From the downloaded zip file - extract the one file "unoconv-0.7\unoconv" (no file extension). This is the unoconv script - none of the other files in the package are required.

Rename the downloaded script to C:\unoconv\unoconv.py

Create a batch file C:\unoconv\unoconv.bat with these contents:

@"C:\Program Files\LibreOffice 5\program\python.exe" c:\unoconv\unoconv.py %*

Set paths in Moodle.

Login as admin and go to Site administration > Server > System paths

Set pathtogs setting to your ghostscript installation binary, (C:\gs\bin\gswin32c.exe) (Do not use gswin32.exe or gswin64.exe, these are not command line programs - use gswin32c.exe or gswin64c.exe)

Set pathtounoconv to the batch file created above (C:\unoconv\unoconv.bat)

Test ghostscript and unoconv are working correctly in the admin test pages "Site administration > Plugins > Activity modules > Assignment > Feedback plugins > Annotate PDF".

Run a unoconv listener

Unoconv utilises a client/server process when converting documents. By default, when there is no running server process - each time unoconv runs it will start a server process, send its request and shut down the server process when the request is complete. The drawback of this mode is that if 2 requests are submitted simultaneously - this can cause the first request to shutdown the server process when the second request is still in progress - and the second conversion request fails. A more robust way to configure unoconv is to start a server process at boot time, and/or run a script to monitor it and restart it if it crashes.

To start a unoconv listener at boot time - you need a start up script. Different operating systems and Linux distributions use different startup scripts - but here are some examples of startup scripts for different systems.

Upstart script for Ubuntu based systems

Launchd script for OS X

Init script for Debian

Init script for CentOS/RedHat 6.x

SystemD service script for CentOS/RedHat 7.x

Offload processing to a different server

Processing office documents can put increased load on your webserver, which may impact on the responsiveness of your site. If you are installing unoconv on a large site you may want to consider running unoconv on a server that is not also serving web requests.

How to do this:

Install unoconv on each webservers and the remote server following the installation instructions above.

Make sure unoconv is started at boot time on the remote server with the "--listener" argument and is monitored and restarted if it exits (see Debian init script for an example of how to do this).

Open the firewall port 2002 between the moodle webservers and the machine running unoconv.

Share the moodle data root between the webservers and the machine running unoconv. This folder must be mounted at the same path on all servers.

Install a wrapper for unoconv on the webservers that forwards the requests to the remote server. Example:

#!/bin/bash
# Wrapper script for unoconv to forward processing.
# Install to /usr/bin/unoconv-remote with 755 permissions
/usr/bin/unoconv --server=<ip of remote server> "$@"

Configure the path to unoconv in the Moodle admin settings to point to this wrapper script.

Additional resources

GitHub dagwieers/unoconv has additional information on installation of unoconv and troubleshooting tips.

Troubleshooting

See also