Universal Office Converter (unoconv)

From MoodleDocs
Revision as of 14:50, 21 March 2018 by Matt Rice (talk | contribs) (→‎Offload processing to a different server: Add example command to for remote unoconv listen server)

What is unoconv?

"unoconv" is a command line program that is used to convert between different office document file formats. It uses an instance of LibreOffice to do the conversion and is used by the Assignment activity to convert documents to pdf so that they can be annotated. If unoconv is not installed - the only impact is that the assignment activities will only allow annotations when students upload a pdf document.

The steps required to install unoconv are different depending on the operating system that you have installed Moodle on.

Installing unoconv on Linux

The required version of unoconv is at least 0.7. Depending on your flavour of linux, this may be available in your package manager and you can install it directly with:

Ubuntu 16.04 LTS

apt-get install unoconv

If your package manager contains an older version of the package, you will have to find a newer version and install it manually (Debian Testing). Unoconv itself is just a python script, so it has few dependencies.

Potential problems:

  • On some systems the apache user home directory is set to a non existent folder. This can cause unoconv to fail. There are 2 solutions to this - one is to make a (writable) home directory for the apache user (like /home/www-data). The other is to run a unoconv listener (described below) as another user other than the apache user (someone with a valid, writable home directory).
  • If you are still running 14.04LTS then unoconv won't work as shipped. This might not be the most efficient route but it worked by first installing unoconv (version 0.6) from the package manager as above. You will then need to grab unoconv 0.7 from Github (https://github.com/dagwieers/unoconv), then upgrade to the latest libreoffice using the PPA (https://launchpad.net/~libreoffice/+archive/ubuntu/ppa). Point moodle at the Github version of unoconv. You need to modify the Python unoconv file by changing 'python' in the first line to 'python3'. You also need to change the permissions on the directory /var/www so that the user www-data can write to it (www-data needs to write to its home directory which it cannot do by default).

On Debian Stable, the cleanest method to install unoconv is using Jessie-backports. First, enable backports repo line in /etc/apt/sources.list:

#### Jessie-backports  ####
deb http://ftp.debian.org/debian jessie-backports main

Then, update and install unoconv from jessie-backports:

apt-get update
apt-get install -t jessie-backports unoconv

The package will bring all necessary dependencies for you.


Ubuntu 14.04 LTS

1) Navigate to opt directory

cd /opt

2) Download unoconv

sudo wget https://raw.githubusercontent.com/dagwieers/unoconv/master/unoconv

3) Modify the Python unoconv file by changing 'python' in the first line to 'python3'

sudo nano /opt/unoconv


eg.

#!/usr/bin/env python3

4) Make unoconv executable

sudo chmod ugo+x /opt/unoconv

5) Add LibreOffice PPA to your system and install the latest version

sudo add-apt-repository ppa:libreoffice/ppa
sudo apt-get update
sudo apt-get install libreoffice

6) Change permissions so apache can write to its home directory

sudo chown www-data /var/www

7) From your browser navigate to Site administration > Server > System paths and add the path to unoconv
/opt/unoconv

  • Note: if you would like to preserve the default path add a symbolic link to /usr/bin:
sudo ln -s /opt/unoconv /usr/bin/

8) Navigate to Site administration > Plugins > Activity modules > Assignment > Feedback plugins > Annotate PDF > Test unoconv path
You should see:
"The unoconv path appears to be properly configured."

  • Download the converted pdf test file. (if the PDF fails to load ensure that www-data can write to its home directory: /var/www)

CentOS / RedHat

Just before you start, you might like to consider installing the latest LibreOffice 5.2 directly from RPM packages, that are not part of the distribution you are using. As of nov-2016, CentOS and RedHat 7.2 comes with OpenOffice 4.3 . so if you are not interested in using this version and would like to install latest 5.2 independent LibreOffice 5.2 , please remove any openoffice packages you might have on your OS by issuing:

yum remove openoffice* libreoffice*

And then follow the install instructions LibreOffice 5.2. It is recommended to chose your localized libreoffice version for better document conversions. and also please skip the following "yum install openoffice* ..." command.

yum install openoffice* openoffice-pyuno
git clone https://github.com/dagwieers/unoconv.git
# copy 
cp unoconv/unoconv /usr/bin
# or link unoconv to /usr/bin
ln -s unoconv/unoconv /usr/bin/unoconv
Note: depends on what version you are installing, openoffice or libreoffice, make sure you installed the *-pyuno package. (the headless package is already compiled into the core)

Make sure it is properly configured: http://your-moodle/admin/search.php?query=unoconv

Production servers should consider running unoconv in listener mode, see Installing_unoconv#Run_a_unoconv_listener or follow directions bellow

vi /etc/systemd/system/unoconv.service

And then copy and paste the following configuration into it:

[Unit]
Description=Unoconv listener for document conversions
Documentation=https://github.com/dagwieers/unoconv
After=network.target remote-fs.target nss-lookup.target

[Service]
Type=simple
Environment="UNO_PATH=/usr/lib64/libreoffice/program"
ExecStart=/usr/bin/unoconv --listener

[Install]
WantedBy=multi-user.target

And then enable and start the above service

systemctl enable unoconv.service
systemctl start unoconv.service

If your selinux is enable yous should set

#setsebool -P httpd_execmem on

Installing unoconv on OS X

Download and install LibreOffice for Mac. Unfortunately - newer versions of LibreOffice are not currently compatible with unoconv for mac and you will have to install LibreOffice 4.2 (Direct download link - https://downloadarchive.documentfoundation.org/libreoffice/old/4.2.5.2/mac/x86_64/LibreOffice_4.2.5.2_MacOS_x86-64.dmg).

Get the latest version of the unoconv python script. One way to do this is with http://brew.sh/ brew.

brew install unoconv

If you haven't done it already - install ghostscript. One way to install ghostscript is also with http://brew.sh/ brew

brew install ghostscript

Set the paths to unoconv and ghostscript in Moodle (Site administration > Server > System paths). If you used brew, they will both be installed to /usr/local/bin.

LibreOffice needs write access to the current users home directory to create some temporary files. When unoconv is run as the webserver user (_www) it does not normally have this permission.

There are some ways to get around this - one way is just to give the "_www" user write access to /Library/WebServer.

Another solution is to convince LibreOffice that this users home directory is somewhere else. This can be done by inserting this code into the top of the unoconv python script. Code to insert:

# Set home to a writable folder. 
os.environ['HOME'] = '/tmp/'                                                                                                        

This needs to be inserted at line 36 immediately after the line "exitcode = 0"

Installing unoconv on Windows

Download and install LibreOffice for windows.

Download the latest version of the unoconv script from https://github.com/dagwieers/unoconv/releases (download the zip version).

From the downloaded zip file - extract the one file "unoconv-0.7\unoconv" (no file extension). This is the unoconv script - none of the other files in the package are required.

Rename the downloaded script to C:\unoconv\unoconv.py

Create a batch file C:\unoconv\unoconv.bat with these contents:

@"C:\Program Files\LibreOffice\program\python.exe" c:\unoconv\unoconv.py %*

Set paths in Moodle.

Login as admin and go to Site administration > Server > System paths

Set pathtogs setting to your ghostscript installation binary, (C:\gs\bin\gswin32c.exe) (Do not use gswin32.exe or gswin64.exe, these are not command line programs - use gswin32c.exe or gswin64c.exe)

Set pathtounoconv to the batch file created above (C:\unoconv\unoconv.bat)

Test ghostscript and unoconv are working correctly in the admin test pages "Site administration > Plugins > Activity modules > Assignment > Feedback plugins > Annotate PDF".

Run a unoconv listener

Unoconv utilises a client/server process when converting documents. By default, when there is no running server process - each time unoconv runs it will start a server process, send its request and shut down the server process when the request is complete. The drawback of this mode is that if 2 requests are submitted simultaneously - this can cause the first request to shutdown the server process when the second request is still in progress - and the second conversion request fails. A more robust way to configure unoconv is to start a server process at boot time, and/or run a script to monitor it and restart it if it crashes.

To start a unoconv listener at boot time - you need a start up script. Different operating systems and Linux distributions use different startup scripts - but here are some examples of startup scripts for different systems.

Upstart script for Ubuntu based systems

Launchd script for OS X

Init script for Debian

Init script for CentOS/RedHat 6.x

SystemD service script for CentOS/RedHat 7.x

Offload processing to a different server

Processing office documents can put increased load on your webserver, which may impact on the responsiveness of your site. If you are installing unoconv on a large site you may want to consider running unoconv on a server that is not also serving web requests.

How to do this:

Install unoconv on each webservers and the remote server following the installation instructions above.

Make sure unoconv is started at boot time on the remote server with the "--listener" argument and is monitored and restarted if it exits (see Debian init script for an example of how to do this). By default, unoconv will only listen on localhost (127.0.0.1): if you want to connect to the listener process from another server, you need to start the unoconv listener process with the "--server" argument too!

An example command for starting a listener on a remote server (0.0.0.0 listens on all interfaces):

unoconv --listener --server 0.0.0.0 --port 2002

Open the firewall port 2002 between the moodle webservers and the machine running unoconv.

Share the moodle data root between the webservers and the machine running unoconv. This folder must be mounted at the same path on all servers.

Install a wrapper for unoconv on the webservers that forwards the requests to the remote server. Example:

#!/bin/bash
# Wrapper script for unoconv to forward processing.
# Install to /usr/bin/unoconv-remote with 755 permissions
/usr/bin/unoconv --server=<ip of remote server> "$@"

Configure the path to unoconv in the Moodle admin settings to point to this wrapper script.

Additional resources

GitHub dagwieers/unoconv has additional information on installation of unoconv and troubleshooting tips.

Troubleshooting

See also