Note:

If you want to create a new page for developers, you should create it on the Moodle Developer Resource site.

External services security: Difference between revisions

From MoodleDocs
(196 intermediate revisions by 5 users not shown)
Line 5: Line 5:
=Overview=
=Overview=


==Current solutions==
Current solutions:
* user keys for gradebook import and export - see require_user_key_login() and db table ''user_private_key''
* user keys for gradebook import and export - see require_user_key_login() and db table ''user_private_key''
* open RSS feeds - no security at all
* open RSS feeds - no security at all
* chat_sid tokens - generated separately for each user in each chat
* chat_sid tokens - generated separately for each user in each chat
* calendar hash from user name, password and salt
* calendar export - hash from user name, password and salt
* hacky cookie emulation in visual gradebook plugin
* hacky cookie emulation in visual gradebook plugin


==Types of token==
=Design=
We need several types of tokens
# token sharing/linked to active session, should time out or be destroyed at the same time as session (ex.: chat) - shared $SESSION and $USER
# permanent token, revokeable by user (ex.: RSS feeds, web services) - emulated $SESSION and $USER


In the second case we need to deal with performance problems if many repeated request expected. This can be dealt with later.
==Different uses==
The external API may be used from different places:
# directly from PHP - no authentication, current user session is used ($USER, $SESSION)
# simple web service layer - easy to use and setup WS layer designed for interaction with enterprise systems (each having own single purpose user account), the protocol is stateless, username and password is sent with each request; security is enforced on multiple levels
# full WS layer with token authentication - it is more flexible and enables normal users to use some restricted parts of the WS APIs, performance might be significantly improved by emulation of sessions
# when embedding external applications - external application receives unique token which is used instead of normal browser session cookie, the session is linked to the current user session in browser, the token is automatically invalidated after logout
# RSS feeds, iCals, etc. - token login, no permanent session


==API layers==
==API layers==


Three layers:
Three layers:
# external server interface (SOAL, REST, RSS, etc.) - deals with tokens, emulates user session, parameter processing
# external server interface (SOAP, REST, RSS, etc.) - deals with tokens, emulates user session, parameter processing
# public PHP API - functions usable directly from PHP, list generated from inline PHP docs, need to verify '''all''' parameters and access control, may access $USER, should not manipulate $SESSION directly, must not read $_POST or $_GET
# public PHP API - functions usable directly from PHP, list generated from inline PHP docs, need to verify '''all''' parameters and access control, may access $USER, should not manipulate $SESSION directly, must not read $_POST or $_GET
# low level internal API - as fast as possible, basic param validation, no access control, must not touch $USER, $SESSION, $_GET or $_POST, must not use has_capability() or require_login()!
# low level internal API - as fast as possible, basic param validation, no access control, must not touch $USER, $SESSION, $_GET or $_POST, must not use has_capability() or require_login()!
==Context restrictions==
Context restriction of token validity should be effective against security problems in external applications interacting with Moodle. Some external applications do not have any access to http cookies, solution is to create temporary tokens. Context restrictions would allow us to grant external access to individual activities, courses ,etc..


=Implementation=
=Implementation=
==Updated user_private_key table==
 
Stores active tokens for cookieless access.
==PHP API==
This simplest case of using the external API. The calling call has to make sure the current $USER and $SESSION is valid. Optionally the caller may set up the context restriction manually. This should be the easiest way to start with Moodle modifications and integration with other PHP software.
 
==Simplified web services==
This type of web service is using simple username+password authentication (in future could be username+certificate). The communication integrity/privacy can be protected using https. Each external application is using own user account. These user accounts can not be used for normal login into moodle vie web interface.
 
This type of WS is intended primarily for integration with student information systems or other enterprise software.
 
===webservice auth plugin===
It is recommended to create separate user for each external application. The benefit of this auth type is that it can not be used for normal log-in from the web interface.
 
Several people questioned the security of sending username+password with each request - it is not a problem because this password is used only for web services, it can not be used for anything else. This means that the password is effectively a standard authentication token ;-) We coudl replace this password by a public certificate stored in the user profile field.
 
===WS entry point===
Separate php file /webservice/xxx/simpleserver.php
 
===external_services_users table===
Specifies which user may use each external service when restrictedusers flag set in service.


{| class="nicetable"
{| class="nicetable"
Line 41: Line 64:
|  
|  
|-
|-
| '''type'''  
| '''externalserviceid'''
| int(10)
|
| foreign key, reference external_services.id
|-
| '''userid'''  
| int(10)
| int(10)
| 0
|  
| 0 means original permanent key, 1 means key linked to current session
| foreign key, reference user.id
|-
| iprestriction
| varchar(255)
| NULL
| restrict access to some ips or subnets only
|-
|-
| '''value'''  
| validuntil
| int(10)
| NULL
| invalidate after (once the date is reached the user cannot access the service)
|-
| timecreated
| int(10)
|
| time when this record added (date of the user/service relation creation)
|}
 
===Steps needed to configure simple web service access===
 
# enable web services
# create new service in some local plug-ip (add code into /local/yourplugin/db/services.php) or manually add new service with some functions
# add new user, set auth plugin to '''webservice''' (normal add user UI)
# enable web service layer plugin (admin UI)
# optionally add user into list of users that are allowed to use specific external services (admin UI)
# set up user permissions needed in function implementations. All required capabilities should be listed in the external function description.
# set up permission to use specific web services (optional, services do not need to specify required capability)
 
 
#Log as admin
#Add a new user
#Enable the Web Service Authentication plugin
#In the new user profil, change authentication method for web services
#In advance feature, enable web services
#In Plugin/Web Services/Manage protocol, enable the protocol you need
#In Plugin/Web Services/External Services, Add a new service, activate it and add some function to this service.
#If the selected service has restricted user option activated, assign the new user to this service
#If the selected service has required capability, assign this capability to the new user
#Assign the capability to use the selected protocol - you'll probably want to create a Web Service role for that
#the new user can now access web service from the enabled protocol with his username/password
 
==General web services==
 
More flexible than the WS with simplified authentication and setup. The performance may be significantly improved by emulation of sessions. The authentication is based on security tokens (user, context and purpose specific) that are generated in normal Moodle interface and then used in other external applications that are using our web services. This allows us to give only partial access to some parts of WS api, external applications and services do not need to be fully trusted.
 
The main drawback is that the complex configuration does not encourage admins to keep everything secured.
 
===external_tokens===
Stores permanent tokens for cookieless access, script runs without real session. (Later might have to add emulated session for performance reasons.)
 
{| class="nicetable"
! Field
! Type
! Default
! Description
|-
| '''id'''
| int(10)
| auto-incrementing
|
|-
| '''token'''  
| varchar(128)
| varchar(128)
|  
|  
| private access key value
| private access key value
|-
|-
| '''sid'''
| tokentype
| varchar(128)
| int(3)
| null
|  
| optional session identifier when key linked to active session
| type of token: 0=permanent, no session; 1=linked to current browser session via sid; 2=permanent, with emulated session
|-
|-
| service
| userid
| varchar(150)
| int(10)
|  
|  
| name of service (ex. mod/forum/rss)
| foreign key, references user.id
|-
| creatorid
| int(10)
| the token creator (an administrator)
| foreign key, references user.id
|-
|-
| '''userid'''
| externalserviceid
| int(10)
| int(10)
|  
|  
| foreign key, references user.id
| foreign key, references external_services.id
|-
| sid
| varchar(128)
| NULL
| links to browser or emulated session
|-
|-
| contextid  
| contextid
| int(10)
| int(10)
| null
|
| security restriction, key usable only in this context, references context.id
| security restriction, token usable only in this context, references context.id
|-
|-
| iprestriction
| iprestriction
| varchar(255)
| varchar(255)
| null
| NULL
| IP address restriction, list of allowed addresses
| IP address restriction, list of allowed addresses
|-
|-
| validuntil  
| validuntil  
| int(10)
| int(10)
| null
| NULL
| timestampt - valid until date
| timestampt - valid until date
|-
|-
Line 92: Line 189:
|}
|}


Alternative solution would be to have two separate tables each for different key type.
===Steps needed to configure general web service access===
 
# enable general web services - global switch affecting all general web services (admin UI)
# enable web service layer plugins (admin UI)
# setup user permission which allows them to create tokens in some specific context for some specific purpose (roles UI)
# set up user permissions needed in function implementations
# set up permission to use specific web services (optional, services do not need to specify required capability)
# each user must create permanent token in specific context for specific purpose (user UI)
# user must copy/past this token to external application
 
==== Random all-time valid token ====
A user can use the same token to call a web service. We call this token an all-time valid token.
 
'''Specificity'''
* A token can be created by a Moodle administrator (administration page) or a simple Moodle user (user profil).
* An administrator has the ability to create a token for another Moodle user but not for himself neither for another administrator (for security purpose).
* A token is always linked to a Moodle user.
* A token cannot be linked to an administrator .
* A Moodle user create his own token on his profil page.
* To create a token, a user need "moodle/webservice:createtoken" capability.
* An administrator create token for other on a web service token page.
* An administrator cannot see token that he didn't create.
* A token is generated randomly and is long enough.
* A token can be used by an external web application. (by browser)
* A token can be used by a mobile application. (i.e. iphone native application)
* A token is sent by the external application at the same time that the function parameters are sent.
* When the user create a token, he must select a service (to link the token to a service). Only services that the user has capability on are displayed.
* ''ip restriction'' and ''valid until'' fields are optional.
* A Moodle user must select an aera on which the token can be valid. (course, ...) - TBD - unclear yet
* A user can only see token string if https is enabled.
* A token can be linked to only one service.
 
 
 
Administration: token management (valid until field is missing here):<br/>
[[Image:Admin_token.png]]
 
 
Administration: create token:<br/>
[[Image:Create_token.png]]
 
==Application embedding==
 
Similar to general WS, the token is linked to current session - when user logs out, the external token is immediately invalidated. The administration is much easier and the tokens are usually created automatically.


==New public_functions table==
The external applications maybe be embedded into browser or they can be launched as separate programs (ex.: Java Web Start, Flex, external media players, etc.)
List of public functions. Created automatically by parsing external files.


==New public_services table==
It is not necessary to use WS layers defined in /webservices/xxx/, the same infrastructure may be used in all ajax scripts that do not have access to browser cookies.
''Service'' is defined as a group of functions.


==New capabilities==
==RSS, iCALs and other feeds==
New capability for each public service above.


==New auth plugins==
All cookie-less scripts such as RSS can use the external API and together with the permanent tokens. In fact ATOM is a nice example of RESTful protocol :-)
Used for system user accounts the web services are running under. Intended for services that need to supply username/password. Alternative solution is to generate normal user token and use it instead of login/password authentication.


=See also=
=See also=
* [[Web services]]
* [[Web services]]
* [[External services description]]


[[Category:Web Services]]
[[Category:Web Services]]

Revision as of 02:59, 14 January 2010

Moodle 2.0


Descriptions of security framework for web services, also used for RSS feeds, embedded application and similar parts that can not use normal HTTP cookies.

Overview

Current solutions:

  • user keys for gradebook import and export - see require_user_key_login() and db table user_private_key
  • open RSS feeds - no security at all
  • chat_sid tokens - generated separately for each user in each chat
  • calendar export - hash from user name, password and salt
  • hacky cookie emulation in visual gradebook plugin

Design

Different uses

The external API may be used from different places:

  1. directly from PHP - no authentication, current user session is used ($USER, $SESSION)
  2. simple web service layer - easy to use and setup WS layer designed for interaction with enterprise systems (each having own single purpose user account), the protocol is stateless, username and password is sent with each request; security is enforced on multiple levels
  3. full WS layer with token authentication - it is more flexible and enables normal users to use some restricted parts of the WS APIs, performance might be significantly improved by emulation of sessions
  4. when embedding external applications - external application receives unique token which is used instead of normal browser session cookie, the session is linked to the current user session in browser, the token is automatically invalidated after logout
  5. RSS feeds, iCals, etc. - token login, no permanent session

API layers

Three layers:

  1. external server interface (SOAP, REST, RSS, etc.) - deals with tokens, emulates user session, parameter processing
  2. public PHP API - functions usable directly from PHP, list generated from inline PHP docs, need to verify all parameters and access control, may access $USER, should not manipulate $SESSION directly, must not read $_POST or $_GET
  3. low level internal API - as fast as possible, basic param validation, no access control, must not touch $USER, $SESSION, $_GET or $_POST, must not use has_capability() or require_login()!

Context restrictions

Context restriction of token validity should be effective against security problems in external applications interacting with Moodle. Some external applications do not have any access to http cookies, solution is to create temporary tokens. Context restrictions would allow us to grant external access to individual activities, courses ,etc..

Implementation

PHP API

This simplest case of using the external API. The calling call has to make sure the current $USER and $SESSION is valid. Optionally the caller may set up the context restriction manually. This should be the easiest way to start with Moodle modifications and integration with other PHP software.

Simplified web services

This type of web service is using simple username+password authentication (in future could be username+certificate). The communication integrity/privacy can be protected using https. Each external application is using own user account. These user accounts can not be used for normal login into moodle vie web interface.

This type of WS is intended primarily for integration with student information systems or other enterprise software.

webservice auth plugin

It is recommended to create separate user for each external application. The benefit of this auth type is that it can not be used for normal log-in from the web interface.

Several people questioned the security of sending username+password with each request - it is not a problem because this password is used only for web services, it can not be used for anything else. This means that the password is effectively a standard authentication token ;-) We coudl replace this password by a public certificate stored in the user profile field.

WS entry point

Separate php file /webservice/xxx/simpleserver.php

external_services_users table

Specifies which user may use each external service when restrictedusers flag set in service.

Field Type Default Description
id int(10) auto-incrementing
externalserviceid int(10) foreign key, reference external_services.id
userid int(10) foreign key, reference user.id
iprestriction varchar(255) NULL restrict access to some ips or subnets only
validuntil int(10) NULL invalidate after (once the date is reached the user cannot access the service)
timecreated int(10) time when this record added (date of the user/service relation creation)

Steps needed to configure simple web service access

  1. enable web services
  2. create new service in some local plug-ip (add code into /local/yourplugin/db/services.php) or manually add new service with some functions
  3. add new user, set auth plugin to webservice (normal add user UI)
  4. enable web service layer plugin (admin UI)
  5. optionally add user into list of users that are allowed to use specific external services (admin UI)
  6. set up user permissions needed in function implementations. All required capabilities should be listed in the external function description.
  7. set up permission to use specific web services (optional, services do not need to specify required capability)


  1. Log as admin
  2. Add a new user
  3. Enable the Web Service Authentication plugin
  4. In the new user profil, change authentication method for web services
  5. In advance feature, enable web services
  6. In Plugin/Web Services/Manage protocol, enable the protocol you need
  7. In Plugin/Web Services/External Services, Add a new service, activate it and add some function to this service.
  8. If the selected service has restricted user option activated, assign the new user to this service
  9. If the selected service has required capability, assign this capability to the new user
  10. Assign the capability to use the selected protocol - you'll probably want to create a Web Service role for that
  11. the new user can now access web service from the enabled protocol with his username/password

General web services

More flexible than the WS with simplified authentication and setup. The performance may be significantly improved by emulation of sessions. The authentication is based on security tokens (user, context and purpose specific) that are generated in normal Moodle interface and then used in other external applications that are using our web services. This allows us to give only partial access to some parts of WS api, external applications and services do not need to be fully trusted.

The main drawback is that the complex configuration does not encourage admins to keep everything secured.

external_tokens

Stores permanent tokens for cookieless access, script runs without real session. (Later might have to add emulated session for performance reasons.)

Field Type Default Description
id int(10) auto-incrementing
token varchar(128) private access key value
tokentype int(3) type of token: 0=permanent, no session; 1=linked to current browser session via sid; 2=permanent, with emulated session
userid int(10) foreign key, references user.id
creatorid int(10) the token creator (an administrator) foreign key, references user.id
externalserviceid int(10) foreign key, references external_services.id
sid varchar(128) NULL links to browser or emulated session
contextid int(10) security restriction, token usable only in this context, references context.id
iprestriction varchar(255) NULL IP address restriction, list of allowed addresses
validuntil int(10) NULL timestampt - valid until date
timecreated int(10) time when key created
lastaccess int(10) time when key last used for access

Steps needed to configure general web service access

  1. enable general web services - global switch affecting all general web services (admin UI)
  2. enable web service layer plugins (admin UI)
  3. setup user permission which allows them to create tokens in some specific context for some specific purpose (roles UI)
  4. set up user permissions needed in function implementations
  5. set up permission to use specific web services (optional, services do not need to specify required capability)
  6. each user must create permanent token in specific context for specific purpose (user UI)
  7. user must copy/past this token to external application

Random all-time valid token

A user can use the same token to call a web service. We call this token an all-time valid token.

Specificity

  • A token can be created by a Moodle administrator (administration page) or a simple Moodle user (user profil).
  • An administrator has the ability to create a token for another Moodle user but not for himself neither for another administrator (for security purpose).
  • A token is always linked to a Moodle user.
  • A token cannot be linked to an administrator .
  • A Moodle user create his own token on his profil page.
  • To create a token, a user need "moodle/webservice:createtoken" capability.
  • An administrator create token for other on a web service token page.
  • An administrator cannot see token that he didn't create.
  • A token is generated randomly and is long enough.
  • A token can be used by an external web application. (by browser)
  • A token can be used by a mobile application. (i.e. iphone native application)
  • A token is sent by the external application at the same time that the function parameters are sent.
  • When the user create a token, he must select a service (to link the token to a service). Only services that the user has capability on are displayed.
  • ip restriction and valid until fields are optional.
  • A Moodle user must select an aera on which the token can be valid. (course, ...) - TBD - unclear yet
  • A user can only see token string if https is enabled.
  • A token can be linked to only one service.


Administration: token management (valid until field is missing here):
Admin token.png


Administration: create token:
Create token.png

Application embedding

Similar to general WS, the token is linked to current session - when user logs out, the external token is immediately invalidated. The administration is much easier and the tokens are usually created automatically.

The external applications maybe be embedded into browser or they can be launched as separate programs (ex.: Java Web Start, Flex, external media players, etc.)

It is not necessary to use WS layers defined in /webservices/xxx/, the same infrastructure may be used in all ajax scripts that do not have access to browser cookies.

RSS, iCALs and other feeds

All cookie-less scripts such as RSS can use the external API and together with the permanent tokens. In fact ATOM is a nice example of RESTful protocol :-)

See also