Difference between revisions of "External services security"

Jump to: navigation, search
(New table external_sessions)
(Random all-time valid token)
 
(151 intermediate revisions by 5 users not shown)
Line 5: Line 5:
 
=Overview=
 
=Overview=
  
==Current solutions==
+
Current solutions:
 
* user keys for gradebook import and export - see require_user_key_login() and db table ''user_private_key''
 
* user keys for gradebook import and export - see require_user_key_login() and db table ''user_private_key''
 
* open RSS feeds - no security at all
 
* open RSS feeds - no security at all
 
* chat_sid tokens - generated separately for each user in each chat
 
* chat_sid tokens - generated separately for each user in each chat
* calendar hash from user name, password and salt
+
* calendar export - hash from user name, password and salt
 
* hacky cookie emulation in visual gradebook plugin
 
* hacky cookie emulation in visual gradebook plugin
  
==Types of token==
+
=Design=
We need several types of tokens
 
# token sharing/linked to active session, should time out or be destroyed at the same time as session (ex.: chat) - shared $SESSION and $USER
 
# permanent token, revokeable by user (ex.: RSS feeds, web services) - emulated $SESSION and $USER
 
  
In the second case we need to deal with performance problems if many repeated request expected. This can be dealt with later.
+
==Different uses==
 +
The external API may be used from different places:
 +
# directly from PHP - no authentication, current user session is used ($USER, $SESSION)
 +
# simple web service layer - easy to use and setup WS layer designed for interaction with enterprise systems (each having own single purpose user account), the protocol is stateless, username and password is sent with each request; security is enforced on multiple levels
 +
# full WS layer with token authentication - it is more flexible and enables normal users to use some restricted parts of the WS APIs, performance might be significantly improved by emulation of sessions
 +
# when embedding external applications - external application receives unique token which is used instead of normal browser session cookie, the session is linked to the current user session in browser, the token is automatically invalidated after logout
 +
# RSS feeds, iCals, etc. - token login, no permanent session
  
 
==API layers==
 
==API layers==
Line 25: Line 28:
 
# public PHP API - functions usable directly from PHP, list generated from inline PHP docs, need to verify '''all''' parameters and access control, may access $USER, should not manipulate $SESSION directly, must not read $_POST or $_GET
 
# public PHP API - functions usable directly from PHP, list generated from inline PHP docs, need to verify '''all''' parameters and access control, may access $USER, should not manipulate $SESSION directly, must not read $_POST or $_GET
 
# low level internal API - as fast as possible, basic param validation, no access control, must not touch $USER, $SESSION, $_GET or $_POST, must not use has_capability() or require_login()!
 
# low level internal API - as fast as possible, basic param validation, no access control, must not touch $USER, $SESSION, $_GET or $_POST, must not use has_capability() or require_login()!
 +
 +
==Context restrictions==
 +
Context restriction of token validity should be effective against security problems in external applications interacting with Moodle. Some external applications do not have any access to http cookies, solution is to create temporary tokens. Context restrictions would allow us to grant external access to individual activities, courses ,etc..
  
 
=Implementation=
 
=Implementation=
  
==New external_functions table==
+
==PHP API==
List of external functions. Created automatically by parsing of external files.
+
This simplest case of using the external API. The calling call has to make sure the current $USER and $SESSION is valid. Optionally the caller may set up the context restriction manually. This should be the easiest way to start with Moodle modifications and integration with other PHP software.
 +
 
 +
==Simplified web services==
 +
This type of web service is using simple username+password authentication (in future could be username+certificate). The communication integrity/privacy can be protected using https. Each external application is using own user account. These user accounts can not be used for normal login into moodle vie web interface.
 +
 
 +
This type of WS is intended primarily for integration with student information systems or other enterprise software.
 +
 
 +
===webservice auth plugin===
 +
It is recommended to create separate user for each external application. The benefit of this auth type is that it can not be used for normal log-in from the web interface.
 +
 
 +
Several people questioned the security of sending username+password with each request - it is not a problem because this password is used only for web services, it can not be used for anything else. This means that the password is effectively a standard authentication token ;-) We coudl replace this password by a public certificate stored in the user profile field.
 +
 
 +
===WS entry point===
 +
Separate php file /webservice/xxx/simpleserver.php
  
 +
===external_services_users table===
 +
Specifies which user may use each external service when restrictedusers flag set in service.
  
 
{| class="nicetable"
 
{| class="nicetable"
Line 43: Line 64:
 
|  
 
|  
 
|-
 
|-
| component
+
| '''externalserviceid'''
| varchar(100)
+
| int(10)
 
|  
 
|  
| Component where function defined.
+
| foreign key, reference external_services.id
 
|-
 
|-
| '''name'''  
+
| '''userid'''  
| varchar(150)
+
| int(10)
 
|  
 
|  
| Name of function
+
| foreign key, reference user.id
 
|-
 
|-
| enabled
+
| iprestriction
| int(4)
 
|
 
| function enabled, functiosn might be disabled for security reasons
 
|-
 
| phpfile
 
 
| varchar(255)
 
| varchar(255)
|  
+
| NULL
| location where function defined
+
| restrict access to some ips or subnets only
 
|-
 
|-
| functionparams?
+
| validuntil
| text
+
| int(10)
| null
+
| NULL
| ?some form of param description?
+
| invalidate after (once the date is reached the user cannot access the service)
 
|-
 
|-
| requireslogin
+
| timecreated
| int(1)
+
| int(10)
 
|
 
|
| extra information - some external functions may not require $USER
+
| time when this record added (date of the user/service relation creation)
 
|}
 
|}
  
==New external_services table==
+
===Steps needed to configure simple web service access===
''Service'' is defined as a group of functions.
+
 
 +
# enable web services
 +
# create new service in some local plug-ip (add code into /local/yourplugin/db/services.php) or manually add new service with some functions
 +
# add new user, set auth plugin to '''webservice''' (normal add user UI)
 +
# enable web service layer plugin (admin UI)
 +
# optionally add user into list of users that are allowed to use specific external services (admin UI)
 +
# set up user permissions needed in function implementations. All required capabilities should be listed in the external function description.
 +
# set up permission to use specific web services (optional, services do not need to specify required capability)
  
  
{| class="nicetable"
+
#Log as admin
! Field
+
#Add a new user
! Type
+
#Enable the Web Service Authentication plugin
! Default
+
#In the new user profil, change authentication method for web services
! Description
+
#In advance feature, enable web services
|-
+
#In Plugin/Web Services/Manage protocol, enable the protocol you need
| '''id'''
+
#In Plugin/Web Services/External Services, Add a new service, activate it and add some function to this service.
| int(10)
+
#If the selected service has restricted user option activated, assign the new user to this service
| auto-incrementing
+
#If the selected service has required capability, assign this capability to the new user
|
+
#Assign the capability to use the selected protocol - you'll probably want to create a Web Service role for that
|-
+
#the new user can now access web service from the enabled protocol with his username/password
| component
+
 
| varchar(100)
+
==General web services==
|
+
 
| Component where service defined.
+
More flexible than the WS with simplified authentication and setup. The performance may be significantly improved by emulation of sessions. The authentication is based on security tokens (user, context and purpose specific) that are generated in normal Moodle interface and then used in other external applications that are using our web services. This allows us to give only partial access to some parts of WS api, external applications and services do not need to be fully trusted.
|-
+
 
| '''name'''
+
The main drawback is that the complex configuration does not encourage admins to keep everything secured.
| varchar(150)
 
|
 
| Name of service (gradeexport/xml, mod/chat/export)
 
|-
 
| enabled
 
| int(4)
 
|
 
| service enabled, for security reasons some services may be disabled
 
|-
 
| version
 
| varchar(10)
 
|
 
| version string
 
|-
 
| requiredcapability
 
| varchar(150)
 
| null
 
| Required capability (tested in security context or system context if no security context specified)
 
|}
 
  
==New external_services_functions table==
+
===external_tokens===
Specifies functions used by services.
+
Stores permanent tokens for cookieless access, script runs without real session. (Later might have to add emulated session for performance reasons.)
  
 
{| class="nicetable"
 
{| class="nicetable"
Line 129: Line 133:
 
|  
 
|  
 
|-
 
|-
| '''externalserviceid'''  
+
| '''token'''  
| int(10)
+
| varchar(128)
 
|  
 
|  
| foreign key, reference external_services.id
+
| private access key value
|-
 
| '''externalfunctionid'''
 
| int(10)
 
|
 
| foreign key, reference external_functions.id
 
|}
 
 
 
==New external_tokens table==
 
Stores tokens for cookieless access, script runs without real session, $USER and $SESSION is emulated. Use is relatively expensive because each scripts has to initialize accessdata in acceslib.php again. Existing data from ''user_private_key'' table are migrated here.
 
 
 
{| class="nicetable"
 
! Field
 
! Type
 
! Default
 
! Description
 
 
|-
 
|-
| '''id'''
+
| tokentype
| int(10)
+
| int(3)
| auto-incrementing
 
 
|  
 
|  
 +
| type of token: 0=permanent, no session; 1=linked to current browser session via sid; 2=permanent, with emulated session
 
|-
 
|-
| '''userid'''
+
| userid
 
| int(10)
 
| int(10)
 
|  
 
|  
 
| foreign key, references user.id
 
| foreign key, references user.id
 
|-
 
|-
| '''token'''
+
| creatorid
| varchar(128)
 
|
 
| private access key value
 
|-
 
| '''contextid'''
 
 
| int(10)
 
| int(10)
|
+
| the token creator (an administrator)
| security restriction, key usable only in this context, references context.id
+
| foreign key, references user.id
 
|-
 
|-
| '''externalserviceid'''
+
| externalserviceid  
 
| int(10)
 
| int(10)
 
|  
 
|  
 
| foreign key, references external_services.id
 
| foreign key, references external_services.id
 
|-
 
|-
| '''itemid'''
+
| sid
 +
| varchar(128)
 +
| NULL
 +
| links to browser or emulated session
 +
|-
 +
| contextid
 
| int(10)
 
| int(10)
 
|
 
|
| Service specific item id
+
| security restriction, token usable only in this context, references context.id
 
|-
 
|-
 
| iprestriction
 
| iprestriction
 
| varchar(255)
 
| varchar(255)
| null
+
| NULL
 
| IP address restriction, list of allowed addresses
 
| IP address restriction, list of allowed addresses
 
|-
 
|-
 
| validuntil  
 
| validuntil  
 
| int(10)
 
| int(10)
| null
+
| NULL
 
| timestampt - valid until date
 
| timestampt - valid until date
 
|-
 
|-
Line 200: Line 189:
 
|}
 
|}
  
==New external_sessions table==
+
===Steps needed to configure general web service access===
Alternative session handling for scripts that are not allowed to access http cookies such as flex apps or external applications. The $SESSION is a true session which shares locking with the real http session. The token is automatically destroyed when http session ends (timeout, logout, login, etc.)
 
  
{| class="nicetable"
+
# enable general web services - global switch affecting all general web services (admin UI)
! Field
+
# enable web service layer plugins (admin UI)
! Type
+
# setup user permission which allows them to create tokens in some specific context for some specific purpose (roles UI)
! Default
+
# set up user permissions needed in function implementations
! Description
+
# set up permission to use specific web services (optional, services do not need to specify required capability)
|-
+
# each user must create permanent token in specific context for specific purpose (user UI)
| '''id'''
+
# user must copy/past this token to external application
| int(10)
+
 
| auto-incrementing
+
==== Random all-time valid token ====
|
+
A user can use the same token to call a web service. We call this token an all-time valid token.
|-
+
 
| '''userid'''  
+
'''Specificity'''
| int(10)
+
* A token can be created by a Moodle administrator (administration page) or a simple Moodle user (user profil).
|
+
* An administrator has the ability to create a token for another Moodle user but not for himself neither for another administrator (for security purpose).
| foreign key, references user.id
+
* A token is always linked to a Moodle user.
|-
+
* A token cannot be linked to an administrator .
| '''token'''
+
* A Moodle user create his own token on his profil page.
| varchar(128)
+
* To create a token, a user need "moodle/webservice:createtoken" capability.
|
+
* An administrator create token for other on a web service token page.
| private access key value
+
* An administrator cannot see token that he didn't create.
|-
+
* A token is generated randomly and is long enough.
| '''contextid'''  
+
* A token can be used by an external web application. (by browser)  
| int(10)
+
* A token can be used by a mobile application. (i.e. iphone native application)
|
+
* A token is sent by the external application at the same time that the function parameters are sent.
| security restriction, key usable only in this context, references context.id
+
* When the user create a token, he must select a service (to link the token to a service). Only services that the user has capability on are displayed.
|-
+
* ''ip restriction'' and ''valid until'' fields are optional.
| '''externalserviceid'''
+
* A Moodle user must select an aera on which the token can be valid. (course, ...) - TBD - unclear yet
| int(10)
+
* A user can only see token string if https is enabled.
|
+
* A token can be linked to only one service.
| foreign key, references external_services.id
+
 
|-
+
 
| '''itemid'''
+
 
| int(10)
+
Administration: token management (valid until field is missing here):<br/>
|
+
[[Image:Admin_token.png]]
| Service specific item id
+
 
|-
+
 
| '''sid'''
+
Administration: create token:<br/>
| varchar(128)
+
[[Image:Create_token.png]]
|
+
 
| PHP session id - links sessions table or files directly for legacy session types
+
==Application embedding==
|}
+
 
 +
Similar to general WS, the token is linked to current session - when user logs out, the external token is immediately invalidated. The administration is much easier and the tokens are usually created automatically.
  
==New capabilities==
+
The external applications maybe be embedded into browser or they can be launched as separate programs (ex.: Java Web Start, Flex, external media players, etc.)
New capabilities for external services above. Each function could define capabilities too.
 
  
==New auth plugins==
+
It is not necessary to use WS layers defined in /webservices/xxx/, the same infrastructure may be used in all ajax scripts that do not have access to browser cookies.
Some external applications may require creation of new user account types that are not able to login interactively but only using login/pass or certificate. New authentication plugin could solve this problem.
 
  
The current design problem that any external actions such as role assign are carried out as nobody or root, it is later not possible to find out who changed what. Another problem is restrictions of external services to some contexts only.
+
==RSS, iCALs and other feeds==
  
User preferences may be also used for storage of extra data such as public certificates.
+
All cookie-less scripts such as RSS can use the external API and together with the permanent tokens. In fact ATOM is a nice example of RESTful protocol :-)
  
 
=See also=
 
=See also=
 
* [[Web services]]
 
* [[Web services]]
 +
* [[External services description]]
  
 
[[Category:Web Services]]
 
[[Category:Web Services]]

Latest revision as of 02:59, 14 January 2010

Moodle 2.0


Descriptions of security framework for web services, also used for RSS feeds, embedded application and similar parts that can not use normal HTTP cookies.

Overview

Current solutions:

  • user keys for gradebook import and export - see require_user_key_login() and db table user_private_key
  • open RSS feeds - no security at all
  • chat_sid tokens - generated separately for each user in each chat
  • calendar export - hash from user name, password and salt
  • hacky cookie emulation in visual gradebook plugin

Design

Different uses

The external API may be used from different places:

  1. directly from PHP - no authentication, current user session is used ($USER, $SESSION)
  2. simple web service layer - easy to use and setup WS layer designed for interaction with enterprise systems (each having own single purpose user account), the protocol is stateless, username and password is sent with each request; security is enforced on multiple levels
  3. full WS layer with token authentication - it is more flexible and enables normal users to use some restricted parts of the WS APIs, performance might be significantly improved by emulation of sessions
  4. when embedding external applications - external application receives unique token which is used instead of normal browser session cookie, the session is linked to the current user session in browser, the token is automatically invalidated after logout
  5. RSS feeds, iCals, etc. - token login, no permanent session

API layers

Three layers:

  1. external server interface (SOAP, REST, RSS, etc.) - deals with tokens, emulates user session, parameter processing
  2. public PHP API - functions usable directly from PHP, list generated from inline PHP docs, need to verify all parameters and access control, may access $USER, should not manipulate $SESSION directly, must not read $_POST or $_GET
  3. low level internal API - as fast as possible, basic param validation, no access control, must not touch $USER, $SESSION, $_GET or $_POST, must not use has_capability() or require_login()!

Context restrictions

Context restriction of token validity should be effective against security problems in external applications interacting with Moodle. Some external applications do not have any access to http cookies, solution is to create temporary tokens. Context restrictions would allow us to grant external access to individual activities, courses ,etc..

Implementation

PHP API

This simplest case of using the external API. The calling call has to make sure the current $USER and $SESSION is valid. Optionally the caller may set up the context restriction manually. This should be the easiest way to start with Moodle modifications and integration with other PHP software.

Simplified web services

This type of web service is using simple username+password authentication (in future could be username+certificate). The communication integrity/privacy can be protected using https. Each external application is using own user account. These user accounts can not be used for normal login into moodle vie web interface.

This type of WS is intended primarily for integration with student information systems or other enterprise software.

webservice auth plugin

It is recommended to create separate user for each external application. The benefit of this auth type is that it can not be used for normal log-in from the web interface.

Several people questioned the security of sending username+password with each request - it is not a problem because this password is used only for web services, it can not be used for anything else. This means that the password is effectively a standard authentication token ;-) We coudl replace this password by a public certificate stored in the user profile field.

WS entry point

Separate php file /webservice/xxx/simpleserver.php

external_services_users table

Specifies which user may use each external service when restrictedusers flag set in service.

Field Type Default Description
id int(10) auto-incrementing
externalserviceid int(10) foreign key, reference external_services.id
userid int(10) foreign key, reference user.id
iprestriction varchar(255) NULL restrict access to some ips or subnets only
validuntil int(10) NULL invalidate after (once the date is reached the user cannot access the service)
timecreated int(10) time when this record added (date of the user/service relation creation)

Steps needed to configure simple web service access

  1. enable web services
  2. create new service in some local plug-ip (add code into /local/yourplugin/db/services.php) or manually add new service with some functions
  3. add new user, set auth plugin to webservice (normal add user UI)
  4. enable web service layer plugin (admin UI)
  5. optionally add user into list of users that are allowed to use specific external services (admin UI)
  6. set up user permissions needed in function implementations. All required capabilities should be listed in the external function description.
  7. set up permission to use specific web services (optional, services do not need to specify required capability)


  1. Log as admin
  2. Add a new user
  3. Enable the Web Service Authentication plugin
  4. In the new user profil, change authentication method for web services
  5. In advance feature, enable web services
  6. In Plugin/Web Services/Manage protocol, enable the protocol you need
  7. In Plugin/Web Services/External Services, Add a new service, activate it and add some function to this service.
  8. If the selected service has restricted user option activated, assign the new user to this service
  9. If the selected service has required capability, assign this capability to the new user
  10. Assign the capability to use the selected protocol - you'll probably want to create a Web Service role for that
  11. the new user can now access web service from the enabled protocol with his username/password

General web services

More flexible than the WS with simplified authentication and setup. The performance may be significantly improved by emulation of sessions. The authentication is based on security tokens (user, context and purpose specific) that are generated in normal Moodle interface and then used in other external applications that are using our web services. This allows us to give only partial access to some parts of WS api, external applications and services do not need to be fully trusted.

The main drawback is that the complex configuration does not encourage admins to keep everything secured.

external_tokens

Stores permanent tokens for cookieless access, script runs without real session. (Later might have to add emulated session for performance reasons.)

Field Type Default Description
id int(10) auto-incrementing
token varchar(128) private access key value
tokentype int(3) type of token: 0=permanent, no session; 1=linked to current browser session via sid; 2=permanent, with emulated session
userid int(10) foreign key, references user.id
creatorid int(10) the token creator (an administrator) foreign key, references user.id
externalserviceid int(10) foreign key, references external_services.id
sid varchar(128) NULL links to browser or emulated session
contextid int(10) security restriction, token usable only in this context, references context.id
iprestriction varchar(255) NULL IP address restriction, list of allowed addresses
validuntil int(10) NULL timestampt - valid until date
timecreated int(10) time when key created
lastaccess int(10) time when key last used for access

Steps needed to configure general web service access

  1. enable general web services - global switch affecting all general web services (admin UI)
  2. enable web service layer plugins (admin UI)
  3. setup user permission which allows them to create tokens in some specific context for some specific purpose (roles UI)
  4. set up user permissions needed in function implementations
  5. set up permission to use specific web services (optional, services do not need to specify required capability)
  6. each user must create permanent token in specific context for specific purpose (user UI)
  7. user must copy/past this token to external application

Random all-time valid token

A user can use the same token to call a web service. We call this token an all-time valid token.

Specificity

  • A token can be created by a Moodle administrator (administration page) or a simple Moodle user (user profil).
  • An administrator has the ability to create a token for another Moodle user but not for himself neither for another administrator (for security purpose).
  • A token is always linked to a Moodle user.
  • A token cannot be linked to an administrator .
  • A Moodle user create his own token on his profil page.
  • To create a token, a user need "moodle/webservice:createtoken" capability.
  • An administrator create token for other on a web service token page.
  • An administrator cannot see token that he didn't create.
  • A token is generated randomly and is long enough.
  • A token can be used by an external web application. (by browser)
  • A token can be used by a mobile application. (i.e. iphone native application)
  • A token is sent by the external application at the same time that the function parameters are sent.
  • When the user create a token, he must select a service (to link the token to a service). Only services that the user has capability on are displayed.
  • ip restriction and valid until fields are optional.
  • A Moodle user must select an aera on which the token can be valid. (course, ...) - TBD - unclear yet
  • A user can only see token string if https is enabled.
  • A token can be linked to only one service.


Administration: token management (valid until field is missing here):
Admin token.png


Administration: create token:
Create token.png

Application embedding

Similar to general WS, the token is linked to current session - when user logs out, the external token is immediately invalidated. The administration is much easier and the tokens are usually created automatically.

The external applications maybe be embedded into browser or they can be launched as separate programs (ex.: Java Web Start, Flex, external media players, etc.)

It is not necessary to use WS layers defined in /webservices/xxx/, the same infrastructure may be used in all ajax scripts that do not have access to browser cookies.

RSS, iCALs and other feeds

All cookie-less scripts such as RSS can use the external API and together with the permanent tokens. In fact ATOM is a nice example of RESTful protocol :-)

See also