Note: You are currently viewing documentation for Moodle 1.9. Up-to-date documentation for the latest stable version is available here: admin/environment/php extension/tokenizer.
Beginning with PHP 4.3.0 these functions are enabled by default. For older versions you have to configure and compile PHP with --enable-tokenizer. You can disable tokenizer support with --disable-tokenizer.
The windows version of PHP has built in support for this extension. You do not need to load any additional extension in order to use these functions.
Note: Builtin support for tokenizer is available with PHP 4.3.0.