Note:

If you want to create a new page for developers, you should create it on the Moodle Developer Resource site.

Paint tool

From MoodleDocs
Revision as of 19:43, 24 April 2009 by Mihai Sucan (talk | contribs) (correction)

Proposal for Google Summer of Code 2009

Mihai Şucan, Aurel Vlaicu University, Romania.

Executive summary: The goal is to develop a new online painting tool inside Moodle, one which allows users to edit images in the HTML editor, in the file manager and in other places.

The paint tool will be tightly integrated into TinyMCE and into Moodle, providing an API for embedding in several ways described below. Additionally, the tool will be developed such that it allows other non Moodle developers to use the Web application.

For the implementation of the painting Web application I propose we use the HTML 5 Canvas API.

I have already developed a proof-of-concept / demo Web application, named PaintWeb.

Note: The current PaintWeb demo is underdeveloped, it's simply only a proof of concept. It lacks any browser-specific work arounds for limitations and bugs. For example, the Text tool only works in Safari 4, Firefox 3.5 and similarly new browsers.

Motivation and use-cases

Children like to draw very much. They like to express themselves using colors, and drawing anything they imagine. Currently Moodle does not allow children to draw. Teachers cannot create assignments which require users to draw something. Any user who wants to insert a drawing has to go through great lengths to draw the image using some offline drawing tool, then upload and insert it into Moodle.

During online chats many courses would benefit having a paint tool integrated. For example, in chats about geometry students and teachers need to draw the shapes they are referring to, important lines like perpendiculars/bisectors/etc. This is not yet possible in Moodle. Having a paint tool would allow the development of a "live sketchboard" during chat sessions.

The same way of working and thinking about the educational act itself would enhance courses from wide ranges, such as, but not limited to chemestry, mathematics, geometry and painting.

Why Canvas

Canvas is a new element in the HTML 5 specification, which provides a 2D context API.

  • Drawing is bitmapped, therefore no additional DOM elements needed, Image data management/manipulation becomes simple and straight-forward.
  • The entire API is native to the browser. No extensions needed, no plugins.
  • "Communication" between the Web page and the Canvas element is trivial, not even worth mentioning. Fact is, Canvas lives in the native space of the browser. You do not have any sandboxing, like in plugins (Flash/Java Applets). This also contributes to improved performance.
  • Loading and saving images is trivial. The Canvas 2D context API provides simple methods for such purposes.
  • Communication between JS and the server can be done via AJAX. It's much easier than doing it from Flash/Java Applets.
  • Drawing is faster than using HTML, Flash, and Java Applets. Not-so-distant future plans for major Web browsers include OpenGL/DirectX hardware acceleration for most operations which means chosing Canvas is a very good decision in the longer term.
  • Even if drawing is done on a bitmapped 2D canvas, the specification allows future browsers to implement saving the image as SVG, not just in a bitmapped format. The availability of SVG export can come with no cost for the Moodle developers, given a browser implementation which retains sufficient information to allow exporting the image to SVG.
  • Better keyboard accessibility. Unlike Flash, Java Applets and other plugins, Canvas does not steal away the keyboard focus (one of the most annoying problems with plugins). Given the OLPC XO laptop and children users, this is very important for them.
  • Zero performance impact for loading the Canvas-based Web application. Java Applets load very slow, freezing the browser for a few seconds (on AMD Athlon XP 1800+), and they require a lot more memory, because you have to run the entire JVM inside the browser. Flash also requires more memory to run. Again, users of OLPC XO laptop will most likely disable Flash/Gnash entirely, because it crawls down the entire browser when many Flash banners are shown on a single page.
  • Flash is known, even by Google, to cause numerous crashes, and I quote: "Plugins are a major source of browser crashes. (Flash is in the callstack of a nontrivial percentage of Firefox crashes.)" [5] It is, therefore, to be desired to avoid any kind of plugins, for stability reasons, not just performance.
  • Canvas is a major browser feature being improved constantly. Each new version of each major Web browser improved the performance of the API.
  • Almost no learning curve associated with developing a Canvas-based Web application. Existing JavaScript developers can further improve and fix the project at any time. They do not need Flash Professional (which is not free), they do not need to learn ActionScript, they do not need to learn Java. In general, it's better to keep the codebase compact, instead of each project using its own language. It's like using Python instead of PHP for some server-side parts.
  • Cross-browser and cross-platform compatibility. Users do not need to install a different browser (it works in almost all major browsers, including, but not limited to Safari, Chrome, Opera, Firefox, Konqueror, Epiphany), and they do not need additional plugins, nor additional settings rather than the defaults. No specific operating system requirements are made either. Developers who want to work on the paint tool do not need to switch to Windows in order to edit some Flash project, and they do not need a completely different development stack for editing some Java project.
  • Given the fact that all major browsers implement the Canvas 2D context API, except Microsoft Internet Explorer, makes it reasonable to expect that an upcoming release of MSIE will indeed support Canvas. Major companies already use Canvas in their projects (see Yahoo Pipes). Ultimately, it is in Microsoft's best interest to support a wider range of Web applications.
  • The final version of Internet Explorer 8 was released in March. While it does not yet support the new Canvas element, this release has proven the renewed interest of Microsoft to implement new Web technologies, in an interoperable way. They have many standards-related improvements.
  • Internet Explorer compatibility can be tackled in multiple ways. For example, very recently a new release of the ExplorerCanvas project from Google has been made available. Such layers of compatibility are simple to use and do not generally require any additional changes to the drawing tool itself. Once IE will natively implement the Canvas support, we can simply remove the compatibility layer for the new version.

I should also note that performance is critical not only for the OLPC XO. Many mobile devices like netbooks, notebooks and cell phones are getting connected to the Internet. All these devices have limited resources.

It's not very long time since Adobe released an up-to-date Flash player for Linux. Until then we were stuck with an outdated Flash player. However, now Flash player is much slower on Linux than the Flash player for Windows. We shouldn't be quick to forget the harm of closed source technologies.

Regarding the use of more JavaScript libraries: the Moodle community and developers have made their choice. They use the Yahoo UI library. I do not want to introduce yet another library which only make things worse and slower. Performance is important and we also don't want bugs between conflicting JS libraries. I will use the YUI library when I consider it's needed inside PaintWeb.

Implementation details

The paint tool

We will reuse the open-source code from PaintWeb. I will completely reorganize the code, make a new GUI, and implement new functionality as desired by the community and the mentor.

I plan to have a separate JS holding the implementation of the painting tools themselves. A separate JSON file for each language. We want PaintWeb to be internationalized. A separate file holding the implementation of the color editor, which will be a reusable component (maybe we can replace the color chooser from TinyMCE).

Suggested file structure:

  • lib/paintweb/paintweb.js - the main codebase.
  • lib/paintweb/paintweb.min.js - the minified/compressed source code which includes the tools and the color editor packed together, for performance reasons. Optionally, this file can also include the language file, the config and the interface.
  • lib/paintweb/paintweb.php - which provides API for easily including paintweb in any place across the entire Moodle application.
  • lib/paintweb/lang/en.json
  • lib/paintweb/lang/ro.json ... etc
  • lib/paintweb/coloreditor.js
  • lib/paintweb/interface.xhtml - the interface markup will not be included in the js, it's separate, for easy GUI tweaks.
  • lib/paintweb/config.json - keyboard shortcuts and more settings.
  • lib/paintweb/interface.css
  • lib/paintweb/tools.js
  • lib/paintweb/images/ - any user interface images/icons needed.
  • lang/*/paintweb.php
  • lang/*/help/paintweb/ - folder holding user help pages for PaintWeb.

All code will use jsdoc-toolkit comments, and an API reference documentation will be generated, to make it easy for new developers to understand the structure.

For the language files, we can use PHP to properly connect to the global language system from Moodle (lang/*/paintweb.php).

For the OLPC XO we might go for "hard-core" performance optimisations. Ideas:

  • Use data URIs for all images inside the CSS - less network requests, known to improve performance.
  • Compact more files into a single one. We might need paintweb.olpc.js which includes as much as code as possible inside a single file. To do this, we will need an automatic packaging php script which takes all the separate files to build a single one.
  • We might propose a different user interface for the OLPC, which is more minimalistic, and less resource-intensive.

TinyMCE integration

TinyMCE 3 in Moodle 2 showing an image in the article being edited.
The "Insert/edit image" dialog showing the new "Edit" button which allows invoking the paint tool
PaintWeb inside Moodle 2

Looking through the source code of TinyMCE and on their official site: TinyMCE provides an API for extending the editor. Therefore, I propose we implement a new button which when clicked allows the user to edit the image he/she has selected in the HTML document. Once clicked, the PaintWeb tool loads, and changes all the toolbars, switching to a so-called "image editing" mode. Once done, the user can save the image (or else Cancel for no saving). We will need a sever-side script to handle the image save operation, some lib/paintweb/editor.php which takes the HTTP POST request, saves the file on the server in a folder we need to decide (mentor decision needed). Once the image is saved using AJAX, the user is returned back to the TinyMCE editor, exactly where he was. He will see the updated image.

In order to implement this tight integration we need to add a new plugin in the lib/editor/tinymce/jscripts/tiny_mce/plugins folder. We will create the paintweb folder which will contain all the required JS to properly bind PaintWeb into TinyMCE.

In a similar way, one can integrate PaintWeb into HTMLArea. Myself, I offer to integrate PaintWeb into TinyMCE, due to time restrains, and due to the fact HTMLArea is rather outdated.

Mockup screenshots

The current TinyMCE 3 editor has an "Insert/edit image" button on the toolbar. I suggest we add a new "Edit" button in the "Insert/edit image" dialog. This new button would allow the user to invoke PaintWeb to edit/re-edit any image he has selected inside the HTML editor. The image transfer will be automatic to the canvas element (canvas_element.getContext('2d').drawImage(selected_image_element)).

Additionally, we can also insert a new button into the toolbar of TinyMCE 3 to directly invoke PaintWeb.

The integration of the paint tool will be seamless to the user. He will be able to edit any image, at any moment. Once the user completes editing, he can pick the Save option to save the modified image, and then return to TinyMCE 3 by clicking "Finish". All of this will use AJAX, and will not require entire page reloads.

PaintWeb will also work inside the fullscreen mode of TinyMCE 3.

Note: These are not screenshots from an actual running PaintWeb. This is a "photoshopped" rendition of how I would like to integrate PaintWeb into Moodle 2 and TinyMCE 3. For the actual implementation I will do further adjustments and improvements.

Moodle integration

PaintWeb.php will provide an API which will allow the embedding of PaintWeb in a standalone version, all by itself, on any page where a Moodle developer wants. It will provide an API for easy loading and saving of images.

Additionally, an API for dynamic inclusion of PaintWeb will be provided, via AJAX. Such that, from any place, PaintWeb can be loaded dynamically.

I would like us to integrate PaintWeb inside the file manager, to allow editing of images there.

Goals

We need to have a clear focus, and a proper set of goals for the end of the summer. As such, my goal is to have the paint tool usable from TinyMCE, and optionally from the file manager.

I can offer to further help the Moodle community after the summer with any new code I will work on from there on for PaintWeb. I will try my best to fix any of the bugs reported in the component, and to upgrade it whenever I have a new release.

Timeline

Once I am accepted for GSoC, I want to get started working on the project. This means I will start working from April 21 (I will not wait for May 23).

Based on experience I estimate the following preliminary implementation timeline:

  • April 21 - May 4. Two weeks for reorganizing, extending, improving the entire current demo / proof of concept code from PaintWeb. This means I will do code separation into multiple files (like described above) and add jsdoc comments. While doing code separation, I will also make sure it's easier to extend/add new tools (making things more modular). I will make sure the paint tool can be easily configured to use a different GUI and different keyboard shortcuts.
  • May 5 - May 12. One week for testing and reviewing. Report any bugs found and code review feedback. Testing can continue afterwards in parallel, while I'm doing other work.
  • May 13 - May 20. One week for making the code cross-browser compatible, on those browsers with native Canvas API support. Basically, this means working around the lack of the native Text API in Opera and Firefox 3.0.
  • May 21 - May 28. One-two weeks for implementing the new proposed GUI, with further improvements. This requires working on the HTML markup, CSS code and any JS associated with user interaction.
  • May 29 - June 5. One week for testing and reviewing of the GUI. Again, once the week is over any testing can continue in parallel.
  • June 6 - June 20. Two weeks for integrating PaintWeb into TinyMCE and Moodle, both client-side and server-side work. This means I will add the new paint tool button into TinyMCE, and I will make sure editing and saving images within the editor works. Obviously, this is where the most help / consulting will be needed from my mentor (Martin).
  • June 21 - June 28. One week for testing and reviewing the PaintWeb integration. This will also continue in parallel after the week is over.
  • (Optional) Around August. One-two weeks for making the paint tool work in MSIE. During this stage I will test ExplorerCanvas and other projects which implement Canvas support in IE.
  • (Optional) Around September. One week or more for implementing new features: bucket fill tool, rectangles with rounded corners, spray tool, grid lines, image/selection rotate, basic image filters (like invert, grayscale, etc), and maybe layers as well.

The timeline depends a lot on community help and any additional requirements made while working on the project.

Additionally, university exams are held some time during June here. I do not know precisely when, it depends on each professor to pick the date. I'll have 5 exams. This will stretch the above timeline by about three weeks. I did not include these delays in the timeline above because I do not know where to insert them. Thus, the above dates are more of 'ideal' dates.

In any case, by mid-term evaluation (July 6th - July 13th) I want to have the first stages completed (code reorganization + new GUI). By 'pencils down' date (August 10 - August 17) I want to have the PaintWeb integration done and tested.

I will announce on the dev forums when each stage is complete.

Once GSoC 2009 has ended I would like to continue working, when I have time.

Why me

I have a keen interest into Web standards and educational software. I am currently pursuing the Bachelor of Science degree at UAV.

My expertise into Web standards and technologies has been gradually building for the last 10 years. I have come to code my own content management system, and my own WYSIWYG HTML editor. I am also coding my own JavaScript library.

Additionally, I am a constant user of educational software since I started school. I continue to use such software even now during University studies. I have experience with Keypress Geometry Sketchpad, and with other mathematical editor tools (such as Mathematica, LyX and more). I have come to know what children and students need improved in the educational software available today. Last year I had a presentation at the Vasile Goldiş University where I have presented several of my ideas for improving current educational software. One of the applications tested was Moodle itself.

When time allowed, I have made contributions to several Web specifications (among them: HTML 5 and XBL 2). My contributions/reviews prompted the editor of the specifications in question to make changes as suggested.

Opera Software published several tutorials I wrote, which detail technical aspects of some new Web technologies, like VoiceXML and Canvas.

In previous national contests of Web applications I have ranked among the first, if not the first.

I closely watch the entire Web browser market and technologies, keeping myself up to date. I am aware of many emerging Web technologies. Thus, I can base my decisions better on which technologies to use.

Last, but not least, I have a fairly good deal of head-start when it comes to developing a paint tool for Moodle. My PaintWeb demo/proof of concept is fairly usable and I can get to work from day one, no need to do any catchup with any APIs, no need to learn anything.

In the future

Ultimately, I would like to see Moodle have a paint tool which allows editing of multiple file formats: bitmapped images and SVGs (vector drawing as well), If we do the project described here, by the end of the summer we will be half way through the ultimate goal. By the end of the year, with sufficient dedication and help from other developers, SVG editing should be doable.

In the future, I would also like to help improve the HTML markup and the entire JavaScript codebase used by Moodle.

For PaintWeb I plan to make its codebase versatile. Meaning, I want it to be able to use multiple different GUIs and keyboard shortcuts just by simply changing a single config string. This would allow PaintWeb to be used on systems with fewer resources with a simpler UI, and in different configurations (say in a realtime chat room). I also want it to be able to use different backends for storage (e.g. Google Gears, Adobe AIR, etc), and different plugins (e.g. additional plugins for a realtime chat room would be needed).

Conclusion

I am looking forward to a fruitful collaboration with the Moodle community of users and developers for the common goal of improving educational software for everyone.

I should also note that I am committing full-time to this project over the summer. It is my desire to have a a well done project within a friendly community as this one. I will be online daily available on Jabber chat, IRC chat and email for any communication.

Any questions and suggestions are welcome.

Retrieved from "https://docs.moodle.org/dev/index.php?title=Paint_tool&oldid=13063"
Powered by MediaWiki