2/2008

Morphic 3.0: The Future of the Graphic User Interface

1. Introduction

Application developers constantly face the need to develop user interfaces for their applications. Many of the problems that need to be solved appear again and again. Therefore, libraries and frameworks were developed to help on these problems, usually as part of programming tools. Most of them are tied to specific operating systems, and almost all of them consist of a pre made set of widgets.

Morphic is the GUI framework in Squeak Smalltalk. It was originally developed as part of the Self project and later ported to Squeak. Designed with the ideas of simplicity and uniformity that are central to Smalltalk, it is both simpler, more powerful and more general than regular window managers and widget toolkits.

In a unique manner, Morphic puts widgets and graphics on equal footing. This gives unparalleled flexibility for widgets, and the possibility of building high level, powerful abstractions for graphic objects, all in a simple, integrated framework. The Morphic framework in Squeak is the only GUI and graphics toolkit that allows building complex applications, including both conventional looking widgets and distinct, unique graphic objects, allowing the programmer to fully control the behavior and the look up to the pixel.

The main drawback of Morphic is that morphs must specify all the drawing operations in pixels, in the Display coordinate system. This ties the design of any GUI to the Display pixel resolution the developer used when designing it. However, most applications have specific units for measures and they map the available screen space to the ranges of values that make sense to them. Therefore, they must include the code to do this.

Other items that need redesign are the dependencies in the framework itself upon the applications built with it. They were the result of the lack of good quality control for the code included in Squeak. 

This document presents the Morphic 3.0 project. Morphic 3.0 is built as an evolution of Morphic in Squeak (which I call Morphic 2.0), but it is also a deep redesign, as many of the design decisions are rethought, leading to a framework that is simpler but more advanced and powerful.

1.1 Coordinate Systems

Any Gui that goes beyond widgets on a form needs to handle its own specific coordinate systems to convert from domain units to display pixels. Some examples are:

• Linear coordinate systems on graphics and photo editors, audio editors, music sequencers, video editors, page editors, etc.

• Logarithmic scales on audio editors, technical graphs, etc.

• Special scales like the staff used in musical notation

• Cartographic projections in maps

• Geographic coordinate systems as in GIS. 

However, most of the available tools only offer Display coordinates. A few (i.e. Cairo / Rome) offer linear coordinate systems and affine transformations. None can handle arbitrary, non-linear coordinate systems. This means that it is the application programmer who must translate the points from the coordinate system of his domain to the Display. This leads to lots of duplicated effort, and to many bugs and inconsistencies between applications. 

Even inside applications the behavior is not consistent. For example MsWord does a great job at rendering the document properly on any display size. But you can't adjust the look of tool bars, icons, menus and such to look good on a display with very high resolution and small size. 

Handling the coordinate systems applications might need in the framework itself will reduce and simplify their code, and make them behave consistently.

1.2 Rendering

I want high quality rendering on any Display, regardless of size or pixel resolution. Therefore, I need complete independence of those Display properties. The programmer must never deal with the concept of pixel. The GUI is thought at a higher level. All the GUI is independent of pixel resolution. All the rendering is anti aliased. But in order to be able to render equally well on very high resolution as on medium resolution devices, the objects to be rendered (i.e. morphs) must be specified in a way that doesn't depend on the resolution of the target device at all. The ultimate way to do this is by thinking of them as continuous functions. This applies to geometric shapes, but also to digital images (photos) and textures.

For all this, we could use the Balloon engine in Squeak, or Rome, the Cairo interface in the Sophie project, right? Unfortunately not. They can only handle Cartesian coordinate systems and linear transformations. Therefore they can not be used if we handle general coordinate systems as I said before. Besides, with them, as with Flash and any vector graphics format, it is easy to build cartoon-like scenes, but they don't help much in producing rich textures and photorrealistic images.

We need a new rendering engine. What features / services should it provide?

• Draw simple shapes, like polygons, ellipses, straight lines, splines, Bezier and other curves. The rendering should be anti aliased. The shapes should be appropriately deformated to honor the coordinate system conversions. The width of the nib of the pen should also adapt to coordinate system conversions. It must not be necessary to implement different algorithms for drawing the same basic shape just for having different coordinate systems. Shapes and coordinate systems are different objects, and they are independent of each other.

• Paint objects with natural looking textures. Textures should also be independent of pixel resolution, and should adapt to coordinate system changes and the corresponding geometric deformations.

• Draw photos and photorrealistic art. They should adapt to the target pixel resolution and they should honor the deformations due to the coordinate systems used.

1.3 Direct Manipulation

The kind of direct manipulation allowed in Morphic 2 is good. The only part that will change is resizing. I believe that it will be more intuitive for end users if objects on the screen behaved more like in the real world. Real objects usually have some fixed size, and we can't easily change it. What we can do is to take a closer look at some object, to see it better and interact more comfortably with it. The resize operation is replaced by a zoom operation that doesn't change the appearance of the morph or its size, but makes it look bigger or smaller. These ideas are related to Zoomable User Interfaces, but in my opinion, they can only be properly implemented with pixel independence. Besides they only realize their full power when they are the basis of the GUI framework, and not a particular kind of image browser.

1.4 Framework Structure

*       The Morph hierarchy should not be a hierarchy of shapes. The shape of an object doesn’t need to be constant, and shapes are not that important. como para estructurar la jerarquia de herencia a partir de ellas.
Elaborar.
 *       Class Morph should not include general concepts of color, border, etc. Each morph class decides what are the relevant properties for its instances. The same goes for submorph aligning.
·    A Morph is placed somewhere in an owner. Class Morph has an instance variable ‘location’ to hold an instance of MorphicLocation. A MorphicLocation specifies a position, extent, and rotation angle expressed in the owner’s coordinate system. Separating them eases the moving, zooming and rotation of morphs. The user can easily move, zoom and rotate morphs. Please note that we are talking about zooming and not resizing.
·    The Morph hierarchy is not a shape hierarchy. Morphs don't have a concept of a border or color. There is no general concept of submorph aligning. A particular morph may implement these features in any way that makes sense for itself.

(ELABORATE)

1.5 Framework Use: GUI building

write it!