Sergey Andreyev

New design paradigm - user-driven applications

Abstract. Programs for engineering and scientific tasks always have to deal with numerous graphical results. The problem of limited screen space and often opposite requirements from different users are the cause of the infinite discussions between designers and users about the best results’ visualization, but the source of this ongoing conflict is not in the level of interface design, but in the basic principle of current graphical output: it is always decided and fixed by the designer and users may only change some views and details. I am writing about the algorithm that will eliminate this problem thus allowing to shift from designer-driven applications to user-driven. It’s not simply another not bad result in interface design. New type of applications in which user is deciding what, when and how to show on the screen – this is the dream of scientists and engineers working on the most complicated tasks. The new paradigm is based on movable and resizable graphics, and such type of graphics can be widely used not only for scientific and engineering applications, but in other absolutely different areas.

Introduction

For the last 20 years users are communicating with PC on two levels. On the upper level the screen space is shared by all the applications; any application is showing information inside the rectangular window. Each window is movable and resizable; at any time windows can be moved around the screen, made bigger or smaller, put side by side or overlapped. This is axiom 1 in modern day programming design: on the upper level all objects are movable and resizable. To make these features obvious the upper level windows have title bars (to grab and move the window) and borders (to resize the window). Movable and resizable are standard features of each window, and only for special purposes they can be eliminated.

By starting any application we enter inner level with slightly different, but also strict rules: here are two different types of objects - controls, inheriting everything from windows, and graphical objects that have no inheritance from them and are absolutely different. The inheritance of controls from windows is not obvious, as controls often do not look like windows. Controls have no title bars, so there is no indication, that they can be moved; usually there are no such borders that inform about the possibility of resizing. But these inherited features of all controls (movable and resizable) can be easily used by the programmers, for example, in the form of anchoring and docking. Using these features or not is defined by the designer of the program, and if it is allowed, then it is often done indirectly. It is organized through one or another form of interface design, every two or three years the most popular word for such form of design will change; now it is adaptable interface [1]. I mentioned only one paper, but I could include many others. The reason that I am not doing it that though it is a good and very popular at the moment way of improving interface it has nothing to do with the main idea of this article. The most important thing is not how controls can be changed, but that for them moving and resizing can be done without problems.

Graphical objects are of absolutely different origin than controls and by default they are neither movable, nor resizable. There are always some tricks to make things look not what they are in reality (the programmers are even paid for some knowledge of such tricks), and one of the often used tricks is painting on top of the control: any panel is a control, so it is resizable, and with the help of anchoring / docking features it is fairly easy to make an impression as if you have a resizable graphics. By default panels have no border, and if the back color of the panel is the same as of the parent window, then there is no way to distinguish between painting in the window or on the panel which resides on it. Certainly, such “resizable” graphics is only an imitation, but in some cases it is just enough; all depends on the purpose of application. Another solution for resizing of rectangular graphical objects is the use of bitmap operations, but in most cases it can’t be used because of quality problems, especially, while enlarging the image. Both of these tricky solutions (painting on panel or using bitmap operations) have one common defect – they can be used only with the rectangular objects.

If any limited area is populated with two different types of tenants that prefer to live under different rules (in our case - controls and graphical objects), then the only way to organize their peaceful residence and avoid any mess is to force them to live under ONE law. Because the currently used graphics is neither movable, nor resizable, then the easiest solution is to strip all controls of those features. That is also the reason, why the percent of applications that are allowing users to move around any inner parts is so small.

Thus we have axiom 2: on the inner level the objects are usually neither movable, nor resizable.

What is interesting, that these two axioms bring us to the absolutely paradoxical situation:

· on the upper level, which is not so important for real work, any user has an absolute control of all the components, and any changes are done easily;

· on the inner level, which is much more important for any user, because the real tasks are solved here, user has nearly no control at all, and if he has, then it is very limited and is always organized indirectly through some additional windows.

What is amazing, that this paradoxical situation is really looked at as an axiom, so I can’t remember (and can’t find) even a single paper that will doubt the fairness of this situation and will show an attempt to change axiom 2 to something better, but there are tons of papers and numerous discussions about the better way of organizing new interface for dealing with the parameters of graphical objects. I have no doubt in the importance of such discussions, as I am working on interface design for many years, but for many applications the question of finding an easy way of moving and resizing of graphical objects is much more important than anything else. If such algorithm would be found, then it will have very interesting and far going consequences; the whole programming of such applications can be changed to the huge users’ benefit.

Axioms I mentioned were never declared as axioms in strict mathematical way; at the same time I never saw, read or heard even about a single attempt to look at this not as an axiom and to design any kind of application on different foundation. Programmers got these undeclared axioms from the main software tools developer (Microsoft) and are working under these rules for years without questioning them.

Certainly, anyone can easily remember one or another example of resizing a graphical object, for example, in Paint the dotted line is moving with your mouse. I was doing such things in the programs for years; it is fairly easy trick that has nothing to do with the real moving or resizing of the objects.

“In science, finding the right formulation of a problem is often the key to solving it…” [2]. When I started to work on the problem of transferring unmovable graphics into movable and resizable, I began with the analysis of their principal differences. The goal was not to make some kind of graphical objects movable; for individual object of particular type anything can be designed. Though initially I was looking for solution for scientific and engineering plotting, which has always a rectangular shape, that was only some kind of experimental model. But I was looking for the general solution and I found it. Before describing the algorithm and, what is from my point of view is much more important - the consequences of using such algorithm - I want to emphasize that:

1. Though the main goal was to find the solution for ordinary plotting, the algorithm can be used in different areas and arbitrary forms.

2. The algorithm must be (and can be) looked on separately from the consequences of using it. Classes and algorithm may be different, but if there is any form of movable and resizable graphics, then it is the base for absolutely new paradigm – user-driven applications.

3. It is not simply an idea of “would be nice to have” type. The classes and algorithm of new plotting were designed and are working now; at the end of this article there are several links to applications and documents:

· a program, demonstrating the use of this algorithm in absolutely different areas;

· DLL with the description of classes;

· the whole project with all the codes and step by step description of how to design the new class of graphical objects with such features.

User-driven applications

Let’s assume that there is such graphics that allows to move and resize any plot while the application is running. How this can affect the design process and the user’s work – the main purpose of any engineering (scientific) application? To analyze the situation let’s look at the well known case of the visualization of mathematical functions.

There are several well known programs which allow viewing of variety of mathematical functions or show the plot(s) based on previously calculated data. Each program sets some limit on the number of shown plots (one or several), on their placement and size (in the same single area; side by side; in several rows and columns). It doesn’t matter how many plots can be shown simultaneously and how many choices the user has to place the plots – each case is predefined by the developer of the program and for each case you can say without running the application where and how each plot would be shown. This is an example of designer - driven application, and all applications for engineering and scientific tasks are developed in this and only this way; users are always working inside the scenario written by the designer.

For user-driven applications it would be absolutely different. On figure 1 is shown the view of TuneableGraphics application, when it is started. It looks like several partly overlapping plots, only user can grab, move and resize each of them at any moment; the new plots can be added and the existing can be deleted. The group of controls can be also moved around the screen. The special lines and areas (connections and nodes – I’ll explain them later) which are sensitive and allow moving and reconfiguring can be switched ON / OFF by the small button in the corner, but while the mouse is moved around the screen the changing of the mouse cursor will signal the possible moving / reconfiguring of the plot even without visualizing those special areas.

What makes this application absolutely different from standard designer-driven applications that it is impossible to predict the view of the application beforehand. User has an absolute control of what, when and how must be shown at any moment; this is the main idea of user-driven applications. It also means that inside this type of applications user gets the same degree of flexibility as now he has only on the upper level. This is absolutely new in design of applications.

Here the developer is not designing the output view, but has to give to the user an instrument to put on the screen whatever he –USER - wants. This is the dream of users of all engineering or scientific program: to have at the screen only what they really needs and in the way they prefer. So what was changed in the main design and what makes it possible to move from designer-driven applications to user-driven?

Such transformation is possible if and only if we have movable and resizable graphics.

Movable and resizable graphics

My idea of making graphical objects movable and resizable is based on contour presentation. The standard words from the graph theory often conflict with the meaning of the same words used by those who are programming under Windows for many years, so I’ll try to use different words with obvious meanings. Any graphical object that needs to be resizable and / or movable will have a contour consisting of nodes and connections. Nodes are used as sensitive areas which can be moved separately; by moving any of them the sizes and shape of the whole object can be changed. Connections are used for grabbing and moving the whole object. Sizes and forms of the sensitive nodes can vary and the same with the sizes of sensitive areas around connections.

Contours do not necessarily copy the shape of the object. Contours and nodes have to organize an easy way of resizing and moving the object and for this purpose they are taking under their control some part of the screen; at the same time they must not interfere with all other mouse clicks aimed at changing other parameters of the graphical objects. For example, in all standard XY plots (figure 1) double click on the main area of any of them will open special window for setting the area’s parameters, and because of this the contour is moved slightly out of the main plotting area.

When the designer decides that moving / resizing would be useful parameters of some type of graphical objects, then he organizes the contour taking into consideration the shape of the object, possible transformations and the set of already used mouse clicks associated with such objects. Two other main functions for moving the whole object and reconfiguring it depend on how this moving / resizing is planned to be used.

If the graphical object has to be unmovable but resizable, then there will be no connections and only nodes. If on the contrary moving is allowed but not resizing, then the contour will have a set of connections between empty nodes. Nodes can be absolutely different in sizes of their sensitive areas and possible movements, thus allowing different types of reconfiguring. This very flexible combination of nodes and connections covers all the possible scenarios, but in each particular case the contour is based on the type and form of the graphical object and how you need to work with it. For some objects the shape of the contour can be the same as an outer frame of an object; in general contour can be of absolutely different shape and looks more like a skeleton. I especially included into TuneableGraphics application different types of contours to show the wide variety of objects and different types of applications where this algorithm can be used.

Identical rules are applied not only to graphical objects, but to controls or groups of controls as well. This is one of the main ideas: similarity in behavior of all the applications’ parts without any division on graphical objects and controls makes it easier for any user to deal with the application. There can be some limitations set by the developer, but these limitations would be based only on the ideas of best design: I don’t think that users would be happy to move around the screen each control separately, especially if there are a lot of them, but if there is an easy way to move the whole group of controls and organize the view in the best way – this would be an extremely useful feature of any application. The best candidates for such movements will be panels with everything residing in them or group boxes. In the application shown on figure 1, there are no problems at all to make movable separately the list with the information and two buttons at the side of it, but because these buttons are used for changing the order of items in the list (thus changing the order of plots drawing) then the best place for the buttons is always at the side of the list, so it makes more sense to give them one contour, thus allowing their simultaneous movement.

The whole algorithm is very easy in use. There is a special class, which organizes the whole process of moving and resizing – Mover, that can do everything now and is ready to work with any objects that we’ll design in the future. We only need to initialize it

Mover Movers = new Mover ();

The only requirement from the Mover is that any object with which he would have to work would be of movable type. All controls are of such type by default; graphical objects have to be derived from GraphicalObject and to implement (override) three methods. (For further details of design there is a link at the end of this paper.)

When Mover is initialized we can add to it every object that we would like to make movable. While starting the above mentioned application I added the panel, on which the list and two buttons reside.

MovableObject mover = new MovableObject (panelWithList);

Movers .Add (mover);

While adding the new plot, I am doing the same

DemoFunction demoFunc = new DemoFunction (…);

Movers .Add (new MovableObject (demoFunc));

Now Movers has everything to do the whole job, and I need only to include three handles.

To grab any object I usually use L_Press

private void OnMouseDown (object sender, MouseEventArgs mea)

{

if (mea .Button == MouseButtons .Left)

{

Movers .CatchMover (mea .Location);

…

}

}

To move already grabbed object or inform the user that something underneath the mouse cursor can be grabbed

private void OnMouseMove (object sender, MouseEventArgs mea)

{

Movers .MovingMover (mea .Location);

}

To release the object

private void OnMouseUp (object sender, MouseEventArgs mea)

{

Movers .ReleaseMover ();

}

These simple pieces of code cover all moving / resizing operations that I was using with absolutely different objects. Only in cases of both forward movement and rotation I had to use a bit more complicated code for OnMouseDown event using CatchMover method with additional parameter – pressed mouse button.

Because there are no limitations on the forms and sizes of nodes and the complexity (or simplicity) of connections this algorithm can be used for making movable / resizable a wide variety of objects. Let’s look at some samples which are far away from the standard rectangular shape.

Samples of movable / resizable objects

The first nonrectangular object on which I was checking the new algorithm was XYZ coordinate system. To make it more interesting I added the type of plot, which is never used in engineering projects, but very often in financial analysis; I call it skyscrapers (Fig.2). Depending on the set of data there is a very high probability that at any fixed angles some of the towers can be closed from view by the higher towers in front. With the axes’ system movable and resizable (see those four sensitive nodes and their connections) for any combination of data the user will immediately set the angles and size for best viewing.

This is also a good example of interconnection between the graphical object and its contour. For XYZ plotting it would be the obvious decision to organize a contour as a copy of the axes. At the same time the user of such skyscrapers’ plot can go into special window for tuning the skyscrapers by double clicking anywhere inside their area. By adding this movable / resizable feature the designer excludes the thin strips around axes from the area which will open the tuning window, so this is the designer’s responsibility to define the width of these strips (though it can be easily given to the user as another parameter).

Another sample is of absolutely different shape and area; similar pictures are often seen in applications for chemistry or medical projects (Fig.3). No limitations on number of balls or connections; users receive an instrument to design any structure. Balls and connections can be added, deleted, changed. In this case it looks like an obvious thing to define the contour as the copy of the graph, but this is not compulsory.

The situation with movable but not resizable objects is very widely used, for example, placement of arbitrary form objects inside some fixed area. In the mentioned application there is an example of one logical game, which is using this algorithm to move the tiles around the screen. There is also another sample where user can draw any number of houses on the screen and place them in absolutely arbitrary positions.

These samples also illustrate the flexibility of the used algorithm. Logical game or houses in town includes the big number of independent graphical objects; the sample from figure 3 includes only one object, but it can consist of huge number of elements; on figure 1 the number of objects is not limited and the complexity of each object can vary. It doesn’t matter how many objects are there and what is the complexity of each of them; all of them are treated in the same way with the handles that were mentioned above.

Let’s summarize some features of this extremely flexible contour design (nodes plus connections), which from my point of view can cover any possible situation to organize moving and resizing of the objects.

· Contour can consist of any number of nodes and their connections.

· Nodes can be moved individually thus allowing to reconfigure the object (fig. 3); by grabbing any connection the whole object can be moved (fig. 1).

· Contour may consist of a serious of connections between empty nodes. That makes an object movable, but not resizable (fig. 1; several controls on top of the panel).

· Each of the nodes has its own parameters, and by connecting nodes of different types it is easy to allow, for example, resizing along one direction but prohibit along another. This means organizing limited reconfiguring. (Such type of objects – scale - is included into application, but is not shown here.)

· It doesn’t matter that some contours may represent graphical objects and others controls or groups of controls. All contours are treated in the same way, thus allowing the user to change easily the inner view of any application.

New graphics, new design, new possibilities

There are two different parts in these described new results: graphics itself and applications, based on it. I am emphasizing the difference between these things as not always they are strictly linked with one another.

The kernel of the new things is definitely the new graphics. I constructed it on the base of contour presentation so, that it can cover any possible demand for moving and resizing objects on the screen. At the same time this new technique doesn’t substitute anything that was used before; it is simply added as a very powerful new feature, so I added it to the graphical classes that I was using for at least a decade.

As any other feature it can be used or not used, and the switch ON / OFF of this feature can be done in the same easy way as with all other features. (Don’t confuse it with switching ON / OFF the visualization of nodes and connections in the mentioned application; these are absolutely different things.) Moving and resizing is simply the feature of any object, so there is no demand that either all objects must be movable or none.

The moving / resizing feature can be used by itself without any changes in design of the whole application. It would be like improving the interface, but the application will be still designer-driven. There are some applications which will improve even in this simple way.

But there are a lot of applications, especially very complicated applications with huge amount of output plotting, which can be significantly changed with the use of such graphics; from author’s experience this would be the programs for engineering tasks and scientific research. The use of new graphics can be a real revolution in design of such complicated engineering (scientific) applications, but to receive the best results designers would have to switch from controlling the output view of an application to giving to the users the whole access to all results and producing them an easy mechanism of visualizing those results in any way and in any place. Thus shifting from designer-driven to user-driven applications. The movable / resizable graphics can be achieved in different ways, but user-driven applications can be based only on such type of graphics.

The consequences of using the movable graphics are definitely much more important than simply adding one new feature to existing graphics. And it is nearly impossible to predict all these consequences without starting to use such graphics and writing the programs which will especially benefit on such feature; in the same mentioned application I’m showing a lot of absolutely different areas, where moveable and resizable graphics can show astonishing results, and this is only a beginning. May be one of the most demanding areas can be financial graphics. Millions of people around the world are analyzing financial information mostly by visualization and comparison of plots. If each individual working with the most popular systems can do this comparison in the best way especially for him, then this would be for benefits of millions of people.

To estimate the difference between simply new movable / resizable feature and consequences of using it lets look at another situation (from my point of view very similar).

We are living in the houses that are fixed on some pieces of property and have fixed outside sizes and inner planning. Some changes can be made from time to time, but they need a lot of efforts and money. And in cases when we really need bigger or smaller house we are usually forced to relocate, which is not always the best thing to do but always have a lot of sociological consequences. Now there is the new unknown company in the market, which is selling the new adjustable buildings, advertising that WITHOUT ANY EXTRA COST you can receive the house which you can stretch or shrink at any moment. Just touch the wall with the wand and tell it to move. (Let’s not mention here what the solid well known constructors would like to do with the new developer - this is not the theme for discussion in programming.) I am absolutely sure that at the beginning 110 percent of the people would be suspicious. The explanation of that construction company, that everything is going to be fine, exactly what they want, will not lessen the suspicions. The consequences of using such constructions can not be predicted now but would be really revolutionary; it would change our whole life, the life of the society as a whole. The consequences are definitely much more important than the mechanism that allowed to achieve them.

I think that movable / resizable graphics can change a lot in the design of programs.

Conclusion. In the modern world of programming we have user-driven very flexible system of windows on the upper level and absolutely designer-driven inner level, on which users are doing the real work. The flexibility of this level is not only very low; for the most complicated engineering and scientific applications this level is hardly adjustable to the users’ demands.

The cause of this awkward situation is the use of the fixed graphics, which is absolutely defined by the designer at the stage of development. Author designed an algorithm, based on contour presentation of graphical objects, which makes arbitrary graphical objects movable and resizable. The same algorithm is used both with graphical objects and controls; as the result applications, based on such algorithm, receive the same level of flexibility as we have now only on the upper level. This means not only working with better graphics, but much more important that the new graphics is the base of the new paradigm – user-driven applications.

In such type of applications user has an absolute control of what, when and how must be shown at any moment. Changing to such type of graphics is easy, but to get all the benefits will require some changes in the design ideas. But the improvements in the output of applications from the users’ point of view can be enormous and their ability to analyze the most complicated and interesting cases will switch to another, much higher level.

Acknowledgments

Many thanks to those with whom I was discussing for years the design of tuneable graphics, beginning with Dr. Stanislav Shabunya from the Heat and Mass Transfer institute. And also to those with whom I was discussing the new movable graphics through the last year. Special thanks to those who didn’t get the new ideas from the beginning and forced me into better and better explanations and further work on the algorithm.

References

1. Stuerzlinger, W., Chapuis, O., Phillips, D., and Roussel, N. User interface façades: towards fully adaptable user interfaces. In Proceedings of UIST '06, pp. 309-318, ACM Press 2006.
2. Hawking, S. The universe in a nutshell. Bantam Press 2001.

Dr. Sergey Andreyev ( andreyev_sergey@yahoo.com )

Other links

Application http://levd.members.winisp.net/TuneableGraphics.zip

An application together with the detailed description of each case illustrated with a lot of pictures.

Test program http://levd.members.winisp.net/Test_MoveGraphLibrary.zip

The whole project of this sample program together with the detailed explanation of all the steps to design and use classes of movable / resizable graphical objects. Another DOC file contains the description of classes from MoveGraphLibrary.dll.

Library http://levd.members.winisp.net/MoveGraphLibrary.zip

Library and the description of classes included into it.