On a More Natural UI: Part 1

I’ve been thinking alot about user interface lately. Not for any particular reason, but because it’s come up incidentally in alot of the articles and websites I’ve looked at over the last week. First off, there’s the release of Bump Top, a fairly innovative desktop management interface. Then there was this article in ars technica, where John L. Hennessy, president of Stanford, bemoaned the fact that there had been no growth or innovation in computing UI in the past two decades. Finally, there was this, an interesting page giving vizualizations for a number of major sorting algorithms. These all kicked around in my head for a few days until I had a few thoughts that eventually lead to a sketch of what the UI of the future might look like.

To explain my thought process, I should start with what I see as the flaws of our current interface. Think about how your desktop looks on a daily basis. How many windows do you usually have open? Is it something like this?

My crowded desktop
My crowded desktop

And this is just when I’m browsing the web. When I’m actually working on something it gets worse. Take for example the response to an RFP I’ve been putting together for work. When I work on it I have to open up the following

  1. Email – I need to have the emails from the colleagues working on the response as well open, so I can review their suggestions and incorporate them.
  2. Visio – I need visio open to work on the graphics that we will be embedding in the response.
  3. One Note – I need one note so that I can review the outlines and notes I’ve made for specific sections
  4. Firefox or IE – I need firefox and IE to browse website for research, open up our company sharepoint site, take screen shots of the application involved in the response, etc.
  5. Excel – I need to have excel open to review the timeline one of my colleagues put together for writing the response.

This is what I have to open up almost every time I work on the project. And what am I making? A simple word document, with some very basic graphics. This is not rocket science or anything. And what makes it worse, is I’ve usually got 10 or more windows open from other projects at the same time. The clutter gets out of control.

Beyond this, there’s the problem of what I’m doing with these windows. Take the RFP response for example. I have to copy and paste data from my emails and notes. I have to take screenshots of the application, and then copy them into the document or an image editor, dealing with word’s complete inability to make intelligent layout decisions. Thankfully here, it’s mostly drag and drop, but even still I sometimes run into issues. But what happens when I start working on the powerpoint presentation we will need for orals? How will I get all my graphics out of the word document and into powerpoint? Manually, by cutting and pasting one at a time for each slide. That’s going to be fun.

Then there’s the challenge of sharing this with my colleagues. Sure I can post it on our sharepoint site, but they still have to open many of the same applications I do, cluttering their desktop. And if they work on it how do we do versioning? Sure Word can track changes, but Visio can’t. Do we have one Word file with all the comments and changes, and then one visio for each edit? Do we just keep the visio’s in word and edit them all from there? There’s really no good answer.

My simple example of a drafting a response to an RFP highlights what I see as some of the fundamental problems of modern user interfaces.

1. They’re application based – Modern UI’s are focused on files and applications. They don’t recognize most projects we work on are multi application, and so they don’t have a good way of dealing with our need to open and manage multiple apps when we are working on a project.

2. They lock data up in a file format – This isn’t just a matter of interoperability (ie opening docx files on Linux and the like.) This is about the fact that the modern UI does not recognize that inside a Word document there might be pictures, and that I might want to extract them for some other purpose.

3. They lack a good way to share and version – Obviously there are exceptions, like with most document processing software. However, the point is that versioning is at an application level, not a OS (this isn’t exactly UI, but I feel like it’s in the same basic realm.)

These three items I would categorize as the biggest failues of modern user interface. Now there has been some progress made on each one in recent years. The Mac has a number of features that assist in window management, and there’s also the famous Compiz Fusion window manager on Linux, which already has an amazing set of tools and is only likely to get better. Copy paste, and drag drop interfaces solve some of the problems I highlight in the second item. And versioning is starting to become a more prevalent part of both the OS and the application. But these things are just baby steps of progress, and it will take a complete rethinking of interface to really solve the problems.

Object Oriented User Interface

I think that a major part of that rethinking will borrow heavily from the realm of object oriented programming. In this section I will outline the basics of what I am calling and Object Oriented User Interface or OOUI (oo-ee).

The OOUI has three parts. Objects, collections, and a canvas. Their relationship can be summed up in the following manner; Collections are a group of objects arranged on a canvas. Those objects can be anything; graphs, images, text files, audio files, web pages, whatever. The canvas can also be very flexible, 2D, 3D, with an image background, with predifined object slots (ie days on a calendar). The user interacts by dragging objects round on the canvas in a very natural manner, much like Bump Top. They can open an object and edit it more fully, or change it’s configuration so that it’s bigger, a different color, transparent, etc. This combination of a very natural user interface, and the ability to save entire collections, helps address the first problem of modern UI’s.

Bump Top offers a good idea of what collections in a OOUI might be like. However an OOUI would go one step further by allowing this type of interaction with more than just icons, but with the actual content (text, images, audio, etc).

The key thing is how the computer manages these objects. Users can save and open collections, so that everything, down even to the layout, is preserved. Furthermore, the OOUI thinks of objects much like objects in object oriented programing. Each object has attributes, it’s content, it’s name, it’s current layout, etc. Each object also has methods that allow the user to manipulate it, such as save, edit, move, group, play, etc.

Here’s where the OOUI begins to solve the second problem. Just like in OOP, objects in the OOUI come in two flavors, primitives and complex objects. The primitives are your basic types of content. Text files, images, sound, video, link, application window, database table are some examples of primitives. Complex objects are then just a collection of primitives, with specially designed methods to bring them together. For example, a programmer could construct a graph object which had one attribute to specify a data source, and then methods to create various kinds of graphs (bar, pie, plot, etc.) Then the user could use this object to create a variety of graphs for their collection. Most importantly, users should be able to easily “explode” these complex objects into their component primitives, letting them say extract all the images from a powerpoint presentation to use somewhere else.

The third problem is a more thorny one, and not as directly related to UI. Addressing it would be more a matter of building some kinda of versioning in at the OS level, so that the collections that a user creates and saves store information about past states. The OOUI doesn’t really address this problem, on it’s own, so we won’t be discussing it anymore. I’ll leave that for someone smarter than me to solve.

However, what we see here is that this idea of an OOUI solves two major problems of user interface. It also goes even further, giving us a new way to think about the idea of an application. I will discuss this in part 2 of the series, coming soon.


4 thoughts on “On a More Natural UI: Part 1

  1. Innnnnteresting! That whole Bump Top thing looks pretty freaking cool! You’re right, my own computer desktop is usually a nightmare to deal with, and really hinders my work progress. Something more intuitive would be awesome…

    (PS I’m trying to do better with commenting, rather than just lurking. It’s nice to have an interactive aspect to blogs I think…)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s