KATI

The last two installments in this series focused on the challenge of integrating NSViewController into the controller layer of the current Cocoa MVC application architecture. This is an important step to take, especially when developing a Cocoa application with a single window interface design. In a single window design, users will access most of the application’s features through one window rather than opening and closing several smaller windows as they are needed. Since this one window will never close during a session, the multiple view controllers in the window will fill the role previously held by multiple window controllers in creating a dynamic Cocoa application.

The application’s view controllers will perform tasks like loading/unloading the content of nib files and their controllers as needed and validating/invalidating menu items as features are added or removed from the window. Plugging NSViewController into the existing controller architecture is a simple way to provide it with the support it needs to fulfill these standard duties for your application.

However important these window controller-type tasks are, keep in mind that they are not the principle duty of a view controller. Unlike NSWindowController, NSViewController has a close relationship to the view layer of the Cocoa MVC design. Each view controller manages a view in a view hierarchy. A system of view controllers, like the system Jonathan Dann and I described in the previous installment, manages the entire view hierarchy of a window. One of the most mission-critical jobs of a system of view controllers is to build and maintain the structure and layout of a complex view hierarchy in a dynamic environment.

Above: Adobe Photoshop Lightroom is an example of a feature rich application with a single window interface design.

Quick Quiz: what is a view hierarchy?

Quick answer from the docs (emphasis is mine):

In addition to being responsible for drawing and handling user events, a view instance can act as a container, enclosing other view instances. Those views are linked together creating a view hierarchy. Unlike a class hierarchy, which defines the lineage of a class, the view hierarchy defines the layout of views relative to other views.

V + C, the dynamic layout duo

There is a key difference between developing a Cocoa application that spreads its interface over several windows and one that uses a single window interface. In a single window design, the view hierarchy of the application’s main window is much more complex and, most importantly, dynamic than Cocoa windows normally are. In fact, the view hierarchy is a prominent character in this type of development. Managing its structure and layout throughout the application’s runtime is a significant design problem that should be addressed in a systematic way by the software’s MVC architecture – in both the controller and view layers.

NSViewController addresses this issue by providing support for building and managing the structure of a view hierarchy through its “view” instance variable. Once you organize your view controllers into a coherent system, you have a convenient mechanism for accessing and adjusting the hierarchy from its significant branches. But the problem is only half solved since changes in the view hierarchy are reflected by changes in the layout, and unfortunately, the issue of layout in a dynamic view environment remains to be addressed by Cocoa.

Building and maintaining a complex layout, like the one pictured above, requires cooperation from both the views and the controllers of a window. The controllers are the dictators of the layout. They tell the views contained in their domain of the hierarchy where to go and how to behave. Unless the circumstances are very special, it’s up to the view hierarchy itself to do the calculations necessary to maintain the integrity of that layout as the forces within it change over time. If either member of the layout duo fails to perform their duty, the interface will simply break.

The problem with V

In Cocoa, NSView fails to perform its duty in this kind of system because it isn’t equipped with the right tools. NSView has something called an “autoresizing mask”, which is meant to define the layout behavior of a view when its superview’s layout changes. However, Autoresizing was never designed to function in a view hierarchy with a dynamic structure. It is simply not the right tool for this job. This is evidenced by one simple thing: Autoresizing requires that a view is already laid out within its superview before you are supposed to set the desired resizing behavior.

This is well and good when you’re creating your entire layout in Interface Builder, where all the views are present and accounted for, but it’s simply not a practical limitation to make in a system of views and view controllers where elements of the layout will be added and removed during the application’s runtime.

It means that a view controller must have knowledge of it’s view’s superview and its frame before it can technically set a layout, but view controllers don’t necessarily have that information. For instance, when a view controller is initialized, it will create a view that will eventually be added to the view hierarchy. At that moment, however, it doesn’t know anything about the view/controller hierarchy that it and its view are about to become a part of. Technically, there’s no way for the controller to guarantee that the layout it dictates to its view (or more specifically, the view’s subviews) won’t be damaged by the layout of the view above it in the hierarchy. Since NSViewController can’t always meet the requirements set forth by Autoresizing, the layout is exposed to the glitches and bugs that I’ve described in previous posts by default, and there’s nothing to be done about it except to take extra special care to explicitly size and position new views when adding them into the hierarchy and to set limits on how much the user is allowed to resize elements like split views and the window itself. This is just extra grunt-work code that isn’t *really* necessary. It’s a byproduct of the bugs in Autoresizing.

The bigger problem, one that does sometimes affect the design of the controllers, is that when a view controller makes a change to the existing view hierarchy, by either adding or removing views or manually adjusting the size or position of a view, it needs to take the view’s siblings into consideration. The quote I pasted earlier, defining a view hierarchy, states that:

the view hierarchy defines the layout of views relative to other views.

What the docs really mean is that the layout of views is defined relative to their superviews. The sibling views aren’t considered by the view hierarchy at all during autoresizing. So, any sibling that needs to move or resize as a result of the change needs to have its layout recalculated and its autoresizing mask re-set by the view controller that propagated the change – even if the affected views aren’t in the controller’s domain of the view hierarchy. In this situation, the dynamic layout behavior of the view hierarchy becomes the responsibility of the view controller to maintain. This slippage of responsibility results in case-by-case workaround code that’s not only a pain in the ass to maintain, but that sometimes requires dependencies to views contained by other view controllers to be hard coded into the view controller. All this does is make your view controllers less reusable by no fault of their own. Again, none of this is necessary, it’s a byproduct of the limitations of Autoresizing.

Absolute vs. Relative

At the heart of the problem with Autoresizing, specifically in a dynamic system of views and view controllers, is the fact that there’s just nothing dynamic about it. The mechanism is based on absolute positions and sizes, which makes it an inappropriate layout tool for this kind of interface development. There are very few absolutes in a single window interface design. Absolutes might include the height of a toolbar or the width of a control, but the state of the view hierarchy as a whole, at any given time during the app’s runtime, is variable.

A dynamic view hierarchy and its controller counterparts would be better served by a layout tool that can accommodate relative sizes and positions. In a relative system, a view controller could just tell a newly created view to, for example, *fill* the width and height of its superview or *float* upwards in its superview – without having to specify any specific size or position (which might not be known anyway) – and the view hierarchy would work out the details once the view is in there.

Remember, view controllers have A LOT of responsibility in a Cocoa applicaiton. They handle action events, load nib files, validate menu items, and sync up views to their data. Why add managing the autoresizing behavior of the view hierarchy to their already long list of chores?

Still more to come…

So, yeah, I warned about the rant Now I’ll try to offer a solution.

I’ve prepared an example project that illustrates how to use Jonathan’s and my controller subclasses, XSWindowController and XSViewController to build and manage a dynamic view hierarchy. The project introduces an NSView subclass, KTView, into the mix. Naturally, KTView uses an alternative to the Autoresizing mechanism that’s specifically designed to work within a changing view hierarchy. It’s not a perfect solution (only Apple can do that), but it does get rid of the extra code from the view controllers that’s only there as a byproduct of Autoresizing.

The goal of the project is to bring together the concepts and opinions that have been covered in these three posts through examples of NSViewController in action.

I’ve decided that I don’t want to release KTView without an Interface Builder plugin, so I’ll publish the project with a tutorial as soon as that’s finished.

Until then, I found the original Smalltalk MVC Design paper, How to use Model-View-Controller. It’s interesting to read how Cocoa’s MVC design deviated from the original idea, especially when it comes to views and controllers. Enjoy!

In Computer Administrative Debris, I briefly discussed the idea of “directness” in interaction design. I described a “direct” interaction model as one where the visual representation of the content and the interface to the content are the same. A direct interaction model also implies some sort of physical interaction with content. Today, we most frequently use the mouse as an extension of our bodies to “point” at objects on the screen and to “drag” or “drop” them. With gesture and touch-based technologies entering the mainstream, these interactions will become more and more concerned with the movement and/or form of the user’s actual body.

A beautiful and extreme example of directness in an interface design can be found in the the software developed by Golan Levin and Zachary Lieberman for their 2004 audio/visual performances, Manual Input Sessions.

The Manual Input Sessions is a series of audiovisual vignettes which probe the expressive possibilities of hand gestures and finger movements.

Using a custom hardware/software setup, they developed interfaces for creating and composing audio in real-time that incorporate physical objects and the body into the interaction models. Users of this software literally mold the audio with their hands over time.

Also, when you watch the videos, notice how each interface has environmental characteristics that aid the user in creating and composing the audio. This is a clever way to add extra meaning to the input data. It also infuses the interfaces with a sense of play and exploration, so they become much more instrument-like and less tool-like.

[these videos need sound]

I found this 1993 video of Steve Jobs being interviewed about Paul Rand over on Paul Robinson’s site. It means so much to me when I see/read/hear someone that I admire admiring someone else that I admire.

Towards the end of the interview, Jobs is asked what his favorite Paul Rand work is. His answer is also my favorite – an offshoot of his famous IBM logo. It combines the wit and playfulness of his children’s illustrations with the discipline and craftsmanship of his corporate logos. Rand’s personality shines in this image:

Here are some of the children’s illustrations that I lifted from this post. “Sparkle and Spin” is the title of one of the children’s books he created with his wife, Ann.

And some of the corporate logos we all know too well…

There’s an article on Design Observer with a funny story about how the Enron logo got its green after Paul Rand’s death in 1997. The middle prong of the E was originally yellow. The article describes all the fanfare and celebration that surrounded the unveiling of Enron’s new corporate identity but…

Within hours, the world would laugh it off the stage. Houston faxed the logo to Enron’s offices in Europe. But in transmission the middle, yellow prong disappeared, leaving the new design meant to celebrate Enron’s triumphant ascension looking more like an electric plug. Worse, to the Italians it resembled an obscene hand gesture, one that meant about the same thing as shooting a middle finger at an American. The European executives roared with laughter: now they had a new way to win Italian customers.

Back in Houston, dismay grew: the yellow prong also vanished when run through the copying machine. Somehow, Enron had spent millions of dollars on a new business logo without bothering to check if it worked in business. Soon the hallway signs went down, the new cards and letterheads were shredded. With no fanfare, another logo was introduced, replacing the yellow prong with a green one.

The symbol meant to carry Enron into the next millennium hadn’t lasted a week.

I would love to hear Rand’s comments on this teeny graphic design mishap.

One last treat. The design company, Imaginary Forces, did a great job animating Paul Rand’s work. The animations are inter-cut with clips of Paul Rand speaking about design. They’ve really captured his spirit with these animations.

Features from this 1987 video that have been realized:

• Touch interface (iPhone)
• Voice commands (Speakable items)
• Animated views (Core Animation)
• Video conferencing (iChat)
• Looking at documents in a video conference (iChat Theater)

Someone needs to implement the bow-tied digital butler.

This tutorial is about implementing a custom control class. The goal is not to explain how to subclass NSControl or NSCell, but how to think more generally about controls, what they are and how to implement one with an NSView subclass. As with the last tutorial, it will be most useful if you download the example project and follow along in the source code. The comments will provide the implementation details. It’s also fun to give Asimo a pompadour.

The tutorial is based on a similar presentation that I gave at a New York City CocoaHeads meeting a few months ago.

What is a Control?

A control is an interface element that allows users to manipulate data in an application. It’s main responsibilities are to draw itself, handle events, and communicate its values to other parts of the application.

Cocoa’s AppKit framework comes with several controls out of the box:

All of the controls pictured above are decedents of Cocoa’s control class, NSControl. As the documentation states, NSControl works closely with NSCell to provide the basic features of a user interface object. In most cases, you can use an AppKit control with no extra code.

Why Make a Custom Control?

With all of AppKit’s ready-made controls, why would you need to implement your own? Here are two situations where the Cocoa control classes are not a good choice for your interface:

1. None of the AppKit controls support the interaction model you want to implement

Take Photoshop’s curves interface as an example. This interface requires that the user is able to add control points to a bezier curve. They must then be able to select the control points and drag them around to adjust the values of the curve. If they click on the pencil button, the interface must switch to a mode that lets them draw the curve directly into the view.

You’re not going to find an out of the box control that supports this interface in AppKit and NSControl may not be an appropriate place to start your custom implementation. If you look at its API, it is clear that NSControl was designed to create the AppKit controls. They all have fairly straightforward drawing, event handling and target/action needs. Point-and-click. If you were to use NSControl as a starting point for this interface, you would need to create a few custom cell classes that you would manage in your control’s custom cell. At this point, you’re asking for a headache. This interface is not what NSControl was designed for. You’re going to spend more time fighting with the framework than implementing your ideas.

2. You are drawing with OpenGL instead of Quartz

All of the controls in the iTunes Cover Flow interface are drawn with OpenGL textures. That includes the buttons, scrollbar and slider. These are absolutely not NSControls, which can only draw in a Quartz graphics context. If you want users to interact with your NSOpenGL view, you’re going to have to start from scratch with your controls. Don’t let this scare you away from using OpenGL views for your interface. They’re fast fast fast and if your app is displaying lots and lots of images, especially in a full screen situation with animation, they’re the way to go.

Leopard’s LayerKit (Core Animation Layers) technology advertises mixed Quartz and OpenGL drawing, but I just have a hard time getting behind that idea. More on this thought another time.

Using NSView

The subject of creating controls for an OpenGL view is a little out of the scope of this tutorial. Again, I will come back to the issue of OpenGL views in the not-too-distant future.

Working with NSView will let us touch on the very basics of what we need to make a control. For really specialized, one-off interfaces that you can add to your window’s view hierarchy with no fuss, it makes a lot of sense to start here.

As I mentioned before, there are three parts to a control. Let’s make a to-do list for our class design. Our view must be able to handle:

1. Drawing
2. Event Handling
3. Communication with other parts of your app

We’ll handle number one with NSView and number two with its superclass, NSResponder. For number three, we’ll use Cocoa bindings.

What Are we Controlling?

Before we subclass anything, we need to figure out what our control will control. The example project contains a control for adjusting the values of a twirl distortion CoreImage filter. To start the design, I looked at the documentation for CITwirlDistortion. It lists the three attributes of the filter: center postition, radius and angle. I decided to create a circular design for controling all three attributes at once. This seemed to be a nicer interface than providing three sliders. Of course this design has some usability issues that will be obvious once you play around with it a bit, but it’s fun to experiment.

KDTwirlDistortionControl has this design:

It’s basically a circle. Users can drag on the main gray part of the circle to adjust its radius - this will change the radius attribute of the filter. Dragging on the blue center point will change the position of the circle and the center position attribute of the filter. Finally, dragging the small black circle around its perimeter will adjust the angle attribute of the filter.

We have an interface design. Time for code.

Defining the NSView subclass

Our NSView subclass, KDTwirlDistortionControl, needs to keep track of three rectangles (pictured in the diagram) that it will use for drawing and hit detection. These rectangles will also be used to calculate the adjustment values of the filter. It declares the following ivars for the rectangles in the header file:

// drawing and hit detection
NSRect	mPositionControlRect;
NSRect	mAngleControlRect;
NSRect	mRadiusControlRect;

The control also contains 4 float values that can be bound to. It does this by declaring the following ivars and their getters and setters:

// the values we bind to
float	mPositionX;	// a value between 0 and 1
float	mPositionY;	// a value between 0 and 1
float	mRadius;	// a value between 0 and 1
float	mAngle;		// a value between 0 and 6 radians

The getters and setters will always return normalized values for these ivars, which are calculated based on the positions and sizes of the rectangles in the view’s coordinate system. Look in the code for more details about this part of the implementation.

With the getters and setters in place, our control can communicate changes to these values to other classes with no more code. We can check number three off of our to-do list. Thanks Cocoa.

The last thing our control needs to manage is its state. There are four possible states in our design:

1. the control is inactive
2. the position rectangle is active
3. the radius rectangle is active
4. the angle rectangle is active

We’ll use this information when we draw to give the user feedback about what they are adjusting. We’ll also use it in our mouse drag handler so that we know which rectangle the user is adjusting. To define the states of the control, create an enum type like this:

typedef enum
{
	kKDTwirlDistortionControlState_Inactive = 0,
	kKDTwirlDistortionControlState_AngleControlActive,
	kKDTwirlDistortionControlState_RadiusControlActive,
	kKDTwirlDistortionControlState_PositionControlActive

}KDTwirlDistortionControlState;

The view subclass declares an ivar that it will use to keep track of its current state:

// state
KDTwirlDistortionControlState		mControlState;

This is all the data we need for our control. Now we will use NSView’s drawing and NSResponder’s event handler methods to implement the control.

Drawing

All we need to do to draw in an NSView subclass is to override the following NSView method:

- (void)drawRect:(NSRect)theRect;

To keep things organized, KDTwirlDistortionControl uses separate drawing methods for each part of the control. It also creates a separate method to lay out the rectangles in the view. It declares them as private methods in a category in the .m file:

@interface KDTwirlDistortionControl (Private)
- (void)drawPositionControlInContext:(CGContextRef)theContext;
- (void)drawAngleControlInContext:(CGContextRef)theContext;
- (void)drawRadiusControlInContext:(CGContextRef)theContext;
- (void)layoutControls;
@end

KDTwirlDistortionControl’s drawRect method looks like this:

- (void)drawRect:(NSRect)theRect
{
 CGContextRef aCGContextRef = [[NSGraphicsContext currentContext] graphicsPort];

 // draw the background
 CGContextSetRGBFillColor(aCGContextRef, 0.8, 0.8, 0.8, 1.0);
 CGRect aCGBackgroundRect = *((CGRect*)&theRect);
 CGContextFillRect(aCGContextRef, aCGBackgroundRect);

 // layout the controls
 [self layoutControls];

 // draw the radius control rect
 [self drawRadiusControlInContext:aCGContextRef];

 // draw the position control rect
 [self drawPositionControlInContext:aCGContextRef];

 // draw the angle control rect
 [self drawAngleControlInContext:aCGContextRef];
}

If you look at the draw methods, you’ll notice that before they draw, they check the control state ivar to determine what color they will use to draw. This gives the user feedback about the state of the control. You might also notice that the control uses Quartz2D drawing commands. Your control could just as well use Cocoa’s NSBezierPath class to draw itself. It’s good to be familiar with both drawing APIs. There might be cases where you will notice the performance difference, so it’s nice to be able to fall back on Quartz for drawing straight into the view instead of using another object.

We can check number 1 off of our to-do list. The drawing is finished. One more.

Event Handling

NSView is a subclass of NSResponder. The mechanism of the view hierarchy ensures that any view you add to it will be placed in the application’s “Responder Chain”. The window will use the view hierarchy to deliver mouse events to the view that the user clicked on. Keyboard events will be delivered through the responder chain. It’s up to the view to implement the NSResponder event handlers that it is interested in.

Most mouse events will be sent automatically to the appropriate view, but there is one step that a view must take to receive keyboard events. The keyboard events go to the “firstResponder” first and then travel through the responder chain. If your view is to receive keyboard events it must accept first responder status. To do this, the view needs to override the NSResponder method:

- (BOOL)acceptsFirstResponder

to return YES. By default it returns NO.

Here’s a list of the most common mouse event handlers that a custom view can impelment:

- (void)mouseDown:(NSEvent*)theEvent;
- (void)mouseDragged:(NSEvent*)theEvent;
- (void)mouseUp:(NSEvent*)theEvent;
- (void)mouseMoved:(NSEvent*)theEvent;
- (void)mouseEntered:(NSEvent*)theEvent;
- (void)mouseExited:(NSEvent*)theEvent;

And common keyboard event handlers:

- (void)keyDown:(NSEvent*)theEvent;
- (void)keyUp:(NSEvent*)theEvent;

NSEvent

All of the event methods are going to pass us an NSEvent object that describes the event. If it’s a mouse event, we can ask the event for the location of the mouse point using the NSEvent method:

- (NSPoint)locationInWindow;

This is going to give us the location of the mouse event in the window (duh). We need to convert this point to a coordinate in our view’s internal coordinate system. We’ll use NSView’s method

- (NSPoint)convertPoint:(NSPoint)thePoint fromView:(NSView*)theView;

If the event is a keyboard event, we can ask the NSEvent object for a string of characters associated with the event with the following methods:

- (NSString *)charactersIgnoringModifiers;
- (NSString *)characters;

Another useful bit of information we can get from NSEvent is whether or not a modifier key is associated with the event. Use the method

- (unsigned int)modifierFlags

to get the current modifier flags, which you can find documented in the list of NSEvent’s constants. Check this against the modifier flag you are looking for. This bit of code is checking for the command key:

if(([theEvent modifierFlags] & NSCommandKeyMask) != 0)
{
 // do something special
}

Check the NSEvent documentation for more information on getting neat things like tablet data.

Ok, Let’s get back to our control.

Mouse Down

When our view receives a mouse down, we want to check to see if the mouse point is inside any of the rectangles. There’s a handy function we can use to check a point struct against a rectangle struct:

BOOL NSPointInRect(NSPoint thePoint, NSRect theRect);

KDTwirlDistortionControl’s mouse down method checks all of the rects against the mouse point. If it determines that a click has occured inside any of its rects, it sets the control state to indicate which rectangle has been hit, tells itself to redraw and returns. Here’s an excerpt:

// down on the position control rect
if(NSPointInRect(aMousePoint,mPositionControlRect))
{
 mControlState = kKDTwirlDistortionControlState_PositionControlActive;
 [self setNeedsDisplay:YES];
 return;
}

If none of the rectangles have been hit, the view centers the position rectangle around the mouse point. This lets user click the background to set the position - hint: this feature comes in handy if you happen to drag it to an unusable position, like I said there are problems in this design ; )

Mouse Dragged

This might be the most important method our class implements. The mouse drag is going to change the values of our active rectangle. It’s also going to call the setter methods of our control’s float values, which will automatically trigger notification of a value change to any class that is bound to the control.

The first thing the mouse dragged handler does is check the control state to determine which rectangle the user is adjusting. It will then perform the calculations it needs to make adjustments to both the rectangle and the associated filter value. It sets the value, tells itself to redraw and returns. Here’s an excerpt:

// dragging the position control rect
if(mControlState == kKDTwirlDistortionControlState_PositionControlActive)
{
 // center the rect around the mouse point
 float aNewXPosition = aMousePoint.x-mPositionControlRect.size.width*.5;
 float aNewYPosition = aMousePoint.y-mPositionControlRect.size.height*.5;
 mPositionControlRect.origin = NSMakePoint(aNewXPosition, aNewYPosition);

 [self setPositionX:aNewXPosition];
 [self setPositionY:aNewYPosition];
 [self setNeedsDisplay:YES];

 return;
}

Mouse Up

The mouse up simply sets the control state to be inactive and tells itself to redraw.

Key Down and Key Up

The key down event handler looks for arrow key presses. When an arrow key is pressed, it adjusts the origin of the position rectangle a few points in the appropriate direction. It also sets the current control state so that the position rectangle is active and redraws. Here’s an excerpt from the method. It uses a switch statement to find the arrow keys:

case NSUpArrowFunctionKey:
   mControlState = kKDTwirlDistortionControlState_PositionControlActive;
   mPositionControlRect.origin.y = mPositionControlRect.origin.y+5;
   [self setPositionY:mPositionControlRect.origin.y];
   [self setNeedsDisplay:YES];
break;

The key up method also checks for arrow keys. If the event is from one of the arrow keys, it resets the control state to inactive and redraws.

Event handling was the last thing on our to-do list. Scratch it off, we’re done. We have a control.

Hi Asimo! Nice Hair-do!

The KDTwirlDistortionControl example is very specific to the kind of data it is controlling and it could be used with any Core Image filter that has the same attributes, despite its name.

In a future post, I will explore ways to abstract the design a little more to make the control more useful in more situations. This will be necessary for designing OpenGL controls and could be useful if you’re using CALayers as the basis for a control. I haven’t had a chance to work with CALayers, but I notice that they’re not descendants of NSResponder. Odd for such a view-like class.

Resources

The Example Project

Cocoa Event handling Guide

The Responder Chain

Cocoa drawing guide

Quartz2D Programming Guide

NSView - Working With the View Hierarchy

Cocoa Bindings

More advanced bindings examples: mmalc’s bindings examples, look at graphics bindings

I’m still using IB2 at work, but we’ll be making the move to Leopard over the next few months. I feel a slight pang of pre-nostalgia when I think about trading in Interface Builder 2 and its historical code base for its shiny new successor, Interface Builder 3. I know that IB3 is going to be a million times more usable and that it was past time for a real change in the world of IB, but those who know me know that I tend to pine for the old before embracing the new. I’ll pine now and get it over with.

I remember the exact moment when I realized how amazingly cool Interface Builder is. I was making a custom palette for some GUI classes, implementing the drawing and mouse event handling methods for when they would be running in IB. While I was typing, it struck me that what I was doing was a little strange. I was giving these classes a second, meta life only to be invoked by Interface Builder. All of the views and controls in Cocoa have this second IB life. When we drop them into a window, they draw preview versions of themselves and sometimes even respond to our mouse and key events (if we poke around in just the right spots). Interface Builder takes over from there. It guides us as we drag the views around, adjust their sizes and much more. Wow, what a clever and useful application. Who thought of this? They’re a genius.

Jean-Marie Hullot and his Interface Builder

Interface Builder was developed by Jean-Marie Hullot, pictured above during his days at NeXT. There is little written about him or his invention on the internet except that Hullot wrote the original version in some Lispy language and that it was originally called ‘SOS Interface’. Steve Jobs loved it, kidnapped Hullot and Tim Berners-Lee used it to make the first web browser, WorldWideWeb. Now we all use Interface Builder.

While I was scouring the net for information about Hullot, I came across a PDF excerpt from the book “How the Web was Born: The Story of the World Wide Web” by James Gillies and Robert Cailliau. There is a short section about Hullot and Interface Builder, titled “Monsieur Hullot’s holiday’. This is my favorite bit (emphasis is mine):

Hullot was using a Macintosh, and like many developers, he rapidly found that although the Mac’s GUI might have been brilliant, it was a pig of a machine to write applications on. ‘When the Macintosh was invented,’ he explains, ’suddenly, compared to what you had to do before, you had to add at least 60 or 70 percent of your time to make the user interface.’ Before the Macintosh, the interface was pretty basic-you would just type a line of text and the computer would respond with successive lines scrolling up the screen-but that meant that programmers didn’t have to worry about it. All the windows and menus that came with the Mac were great for the people who used the end applications, but for the people who had to write them they were a nightmare. Each time they wanted a menu or a window in their application, they had to program it in from scratch. Hullot’s big idea was to turn the problem into a simple question of drawing lines on the screen. He made a palette of objects for things like menu items and windows. Then to build the outline of an application you would simply take objects from the palette and draw lines between them. So, for example, you could pick up a menu object, and then by drawing lines from the other objects like ‘Open’ or ‘Print’, you would put those functions in the menu.

Hullot called his invention Interface Builder. It was a powerful tool that, like Smalltalk, put the power of objects at the developer’s disposal.

Kind of amazing, huh? These days, it’s easy to take things like windows and menus for granted. They come free with our development tools. On top of that, the responder chain and target/action mechanisms make it simple to dynamically link buttons and menu items to actions. If your app has a simple interface, there’s very little code that needs to be written for UI development. Even if your interface is complex, much of the burden is lifted by Hullot’s technology. Thank you, Jean-Marie.

The book goes on to describe Hullot and Jobs’ first meeting and the rest is history. In case you haven’t seen it, here’s a video of Steve Jobs presenting some applications on NeXTSTEP. He shows Interface Builder at 23:40. Funny how little has changed until now.

So, I’ll say goodbye to Inteface Builder 2 and end my pining now. Bye bye, IB2. Your interface was really strange and awkward but you were really cool and useful. I’ll miss palettes (not really, actually).

Edward Tufte recently posted a video analysis of the iPhone interface on his website. If you haven’t seen it yet, I highly recommend taking a look.

My favorite part is when he offhandedly states, “The white frames are too thick and should be in gray and only a single pixel thick,” as he’s scrolling the thumbnail browser. I imagine Edward Tufte working as a design supervisor at Apple, watching over people’s shoulders, scolding young designers for not displaying enough information. I’m sure he would run a tight ship where every pixel would have a meaningful purpose.

Actually, I lied, my real favorite part is a term Tufte uses frequently in the video, “computer administrative debris”, usually as he’s noting the lack of it on the iPhone’s UI. That’s a great way to describe all the crap on our interfaces, the important stuff that enables users to perform complex tasks in a tiny two-dimensional workspace. Things like windows, menus, tabs, check boxes, toolbars, scroll bars, etc.

One of the challenges of interface design is to reduce computer administrative debris while maintaining a high standard of usability. Of course, the difficulty of this challenge is related to the complexity of the application. Pro apps are going to have more computer administrative debris than consumer apps simply because they have more features. The iPhone and Apple TV-type applications have it easy in this sense, because their users are mostly browsing content. The developers don’t often need to provide them with a context to adjust or create the content. Most desktop apps, on the other hand, are all about providing that context. Still, there are some things that desktop application developers can do to trim the amount of debris on their interfaces.

Top: Flying Meat’s consumer image editor, Acorn

Bottom: Adobe’s pro image editor, Photoshop

Features

As Jeff Atwood points out in his post, Every User Lies, users don’t actually use most of the features of the applications they buy. They want to, and might even tell you that they do, but they usually don’t. Jeff’s advice is good. Observe. Figure out what is critical for your users to get their work done and go from there. Also, don’t be afraid to say no to feature requests if you’re in the position to do so. Keeping the feature set streamlined is a simple way to reduce the amount of computer administrative debris your users will have to wade through to get a simple task done.

A good example of this is Flying Meat’s consumer image editor, Acorn. Pictured above are both Acorn and its comparable pro application, Adobe Photoshop. They’re both shown at their initial states, just after I opened an image. Acorn is clearly more approachable because there is less debris on the screen. Its feature set is not nearly as rich a Photoshop’s, but Acorn’s developer, Gus Mueller, knew that his users were not going to use all of those features anyway. He made a succinct feature list and because of that he is able to minimize computer administrative debris on his interface.

Organization

Pro and prosumer apps, like Apple’s Aperture, are in a constant state of struggle with computer administrative debris. They are the applications with the most features, but with some organization, even they manage improve their computer administrative debris to content ratio over time.

Top: Aperture 2
Bottom: Aperture 1.5.6

Pictured above are Apple’s Aperture 2, which came out this week, and its predecessor, Aperture 1.5.6. The Aperture 2 team did some re-organizing for this release. The most notable organizational improvement is the shifting of the Adjustments and Metadata widgets over to the source list area. The arrangement that we see in 1.5.6 is no longer possible. Users will never see the source list, adjustments, and metadata debris at the same time. The Aperture team understood that the users will never need to access more than one of these features at a time, so they made the decision to consolidate. Using tabs as an organizational tool, they reduced debris.

Directness

Tufte makes an important observation about the iPhone when he talks about computer administrative debris. For many features, the iPhone uses the content as the interface and users interact with it directly rather than through interface widgets. Again, the fact that users of the iPhone are mostly browsing makes it easier to do this, but there’s something to be learned from their model.

A simple example of letting the users interact directly with content to trim debris is drag and drop, something we’re all familiar with by now. By letting users either pick up items from one container and drag them into another or reorder items in a single container, we reduce the need for buttons or action menus. Less debris. Apple’s iMovie 08 boasts, “Drag-and-drop moviemaking”. In fact, Apple’s entire iLife and iWork repertoire is full of examples of interacting directly with content instead of through menus, buttons or text fields, greatly reducing the amount of computer administrative debris on their interfaces.

Apple’s Keynote

Keynote, pictured above, has a very clean interface with little debris because users interact directly with the interface representations of the content. Can your interaction model be more direct? When could users interact directly with content instead of through text fields or action menus? This is a tough one and often requires some creative coding, but its worth it.

Free of Debris (not really)

It’s not possible to completely eliminate widgets from a desktop application, but we have the tools we need to reduce the amount of unsightly and distracting computer administrative debris on our interface designs. With the introduction of gesture events and CALayers in Cocoa, the Mac developer’s toolbox is becoming more sophisticated and we have lots of room to be creative and innovative.

Users are also becoming more sophisticated and their expectations are evolving. It won’t be long until the mere sight of a pop up menu will be enough to make them quit an application and never open it again. Trust that they can handle an unorthodox interaction model. They might be liars, as Jeff Atwood points out, but they learn quick if they’re presented with a well thought out and executed design. The iPhone interface proves that and as Tufte notes over and over in his video, there is often no computer administrative debris in sight.