KATI

This tutorial is about implementing a custom control class. The goal is not to explain how to subclass NSControl or NSCell, but how to think more generally about controls, what they are and how to implement one with an NSView subclass. As with the last tutorial, it will be most useful if you download the example project and follow along in the source code. The comments will provide the implementation details. It’s also fun to give Asimo a pompadour.

The tutorial is based on a similar presentation that I gave at a New York City CocoaHeads meeting a few months ago.

What is a Control?

A control is an interface element that allows users to manipulate data in an application. It’s main responsibilities are to draw itself, handle events, and communicate its values to other parts of the application.

Cocoa’s AppKit framework comes with several controls out of the box:

All of the controls pictured above are decedents of Cocoa’s control class, NSControl. As the documentation states, NSControl works closely with NSCell to provide the basic features of a user interface object. In most cases, you can use an AppKit control with no extra code.

Why Make a Custom Control?

With all of AppKit’s ready-made controls, why would you need to implement your own? Here are two situations where the Cocoa control classes are not a good choice for your interface:

1. None of the AppKit controls support the interaction model you want to implement

Take Photoshop’s curves interface as an example. This interface requires that the user is able to add control points to a bezier curve. They must then be able to select the control points and drag them around to adjust the values of the curve. If they click on the pencil button, the interface must switch to a mode that lets them draw the curve directly into the view.

You’re not going to find an out of the box control that supports this interface in AppKit and NSControl may not be an appropriate place to start your custom implementation. If you look at its API, it is clear that NSControl was designed to create the AppKit controls. They all have fairly straightforward drawing, event handling and target/action needs. Point-and-click. If you were to use NSControl as a starting point for this interface, you would need to create a few custom cell classes that you would manage in your control’s custom cell. At this point, you’re asking for a headache. This interface is not what NSControl was designed for. You’re going to spend more time fighting with the framework than implementing your ideas.

2. You are drawing with OpenGL instead of Quartz

All of the controls in the iTunes Cover Flow interface are drawn with OpenGL textures. That includes the buttons, scrollbar and slider. These are absolutely not NSControls, which can only draw in a Quartz graphics context. If you want users to interact with your NSOpenGL view, you’re going to have to start from scratch with your controls. Don’t let this scare you away from using OpenGL views for your interface. They’re fast fast fast and if your app is displaying lots and lots of images, especially in a full screen situation with animation, they’re the way to go.

Leopard’s LayerKit (Core Animation Layers) technology advertises mixed Quartz and OpenGL drawing, but I just have a hard time getting behind that idea. More on this thought another time.

Using NSView

The subject of creating controls for an OpenGL view is a little out of the scope of this tutorial. Again, I will come back to the issue of OpenGL views in the not-too-distant future.

Working with NSView will let us touch on the very basics of what we need to make a control. For really specialized, one-off interfaces that you can add to your window’s view hierarchy with no fuss, it makes a lot of sense to start here.

As I mentioned before, there are three parts to a control. Let’s make a to-do list for our class design. Our view must be able to handle:

1. Drawing
2. Event Handling
3. Communication with other parts of your app

We’ll handle number one with NSView and number two with its superclass, NSResponder. For number three, we’ll use Cocoa bindings.

What Are we Controlling?

Before we subclass anything, we need to figure out what our control will control. The example project contains a control for adjusting the values of a twirl distortion CoreImage filter. To start the design, I looked at the documentation for CITwirlDistortion. It lists the three attributes of the filter: center postition, radius and angle. I decided to create a circular design for controling all three attributes at once. This seemed to be a nicer interface than providing three sliders. Of course this design has some usability issues that will be obvious once you play around with it a bit, but it’s fun to experiment.

KDTwirlDistortionControl has this design:

It’s basically a circle. Users can drag on the main gray part of the circle to adjust its radius - this will change the radius attribute of the filter. Dragging on the blue center point will change the position of the circle and the center position attribute of the filter. Finally, dragging the small black circle around its perimeter will adjust the angle attribute of the filter.

We have an interface design. Time for code.

Defining the NSView subclass

Our NSView subclass, KDTwirlDistortionControl, needs to keep track of three rectangles (pictured in the diagram) that it will use for drawing and hit detection. These rectangles will also be used to calculate the adjustment values of the filter. It declares the following ivars for the rectangles in the header file:

// drawing and hit detection
NSRect	mPositionControlRect;
NSRect	mAngleControlRect;
NSRect	mRadiusControlRect;

The control also contains 4 float values that can be bound to. It does this by declaring the following ivars and their getters and setters:

// the values we bind to
float	mPositionX;	// a value between 0 and 1
float	mPositionY;	// a value between 0 and 1
float	mRadius;	// a value between 0 and 1
float	mAngle;		// a value between 0 and 6 radians

The getters and setters will always return normalized values for these ivars, which are calculated based on the positions and sizes of the rectangles in the view’s coordinate system. Look in the code for more details about this part of the implementation.

With the getters and setters in place, our control can communicate changes to these values to other classes with no more code. We can check number three off of our to-do list. Thanks Cocoa.

The last thing our control needs to manage is its state. There are four possible states in our design:

1. the control is inactive
2. the position rectangle is active
3. the radius rectangle is active
4. the angle rectangle is active

We’ll use this information when we draw to give the user feedback about what they are adjusting. We’ll also use it in our mouse drag handler so that we know which rectangle the user is adjusting. To define the states of the control, create an enum type like this:

typedef enum
{
	kKDTwirlDistortionControlState_Inactive = 0,
	kKDTwirlDistortionControlState_AngleControlActive,
	kKDTwirlDistortionControlState_RadiusControlActive,
	kKDTwirlDistortionControlState_PositionControlActive

}KDTwirlDistortionControlState;

The view subclass declares an ivar that it will use to keep track of its current state:

// state
KDTwirlDistortionControlState		mControlState;

This is all the data we need for our control. Now we will use NSView’s drawing and NSResponder’s event handler methods to implement the control.

Drawing

All we need to do to draw in an NSView subclass is to override the following NSView method:

- (void)drawRect:(NSRect)theRect;

To keep things organized, KDTwirlDistortionControl uses separate drawing methods for each part of the control. It also creates a separate method to lay out the rectangles in the view. It declares them as private methods in a category in the .m file:

@interface KDTwirlDistortionControl (Private)
- (void)drawPositionControlInContext:(CGContextRef)theContext;
- (void)drawAngleControlInContext:(CGContextRef)theContext;
- (void)drawRadiusControlInContext:(CGContextRef)theContext;
- (void)layoutControls;
@end

KDTwirlDistortionControl’s drawRect method looks like this:

- (void)drawRect:(NSRect)theRect
{
 CGContextRef aCGContextRef = [[NSGraphicsContext currentContext] graphicsPort];

 // draw the background
 CGContextSetRGBFillColor(aCGContextRef, 0.8, 0.8, 0.8, 1.0);
 CGRect aCGBackgroundRect = *((CGRect*)&theRect);
 CGContextFillRect(aCGContextRef, aCGBackgroundRect);

 // layout the controls
 [self layoutControls];

 // draw the radius control rect
 [self drawRadiusControlInContext:aCGContextRef];

 // draw the position control rect
 [self drawPositionControlInContext:aCGContextRef];

 // draw the angle control rect
 [self drawAngleControlInContext:aCGContextRef];
}

If you look at the draw methods, you’ll notice that before they draw, they check the control state ivar to determine what color they will use to draw. This gives the user feedback about the state of the control. You might also notice that the control uses Quartz2D drawing commands. Your control could just as well use Cocoa’s NSBezierPath class to draw itself. It’s good to be familiar with both drawing APIs. There might be cases where you will notice the performance difference, so it’s nice to be able to fall back on Quartz for drawing straight into the view instead of using another object.

We can check number 1 off of our to-do list. The drawing is finished. One more.

Event Handling

NSView is a subclass of NSResponder. The mechanism of the view hierarchy ensures that any view you add to it will be placed in the application’s “Responder Chain”. The window will use the view hierarchy to deliver mouse events to the view that the user clicked on. Keyboard events will be delivered through the responder chain. It’s up to the view to implement the NSResponder event handlers that it is interested in.

Most mouse events will be sent automatically to the appropriate view, but there is one step that a view must take to receive keyboard events. The keyboard events go to the “firstResponder” first and then travel through the responder chain. If your view is to receive keyboard events it must accept first responder status. To do this, the view needs to override the NSResponder method:

- (BOOL)acceptsFirstResponder

to return YES. By default it returns NO.

Here’s a list of the most common mouse event handlers that a custom view can impelment:

- (void)mouseDown:(NSEvent*)theEvent;
- (void)mouseDragged:(NSEvent*)theEvent;
- (void)mouseUp:(NSEvent*)theEvent;
- (void)mouseMoved:(NSEvent*)theEvent;
- (void)mouseEntered:(NSEvent*)theEvent;
- (void)mouseExited:(NSEvent*)theEvent;

And common keyboard event handlers:

- (void)keyDown:(NSEvent*)theEvent;
- (void)keyUp:(NSEvent*)theEvent;

NSEvent

All of the event methods are going to pass us an NSEvent object that describes the event. If it’s a mouse event, we can ask the event for the location of the mouse point using the NSEvent method:

- (NSPoint)locationInWindow;

This is going to give us the location of the mouse event in the window (duh). We need to convert this point to a coordinate in our view’s internal coordinate system. We’ll use NSView’s method

- (NSPoint)convertPoint:(NSPoint)thePoint fromView:(NSView*)theView;

If the event is a keyboard event, we can ask the NSEvent object for a string of characters associated with the event with the following methods:

- (NSString *)charactersIgnoringModifiers;
- (NSString *)characters;

Another useful bit of information we can get from NSEvent is whether or not a modifier key is associated with the event. Use the method

- (unsigned int)modifierFlags

to get the current modifier flags, which you can find documented in the list of NSEvent’s constants. Check this against the modifier flag you are looking for. This bit of code is checking for the command key:

if(([theEvent modifierFlags] & NSCommandKeyMask) != 0)
{
 // do something special
}

Check the NSEvent documentation for more information on getting neat things like tablet data.

Ok, Let’s get back to our control.

Mouse Down

When our view receives a mouse down, we want to check to see if the mouse point is inside any of the rectangles. There’s a handy function we can use to check a point struct against a rectangle struct:

BOOL NSPointInRect(NSPoint thePoint, NSRect theRect);

KDTwirlDistortionControl’s mouse down method checks all of the rects against the mouse point. If it determines that a click has occured inside any of its rects, it sets the control state to indicate which rectangle has been hit, tells itself to redraw and returns. Here’s an excerpt:

// down on the position control rect
if(NSPointInRect(aMousePoint,mPositionControlRect))
{
 mControlState = kKDTwirlDistortionControlState_PositionControlActive;
 [self setNeedsDisplay:YES];
 return;
}

If none of the rectangles have been hit, the view centers the position rectangle around the mouse point. This lets user click the background to set the position - hint: this feature comes in handy if you happen to drag it to an unusable position, like I said there are problems in this design ; )

Mouse Dragged

This might be the most important method our class implements. The mouse drag is going to change the values of our active rectangle. It’s also going to call the setter methods of our control’s float values, which will automatically trigger notification of a value change to any class that is bound to the control.

The first thing the mouse dragged handler does is check the control state to determine which rectangle the user is adjusting. It will then perform the calculations it needs to make adjustments to both the rectangle and the associated filter value. It sets the value, tells itself to redraw and returns. Here’s an excerpt:

// dragging the position control rect
if(mControlState == kKDTwirlDistortionControlState_PositionControlActive)
{
 // center the rect around the mouse point
 float aNewXPosition = aMousePoint.x-mPositionControlRect.size.width*.5;
 float aNewYPosition = aMousePoint.y-mPositionControlRect.size.height*.5;
 mPositionControlRect.origin = NSMakePoint(aNewXPosition, aNewYPosition);

 [self setPositionX:aNewXPosition];
 [self setPositionY:aNewYPosition];
 [self setNeedsDisplay:YES];

 return;
}

Mouse Up

The mouse up simply sets the control state to be inactive and tells itself to redraw.

Key Down and Key Up

The key down event handler looks for arrow key presses. When an arrow key is pressed, it adjusts the origin of the position rectangle a few points in the appropriate direction. It also sets the current control state so that the position rectangle is active and redraws. Here’s an excerpt from the method. It uses a switch statement to find the arrow keys:

case NSUpArrowFunctionKey:
   mControlState = kKDTwirlDistortionControlState_PositionControlActive;
   mPositionControlRect.origin.y = mPositionControlRect.origin.y+5;
   [self setPositionY:mPositionControlRect.origin.y];
   [self setNeedsDisplay:YES];
break;

The key up method also checks for arrow keys. If the event is from one of the arrow keys, it resets the control state to inactive and redraws.

Event handling was the last thing on our to-do list. Scratch it off, we’re done. We have a control.

Hi Asimo! Nice Hair-do!

The KDTwirlDistortionControl example is very specific to the kind of data it is controlling and it could be used with any Core Image filter that has the same attributes, despite its name.

In a future post, I will explore ways to abstract the design a little more to make the control more useful in more situations. This will be necessary for designing OpenGL controls and could be useful if you’re using CALayers as the basis for a control. I haven’t had a chance to work with CALayers, but I notice that they’re not descendants of NSResponder. Odd for such a view-like class.

Resources

The Example Project

Cocoa Event handling Guide

The Responder Chain

Cocoa drawing guide

Quartz2D Programming Guide

NSView - Working With the View Hierarchy

Cocoa Bindings

More advanced bindings examples: mmalc’s bindings examples, look at graphics bindings

I’m still using IB2 at work, but we’ll be making the move to Leopard over the next few months. I feel a slight pang of pre-nostalgia when I think about trading in Interface Builder 2 and its historical code base for its shiny new successor, Interface Builder 3. I know that IB3 is going to be a million times more usable and that it was past time for a real change in the world of IB, but those who know me know that I tend to pine for the old before embracing the new. I’ll pine now and get it over with.

I remember the exact moment when I realized how amazingly cool Interface Builder is. I was making a custom palette for some GUI classes, implementing the drawing and mouse event handling methods for when they would be running in IB. While I was typing, it struck me that what I was doing was a little strange. I was giving these classes a second, meta life only to be invoked by Interface Builder. All of the views and controls in Cocoa have this second IB life. When we drop them into a window, they draw preview versions of themselves and sometimes even respond to our mouse and key events (if we poke around in just the right spots). Interface Builder takes over from there. It guides us as we drag the views around, adjust their sizes and much more. Wow, what a clever and useful application. Who thought of this? They’re a genius.

Jean-Marie Hullot and his Interface Builder

Interface Builder was developed by Jean-Marie Hullot, pictured above during his days at NeXT. There is little written about him or his invention on the internet except that Hullot wrote the original version in some Lispy language and that it was originally called ‘SOS Interface’. Steve Jobs loved it, kidnapped Hullot and Tim Berners-Lee used it to make the first web browser, WorldWideWeb. Now we all use Interface Builder.

While I was scouring the net for information about Hullot, I came across a PDF excerpt from the book “How the Web was Born: The Story of the World Wide Web” by James Gillies and Robert Cailliau. There is a short section about Hullot and Interface Builder, titled “Monsieur Hullot’s holiday’. This is my favorite bit (emphasis is mine):

Hullot was using a Macintosh, and like many developers, he rapidly found that although the Mac’s GUI might have been brilliant, it was a pig of a machine to write applications on. ‘When the Macintosh was invented,’ he explains, ’suddenly, compared to what you had to do before, you had to add at least 60 or 70 percent of your time to make the user interface.’ Before the Macintosh, the interface was pretty basic-you would just type a line of text and the computer would respond with successive lines scrolling up the screen-but that meant that programmers didn’t have to worry about it. All the windows and menus that came with the Mac were great for the people who used the end applications, but for the people who had to write them they were a nightmare. Each time they wanted a menu or a window in their application, they had to program it in from scratch. Hullot’s big idea was to turn the problem into a simple question of drawing lines on the screen. He made a palette of objects for things like menu items and windows. Then to build the outline of an application you would simply take objects from the palette and draw lines between them. So, for example, you could pick up a menu object, and then by drawing lines from the other objects like ‘Open’ or ‘Print’, you would put those functions in the menu.

Hullot called his invention Interface Builder. It was a powerful tool that, like Smalltalk, put the power of objects at the developer’s disposal.

Kind of amazing, huh? These days, it’s easy to take things like windows and menus for granted. They come free with our development tools. On top of that, the responder chain and target/action mechanisms make it simple to dynamically link buttons and menu items to actions. If your app has a simple interface, there’s very little code that needs to be written for UI development. Even if your interface is complex, much of the burden is lifted by Hullot’s technology. Thank you, Jean-Marie.

The book goes on to describe Hullot and Jobs’ first meeting and the rest is history. In case you haven’t seen it, here’s a video of Steve Jobs presenting some applications on NeXTSTEP. He shows Interface Builder at 23:40. Funny how little has changed until now.

So, I’ll say goodbye to Inteface Builder 2 and end my pining now. Bye bye, IB2. Your interface was really strange and awkward but you were really cool and useful. I’ll miss palettes (not really, actually).

NSTableView is a great class for displaying long lists of data, but its default implementation is kind of boring looking. This post will show you how give your table views extra sex appeal. I’m categorizing it as a “don’t try this at home” tutorial because it sometimes involves overriding private NSTableView methods. Be warned that Apple can and will change the private framework code without notice and that some of the techniques used in this tutorial may no longer work. The tutorial and the example project are meant to be educational and exploratory. They’re not necessarily “right”, so use them wisely.

If anyone out there knows how to get comparable results without using NSTableView private methods, please please leave a comment.

The Example Project

The CustomTableView Xcode project can be dowloaded here.

If you’re new to Cocoa or subclassing NSTableView, the tutorial will be most useful if you download the project and follow along in the source code. The comments will help explain implementation details. The project has 3 classes:

• MyController (the application’s main controller and the table’s delegate and data source)
• MyTableView (the NSTableView subclass)
• MyCell (the NSCell subclass)

It also adds categories to NSColor and NSBezierPath for custom alternating background colors and rounded rectangles.

In the sample project, customizations are made to the following visual properties of the table:

• Alternating background colors
• Selection highlight
• Drag image
• Drop highlight on rows
• Drop highlight between rows

The table also uses an NSCell subclass to draw an icon and two rows of text. Let’s start with NSCell.

NSCell

Table views use the NSCell class to draw content in their rows. The first step to customizing the look of your table is to create an NSCell subclass and override the method

-(void)drawInteriorWithFrame:(NSRect)theFrame inView:(NSView *)theView

In the example project, the MyCell class divides the NSRect passed in the frame argument into three smaller rectangles that it uses to position the image, title and subtitle. The image and text are then drawn into those rectangles using NSImage and NSString’s respective -(void)drawInRect methods.

After you make your subclass, you need to let the table view know about it. You can set the cell for the columns of the table using NSTableColumn’s method

-(void)setDataCell:(NSCell *)theCell

In the example project, the MyController class sets the data cell for the table’s column with the following lines of code in awakeFromNib:

MyCell * aCustomCell = [[[MyCell alloc] init] autorelease];
[[oMyTableView tableColumnWithIdentifier:@"theTableColumn"] setDataCell:aCustomCell];

NSTableView

After you have an NSCell subclass, you may also want to customize some of NSTableView’s visual properties. You can set the background color, row height, whether or not the table should draw horizontal or vertical lines and the color of the lines using the NSTableView inspector palette in Interface Builder. Other adjustments take a little bit of code.

Alternating Background Colors

If you choose to display alternating background colors, the default colors are light blue and white. A simple way to customize the alternating background color scheme in your application is to add a category to NSColor that overrides the method

+(NSArray *)controlAlternatingRowBackgroundColors 

In your implementation of this method, return an NSArray containing the two colors you want your tables to use.

Selection Highlight

In the example project, MyTableView draws a rounded rectangle in the selected rows. If you want to draw your own selection highlight, override the following method in your NSTableView subclass:

-(void)highlightSelectionInClipRect:(NSRect)theClipRect

This method is asking you to draw the highlights for the rows that are currently selected within the “clip rect”, or the area of the table that is visible. One way to do this is to:

1. Use NSTableView’s method

   -(NSRange)rowsInRect:(NSRect)theRect;

   to get the range of row indexes that are currently visible in the clip rect.

2. Use NSTableView’s method

   -(NSIndexSet)selectedRowIndexes;

   to get a list of all of the selected rows in the table.

3. Go through the list of visible row indexes. If an index is included in the list of selected indexes, get the NSRect for that row with the NSTableView method:

   -(NSRect)rectOfRow:(int)theRowIndex;

4. Draw your custom highlight style inside this rectangle.

If your application is running on Tiger or any previous OS and you only override this method, you’ll notice that the table still draws its default highlight on top of your drawing. The second thing you must do to draw a custom selection highlight in this case is override the private method:

-(id)_highlightColorForCell:(NSCell *)cell
{
    return nil;
}

Returning nil in this method will “stop” the table view from drawing the default highlight rectangle (weird, huh?).

If you only want to change the highlight selection color instead of drawing a custom highlight, you can simply override the method

-(id)_highlightColorForCell:(NSCell *)cell

to return the color of your choice.

Drag Image

If your NSTableView subclass supports dragging, you can create a custom drag image by overriding the method

- (NSImage *)dragImageForRowsWithIndexes:(NSIndexSet *)theRowIndexes
			    tableColumns:(NSArray *)theTableColumns
				   event:(NSEvent *)theEvent
				  offset:(NSPointPointer)theOffset

In this method make an NSImage, draw what you want into the image and return it. You can get your NSTableView subclass to give you an NSImage with the content of the cell drawn inside by calling super’s implementation of the method.

In the example project, MyTableView composites the NSImage returned by super onto another NSImage with a rounded rectangle drawn inside. The new NSImage is returned as the drag image.

Drop Highlights

Drop highlights in NSTableView are notoriously fugly and they are not “legally” customizable. In the example project, MyTableView overrides some private methods to draw a black rounded rectangle to highlight drags over rows and a line with a small accent circle (like NSOutlineView) to highlight drags between rows.

For Tiger and earlier, there are two private methods that your NSTableView subclass can override to draw custom drop highlights on and between rows:

-(void)_drawDropHighlightOnRow:(int)theRowIndex;
-(void)_drawDropHighlightBetweenUpperRow:(int)theUpperRowIndex
			     andLowerRow:(int)theLowerRowIndex
				atOffset:(float)theOffset; 

In both cases, use the NSTableView method

-(NSRect)rectOfRow:(int)theRowIndex

to retrieve the rectangle of the row you want to highlight.

In Leopard, things get a little more complicated. There are several private methods that NSTableView might use to draw the drop highlight between rows. Through trial and error, I found that overriding the following method does the trick:

- (void)_drawDropHighlightBetweenUpperRow:(int)theUpperRowIndex
			      andLowerRow:(int)theLowerRowIndex
				    onRow:(int)theRow
				 atOffset:(float)theOffset

A new NSTableView feature in Leopard is the drop highlight color. Like the rest of the drop highlight visual properties, it is only customizable by overriding a private method. If you don’t override this method, the default implementation will draw a light blue background over the row that is the current drop target. If light blue isn’t your thing, you can override

+(id)_dropHighlightBackgroundColor;

to return the color of your choice. If you want your table to have the same highlights running in Leopard as it does in Tiger, return [NSColor clearColor].

Do Not Try This At Home!!!

I hope that this post will help you build a beuatiful table view that your users will love to look at, but once again, please please be careful when using private NSTableView methods for selection and drop highlights. While it’s fun and educational to explore the AppKit framework, you absolutely should not override private methods in your shipping app. I’ve filed an enhancement radar in the hopes that Apple will make these methods public:

rdar://problem/5758731

If anyone knows of other techniques that are more legal, please share them in the comments.

Resources

The source code in the CustomTableView example project has detailed comments that should help answer any questions you may have about implementing the techniques described in this post.

Other useful projects with alternate approaches to customizing the visual elements of a table:

• Mark Alldritt’s SourceListView
• Cocoa Pirate’s Source List implementation
• Apple’s ImageAndTextCell example code

If you are new to Cocoa drawing, I’d suggest reading the Quartz 2D Programming Guide as well the Learn Quartz tutorials at CocoaDevCentral.

For more information on NSTableView, start with the Introduction to Table Views Programming Guide. There is an NSTableView tutorial on the CocoaDev wiki as well as one at CocoaDevCentral. These will help with the less superficial parts of programming an NSTableView (like how to really implement drag and drop and data sources).

For more information on NSCell, read the Control and Cell Programming Topics for Cocoa.

If NSTableView isn’t exactly what you’re looking for and you’re targeting OS 10.5+, you might want to look into the new Cocoa class, NSCollectionView.

Also, If you’re targeting OS 10.5+, take a look at the new source list selection style for NSTableView.

Finally, class-dump lets you look at stuff you shouldn’t be looking at.

Edward Tufte recently posted a video analysis of the iPhone interface on his website. If you haven’t seen it yet, I highly recommend taking a look.

My favorite part is when he offhandedly states, “The white frames are too thick and should be in gray and only a single pixel thick,” as he’s scrolling the thumbnail browser. I imagine Edward Tufte working as a design supervisor at Apple, watching over people’s shoulders, scolding young designers for not displaying enough information. I’m sure he would run a tight ship where every pixel would have a meaningful purpose.

Actually, I lied, my real favorite part is a term Tufte uses frequently in the video, “computer administrative debris”, usually as he’s noting the lack of it on the iPhone’s UI. That’s a great way to describe all the crap on our interfaces, the important stuff that enables users to perform complex tasks in a tiny two-dimensional workspace. Things like windows, menus, tabs, check boxes, toolbars, scroll bars, etc.

One of the challenges of interface design is to reduce computer administrative debris while maintaining a high standard of usability. Of course, the difficulty of this challenge is related to the complexity of the application. Pro apps are going to have more computer administrative debris than consumer apps simply because they have more features. The iPhone and Apple TV-type applications have it easy in this sense, because their users are mostly browsing content. The developers don’t often need to provide them with a context to adjust or create the content. Most desktop apps, on the other hand, are all about providing that context. Still, there are some things that desktop application developers can do to trim the amount of debris on their interfaces.

Top: Flying Meat’s consumer image editor, Acorn

Bottom: Adobe’s pro image editor, Photoshop

Features

As Jeff Atwood points out in his post, Every User Lies, users don’t actually use most of the features of the applications they buy. They want to, and might even tell you that they do, but they usually don’t. Jeff’s advice is good. Observe. Figure out what is critical for your users to get their work done and go from there. Also, don’t be afraid to say no to feature requests if you’re in the position to do so. Keeping the feature set streamlined is a simple way to reduce the amount of computer administrative debris your users will have to wade through to get a simple task done.

A good example of this is Flying Meat’s consumer image editor, Acorn. Pictured above are both Acorn and its comparable pro application, Adobe Photoshop. They’re both shown at their initial states, just after I opened an image. Acorn is clearly more approachable because there is less debris on the screen. Its feature set is not nearly as rich a Photoshop’s, but Acorn’s developer, Gus Mueller, knew that his users were not going to use all of those features anyway. He made a succinct feature list and because of that he is able to minimize computer administrative debris on his interface.

Organization

Pro and prosumer apps, like Apple’s Aperture, are in a constant state of struggle with computer administrative debris. They are the applications with the most features, but with some organization, even they manage improve their computer administrative debris to content ratio over time.

Top: Aperture 2
Bottom: Aperture 1.5.6

Pictured above are Apple’s Aperture 2, which came out this week, and its predecessor, Aperture 1.5.6. The Aperture 2 team did some re-organizing for this release. The most notable organizational improvement is the shifting of the Adjustments and Metadata widgets over to the source list area. The arrangement that we see in 1.5.6 is no longer possible. Users will never see the source list, adjustments, and metadata debris at the same time. The Aperture team understood that the users will never need to access more than one of these features at a time, so they made the decision to consolidate. Using tabs as an organizational tool, they reduced debris.

Directness

Tufte makes an important observation about the iPhone when he talks about computer administrative debris. For many features, the iPhone uses the content as the interface and users interact with it directly rather than through interface widgets. Again, the fact that users of the iPhone are mostly browsing makes it easier to do this, but there’s something to be learned from their model.

A simple example of letting the users interact directly with content to trim debris is drag and drop, something we’re all familiar with by now. By letting users either pick up items from one container and drag them into another or reorder items in a single container, we reduce the need for buttons or action menus. Less debris. Apple’s iMovie 08 boasts, “Drag-and-drop moviemaking”. In fact, Apple’s entire iLife and iWork repertoire is full of examples of interacting directly with content instead of through menus, buttons or text fields, greatly reducing the amount of computer administrative debris on their interfaces.

Apple’s Keynote

Keynote, pictured above, has a very clean interface with little debris because users interact directly with the interface representations of the content. Can your interaction model be more direct? When could users interact directly with content instead of through text fields or action menus? This is a tough one and often requires some creative coding, but its worth it.

Free of Debris (not really)

It’s not possible to completely eliminate widgets from a desktop application, but we have the tools we need to reduce the amount of unsightly and distracting computer administrative debris on our interface designs. With the introduction of gesture events and CALayers in Cocoa, the Mac developer’s toolbox is becoming more sophisticated and we have lots of room to be creative and innovative.

Users are also becoming more sophisticated and their expectations are evolving. It won’t be long until the mere sight of a pop up menu will be enough to make them quit an application and never open it again. Trust that they can handle an unorthodox interaction model. They might be liars, as Jeff Atwood points out, but they learn quick if they’re presented with a well thought out and executed design. The iPhone interface proves that and as Tufte notes over and over in his video, there is often no computer administrative debris in sight.