MIDP user interface development

A core feature of the MIDP (Mobile Information Device Profile) technology is its support for developing mobile phone user interfaces. The MIDP provides a set of Java APIs known as the LCDUI, which has functionalities similar to the Java Abstract Windows Toolkit (AWT) and Swing APIs in the desktop world. This article covers mobile phone UI development using the MIDP APIs. Our focus is the MIDP 2.0 API, which is standard on all Nokia Developer Platform 2.0 devices. At the time of writing (late 2004), Developer Platform 1.0 devices, which are based on MIDP 1.0, still have a large installed base. Hence, the key differences between the MIDP 2.0 and MIDP 1.0 APIs are also briefly covered throughout the discussion. The topics in this article are as follows:

• The design of the MIDP UI API: covers the overall design of the LCDUI API, including the distinction between the high-level and low-level APIs. We introduce the common display and UI event models for all LCDUI Screen and Canvas classes.
• The low-level API: covers the pixel-based API. We discuss how to draw on the Canvas and how to handle keypad events. A useful example illustrates how to add animation functionality to Canvas applications.

The design of the MIDP UI API

Existing Java developers can get started with the MIDP very quickly because it leverages some proven design patterns in standard Java UI APIs, such as the AWT and Swing. The MIDP LCDUI supports two programming models: the widget-based, high-level model and the pixel-based, low-level model.

The UI models

The LCDUI high- and low-level APIs satisfy the needs of different developers. The high-level API is very easy to learn and to use. It aims to support fast development cycles, and it is easier to write portable code with the high-level API. The low-level API provides raw power and flexibility to developers who need to control every aspect of the user experience. In this section, we look at those two APIs from a bird’s-eye view.

The high-level API

The design goal of the high-level API is to support programming applications that are portable across devices. It provides a set of standard UI classes representing basic UI components at a high abstraction level. Devices implement the components, and, therefore, an application programmed with high-level APIs adopts the native look and feel of that device. For example, the Series 40 and Series 60 devices often render the same widget differently according to their own UI guidelines. The high-level API is mostly suitable for business applications, which do not differentiate themselves with UI innovations. Such applications offer a “no-surprise” UI to users and have minimal learning curves. Their value comes from the content and functionalities behind the user interface.

However, the drawback with the high-level UI approach is also obvious: the developer has little control over the drawing details and cannot go beyond the predefined set of widget components. For instance, it would be hard to develop an animation screen using the high-level API alone.

The low-level API

The low-level API gives developers complete control of the entire device display, including drawing on any pixel, rendering basic shapes, and drawing text with specific fonts. The low-level API also supports richer user interactions than the high-level API, which only captures the soft-key events. The low-level API provides mechanisms for developers to handle all keypad key events and pointer movements. Custom rendering and event handling are crucial to game developers.

A low-level UI application needs to render itself rather than delegate the task to the runtime library. Hence, low-level API applications usually require much more code than the high-level ones. Porting low-level UI applications to different devices can sometimes be a tedious task. For instance, it is hard to implement a native look-and-feel text input box using pixel-level tools. In addition, a text input box implemented with a low-level API would not have the same performance as the ready-to-use high-level API component, since we now need to do all the font calculation and rendering on the Java level instead of the optimized native level. So, we should use the high-level components whenever possible and leave only the parts that require custom rendering to low-level APIs.

Architecture of the LCDUI

Figure 1 shows important classes and interfaces in the MIDP UI package. In this section, we focus on the API design of the four important classes: Display, Displayable, Command,, and CommandListener.

Display

The Display class provides access to the physical screen. To avoid complications in multithread applications, only one instance of the Display object is allowed for each MIDlet. For this reason, the Display class constructor is private, and the single instance policy (the Singleton pattern) is enforced by a factory method. We can obtain the instance of the Display class by passing the MIDlet object to the static Display.getDisplay() method.

 public static Display getDisplay (MIDlet m) 

The most important methods in the Display class display Displayable objects to the physical screen. The Displayable object represents a view of the screen. In MIDP, there is only one Displayable object visible at any given time. It is different from the Windows programming model in which the forms display themselves and more than one window can be visible on the same screen. Hence, the setCurrent() method is used to switch the view.

 public void setCurrent(Displayable d)
public void setCurrent(Alert a, Displayable d)
public void setCurrentItem(Item item)
public Displayable getCurrent()

The first method simply displays a Displayable instance, such as a List or a Form to the screen. The second method pops up an Alert note. Upon dismissal, the specified Displayable object is shown. The third method displays the Displayable object that contains the specified UI component item (available on MIDP 2.0 devices only). The Display class makes sure that the item is properly focused and scrolled to be visible.

The setCurrent() method call is asynchronous. That means the screen change may not take place immediately upon the application calling setCurrent(). The screen is only updated and rendered when the UI thread is idle. For example, if we call setCurrent() within a method running in the UI thread (e.g., a UI event callback method), the screen is not updated until the method returns. The getCurrent() method is hence a useful means of finding out whether the change has actually occurred.

Displayable

The Displayable class is an abstract representation of a full screen of display content. Despite being an abstract class, the Displayable does not contain any abstract method. It is abstract because the Displayable class does not contain any rendering logic and hence cannot be used directly. The abstract modifier and nonpublic constructor of the Displayable class force developers to use its concrete subclasses in the LCDUI, which do contain the rendering logic. We can query whether a Displayable object is shown on the LCD by calling its isShown() method. The Displayable class allows us to query the size of the screen display area. The returned values do not necessarily indicate the physical size of the LCD. Rather, it is the available screen area for the MIDlet, which is always smaller than the physical LCD size.

 public int getWidth()
public int getHeight()
public boolean isShown() 

The Displayable class also provides methods to add and manipulate the title and ticker of the display area. Figure 2 shows the title and ticker on Series 40 and Series 60 devices.

 public void setTitle(String title)
public String getTitle()
public void setTicker (Ticker ticker)
public Ticker getTicker () 

The most important functionality of the Displayable class is to provide the basic infrastructure for user interactions through the Observer design pattern. A key interface in the infrastructure is CommandListener, which has only one method declared.

 public interface CommandListener {
   void commandAction(Command command, Displayable
displayable);
} 

The developer provides an implementation of the commandAction() method to specify what to do when a specific command is invoked by the user. Figure 3 illustrates this pattern.

The Command and CommandListener classes work together with the Displayable as follows.

1. We can associate Command objects with a Displayable screen.
2. Each Command object is mapped to a visual element, such as a soft key, on a Displayable screen.
3. When the user presses the soft key, a UI event is generated against the current Displayable object.
4. The Displayable object calls the commandAction() method in the registered CommandListener object and passes in the Command object mapped to the soft key. The listener object changes the display to the next screen by calling Display.setCurrent() from within the commandAction() method.

The Command-related methods in the Displayable class are used in all the examples throughout this book. We can add a Command, remove a Command, or register a CommandListener in the Displayable object. The Command-related methods are as follows.

 public void addCommand(Command cmd)
public void removeCommand(Command cmd)
public void setCommandListener(CommandListener l) 

Command

When a Command object is added to a Displayable object, it is mapped to a soft key on the phone. If the Displayable object contains more than two commands, the menu label for one soft key automatically becomes “Options.” If the user presses that key, the rest of the commands are displayed in a menu. Figure 4 shows how the Command objects are mapped.

Notice that the Options menu automatically gets the native look and feel on different devices. For each command, the MIDP API provides a mechanism for developers to specify its functional nature and priority. Based on that information, the Java runtime decides how to map the Command object to soft keys or Options menus.

The Command class constructors are as follows.

 Command (String label, int cmdType, int priority)
Command (String shortLabel, String longLabel,
   int cmdType, int priority) 

Now, let’s examine the second version of the constructor, which contains all the optional arguments. The first argument in the constructor is a short text label for the Command. It is displayed when the Command is mapped to a soft key. The second argument is a long text label for the Command. It is displayed when the Command is shown in an expanded menu. On Nokia devices, the long text label is displayed in the expanded options menu only when it can fit into one line (typically three or four words). If it needs to be truncated, the short text label is used in the Options menu. The third argument is the type of the Command. It determines the mapping of the Command to soft keys. For example, on Nokia devices, the EXIT type Commands always map to a single soft key on the rightmost soft key or appear in the Options menu. The MIDP specification defines the following Command types: SCREEN, BACK, OK, CANCEL, HELP, STOP, EXIT, and ITEM. The last argument is the priority level of the Command. Higher priority corresponds to a smaller value here. Given the same type, a high priority Command is displayed in a more accessible position.

A sample application

In this article, we use a sample application to display various demo screens and demonstrate the usage of the MIDP UI APIs. The entry MIDlet class DriverMidlet holds the Display object and provides two shared commands for each of the 14 demonstration screens in the application. The Next command invokes the DriverMidlet.next() method to instantiate and display the next demonstration screen; the Exit command exits the application. The source code of the DriverMidlet is listed below. Notice that we add the shared commands to each of the demo screens.

 

public class DriverMidlet extends MIDlet {
      implements CommandListener {


static Display display;
   private static Displayable demo;
   // static variable keeps track of the demo state
   private static int index = -1;


Command exit = new Command ("Exit", Command.EXIT, 1);
   Command next = new Command ("Next", Command.OK, 1);


public DriverMidlet () {
      display = Display.getDisplay(this);
   }


public void commandAction (Command c, Displayable d) {
      if (c == exit) {
         exit ();
      } else if (c == next) {
         next ();
      }
   }


protected void startApp () {
      next ();
   }


protected void pauseApp () {
      // Do nothing
   }


protected void destroyApp (boolean unconditional) {
      notifyDestroyed ();
   }


public void next () {
      index++;
      if (index > 16) {
         index = 0;
      }


switch (index) {
         case 0: demo = new DeviceInfoDemo (this); break;
         case 1: demo = new AlertDemo (this); break;
         case 2: demo = new ListDemo (this); break;
         case 3: demo = new MenuDemo (this); break;
         case 4: demo = new TextBoxDemo (this); break;
         case 5: demo = new TextDateDemo (this); break;
         case 6: demo = new ChoiceGroupDemo (this); break;
         case 7: demo = new PopupDemo (this); break;
         case 8: demo = new GaugeDemo (this); break;
         case 9: demo = new StringImageDemo (this); break;
         case 10: demo = new LayoutDemo (this); break;
         case 11: demo = new ItemCommandDemo (this); break;
         case 12: demo = new CanvasDemo (this); break;
         case 13: demo = new AnimationDemo (this); break;
         case 14: demo = new ScrollCanvasDemo (this); break;
         case 15: demo = new WrapTextDemo (this); break;
         case 16: demo = new ImageButtonDemo (this); break;
     }
      // Add shared commands
      demo.addCommand (exit);
      demo.addCommand (next);
       display.setCurrent(demo);
   }
    public void exit () {
      destroyApp (true);
   }
}

 

The next listing shows how a generic demonstration class is structured. Notice that we pass the DriverMidlet reference to each demo class. In the demo class’s commandAction() method, we always delegate to DriverMidlet.commandAction() to reuse the handler logic for the shared Next and Exit commands.

 

public class UIDemo extends Form
      implements CommandListener {
   // declare commands that are specific to this demo
   // ...


// Handler for shared commands ("next" and "exit")
   private CommandListener comm;


public UIDemo (CommandListener c) {
      super ("Title");
      comm = c;


// instantiate and add demo specific commands
      this.setCommandListener(this);
   }


public void commandAction (Command c, Displayable d) {
      // Handle demo specific commands ...


// delegate "next" and exit" to DriverMidlet
      comm.commandAction(c, d);
   }


// Other methods ...
}

 

The low-level API

The low-level UI API consists of four classes, the Canvas class, the Graphics class, the Image class, and the Font class. Additional classes to manipulate the low-level drawing are available in the MIDP Game API. Unlike widgets in the high-level API, which are drawn by the Java runtime, the Canvas object draws itself on the screen. It works as follows.

• The Canvas abstract class defines an abstract method paint(Graphics g).
• Each concrete Canvas subclass must implement the paint() method to specify how to draw itself on the screen.
• When a Canvas object is displayed, the paint() method is automatically invoked. The Graphics object is passed in by the Java runtime. Developers do not invoke the paint() method directly.
• When the screen needs updating, the application calls repaint(), which schedules the runtime to call the paint() method. Note that the repaint() method is asynchronous. The paint() method might not be called immediately after the repaint() method returns. In addition, the Canvas class provides a second version of the repaint() method, which takes in four integer arguments (x, y, width, height), to repaint only the specified rectangle region of the screen.
• The serviceRepaints() method forces the runtime to service all pending repaint() requests. We have to use it with caution, since it compromises the automaticity built into the platform and could cause deadlock. For example, we should never call serviceRepaints() from within a paint() method.

Inside the paint() method, all the actual drawing work is done with methods in the Graphics object.

Graphics

The Graphics class provides methods to do the actual drawing on a Canvas. It is usually passed into the Canvas.paint() method. We never call the paint() method or instantiate the Graphics object in our applications. They are jobs for the Java runtime. The following methods in the Graphics class set the drawing mode such as the current color, font, and stroke style. Two strokestyle constants are defined in the Graphics class: SOLID and DOTTED. The corresponding get methods are also available.

 public void setColor(int red, int green, int blue)
public void setColor(int RGB)
public void setGrayScale(int value)
public void setStrokeStyle(int style)
public void setFont(Font font) 

The drawing methods can draw the following objects and shapes at the specified location.

• Lines
• Outline of geometric objects such as arcs and rectangles
• Filled geometric objects
• Text strings
• Images

 

public void drawChar(char c, int x, int y, int anchor)
public void drawChars(char[] data, int x, int y, int anchor)
public void drawString(string str, int x, int y, int anchor)
public void drawSubstring(String str, int offset, int len,
         int x, int y, int anchor)
public void drawImage(Image img, int x, int y, int anchor)


public void drawLine(int x1, int y1, int x2, int y2)
public void drawArc(int x, int y, int width, int height,
         int startAngle, int arcAngle)
public void fillArc(int x, int y, int width, int height,
         int startAngle, int arcAngle)
public void drawRect(int x, int y, int width, int height)
public void fillRect(int x, int y, int width, int height)
public void drawRoundRect(int x, int y, int width,
         int height, int arcWidth, int arcHeight)
public void fillRoundRect(int x, int y, int width,
         int height, int arcWidth, int arcHeight)
public void fillTriangle(int x1, int y1,
         int x2, int y2, int x3, int y3)

 

The text- and image-drawing method takes in an anchor argument, which specifies where the object should be drawn relative to the specified location. The available anchor values are static final fields defined in the Graphics class. The available anchors are listed in Table 1.

Table 1. Static constants for anchors

We can limit the effect of any rendering method by specifying a rectangular clipping area via the setClip(int x, int y, int width, int height) method. Only the screen area within the clip would be affected by any subsequent rendering method.

Fonts

The MIDP specification defines a Font class, which represents available text fonts we can use in the StringItem component or the Graphics.drawString() method. The most important method is the getFont() method.

 Font getFont (int face, int style, int size)
Font getFont (int fontSpecifier)

In the first version of the getFont() method, the available face values are FACE_MONOSPACE, FACE_PROPORTIONAL, and FACE_SYSTEM; the available style values are STYLE_PLAIN, STYLE_BOLD, STYLE_ITALIC, and STYLE_UNDERLINED; the available size values are SIZE_SMALL, SIZE_MEDIUM, and SIZE_LARGE. Considerable design effort has been made to make sure that Nokia fonts are easy to read on Series 40 devices. All Series 40 fonts are proportional. In the second version of the getFont() method, the fontSpecifier takes either of the following two values:

• FONT_INPUT_TEXT: the returned font must be the same as used in TextField in Form
• FONT_STATIC_TEXT: the return font must be the same as used in StringItem body (not in the label) in Form

In addition, the Font class provides methods, such as getHeight() and getWidth(), to measure the width and height of any given string in any supported font. Those methods are very handy when we need to manually control text layout on a Canvas.

Key-event model

A concrete Canvas subclass not only draws itself on the screen but also handles events from the keypad and touch screen. For example, when the user presses a key, the MIDP runtime calls the Canvas.keyPressed() method and passes the key code. By default, the keyPressed() method does nothing. It just ignores the event. The Canvas subclass overrides it to respond to the event. The Canvas class has multiple event-handler methods mapped to different types of events. The following key-event handlers are called when the user presses, releases, or holds a key.

 public void keyPressed(int keyCode)
public void keyReleased(int keyCode)
public void keyRepeated(int keyCode) 

The keyCode argument passed into the handler methods is an integer value representing the event key. For simple keys, the keyCode value corresponds directly to static final fields in the Canvas class. Nokia devices may deliver key codes other than the MIDP-defined ones to the MIDlet. Some Nokia devices (e.g., Nokia 6800) support full keyboards and can deliver a large number of key codes. All MIDP standard and Nokia-specific key codes are listed in Table 2.

Table 2. MIDP standard and Nokia-specific key codes

For game action keys, the keyCode value must be mapped to game action constants via the getGameAction() method. The mapping is necessary because multiple keys might correspond to the same game action. For example, the numeric key 4 and the scroll-left key are often both mapped to the move-left action in a game. For more portable applications, you are strongly encouraged to use game action values rather than raw key codes whenever possible. Table 3 lists the game action constants in the Canvas class. Figure 5 shows those keys on a device.

Table 3. Static constants in the Canvas class for game action code mapped by the getGameAction() method (on an ITU-T numeric keypad)

For devices with touch screen and pointer (e.g., the Nokia 7710 in Series 90), the MIDP runtime can also capture pointer events when the user presses, releases, or drags the pointer across the screen. The arguments passed into these methods are the pixel coordinates of the pointer position on the screen.

 public void pointerPressed(int x, int y)
public void pointerReleased(int x, int y)
public void pointerDragged(int x, int y) 

Canvas in action

In this section, we demonstrate the use of the Canvas class via three examples.

The movable text

The following example shows how to draw an image on a Canvas and a string of text on top of it. When the user presses the scroll keys, the key-event handler changes the location of the string accordingly and calls the repaint() method to update the display. Hence, we appear to move the string around on the screen using scroll keys. Figure 6 shows the application in action.

 

public class CanvasDemo extends Canvas
         implements CommandListener {


private CommandListener comm;
   private Image img;
   private Font font;
   public int currentPosX, currentPosY;
   public int width, height;


public CanvasDemo (CommandListener c) {
      comm = c;
      width = getWidth ();
      height = getHeight ();
      // The current cursor position for the text
      currentPosX = width / 2;
      currentPosY = height / 2;
      // The text font
      font = Font.getFont(Font.FACE_PROPORTIONAL,
         Font.STYLE_BOLD, Font.SIZE_LARGE);


// setFullScreenMode (true);
      setCommandListener (this);


try {
         img = Image.createImage("/" + "telescope.png");
      } catch (Exception e) {
         e.printStackTrace ();
         img = null;
   }
}


public void commandAction (Command c, Displayable d) {
   comm.commandAction(c, d);
}
 public void paint (Graphics g) {
   // Fill a white background
   g.setColor(0xffffff);
   g.fillRect(0, 0, width, height);
   // Set text color
   g.setColor(0x000000);
   g.drawImage (img, width / 2, height / 2,
      Graphics.HCENTER | Graphics.VCENTER);
   g.setFont (font);
   g.setColor (255, 0, 0);
   g.drawString ("Telescope", currentPosX, currentPosY,
      Graphics.BASELINE | Graphics.HCENTER);
   }
    // Move the text around when the game keys are pressed
   public void keyPressed (int keyCode) {
      int gameCode = getGameAction (keyCode);
      if (gameCode == UP) {
         currentPosY -= 10;
         repaint ();
      } else if (gameCode == DOWN) {
         currentPosY += 10;
         repaint ();
      } else if (gameCode == LEFT) {
         currentPosX -= 10;
         repaint ();
      } else if (gameCode == RIGHT) {
         currentPosX += 10;
         repaint ();
      } else {
         super.keyPressed(keyCode);
      }
   }
}

 

We instantiate the font object and retrieve screen width and height values from outside the paint() method. Since the paint() method is called every time the screen refreshes, we should avoid expensive method calls and/or object instantiation/garbage collection in paint().

Animation

The above Canvas example can be expanded to illustrate the basic technique of animation. In the code below, the handler for the Start command starts a new thread. The thread keeps updating the position of the text and calling the repaint() method. That causes the text to scroll up and down on the screen automatically.

 

public class AnimationDemo extends Canvas
      implements CommandListener {
   private Command start, stop;
   private Animator animator;
   private CommandListener comm;
   private Image img;
   private Font font;
   public int currentPosX, currentPosY;
   public int width, height;


public AnimationDemo (CommandListener c) {
      comm = c;
      width = getWidth ();
      height = getHeight ();
      // position of the text
      currentPosX = width / 2;
      currentPosY = 10;
      // text font
      font = Font.getFont(Font.FACE_SYSTEM,
         Font.STYLE_ITALIC, Font.SIZE_LARGE);
      animator = new Animator (this);


// setFullScreenMode (true);
      start = new Command ("Start", Command.SCREEN, 2);
      stop = new Command ("Stop", Command.SCREEN, 2);


addCommand (start);
      addCommand (stop);
      setCommandListener (this);


try {
         img = Image.createImage("/" + "telescope.png");
      } catch (Exception e) {
         e.printStackTrace ();
         img = null;
      }
   }


public void commandAction (Command c, Displayable d) {
      if (c == start) {
         Thread t = new Thread (animator);
         t.start ();
      } else if (c == stop) {
         animator.stopped = true;
      }
      comm.commandAction(c, d);
   }


public void paint (Graphics g) {
      g.setColor(0xffffff);
      g.fillRect(0, 0, width, height);
   
      g.setColor(0x000000);
      g.drawImage (img, width / 2, height / 2,
         Graphics.HCENTER | Graphics.VCENTER);
      g.setFont (font);
      g.setColor (255, 0, 0);
      g.drawString ("Telescope", currentPosX, currentPosY,
         Graphics.BASELINE | Graphics.HCENTER);
      }
   }


class Animator implements Runnable {


AnimationDemo demo;
      public boolean stopped;


public Animator (AnimationDemo demo) {
         this.demo = demo;
         stopped = false;
      }


public void run () {
         while (!stopped) {
         demo.currentPosY += 5;
         if (demo.currentPosY > demo.height) {
            demo.currentPosY = 0;
         }
         demo.repaint ();
      }
   }
}

 

Figure 7 shows the animated example in action.

Summary

MIDP offers an impressive set of rich APIs to build GUI applications on mobile phones. In this article, we focused on the MIDP 2.0 API, which is standard on Nokia Developer Platform 2.0 devices.

The low-level API gives the developer the ultimate control of the screen area. It is great for mobile games and other applications that need custom UIs.

Michael Juntao Yuan is a researcher, developer, author, and advocate for Java and open source technologies. He is a research associate at the Center for Research in Electronic Commerce at the University of Texas at Austin and writes for JavaWorld. Yuan won the 2002 Grand Prize in Nextel, Sun, and Motorola’s national J2ME application contest. Currently, he leads an effort in BuzzPhone.com to develop a series of official blueprint applications for Nokia. Kevin Sharp is senior technical editor and columnist for Supply Chain Systems magazine and a technical editor with Forum Nokia. He is a registered professional engineer with experience in circuit design, optical symbol recognition, radio frequency identification systems, and hazardous environment process monitoring. He founded Accurate Information in 1989 and continues to manage its consulting practices in logistics, supply chain management, and mobile technologies.