The iStuff toolkit of physical devices, and the flexible software infrastructure to support it, were designed to simplify the exploration of novel interaction techniques in the post-desktop era of multiple users, devices, systems and applications collabora

Ft. Lauderdale, Florida, USA April 5-10, 2003 Paper: Between U and I

iStuff: A Physical User Interface Toolkitfor Ubiquitous Computing Environments

Rafael Ballagas, Meredith Ringel, Maureen Stone1, Jan Borchers

Computer Science Department

Stanford UniversityStanford, CA 94305, USA

ballagas@stanford.edu, {merrie, borchers}@cs.stanford.edu, stone@stonesc.com

ABSTRACT

The iStuff toolkit of physical devices, and the flexiblesoftware infrastructure to support it, were designed to sim-plify the exploration of novel interaction techniques in thepost-desktop era of multiple users, devices, systems andapplications collaborating in an interactive environment.The toolkit leverages an existing interactive workspace in-frastructure, making it lightweight and platform independ-ent. The supporting software framework includes a dynami-cally configurable intermediary to simplify the mapping ofdevices to applications. We describe the iStuff architectureand provide several examples of iStuff, organized into adesign space of ubiquitous computing interactioncomponents. The main contribution is a physical toolkitfor distributed, heterogeneous environments with run-timeretargetable device data flow. We conclude with some in-sights and experiences derived from using this toolkit andframework to prototype experimental interaction techniquesfor ubiquitous computing environments.

Keywords: User interface toolkits, ubiquitous comput-ing, tangible user interfaces, input and interaction tech-nologies, wireless devices, development tools, prototyping,programming environments, intermediation.

INTRODUCTION

While the mouse and keyboard have emerged as the pre-dominant input devices for desktop computers, user inputin ubiquitous computing (ubicomp) environments [19]presents a different set of challenges. A desktop environ-ment is targeted for one user, one set of hardware, and asingle point of focus. In a post-desktop, ubicomp environ-ment, complexity is added in every direction; there aremultiple displays, multiple input devices, multiple sys-tems, multiple applications, and multiple concurrent users.The iStuff toolkit was designed to support user interfaceprototyping in ubiquitous computing environments. Ourdomain is explicit interaction [1] with a room-sized envi-ronment consisting of displays of many sizes, plus supportfor wireless technology of various types, integrated using acommon middleware. Our goal is to allow multiple, co-Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and thatcopies bear this notice and the full citation on the first page. To copyotherwise, or republish, to post on servers or redistribute to lists, requiresprior specific permission and/or a fee.

CHI 2003, April 5-10, 2003, Ft. Lauderdale, Florida, USA.Copyright 2003 ACM 1-58113-630-7/03/0004…$5.00.

located users to fluidly interact with any of the displaysand applications in augmented environments such as theStanford iRoom, using for input and output any devicesconveniently at hand.

The toolkit was designed on top of iROS, a TCP- and Java-based middleware that allows multiple machines and appli-cations to exchange information [11]. iROS supportscommunication through the Event Heap, a central serverprocess that receives events from client applications in theroom and redistributes them to the appropriate recipients.The machines in the iRoom run standard operating systemsand applications, rather than custom systems designed ex-clusively for the environment. Applications developed forthe iRoom typically consist of suites of programs thatcombine their own UIs with interaction linked through theiROS. This approach allows for incremental deployment ofcomplex systems, such as those developed for constructionmanagement [6]. However, it exposes a fundamental as-sumption of such operating systems—that each displaycomes with its own dedicated pointing device and key-board.

The iStuff toolkit combines lightweight wireless input andoutput devices, such as buttons, sliders, wands, speakers,buzzers, microphones, etc., with their respective softwareproxies running on a machine in the iRoom in order tocreate iStuff components. Each component can be dynami-cally mapped to different applications running in theiRoom through a software intermediary called the Patch-Panel.

This framework allows HCI researchers to quickly proto-type a non-standard physical user interface and run experi-ments using it without having to run wires, solder upcomponents, or write yet another serial device driver. Eventcommunication takes only a few lines of platform-inde-pendent Java code, making it easy for applications to be-come iStuff-enabled.

It should be emphasized that this paper discusses a physicaltoolkit, aimed at rapidly prototyping and building physicaldevices. This is fundamentally different from a GUItoolkit, which is aimed at rapidly prototyping and buildinggraphical applications. For example, GUI toolkits have a1

StoneSoup Consulting, 191 Pine Lane, Los Altos, CA 94022

Volume No. 5, Issue No. 1 537