Introduction to Surface Computing
Surface computing refers to computing technologies and interfaces that use surfaces like tables, walls or tabletops as display and interaction surfaces. Through the use of interactive displays, digital information and physical objects can be combined to allow natural multi-touch and tangible interactions. These new types of surfaces provide novel ways for people to access and interact with digital information in physical, social environments.
Surface computing builds upon previous interactive tabletop and multi-touch technologies by expanding the interactive surface to larger physical spaces like walls, tables or room-sized environments. Unlike traditional desktop computing which focuses interactions through a mouse and keyboard on a small monitor, surface computing leverages people’s innate abilities to interact physically and collaboratively with surfaces and objects in the real world. Some key attributes of surface computing include:
Large, high-resolution interactive display surfaces that can span entire tables, walls or rooms
Multi-touch sensing to track hands, fingers and objects touching the surface
Tracking physical objects placed on the surface to serve as interfaces or information containers
Support for multiple co-located users to interact simultaneously in a around-the-table or room-scale manner
Integration with other modalities like pen, gestures, projected objects or augmented reality overlays
Hosting of a wide range of applications from information browsing to digital gameplay to data visualization
Surface computing presents many opportunities for collaboration, information sharing, learning and games in public settings. Realizing its full potential will require overcoming technical challenges in sensing, displays, tracking and systems architecture as well as developing compelling application experiences. This paper will explore the origins and key concepts of surface computing, analyze important technical advancements and research directions, and consider applications and implications of this emerging paradigm.
Origins and Concepts
The roots of surface computing can be traced back to early interactive tabletop and digital whiteboard systems developed in research labs in the 1990s. Pioneer projects like DigitalDesk at Xerox PARC explored vertically-oriented graphical displays combined with natural input modalities like pens, physical objects and gestures. Around the same time, researchers at the MIT Media Lab began experimenting with direct touch interfaces on horizontal surfaces using cameras and computer vision for sensing.
Commercial interest in these interactive tabletop concepts increased in the 2000s as multi-touch technology advanced. Microsoft Research demonstrated several prototype tabletop systems like iTable, Mixer and Surface that employed infrared optics for multi-touch input. Meanwhile, startups like Planar Systems, Persuasive Displays and Dolphin helped accelerate the adoption of multi-touch electronic whiteboards.
A key milestone was the introduction of the first-generation Microsoft Surface tabletop device in 2007. Its integrated 3D camera sensing, large pixel-precise multi-touch surface and ability to track physical objects on top helped establish core elements of modern surface computing interfaces. Around the same time, researchers at Mitsubishi explored augmented reality overlays on tabletops to blend physical and digital content.
Several academic research platforms also influenced the direction of surface computing. The DiamondTouch tabletop system from MIT provided intuitive multi-user input through capacitive sensing of users’ bodies touching the edge of the table. The Reactable from Universitat Pompeu Fabra augmented a tangible tabletop musical interface with projected overlays. The iRoom at University of Illinois promoted room-scale interaction including walking, gesturing and object manipulation across large displays.
These projects demonstrated the value of natural, spatially-aware interactions across large horizontal or vertical surfaces. Users could directly touch, grasp, move and combine on-surface virtual elements just as they would with physical objects. Multiple co-located people were supported through transparent multi-touch sensing. Physical tokens or props could serve as interfaces or contain digital information.
Technical Advances Driving Adoption
Ongoing technical improvements are increasing the viability and adoption of surface computing technologies across various applications and domains. Key advances include wide format high-resolution displays, sophisticated tracking techniques, cheaper multi-touch sensing, seamless integration of augmented reality and growing computing power.
Displays: Larger 4K, 5K and 8K LCD, LED and OLED panels with pixel densities over 200 PPI are bringing surface computing interfaces to life at room scale. Multi-panel video walls enable seamless tabletops or wall spaces. New transparent and flexible display technologies promise even more diverse form factors.
Tracking: Sophisticated simultaneous localization and mapping (SLAM) algorithms along with infrared, depth and computer vision techniques power spatial tracking of touch points, objects and people throughout rooms with sub-millimeter accuracy. This enables extremely precise interactions across large areas.
Sensing: Affordable multi-touch sensing through projected capacitive, frustrated total internal reflection and acoustic wave techniques allow smooth differentiation of up to 256 touches without direct contact. Infrared and ultrasonic beacons permit tracking of everyday physical objects while preserving their physicality.
Augmentation: Advancing augmented and mixed reality now lets projected information blend seamlessly with physical surfaces and objects through dynamic spatial registration. Users interact with integrated digital and physical experiences in familiar, tangible ways.
Computing: Higher performance mobile chipsets, embedded systems and cloud computing resources distributed at the edge make very large, interactive surfaces feel responsive while supporting complex recognition algorithms and multimedia applications.
Together, these enabling technologies are expanding the scale, precision and affordability of surface computing interfaces, fueling adoption within industries like automotive design, scientific collaboration, data visualization, healthcare and more. Larger pixel densities enable higher content resolution while wider tracking areas support room-scale interactions across tables, walls or entire spaces. Integration of augmented reality further unites physical and digital worlds without compromising tactility or natural dynamics.
Research Directions
With its focus on horizontal surfaces serving as interactive canvases within physical spaces, surface computing fosters new styles of collaborative, tangible user experiences. Active areas of research aim to advance key technical aspects and better understand emerging interaction paradigms.
Multi-User Interactions: Studying how people fluidly share, transfer and combine digital resources while co-located encourages coordination, turn-taking and social learning. New interaction patterns also emerge across asymmetric shared and personal areas.
Embedded Tracking: Seamlessly embedding tracking into diverse surfaces including wood, plastic or fabric through techniques like ultrasonic signals, conductive meshes or capacitive arrays enhances mobility, aesthetics and naturalness of touch interfaces.
Spatial Interfaces: Interactions extend beyond horizontal surfaces to incorporate vertical panels, wearables, tabletop objects or entire room volumes. Gestures, body movements and spatialized audio/visualizations promote multi-dimensional experiences.
Tangible User Interfaces: Physical tokens, everyday objects and hybrid materials that bridge digital and physical worlds serve as controls, containers of information or mediators for collaborative activities while preserving tactile affordances.
Embedded Computing: Strategic placement of computational and sensing resources within surfaces, objects or spatially distributed throughout environments support responsive interactions across large physical spaces.
Applications and Implications
Initial applications demonstrate surface computing’s unique affordances for social learning, planning and visualization in fields like education, scientific collaboration, design, and information sharing. These applications also surface implications for interaction design principles, work practices and public spaces.
Education: Interactive whiteboards, large tabletops and spatial environments facilitate participatory learning through tangible manipulation of simulated models, maps, diagrams or historical timelines.
Scientific Collaboration: Wide visualization surfaces allow co-located researchers to explore, annotate and dynamically modify complex datasets together, fostering new insights. Wall-sized displays extend this to room-scale modeling and simulation.
Design: Large, multi-touch CAD and whiteboarding tools on tables or walls enhance idea generation, document sharing, iterative prototyping and review cycles within interdisciplinary teams.
Data Visualization: Dynamic graph databases, geospatial datasets or abstract information spaces come to life across tables and walls in ways that support casual observation, facilitated touring and linked contextual views.
Information Sharing: Large public displays and interactive spaces promote serendipitous interactions, wayfinding, notifications and collaborative sense-making of community information and notifications within transit hubs, libraries or museums.
Social norms, privacy considerations and changes to existing work practices will require exploration as surface computing increasingly shapes how people interact, learn and make decisions collaboratively within professional and social settings. Overall, marrying the digital and physical through large interactive surfaces presents many opportunities for innovative experiences and applications across an array of contexts.
Conclusion
Surface computing’s vision of integrating computing capabilities throughout physical spaces continues to take shape through ongoing hardware and software advances. Large interactive surfaces now support natural, co-located collaboration through intuitive multi-touch and tangible interactions spanning tables, walls and entire rooms. Sophisticated tracking and sensing techniques power incredibly precise multi-user experiences while augmented reality seamlessly combines physical and virtual worlds.
Surface computing’s unique blend of direct touch input, spatial interfaces and embedded computing opens up new opportunities for information visualization, scientific modeling, education and digital collaboration across workplaces, public spaces and homes. Key areas like multi-user surface interactions, embedded tracking techniques and spatial interfaces remain active areas for exploring emerging paradigms. Initial applications indicate immense potential for this interactive medium with many implications yet to be discovered. As enabling technologies mature further, surface computing will undoubtedly transform how people work, learn and socialize together with digital information in physical environments.
Andrew Stroud