Projects

A system that augments VR speech transcripts with textual descriptions of ambiguous references to objects (such as "it" or "there").

The paper proposes a computational method to score how tranquil, social, or explorative spaces feel by combining daylight, pathways, and visibility data. Tested on 20 apartments, it found people rate spaces uniformly while the algorithm detects variation, with stronger alignment among experienced participants.

Branched Narrative Fiction (BNF) are non-linear games where players shape the story. WhatIF is a tool that helps developers design, analyze, and refine branching plots, overcoming challenges like writer's block.

In 2023, at Autodesk Research we partnered with The Bentway in Toronto to use wearable sensors and cameras to capture how people experienced the public space under Toronto's Gardiner Expressway. Participants joined "walkshops", recording video, audio, surveys, and environmental data, while algorithms analyzed what they saw and visualized anonymized facial contours. The project shows how participatory, data-driven methods can help design public …

Creators can use WorldSmith, a multi-modal generative AI tool, to quickly design and refine fictional worlds via text, sketches, and region-based edits.

VR users can be watched without knowing, and they can also record bystanders. Vice VRsa alerts VR users when people are nearby and alerts bystanders if the VR user is recording.

Switching between solo and group work in remote settings is tricky. Peek-at-You combines collaboration and communication tools with features like position indicators, peeking at a speaker's view, and pushing views, helping teams stay aware, understand each other, and switch work modes smoothly.

AvatAR is an AR system for exploring human motion data in place, using 3D trajectories, posture-based avatars, and embedded visualizations of gaze and touch. It combines an AR headset and a tablet for both mid-air and touch control, making motion data interactive and immersive.

The Tactile Emoticon uses connected mitts to send touch-based messages through vibration, pressure, and temperature. Studies with 68 participants show how people shape and interpret these tactile signals based on emotion, context, and social norms, offering design insights for more expressive remote touch communication.

AirConstellations uses movable, self-stable armatures to hold and arrange multiple devices in mid-air, letting users flexibly position, combine, or separate them for different tasks. It supports dynamic layouts for collaboration, productivity, and remote work, enabling smooth transitions and context-aware interactions.

OccSim simulates occupant behaviors in 3D building models to assess virus transmission risks over time. Tested with multiple stakeholder groups, it helped identify specific hazards and informed decisions on safer building design, operation, and policy.

EvalMe is a playful tangible tool that helps students reflect on their learning during and after lessons.

SurfaceFleet is a cross-device toolkit that uses always-on-top Applets to let tools and content move freely across devices, apps, users, and time, with cloud-based resilience.

My PhD thesis on cross-device computing: maps the field with a taxonomy, prototypes interactions across devices (for curation and collaboration), and provides tools/methods to evaluate multi-device work.

Taxonomy that analyzes 510 cross-device computing papers to build a unifying taxonomy, chart trends, map tech and applications, synthesize interactions and evaluations, and highlight open issues to guide future research.

EagleView speeds collaboration analysis by pairing video with real-time proxemic visualizations and timeline queries, letting analysts quickly find and interpret key moments.

Modular hardware platform for linking multiple mobile devices into flexible cross-device workspaces. Includes 3D-printed link modules, design strategies, a visual configuration tool, capacitive links for device recognition, flexible joint designs, and a detailed design space for creating new setups.

Multi-device trip planning study: overview displays improved decisions, organization, and insights, with separate overview devices boosting engagement and collaboration.

Interactive tabletops showed promise for collaboration but failed commercially due to cost, bulk, and limited apps, especially once mobile devices with multi-touch and rich ecosystems took over. Today, however, phones and tablets are increasingly used together on tabletops, opening new opportunities for shared interaction.

Wristband.io extends smartwatch interaction beyond the small screen by integrating input and output directly into the wristband. It features back-of-band touch for more accurate control, a low-resolution band display for subtle notifications, and tangible buttons for quick, eyes-free input.

CurationSpace is a cross-device system that supports collaborative curation of digital historical artefacts. Building on the concept of Instrumental Interaction, it lets curators use smartwatches as instruments or modifiers while working on shared interactive surfaces, enabling more expressive ways to create, navigate, and share resources.

Full paper at Pervasive Display '14Extended Abstract at CHI '14Pervasive Displays paper in the ACM DLCHI EA in the ACM DL tl;dr: As large displays become more common in public spaces, they raise risks of exposing personal or sensitive information to passers-by. We address this …

An implicit authentication method that strengthens Android-style password patterns by also verifying how users draw them. Beyond the shape itself, the system analyzes input dynamics to add a transparent security layer without reducing convenience.