Big Data for People

Human-Computer Interaction Researcher
Manager, Human Interfaces Group
Mission Operations
NASA Jet Propulsion Lab

Rss Publications


APS DFD 2013

We report on the outcome of a design-informed visualization project performed by the Caltech/JPL/Art Center Data Visualization Summer Internship Program tasked with creating a visualization of the deconstruction of a full turbulent flow field into subsets of dynamic, interacting building blocks. The goal was to create a visualization suite capable of illuminating the velocity fields associated with individual resolvent modes and the rapid increase in complexity associated with linear superposition of modes. The software, named ModeDynamix has the potential to provide insight into the underlying dynamics of wall turbulence.

CSER 2013

One of the barriers to the success of Model-Based Systems Engineering (MBSE) efforts is realizing effective communication of the output diagrams—i.e., modeling views—that address the concerns of, and inform, a broad spectrum of customer stakeholders. Abstracting and implementing the visual presentation of views—as products of very complex system models—is nearly as important to the effectiveness of these efforts to inform decision-making as the technical competency and completeness of those models. However, the information visualization of data from complex system models is often treated second to the technical considerations. This paper will introduce high-level guidelines for visual presentation of MBSE efforts. These insights will be presented such that they conform to numerous system modeling languages/representation standards. The insights are drawn from best practices of Information Visualization (InfoVis) as applied to aerospace-based applications. For example, the paper will discuss how modelers can to take advantage of functionality in existing modeling notions and software tools that implement them, and also the importance of keeping in mind the final presentation media and historically accepted viewpoint styles. The paper will also present a concept for how to move beyond traditionally static outputs; in turn, allowing non-modeling experts to dynamically manipulate the output views within the context of the stakeholders’ real-time concerns and needs to answer specific questions about the modeled system(s).

IEEE Pervasive 2012

Researchers recently gathered at Tsinghua University in Beijing, China, to share their accomplishments at the 13th annual ACM Conference on Ubiquitous Computing. This multidisciplinary conference featured advances in sensing technologies and activity-recognition algorithms, novel applications, and results from end-user evaluations. This department samples work presented at the conference, providing an overview of the themes addressed as well as reference pointers for those interested in learning more.

DIS 2012

Designing radically new technology systems that people will want to use is complex. Design teams must draw on knowledge related to people's current values and desires to envision a preferred yet plausible future. However, the introduction of new technology can shape people's values and practices, and what-we-know-now about them does not always translate to an effective guess of what the future could, or should, be. New products and systems typically exist outside of current understandings of technology and use paradigms; they often have few interaction and social conventions to guide the design process, making efforts to pursue them complex and risky. User Enactments (UEs) have been developed as a design approach that aids design teams in more successfully investigate radical alterations to technologies' roles, forms, and behaviors in uncharted design spaces. In this paper, we reflect on our repeated use of UE over the past five years to unpack lessons learned and further specify how and when to use it. We conclude with a reflection on how UE can function as a boundary object and implications for future work.

JAISE 2011

The ubiquitous computation that underlies ambient intelligence offers an unprecedented framework for innovation to cycle between the Computer and Social Sciences. Commercial mobile phones effectively function as sensing platforms, effortlessly creating massive datasets. Machine learning and data mining provide powerful analytic lenses to observe and measure people in ways previously not possible. Examining data describing how routines support everyday activities, we can uncover new subjects for sensing and machine learning. I demonstrate the value of this approach using dual-income families. My studies of family logistics show that unexpected changes to plans cause parents the most anxiety. My data collection using mobile phones shows that that less than 20% of all days involve no unexpected changes. Interview studies show that family members sometimes need but do not have access to information about the plans and routines of other family members. Because family members do not document this information, they do not exist as resources family members can turn to when needed.

UbiComp 2011

The complexities and costs of deploying Ubicomp applications seriously compromise our ability to evaluate such systems in the real world. To simplify Ubicomp deployment we introduce the robotic pseudopod (P.Pod), an actuator that acts on mechanical switches originally designed for human control only. P.Pods enable computational control of devices by hijacking their mechanical switches – a term we refer to as mechanical hijacking. P.Pods offer simple, low-cost, non-destructive computational access to installed hardware, enabling functional, real world Ubicomp deployments. In this paper, we illustrate how three P.Pod primitives, built with the Lego MindStorm NXT toolkit, can implement mechanical hijacking, facilitating real world Ubicomp deployments which otherwise require extensive changes to existing hardware or infrastructure. Lastly, we demonstrate the simplicity of P.Pods by observing two middle school classes build working smart home applications in 4 hours.

CHI 2011

Part of being a parent is taking responsibility for arranging and supplying transportation of children between various events. Dual-income parents frequently develop routines to help manage transportation with a minimal amount of attention. On days when families deviate from their routines, effective logistics can often depend on knowledge of the routine location, availability and intentions of other family members. Since most families rarely document their routine activities, making that needed information unavailable, coordination breakdowns are much more likely to occur. To address this problem we demonstrate the feasibility of learning family routines using mobile phone GPS. We describe how we (1) detect pick-ups and drop- offs; (2) predict which parent will perform a future pick-up or drop-off; and (3) infer if a child will be left at an activity. We discuss how these routine models give digital calendars, reminder and location systems new capabilities to help prevent breakdowns, and improve family life.

Thesis 2011

By understanding how routines support people's everyday activities, we can uncover new subjects for sensing and machine learning. This new data creates new ways for end-user applications to support daily life. I demonstrate the value of this approach using dual-income families. My studies of family logistics shows that family members sometimes need but do not have access to information about the plans and routines of other family members. Because family members do not document this information, they do not exist as resources family members can turn to when needed. With only the GPS on commercial mobile phones, we can use machine learning and data mining to automatically document family logistical routines, and present that information to families to help them feel more in control of their lives.

Group 2010

My work looks to comprehend, and (re-)define the appropriate role of intelligent systems in the home. To explore this question, I have focused on routine learners in the context of dual-income family coordination. Families often depend on accurate information about one another’s routines to coordinate. When inaccurate, parents make plans that create conflicts, go to the wrong places, and even leave kids at their activities. Despite this important role, routines, are rarely documented, and so are not available to people for support, or to computational systems as input. My work demonstrates how unsupervised models of family routine can be learned using a single, lightweight sensor. The tacit knowledge of family routine can be captured and exploited by learning systems, providing a new kind of information stream that empowers families and computational systems alike with new capabilities.

UbiComp 2010

Even though the coordination of kids’ activities is largely successful, the modern dual income family still regularly experiences breakdowns in their practices. Families often rely on routines to help them coordinate when plans prove less effective. Routines, however, are rarely documented, challenging to express in detail, and frequently evolving, making them cumbersome to manually describe and so largely unavailable to computational systems as input. This work proposes that this disconnect can be overcome, and argues that unsupervised models of family routine can be learned using a single, lightweight sensor. This way, the successful but tacit knowledge of the routine might be captured and exploited by learning systems, providing a new kind of information for families and computational systems alike. A method is proposed to develop a Bayesian Network to reason about the state of family coordination. This model relies on learned routines of pickup and drop- off at kids’ activities.

CHI 2010

Researchers have detailed the importance of routines in how people live and work, while also cautioning system designers about the importance of people’s idiosyncratic behavior patterns and the challenges they would present to learning systems. We wish to take up their challenge, and offer a vision of how simple sensing technology could capture and model idiosyncratic routines, enabling applications to solve many real world problems. To identify how a simple routine learner can demonstrate this in support of family coordination, we conducted six months of nightly interviews with six families, focusing on how they make and execute plans. Our data reveals that only about 40% of events unfold in a routine manner. When deviations do occur, family members often need but do not have access to accurate information about their routines. With about 90% of their content concerning deviations, not routines, families do not rely on calendars to support them during these moments. We discuss how coordination tools, like calendars and reminder systems, would improve coordination and reduce stress when augmented with routine information, and how commercial mobile phones can support the automatic creation of routine models.

Design & Emotion Moves 2008

For many years technology researchers have promised a smart home that, through an awareness of people’s activities and intents, will provide the appropriate assistance to improve human experience. However, before people will accept intelligent technology into their homes and their lives, they must feel they have control over it (Norman 1994). To address this issue, social researchers have been conducting ethnographic research on families, looking for opportunities where technology can best provide assistance. At the same time, technology researchers studying “end user programming” have focused on how people can control devices in their homes. We observe an interesting disconnect between the two approaches–the ethnographic work reveals that families desire to “feel in control of their lives,” more than in control of their devices. Our work attempts to bridge the divide between these two research communities by exploring the role a smart home can play in the life of a dual-income family. If we first understand the roles a smart home can play, we can then more appropriately choose how to provide families with the control they desire, extending the control of devices to incorporate the control of their lives families say they need.

UbiComp 2007

While the user-centered design methods we bring from human-computer interaction to ubicomp help sketch ideas and refine prototypes, few tools or techniques help explore divergent design concepts, reflect on their merits, and come to a new understanding of design opportunities and ways to address them. We present Speed Dating, a design method for rapidly exploring application concepts and their interactions and contextual dimensions without requiring any technology implementation. Situated between sketching and prototyping, Speed Dating structures comparison of concepts, helping identify and understand contextual risk factors and develop approaches to address them. We illustrate how to use Speed Dating by applying it to our research on the smart home and dual-income families, and highlight our findings from using this method.

DPPI 2007

Dual-income families experience stress as they attempt to manage the conflicting responsibilities of work, school, home, and enrichment activities. Opportunities exist for technology to provide support in managing their children's activities, helping parents feel more in control of their lives. In this paper, we explore opportunities to support children's activities. Based on our contextual fieldwork with dual-income families, we suggest a concept of the Smart Bag, which addresses two design opportunities: (i) a reminder system that helps people remember their schedules and what they need to take, and (ii) a reminder system that allows parents to engage in parenting.

D&E 2006

For many years technology researchers have promised a smart home that will provide the appropriate assistance to improve human experience. While technology researchers have focused on how people can control devices in their homes, our ethnographic research and findings by other social scientists have revealed that families want more control over their lives. We believe a smart home should provide families with a feeling of control over their lives: being relieved from breakdowns in their daily routines, and getting emotional satisfaction from the things they value – identity, time, and relationships. In this paper, we explore the roles that a smart home can play for families to regain control over their lives. We present our human-centered design research on dual- income families and suggest insights about the potential roles of a smart home based on this research.

UbiComp 2006

Much smart home research has produced work that chooses to enhance the home experience by providing more control over domestic devices. Our work with dual-income families suggests that more than devices, families want more control over their lives. We argue that a smart home might help provide that assistance to families in two ways: (1) by helping families navigate the complex network of activities in which they participate; and (2) by providing families more emotional satisfaction from the things they value – their time, their relationships, and their identity. In this position paper, we discuss these two service alternatives, and describe how life control provides a pathway to a nurturing home environment.

UbiComp 2006

Seeking to be sensitive to users, smart home researchers have focused on the concept of control. They attempt to allow users to gain control over their lives by framing the problem as one of end-user programming. But families are not users as we typically conceive them, and a large body of ethno- graphic research shows how their activities and routines do not map well to programming tasks. End-user programming ultimately provides control of de- vices. But families want more control of their lives. In this paper, we explore this disconnect. Using grounded contextual fieldwork with dual-income families, we describe the control that families want, and suggest seven design principles that will help end-user programming systems deliver that control.

IE 2006

The majority of smart home research has focused on novel technical artifacts, but has overlooked the issues surrounding social relationships in the home. We argue in favor of research that is sensitive to and functions within the social constraints of dual income family homes. This paper describes our grounded contextual fieldwork with real families in their homes, and identifies socially-aware concepts smart home systems will need to address.

Pervasive 2005

Researchers developing Ubicomp applications often must make ill-informed but irrevocable decisions early in the design process. While desktop computing researchers have multiple methods at their disposal to manage the risk involved in these decisions, the complexity of Ubicomp research affords few alternatives. We suggest that Ubicomp research faces a poverty of effective design process. We explore alternatives that might supplement existing de-sign processes so that designers can make decisions from positions of information. This suggests an opportunity to develop both tools and techniques that support early-stage evaluation

CHI 2005

Substantial stumbling blocks confront computer-illiterate elders. We introduce a novel user interface technology to lower these start up costs: the book as user interface, or BUI. Book pages contain both step-by-step instructions and tangible controls, turning a complex interaction into a walk-up-and-use scenario. The system expands support past the technical artifact to a go-to relationship. ElderMail users designate an internet-savvy trusted friend or relative to help with complex tasks. In this paper, we conduct a preliminary evaluation of a BUI-based email system, and report our findings. While research has augmented paper artifacts to provide alternate access into the digital world, we find that elders use the BUI as a way to circumvent the digital world.
© 1998 — 2021 by Scott Davidoff, some rights reserved.
Contact Scott Davidoff