ACM Transactions on

Interactive Intelligent Systems (TIIS)

Latest Articles

Special Issue on Highlights of ACM Intelligent User Interface (IUI) 2018

Exploring a Design Space of Graphical Adaptive Menus: Normal vs. Small Screens

Graphical Adaptive Menus are Graphical User Interface menus whose predicted items of immediate use can be automatically rendered in a prediction... (more)

FourEyes: Leveraging Tool Diversity as a Means to Improve Aggregate Accuracy in Crowdsourcing

Crowdsourcing is a common means of collecting image segmentation training data for use in a variety of computer vision applications. However, designing accurate crowd-powered image segmentation systems is challenging, because defining object boundaries in an image requires significant fine motor skills and hand-eye coordination, which makes these... (more)

A Bandit-Based Ensemble Framework for Exploration/Exploitation of Diverse Recommendation Components: An Experimental Study within E-Commerce

This work presents an extension of Thompson Sampling bandit policy for orchestrating the collection... (more)

Exploring Social Recommendations with Visual Diversity-Promoting Interfaces

The beyond-relevance objectives of recommender systems have been drawing more and more attention. For example, a diversity-enhanced interface has been... (more)

A Visual Analytics Approach for Interactive Document Clustering

Document clustering is a necessary step in various analytical and automated activities. When guided by the user, algorithms are tailored to imprint a... (more)

AnchorViz: Facilitating Semantic Data Exploration and Concept Discovery for Interactive Machine Learning

When building a classifier in interactive machine learning (iML), human knowledge about the target class can be a powerful reference to make the classifier robust to unseen items. The main challenge lies in finding unlabeled items that can either help discover or refine concepts for which the current classifier has no corresponding features (i.e.,... (more)

A Roadmap to User-Controllable Social Exploratory Search

Information-seeking tasks with learning or investigative purposes are usually referred to as exploratory search. Exploratory search unfolds as a... (more)

Individualising Graphical Layouts with Predictive Visual Search Models

In domains where users are exposed to large variations in visuo-spatial features among designs, they often spend excess time searching for common... (more)

A Data-Driven Approach to Designing for Privacy in Household IoT

In this article, we extend and improve upon a previously developed data-driven approach to design privacy-setting interfaces for users of household... (more)

Unobtrusive Activity Recognition and Position Estimation for Work Surfaces Using RF-Radar Sensing

Activity recognition is a core component of many intelligent and context-aware systems. We present a... (more)


Call for Papers: Special Issue on Interactive Visual Analytics for Making Explainable and Accountable Decisions. Submission Deadline: Nov. 17th, 2019.

Get more detail.

Indexed by SCI. TiiS is now approved and indexed by SCI. TiiS will have an impact factor when 2019 index is published.

TiiS 2017 Best Paper Award winners are Marius Kaminskas and Derek Bridge for Diversity, Serendipity, Novelty, and Coverage: A Survey and Empirical Analysis of Beyond-Accuracy Objectives in Recommender Systems from TiiS 7(1)! 

TiiS 2016 Best Paper Award is granted to Weike Pan, Qiang Yang, Yuchao Duan, and Zhong Ming, for their article "Transfer Learning for Semi-Supervised Collaborative Recommendation", appeared in TiiS 6(2). Congratulations to all the authors! 

Forthcoming Articles
Modeling Dyslexic Students' Motivation for Enhanced Learning in E-learning Systems

E-Learning systems can support real-time monitoring of learners? learning desires and effects, thus offering opportunities for enhanced personalized learning. Recognition of the determinants of dyslexic users? motivation to use e-learning systems is important to help developers improve the design of e-learning systems and educators direct their efforts to relevant factors to enhance dyslexic students? motivation. Existing research has rarely attempted to model dyslexic users? motivation in e-learning context from a comprehensive perspective. This paper has conceived a hybrid approach, namely combining the strengths of qualitative and quantitative analysis methods, to motivation modeling. It examines a variety of factors which affect dyslexic students? motivation to engage in e-learning systems from psychological, behavioral and technical perspectives, and establishes their interrelationships. Specifically, the study collects data from a multi-item Likert-style questionnaire to measure relevant factors for conceptual motivation modelling. It then applies the Structural Equation Modeling approach to determine the quantitative mapping between dyslexic students? continued use intention and motivational factors, followed by discussions about theoretical findings and design instructions according to our motivation model. Our research has led to a novel motivation model with new constructs of Learning Experience, Reading Experience, Perceived Control and Perceived Privacy. Initial results have indicated direct effects of Attitudes Toward School, Visual Attractiveness, Reading Experience and Utilization on continued use intention.

When People and Algorithms Meet: A Method and Analysis to Elicit User-reported Problems in Intelligent Everyday Applications

The complex nature of intelligent systems motivates work on supporting users during interaction, for example through explanations. However, there is yet little empirical evidence on specific problems users face in such systems in everyday use. This paper contributes a novel method and analyses to investigate such problems as reported by users: We analysed 45,448 reviews of four apps on the Google Play Store (Facebook, Netflix, Google Maps and Google Assistant) with sentiment analysis and topic modelling to reveal problems during interaction that can be attributed to the apps? algorithmic decision-making. We enriched this data with users? coping and support strategies through a follow-up online survey (N=286). In particular, we found problems and strategies related to content, algorithm, user choice, and feedback. We discuss corresponding implications for designing user support, highlighting the importance of user control and explanations of output, not processes.

PRIME: A Personalized Recommender System for Information Visualization Methods via Extended Matrix Completion

Adapting user interface designs for specific tasks performed by different users is a challenging yet important problem. Automatically adapting visualization designs to users and contexts (e.g., tasks, display devices, environments, etc.) can theoretically improve human-computer interaction to acquire insights from complex datasets. However, effectiveness of any specific visualization is moderated by individual differences in knowledge, skills and abilities for different contexts. A modeling framework called Personalized Recommender System for Information visualization Methods via Extended matrix completion (PRIME) is proposed for recommending the optimal visualization designs for individual users in different contexts. PRIME quantitatively models covariates (e.g., psychological and behavioral measurements) to predict recommendation scores (e.g., perceived complexity, mental workload, etc.) for users to adapt the visualization specific to the context. An evaluation study was conducted and showed that the PRIME can achieve satisfactory recommendation accuracy for adapting visualization, even when there are limited historical data. PRIME can make accurate recommendation even for new users or new tasks based on historical wearable sensor signals and recommendation scores. This capability contributes to designing new generation of visualization systems that adapt to users? status in real time. PRIME can support researchers in reducing the sample size requirements to quantify individual differences, and practitioners in adapting visualizations according to user states and contexts in real time.

Photo Sleuth: Identifying Historical Portraits with Face Recognition and Crowdsourced Human Expertise

Identifying people in historical photographs is important for preserving material culture, correcting the historical record, and creating economic value, but it is also a complex and challenging task. In this paper, we focus on identifying portraits of soldiers who participated in the American Civil War (1861--65), the first widely-photographed conflict. Many thousands of these portraits survive, but only 10-20% are identified. We created Photo Sleuth, a web-based platform that combines crowdsourced human expertise and automated face recognition to support Civil War portrait identification. Our mixed-methods evaluations of Photo Sleuth one month and 11 months after its public launch showed that it helped users successfully identify unknown portraits and provided a sustainable model for volunteer contribution. We also discuss implications for crowd-AI interaction and person identification pipelines.

Algorithmic and HCI aspects for explaining recommendations of artistic images

Understanding why automatic recommendation systems make decisions is an important area of research because users' satisfaction improves when they understand the reasoning behind the suggestions. In the area of visual art recommendation, explanation is a critical part of the process of selling artworks. Traditionally, artwork has been sold in galleries where people can see different physical items, and artists have the chance to persuade potential customers into buying their work. Online sales of art only offer the user the action of navigating through the catalog, but nobody plays the persuasive role of the artist. In the music industry, another artistic domain, recommendation systems have been very successful and play a key role by showing users what they would like to hear. There is a large body of research about this field of recommendation, but there is little research about explaining content-based recommendations of visual arts, though both belong to the artistic domain. Current works do not provide a perspective of the many variables involved in the user perception of several aspects of the system such as domain knowledge, perception of relevance, explainability, and trust. In this paper we aim to fill this gap by studying several aspects of the user experience with a recommender system of artistic images. We conducted two user studies in Amazon Mechanical Turk to evaluate different levels of explainability, combined with different algorithms, interfaces and devices, in order to learn about the interaction between these variables and which effects cause these interactions in the user experience. Our experiments confirm that explanations of recommendations in the image domain are useful and increase user satisfaction, perception of explainability and relevance. In the first study, our results show that the observed effects are dependent on the underlying recommendation algorithm used. In the second study, our results show that these effects are also dependent of the device used in the study. Our general results indicate that algorithms should not be studied in isolation, but rather in conjunction with interfaces and the device since all of them play a significant role in the perception of explainability and trust for image recommendation. Finally, using the framework by Knijnenburg et al., we provide a comprehensive model, for each study, which synthesizes the effects between different variables involved in the user experience with explainable visual recommender systems of artistic images.

Personality Sensing: Detection of Personality Traits Using Physiological Responses to Image and Video Stimuli

Personality detection is an important task in psychology, as different personality traits are linked to different behaviours and real-life outcomes. Traditionally it involves filling out lengthy questionnaires which is time consuming, and may also be unreliable if respondents do not fully understand the questions or are not willing to honestly answer them. In this paper, we propose a framework for objective personality detection that leverages humans' physiological responses to external stimuli. We exemplify and evaluate the framework in a case study, where we expose subjects to affective image and video stimuli, and capture their physiological responses using non-invasive commercial-grade eye-tracking and skin conductivity sensors. These responses are then processed and used to build a machine learning classifier capable of accurately predicting a wide range of personality traits. We investigate and discuss the performance of various machine learning methods, the most and least accurately predicted traits, and also assess the importance of the different stimuli, features, and physiological signals. Our work demonstrates that personality traits can be accurately detected, suggesting the applicability of the proposed framework for robust personality detection and use by psychology practitioners and researchers, as well as designers of personalised interactive systems.

Mental Models of Mere Mortals with Explanations of Reinforcement Learning

We present an empirical study with humans not trained in AI, to investigate the impact of two types of explanations on non-experts' understanding of reinforcement learning (RL) agents. The study was a 124 participant, four-treatment experiment to compare participants' mental models of an RL agent, in a context of a simple Real-Time Strategy (RTS) game. The four treatments isolated the two types of explanations vs. neither vs. both together. The two types of explanations were: (1) saliency maps (an "Input Intelligibility Type" that explains the AI's focus of attention), and (2) a recent explanation type, reward-decomposition bars (an "Output Intelligibility Type" that explains the AI's predictions of future types of rewards). Our results show that the combination of both saliency and decomposed reward bars was needed to achieve a statistically significant difference in participants' mental model description scores over the no-explanation treatment. However, this combination of explanations was far from a panacea: They exacted disproportionately high cognitive loads from the participants. Further, in some situations, participants who saw both explanations predicted the agent's next action worse than all other treatments' participants.

Affect-Aware Word Clouds

Word clouds are widely used for non-analytic purposes, such as introducing a topic to students, or creating a gift with personally meaningful text. Surveys show that users prefer tools that yield word clouds with a stronger emotional impact. Fonts and color palettes are powerful typographical signals that may determine this impact. Typically, these signals are assigned randomly, or expected to chosen by the users. We present an affect-aware font and color palette selection methodology that aims to facilitate more informed choices. We induce associations of fonts with a set of eight affects, and evaluate the resulting data in a series of user studies both on individual words as well as in word clouds. Relying on a recent study to procure affective color palettes, we carry out a similar user study to understand the impact of color choices on word clouds. Our findings suggest that both fonts and color palettes are powerful tools contributing to the affect associated with a word cloud. The experiments further confirm that the novel datasets we propose are successful in enabling this. We also find that, for the majority of the affects, both signals need to be congruent to create a stronger impact. Based on this data, we implement a prototype that allows users to specify a desired affect and recommends congruent fonts and color palettes for the word cloud. Our prototype determines the sentiment of the input text automatically when no affective choices are provided.

Smell Pittsburgh: Engaging Community Citizen Science for Air Quality

Urban air pollution has been linked to various human health concerns, including cardiopulmonary diseases. Communities who suffer from poor air quality often rely on experts to identify pollution sources due to the lack of accessible tools. Taking this into account, we developed Smell Pittsburgh, a system that enables community members to report odors and track where these odors are frequently concentrated. All smell report data are publicly accessible online. These reports are also sent to the local health department and visualized on a map along with air quality data from monitoring stations. This visualization provides a comprehensive overview of the local pollution landscape. Additionally, with these reports and air quality data, we developed a model to predict upcoming smell events and send push notifications to inform communities. We also applied regression analysis to identify statistically significant effects of push notifications on user engagement. Our evaluation of this system demonstrates that engaging residents in documenting their experiences with pollution odors can help identify local air pollution patterns, and can empower communities to advocate for better air quality.

Progressive Disclosure: When, Why and How Do Users Want Algorithmic Transparency Information?

It is essential that users understand how algorithmic decisions are made, as we increasingly delegate important decisions to intelligent systems. Prior work has often taken a techno-centric approach, focusing on new computational techniques to support transparency. In contrast, this paper employs empirical methods to better understand user reactions to transparent systems, in order to motivate user-centric designs for transparent systems. We assess user reactions to transparency feedback in four studies of an emotional analytics system. In Study 1, users anticipated that a transparent system would perform better, but unexpectedly retracted this evaluation after experience with the system. Study 2 offers an explanation for this paradox, by showing that the benefits of transparency are context dependent. One the one hand, transparency can help users form a model of the underlying algorithm?s operation. On the other hand, positive accuracy perceptions may be undermined when transparency reveals algorithmic errors. Study 3 explored real-time reactions to transparency. Results confirmed Study 2, in showing that users are both more likely to consult transparency information and to experience greater system insights when formulating a model of system operation. Study 4 used qualitative methods to explore real-time user reactions in order to motivate transparency design principles. Results again suggest that users may benefit from initially simplified feedback that hides potential system errors and assists users in building working heuristics about system operation. We use these findings to motivate new progressive disclosure principles for transparency in intelligent systems, and discuss theoretical implications.

Exploring the Role of Common Model of Cognition in Designing Adaptive Coaching Interactions for Health Behavior Change

Our research aims to develop intelligent collaborative agents that are human-aware - they can model, learn, and reason about their human partner's physiological, cognitive, and affective states. In this paper, we study how adaptive coaching interactions can be designed to help people develop sustainable healthy behaviors. We leverage the common model of cognition - CMC - as a framework for unifying several behavior change theories that are known to be useful in human-human coaching. We motivate a set of interactive system desiderata based on the CMC-based view of behavior change. Then, we propose PARCoach - an interactive system that addresses the desiderata. PARCoach helps a trainee pick a relevant health goal, set an implementation intention, and track their behavior. During this process, the trainee identifies a specific goal-directed behavior as well as the situational context in which they will perform it. PARCcoach uses this information to send notifications to the trainee, reminding them of their chosen behavior and the context. We report the results from a $4$-week deployment with $60$ participants. Our results support the CMC-based view of behavior change and demonstrate that the desiderata for proposed interactive system design is useful in producing behavior change.

Designing an AI Health Coach and Studying its Utility in Promoting Regular Aerobic Exercise

Our research aims to develop interactive, social agents that can coach people to learn new tasks, skills, and habits. In this paper, we focus on coaching sedentary, overweight individuals (i.e., "trainees") to exercise regularly. We employ adaptive goal setting in which the intelligent health coach generates, tracks, and revises personalized exercise goals for a trainee. The goals become incrementally more difficult as the trainee progresses through the training program. Our approach is model-based - the coach maintains a parameterized model of the trainee?s aerobic capability that drives its expectation of the trainee?s performance. The model is continually revised based on trainee-coach interactions. The coach is embodied in a smartphone application which serves as a medium for coach-trainee interaction. We adopt a task-centric evaluation approach for studying the utility of the proposed algorithm in promoting regular aerobic exercise. We show that our approach can adapt the trainee program not only to several trainees with different capabilities, but also to how a trainee?s capability improves as they begin to exercise more. Experts rate the goals selected by the coach better than other plausible goals, demonstrating that our approach is consistent with clinical recommendations. Further, in a 6-week observational study with sedentary participants, we show that the proposed approach. helps increase exercise volume performed each week.

Generating and Understanding Personalized Explanations in Hybrid Recommender Systems

Recommender systems are ubiquitous, and shape the way users access information and make decisions. As these systems become more complex, there is a growing need for transparency and interpretability. In this paper, we study the problem of generating and visualizing personalized explanations for recommender systems which incorporate signals from many different data sources. We use a flexible, extendable probabilistic programming approach, and show how we can generate real-time personalized recommendations. We then turn these personalized recommendations into explanations. We perform an extensive user study to evaluate the benefits of explanations for hybrid recommender systems. We conduct a crowd-sourced user study where our system generates personalized recommendations and explanations for real users of the music platform. First, we evaluate the performance of the recommendations in terms of perceived accuracy and novelty. Next, we experiment with 1) different explanation styles (e.g., user-based, item-based), 2) manipulating the number of explanation styles presented, and 3) manipulating the presentation format (e.g., textual vs. visual). We also apply a mixed-model statistical analysis to consider user personality traits as a control variable and demonstrate the usefulness of our approach in creating personalized hybrid explanations with different style, number, and format. Finally, we perform a post analysis which shows different preferences for explanation styles between experienced and novice users.

PPLearner: Online Personal Preference Modeling for Context-Dependent User Selections

This paper presents an online algorithms for personal preference modeling to estimate context-dependent user selection by using combination of a data-driven approach and sequential user feedback. The proposed approach is based on an algorithm of online prediction with experts in which our online learner uses a set of policies generated by supervised learning with collected other users' data and adapts to a target user by learning and tracking the optimal policy that best predicts the target user's selection. In each time step, the learner compares a user selection and each prediction from all policies. We also propose an alternative algorithm for a more challenging setting in which our learner is allowed to access limited numbers of policies in each time step (limited prediction setting). We applied the proposed approach to image filter selection as a concrete application. A series of evaluations of our algorithm revealed that 1) our online learning approach achieved better prediction accuracy and less regret than traditional supervised learning or bandit approaches, 2) prediction accuracy increased along with the number of policies, and 3) the proposed algorithm for the limited prediction setting outperformed the state-of-the-art algorithm when the learner obtained predictions from fewer than half the total number of policies. Results of our pilot user study also suggest the potential effectiveness of presenting prediction results from our algorithm that makes users' selection processes more efficient.

How Impactful is Presentation in Email: The Effect of Avatars and Signatures

The primary well-controlled study of 900 participants found that personal presentation choices in professional emails (non-content changes such as Profile Avatar & Signature) impact the recipient?s perception of the sender?s personality, and the quality of the email itself. By understanding the role these choices play, employees can gain better control over how they influence the recipient of their messages. Results further indicated that while some variations can positively impact the recipient?s view of the sender, these same variations often also have negative side-efects. This implies that many seemingly innocuous presentation decisions should be made in the context of who is receiving the email, and if these effects negatively impact the content of the message. For example, while statements in a Signature about the email having being written on a phone are included to preemptively apologize for typing mistakes, this causes the sender to appear less agreeable, less conscientious, and less open, and the email itself appears less well written and more poorly formatted even though the email itself did not change in the study.

All ACM Journals | See Full Journal Index

Search TIIS
enter search term and/or author name