Publications

You can also find my articles on my Google Scholar profile.

Journal Articles

Exploring Uni-manual Around Ear Off-Device Gestures for Earables

Published in UbiComp/IMWUT '24, 2024

Small form factor limits physical input space in earable (i.e., ear-mounted wearable) devices. Off-device earable inputs in alternate mid-air and on-skin around-ear interaction spaces using uni-manual gestures can address this input space limitation. Segmenting these alternate interaction spaces to create multiple gesture regions for reusing off-device gestures can expand earable input vocabulary by a large margin. Although prior earable interaction research has explored off-device gesture preferences and recognition techniques in such interaction spaces, supporting gesture reuse over multiple gesture regions needs further exploration. We collected and analyzed 7560 uni-manual gesture motion data from 18 participants to explore earable gesture reuse by segmentation of on-skin and mid-air spaces around the ear. Our results show that gesture performance degrades significantly beyond 3 mid-air and 5 on-skin around-ear gesture regions for different uni-manual gesture classes (e.g., swipe, pinch, tap). We also present qualitative findings on most and least preferred regions (and associated boundaries) by end-users for different uni-manual gesture shapes across both interaction spaces for earable devices. Our results complement earlier elicitation studies and interaction technologies for earables to help expand the gestural input vocabulary and potentially drive future commercialization of such devices.

Recommended citation: Shaikh Shawon Arefin Shimon, Ali Neshati, Junwei Sun, Qiang Xu, and Jian Zhao. 2024. "Exploring Uni-manual Around Ear Off-Device Gestures for Earables." Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 8, 1, Article 3 (March 2024), 29 pages. https://doi.org/10.1145/3643513
Download Paper | Download Slides | Download Bibtex

Conference Papers

Exploring Above-neck Unimanual Swipe Gestures for Off-Device Earable Interaction

Published in Graphics Interface (GI) '26, 2026

Despite their growing popularity, in-ear Earable / Hearable devices (i.e., ear-mounted wearables) face interaction challenges due to limited input space and compact form factors. To enhance interaction capabilities, researchers are exploring off-device hand-based input spaces above the neck using midair and onskin gestures. However, existing literature primarily focuses on axial swipes (i.e., horizontal and vertical), leaving nonaxial swipes (i.e., unidirectional swipes with varied orientations) and angular swipes (e.g., L, U, or V) largely underexplored despite their potential interaction advantages. To address this gap, we conducted a within-subject gesture motion analysis study with 24 participants, analyzing 5,568 swipes of varying shape, orientation, and complexity. Our results revealed preferred starting and ending regions for different unidirectional and angular swipe shapes, as well as intuitive swipe shapes within the off-device, above-neck manual interaction space. We further examine off-device swipe characteristics, discuss the feasibility of recognizing these earable gestures with current sensing technologies, and highlight their potential application in various scenarios. These findings broaden the understanding of off-device earable gestures and provide design insights for integrating suitable nonaxial and angular swipes alongside traditional axial gestures to enhance interaction with in-ear earable devices.

Recommended citation: Shaikh Shawon Arefin Shimon, Ali Neshati, Junwei Sun, Qiang Xu, and Jian Zhao. 2026. "Exploring Above-neck Unimanual Swipe Gestures for Off-Device Earable Interaction." In Proceedings of the 52nd Graphics Interface Conference (GI '26). Association for Computing Machinery, New York, NY, USA, Article 28, 1–21. https://doi.org/10.1145/3831423.3831436
Download Paper | Download Slides | Download Bibtex

MFS Design in Appstore-enabled Smart Featurephones for Low-literate, Marginalized Communities

Published in CHI '23, 2023

Mobile Financial Services (MFS) has gained significant popularity during the COVID-19 pandemic, especially among marginalized and low-income, low-literate communities around the world. Such communities have not been traditionally considered while designing MFS services via smartphone apps or USSD services in featurephones. Financial constraints limit such end-users towards basic featurephones, where recent appstore support has made it possible to deploy app-based MFS solutions beyond USSD. This new featurephone platform is a relatively underexplored area in terms of addressing design issues related to aforementioned end-users while developing MFS solutions. Our work addresses this gap by presenting qualitative findings on barriers to technology access focused on MFS solutions in marginal communities. We present a prototype non-USSD, app-based solution on an appstore-supported featurephone platform designed via a human-centered approach. This work has the potential to increase the financial inclusivity of marginalized communities in cashless MFS transactions via low-cost, appstore-enabled featurephones.

Recommended citation: Mohammad Rayed, Tawfique Elahi, Shaikh Shawon Arefin Shimon, and Nova Ahmed. 2023. "MFS Design in Appstore-enabled Smart Featurephones for Low-literate, Marginalized Communities." In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI '23). Association for Computing Machinery, New York, NY, USA, Article 762, 1–19. https://doi.org/10.1145/3544548.3580661
Download Paper | Download Bibtex

Exploring Non-touchscreen Gestures for Smartwatches

Published in CHI '16, 2016

Although smartwatches are gaining popularity among mainstream consumers, the input space is limited due to their small form factor. The goal of this work is to explore how to design non-touchscreen gestures to extend the input space of smartwatches. We conducted an elicitation study eliciting gestures for 31 smartwatch tasks. From this study, we demonstrate that a consensus exists among the participants on the mapping of gesture to command and use this consensus to specify a user-defined gesture set. Using gestures collected during our study, we define a taxonomy describing the mapping and physical characteristics of the gestures. Lastly, we provide insights to inform the design of non-touchscreen gestures for smartwatch interaction.

Recommended citation: Shaikh Shawon Arefin Shimon, Courtney Lutton, Zichun Xu, Sarah Morrison-Smith, Christina Boucher, and Jaime Ruiz. 2016. "Exploring Non-touchscreen Gestures for Smartwatches." In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). Association for Computing Machinery, New York, NY, USA, 3822–3833. https://doi.org/10.1145/2858036.2858385
Download Paper | Download Slides | Download Bibtex

Exploring User-Defined Back-Of-Device Gestures for Mobile Devices

Published in MobileHCI '15, 2015

Many studies have highlighted the advantages of expanding the input space of mobile devices by utilizing the back of the device. We extend this work by performing an elicitation study to explore users’ mapping of gestures to smartphone commands and identify their criteria for using back-of-device gestures. Using the data collected from our study, we present elicited gestures and highlight common user motivations, both of which inform the design of back-of-device gestures for mobile interaction.

Recommended citation: Shaikh Shawon Arefin Shimon, Sarah Morrison-Smith, Noah John, Ghazal Fahimi, and Jaime Ruiz. 2015. "Exploring User-Defined Back-Of-Device Gestures for Mobile Devices." In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '15). Association for Computing Machinery, New York, NY, USA, 227–232. https://doi.org/10.1145/2785830.2785890
Download Paper | Download Slides | Download Bibtex