Overview

Smartphones equipped with powerful embedded sensors (e.g. accelerometers, GPS, microphones, etc.) can be used to monitor multiple dimensions of human behaviors including physical, mental, and social behaviors of wellbeing. In particular, such enhanced capabilities in mobile devices enable people to better manage their health. Existing mobile health applications mostly rely on manual data entry which may not be sufficient for certain population e.g. seniors and students with emotional behavioral disorders (EBD). This project aims to build a multimodal sensing system which provides continuous and efficient monitoring of users' activities using mobile phones and wearable sensors.

Our system analyzes and correlates different sensor streams to infer certain behaviors as well as possible environmental factors that may trigger such behaviors. Furthermore, our system provides non-intrusive peer assisted localization technique that allows caregivers to track the whereabouts of monitored users. We also develop efficient schemes to infer higher layer information e.g. activity levels of monitored individuals; social relationships among monitored users. Additionally, the communities extracted from a mobile phone enabled social network in our system not only enable mobile healthcare systems but can also be exploited for securing certain components of the system (e.g., coping with clone attacks). Our system allows users to be monitored for their mental, cognitive, and physical well-being and can potentially reduce the cost for special need education as a result of increasing the productivity of teachers and caregivers.

Lehigh Autism Social Alert System

Wireless Treasure Hunt App (Fighting Obesity)

Sensors embedded within smartphones and wearable sensors provide users with the ability to colletively sense the world. This leads to a growing trend of mobile healthcare systems utilizing such sensors. However, such healthcare systems are vulnerable to user spoofing, in which an adversary may use others' collected sensor data to gain benefitis. We have designed a gait-based user verification scheme for smartphone enabled mobile healthcare systems. Our scheme exploits the readily available accelerometers embedded within smartphones for user verification. Specifically, our scheme extract gait patterns from run-time accelerometer measurements to perform robust user verification. Our scheme is designed such that it is robust to users' various walking speeds. Our framework can be implemented in two ways: user centric and service centric. Our results can be found in these two published papers: IEEE SECON 2013 paper and TMC paper .

Mobile devices such as smartphones and tablets play important roles in our daily lives. Users often like to use their smartphone to complete online transactions including payment of any healthcare services they receive. Few studies provide efficient real time user signature verification. In our work, we propose a critical segment based online signature verification system to secure mobile transactions on multi-touch mobile devices. Our system identifies and exploits the segments which remain invariant within a user's signature to capture the intrinsic signing behavior embedded in a user's signature. Experimental evaluation of 25 subjects over 6 months time period shows that our system is highly accurate in providing signature verification and robust to signature forging attacks. Our results can be found in this IEEE CNS 2015 paper .

We have recently developed efficient deep learning based mobile object recognition scheme which allows a group of users to collaborate in improving the object recognition rate in different visual domains. Our related DeepCham paper has been published at First IEEE/ACM Symposium on Edge Computing.

We have recently developed several Android-based applications targetting autistic children/teenagers. Interested readers can find more information in this link .

Two undergraduates have also developed a customizable Kinect-based game via a senior design project. This project won the peer choice award. This game is very popular among autistic children attending Autism Respite events at a nearby church. The poster for this game can be found in this link . A new version of the Kinect game involving two players have been developed by an undergraduate funded by NSF REU funding.

Quick Links

Sponsors - NSF CSR Grant.

Research - Other On-Going Research.

Publications - Related Publications

People - Team Members

WiNS Lab, Lehigh

Smartphone Enabled Social and Physical Compass System (SENSCOPS)

Overview

Quick Links