Activity models play a critical role for activity recognition and assistance in ambient assisted living. Existing approaches to activity modeling suffer from a number of problems, e.g., model reusability, and incompleteness. In an effort to address these problems, we introduce an ontology-based hybrid approach to activity modeling that combines domain knowledge based model specification and data-driven model learning. Central to the approach is an iterative process that begins with “seed” activity models created by ontological engineering. The “seed” models are deployed, and subsequently evolved through incremental activity discovery and model update. While our previous work has detailed ontological activity modeling and activity recognition, this paper focuses on the systematic hybrid approach and associated methods and inference rules for learning new activities and user activity profiles. The approach has been implemented in a feature rich assistive living system. Analysis of the experiments conducted has been undertaken in an effort to test and evaluate the activity learning algorithms and associated mechanisms.
- INTRODUCTION
Smart homes (SH) have been widely accepted as being a promising paradigm for technology-driven assistive living for the aging population. A SH can be described as a home environment augmented with a diversity of multimodal sensors, actuators, and devices along with information and communication technology based services and systems. By monitoring environmental changes and inhabitants’ activities, an assistive system within a SH can process sensor data, infer an inhabitant’s needs, and take appropriate actions to support activities of daily living (ADLs). As such, a SH can help older people prolong their independent living and enhance quality of life within their own homes.
Activity models play a crucial role in the realization of the SH concept. They are required to support reasoning based upon realtime streaming sensor data in order to infer the current activity for application-level functions. This may include, for example, to predict the next action within a specific task or to detect anomalies within the ADLs currently being undertaken. The completeness and accuracy of ADL models is therefore critical for an assistive system to function correctly. If an activity is not modeled or the model is not accurate, the activity will not be recognized by an assistive system. The system will therefore not be able to provide assistance and/or prediction with regard to this activity.
Modeling ADLs is a challenging task due to their unique characteristics. For example, there are a large number of ADLs in a diversity of categories which can all be modeled at multiple levels of granularity. In addition, most ADLs involve performing a number of actions. The sequence of the actions to be performed is usually dependent on an individual’s own preferences. Furthermore, the manner in which an ADL is performed evolves dynamically, for example the change in duration or the order of objects being used within a task. This is particularly the case for older people and those suffering from decline of cognitive capabilities.
Currently, there are two mainstream approaches to modeling ADLs. One approach is to learn an individual’s activity models from existing behavioral datasets using data mining and machine learning techniques. With this approach, activity models are created based on two tasks, namely the creation of a probabilistic or statistical activity model and the training of the model to decide its parameters or mappings. Given that the approach is based on intensive data analysis, it is usually referred to as a data-driven approach. A data-driven approach to ADL modeling has three major drawbacks. The first is the well known cold-start problem, i.e., requiring a large representative dataset to support model training for each ADL. This problem is exacerbated in the context of assistive living as people are reluctant to disclose their behavioral data due to privacy and ethical considerations. The second drawback is related to model applicability and reusability. A data-driven approach is sensitive to unseen data which makes it difficult to apply the ADL models which have been learnt from one person to another person. This means that with a data-driven approach, an activity model for different users has to be learnt separately. The third drawback is the incompleteness of activity models which is closely related to the aforementioned two issues. With the data-driven approach, every activity model for all the ADLs for every user needs to be learnt in order to create complete ADL models. Given the large number of ADLs and the cold-start problem, this is a huge challenge, if indeed not impossible, in practice. To mitigate the aforementioned problems, researchers have recently started applying transfer learning techniques to activity modeling and recognition by reusing resources and knowledge. This involves transferring the source datasets, or features or models, from one user to another in different settings. Nevertheless, such research is still at its infancy with many open challenges.
An alternative to the data-driven approach is to manually define activity models by making use of rich, prior knowledge, and domain heuristics. This approach is motivated by the observation that most ADLs are daily routines which normally take place within a specific circumstance of time, location, and space with relatively fixed types of objects. Using formal knowledge acquisition and modeling technologies, activity models can be created by means of various knowledge modeling tools. As this approach is closely related to knowledge engineering, it is referred to as a knowledge driven approach. A knowledge-driven approach overcomes the cold-start problem and improves model reusability by modeling activities at multiple levels of abstraction to create both generalized and specialized ADL models. For example, ontological activity modeling can model a generic ADL as an ontological activity class and an individual-specific ADL as an instance of the corresponding activity class. Nevertheless, given that ADL models are created manually by domain experts on a case-by case basis, the approach is questionable in relation to its scalability of generating complete ADL models. As such, it also suffers from model incompleteness. In addition, ADL models created by knowledge-driven approaches are perceived as being generic and static. Adapting an individual’s ADL models to their changing behaviors is still an open issue.
Rather than trying to reuse resources and knowledge among different users similar to the scenario with transfer learning based research, this paper introduces an ontology-based hybrid approach by incorporating data-driven learning capabilities into a knowledge-driven approach to address the aforementioned problems of activity modeling. The rationale is to provide generic activity models suitable for all users and then create individual activity models through incremental learning. The approach uses semantic technologies as a conceptual backbone and technology enablers for ADL modeling, classification, and learning. The distinguishable feature of the approach from existing approaches is that ADL modeling is not a one-off effort, instead, a multiphase iterative process that interleaves knowledge-based model specifications and data-driven model learning. The process consists of three key phases. In the first phase, the initial seed ADL models are created through ontological engineering by leveraging domain knowledge and heuristics, thus solving the cold-start problem. Ontological activity modeling creates activity models at two levels of abstractions, namely as ontological activity concepts and their instances, respectively. Ontological activity concepts represent generic coarse-grained activity models applicable and reusable for all users, thus solving the reusability problem. The seed ADL models are then used in applications for activity recognition at the second phase. In the third phase, the activity classification results from the second phase are analyzed to discover new activities and user profiles. These learnt activity patterns are in turn used to update the ADL models, thus solving the incompleteness problem. Once the first phase completes, the remaining two-phase process can be iteratedmany times to incrementally evolve the ADL models, leading to complete, accurate, and up-to-date ADL models This paper makes three main contributions. First, we develop a hybrid approach to activity modeling that combines the strengths of data- and knowledge-driven approaches to support an incremental modeling process. The approach is built upon the work in, however, extends it by incorporating the learning capabilities to provide a viable solution for addressing existing problems relating to ADL modeling. Second, we develop a learning method to discover activities that are performed by users but have not yet been modeled. Third, we define the characteristics of a user activity profile and develop analysis methods and associated inference rules to learn a user’s activity profiles, i.e., the specific way the user performs activities. The learning methods of activity profiles can detect the changing manner an activity is performed, thus allowing ADL models to adapt over time. We have implemented the approach in a feature-rich assistive living system. ADL discovery algorithms and profile learning methods have been tested and evaluated in a number of experiments by participants in a real sensorised environment. Initial results have demonstrated that the approach works and the algorithms are effective.
It is worth noting that the research presented in this paper is based on single-user single-activity scenarios. While complex activity scenarios, e.g., interleaved and concurrent activities, pose many research problems, it is beyond the scope of this paper to address them all. In addition, the activities this research is concerned with are basic ADLs and instrumental ADLs which can be performed within home environments with clear model semantics, such as meal and drink preparation. Instrumental ADLs such as shopping and use of transportation which take place outside residential environments, and money management and housekeeping which do not have meaningful computational models, require special treatment, and are therefore also considered to be beyond the scope of this paper. Activity monitoring in this study is based on dense sensing, i.e., one miniaturized sensor is attached to individual objects that are used for monitoring individual tasks within ADLs. As such, by analyzing an inhabitant’s interactions with objects of interest, it is possible to infer the inhabitant’s activity.
1.3 LITRATURE SURVEY
A REVIEW OF SMART HOMES—PRESENT STATE AND FUTURE CHALLENGES
PUBLICATION: M. Chan, D. Est`eve, C. Escriba, and E. Campo, Comput.Methods Programs Biomed., vol. 91, no. 1, pp. 55–81, 2008.
In the era of information technology, the elderly and disabled can be monitored with numerous intelligent devices. Sensors can be implanted into their home for continuous mobility assistance and non-obtrusive disease prevention. Modern sensor-embedded houses, or smart houses, cannot only assist people with reduced physical functions but help resolve the social isolation they face. They are capable of providing assistance without limiting or disturbing the resident’s daily routine, giving him or her greater comfort, pleasure, and well-being. This article presents an international selection of leading smart home projects, as well as the associated technologies of wearable/implantable monitoring systems and assistive robotics. The latter are often designed as components of the larger smart home environment. The paper will conclude by discussing future challenges of the domain.
EXPERIENCES IN THE DEVELOPMENT OF A SMART LAB
PUBLICATION: C. D. Nugent, M. Mulvenna, X. Hong, and S. Devlin, Int. J. Biomed. Eng. Technol., vol. 2, no. 4, pp. 319–331, 2009.
There is now a growing demand to provide improved delivery of health and social care due to changes in the age profile of our population. One area where these services may be improved is through the development of smart living environments. Within this paper we provide an overview of the drivers behind the development of such environments along with details of the different ways in which they may exist. Finally, we provide details of our initial experiences in the establishment of a Smart Living Environment for the development of assistive technologies to support independent living.
INFERRING ACTIVITIES FROM INTERACTIONS WITH OBJECTS
PUBLICATION: M. Philipose, K. P. Fishkin, M. Perkowitz, D. J. Patterson, D. Fox, H. Kautz, and D. Hahnel, IEEE Pervasive Comput., vol. 3, no. 4, pp. 50–57, Oct.–Dec. 2004.
A key aspect of pervasive computing is using computers and sensor networks to effectively and unobtrusively infer users’ behavior in their environment. This includes inferring which activity users are performing, how they’re performing it, and its current stage. The eldercare field is a prime, yet difficult application area for inferring whether and how people with early-stage cognitive decline are performing activities of daily living. 1 (For more on ADLs, see the “Activities of Daily Living” sidebar.) Recognizing ADLs, particularly in the home, is challenging on several fronts. First, because users can perform ADLs in various ways, models of activities and recognition software must adapt to this variety. Second, the underlying sensors must report the features required of them robustly across various sensing contexts (such as light levels, sound levels, and locations). Third, given the large number of ADLs—20 to 30 classes (such as making a meal) with thousands of instances—a system should model each activity with minimum human effort. Addressing these challenges simultaneously has been a key barrier to success for ADL monitoring systems.
INTERNATIONAL CLASSIFICATION OF FUNCTIONING, DISABILITY AND HEALTH (ICF)
PUBLICATION: World Health Organization, [Online]. Available: http://www.who.int/classifications/icf/en/
The International Classification of Functioning, Disability and Health, known more commonly as ICF, is a classification of health and health-related domains. As the functioning and disability of an individual occurs in a context, ICF also includes a list of environmental factors. ICF is the WHO framework for measuring health and disability at both individual and population levels. ICF was officially endorsed by all 191 WHO Member States in the Fifty-fourth World Health Assembly on 22 May 2001(resolution WHA 54.21) as the international standard to describe and measure health and disability.
CHAPTER 2
2.0 SYSTEM ANALYSIS
2.1 EXISTING SYSTEM:
The Existing system is data driven approach, the drawback of this system is related to model applicability and reusability. A data-driven approach is sensitive to unseen data which makes it difficult to apply the ADL models which have been learnt from one person to another person. This means that with a data-driven approach, an activity model for different users has to be learnt separately. The third drawback is the incompleteness of activity models which is closely related to the aforementioned two issues. With the data-driven approach, every activity model for all the ADLs for every user needs to be learnt in order to create complete ADL models. Given the large number of ADLs this is a huge challenge, if indeed not impossible, in practice. To mitigate the aforementioned problems, researchers have recently started applying transfer learning techniques to activity modeling and recognition by reusing resources and knowledge. This involves transferring the source datasets, or features or models, from one user to another in different settings. An alternative to the data-driven approach is to manually define activity models by making use of rich, prior knowledge, and domain heuristics.
2.1.1 DISADVANTAGES:
2.2 PROPOSED SYSTEM:
This approach is motivated by the observation that most ADLs are daily routines which normally take place within a specific circumstance of time, location, and space with relatively fixed types of objects. Using formal knowledge acquisition and modeling technologies, activity models can be created by means of various knowledge modeling tools. As this approach is closely related to knowledge engineering, it is referred to as a knowledge driven approach. A knowledge-driven approach improves model reusability by modeling activities at multiple levels of abstraction to create both generalized and specialized ADL models. For example, ontological activity modeling can model a generic ADL as an ontological activity class and an individual-specific ADL as an instance of the corresponding activity class. ADL modeling is not a one-off effort, instead, a multiphase iterative process that interleaves knowledge-based model specifications and data-driven model learning. The process consists of two key phases.
Ontological activity modeling creates activity models at two levels of abstractions, namely as ontological activity concepts and their instances, respectively. Ontological activity concepts represent generic coarse-grained activity models applicable and reusable for all users, thus solving the reusability problem. The seed ADL models are then used in applications for activity recognition at the second phase. In the third phase, the activity classification results from the second phase are analyzed to discover new activities and user profiles. These learnt activity patterns are in turn used to update the ADL models, thus solving the incompleteness problem. Once the first phase completes, the remaining two-phase process can be iterated many times to incrementally evolve the ADL models, leading to complete, accurate, and up-to-date ADL models. This paper makes three main contributions. First, we develop a hybrid approach to activity modeling that combines the strengths of data- and knowledge-driven approaches to support an incremental modeling process. The approach is built upon the work in however, extends it by incorporating the learning capabilities to provide a viable solution for addressing existing problems relating to ADL modeling. Second, we develop a learning method to discover activities that are performed by users but have not yet been modeled. Third, we define the characteristics of a user activity profile and develop analysis methods and associated inference rules to learn a user’s activity profiles, i.e., the specific way the user performs activities. The learning methods of activity profiles can detect the changing manner an activity is performed, thus allowing ADL models to adapt over time. We have implemented the approach in a feature-rich assistive living system.
2.2.1
ADVANTAGES:
2.3 HARDWARE & SOFTWARE REQUIREMENTS:
2.3.1 HARDWARE REQUIREMENT:
v Processor – Pentium –IV
- Speed –
1.1 GHz
- RAM – 256 MB (min)
- Hard Disk – 20 GB
- Floppy Drive – 1.44 MB
- Key Board – Standard Windows Keyboard
- Mouse – Two or Three Button Mouse
- Monitor – SVGA
2.3.2 SOFTWARE REQUIREMENTS:
- Operating System : Windows XP
- Front End : Microsoft Visual Studio .NET 2008
- Back End : MS-SQL Server 2005
- Document : MS-Office 2007