How To Do Good Research In Activity Recognition:
Experimental methodology, performance evaluation and reproducibility.
Workshop in conjunction with Pervasive 2010

PDF Version

Call For Papers

Description

In fields such as computer vision or speech recognition the community has a well established understanding of what criteria high quality research has to fulfil. This includes standard test data sets, ways to set up experiments, evaluation criteria, and things that have to be described in order to make experiments reproducible. Similar expectations have not been established in the literature on context and activity recognition.

While some of the techniques from other pattern recognition domains can be applied, the activity recognition problem has some unique characteristics. These include difficulties in collecting large data sets, issues with the definition of the activities themselves, interleaving and overlapping of activities, mixing of signals from different people at the same location, the impact of environment on behaviour, broad range of temporal and spatial scales on which activities occur and different requirements with respect to performance (e.g. high recall vs. high precision or event accuracy vs. timing accuracy).

The objective of the workshop is to get key members of the research community to discuss, debate, and recommend some "best practices" for activity and context recognition research. The resulting guidelines would be put together in a joint document to be distributed within the community (and published in the workshop proceedings). We also aim to submit a concise version for publication in a jointly authored article in a high impact journal or magazine (e.g. IEEE Pervasive Magazine).

Submissions

The workshop will be focused on discussion, assembling the recommendations, and planning the publication. Participants will be selected on the basis of position statements. The submission paper should include each of the following sections:

  1. What is the community doing well?
    What do you see as the main overall strengths of activity recognition work that has been published to date?
  2. Where does the community need to improve?
    What do you see as the main weaknesses of activity recognition work that has been published to date?
  3. My best recommendations
    Imagine you are reviewing an activity recognition paper. The authors have identified a set of activities they wish to automatically detect using a specific sensor system. They have collected training and test data. Being as specific as you can,
    1. Describe how you would ideally like to see their results presented.
    2. Describe what you would not like to see.
  4. Next steps
    Overall, what steps would you recommend that the community take, and how, to improve the overall quality of activity recognition submissions and publications?
  5. My work
    Briefly describe your own activity recognition research. What motivated you to apply for this workshop?

In addition to those accepted on the basis of position statements, there will be a number of invited participants representing key research groups in the area.

Format

Submissions should be 3-4 pages in length and follow the standard IEEE format. Email the final PDF to: paul lukowicz

Important Dates

February 1, 2010: deadline for submissions
March 1, 2010: notification of acceptance to authors
May 17, 2010: workshop at Pervasive 2010 in Helsinki, Finland

Organizers

Paul Lukowicz (University of Passau, Germany)
Stephen Intille (MIT, USA)
Jamie A Ward (Lancaster University, UK)

Further Information

Workshop website: http://ubicomp.lancs.ac.uk/workshops/activity2010

contact:
Paul Lukowicz
University of Passau, Embedded Systems Lab
Innstrasse 43, D-94032 Passau, Germany

email: paul lukowicz