Durable Interaction with Socially Intelligent
by The Danish Council for Independent Research | Technology and
Sciences, Ministry of Science, Innovation and Higher Education (in
Danish: Det Frie ForskningsrĆd | Teknologi og Produktion, Uddannelses-
Grant DFF – 1335-00162
1 August 2013 – 31 December 2017
A mirror project homepage: http://socialrobot.dk
Department of Electronic
Systems, Aalborg University, Denmark
This project develops potentially groundbreaking
methods that make service robots socially intelligent and capable of
establishing durable relationship with their users. This relies on developing
the capabilities to sense and express, which faces grand challenges: the
low-quality signals and the poor context-awareness. We first propose a new
paradigm called reinforcement fusion, which combines sensor signals in an
interactive way: e.g. when a robot detects a sound direction, it turns towards
the direction to see better and moves towards it to hear better.
Reinforcement fusion is analogous to reinforcement learning, a known term in
machine learning. It will dramatically improve robot's sensibility to the
context including social behaviours. Secondly we
propose a concept of social behaviour entrainment to
adapt behaviours. Entrainment is the phenomenon that
dialogue partners tend to adapt their speaking style of each other. Our
hypothesis is that through reinforcement fusion based tracking and social
information extraction, machine learning based context and user modelling, and social behaviour
entrainment, durable social interaction between human and robot is achievable.
The reinforcement fusion paradigm is applicable to all sorts of systems with
Tan, SŅren Holdt Jensen, BŅrge Lindberg, Nicolai Bĺk Thomsen, Xiaodong
Duan, Evgenios Vlachos, Ibrahim Hameed (former member).
iSocioBot (or SocioBot), Nao, Pioneer 3-DX, TurtleBot, and HOVIS Genie.
- Our robotic research with healthy elders and elders suffering from dementia at Fremtidens Plejehjem facility was picked up by Aalborg Kommune LinkedIn page, July 15, 2016.
- International Workshop on Sensing, Processing and Learning for Intelligent Machines (SPLINE 2016), jointly organised by iSocioBot project, was held at Aalborg University, Denmark, July 6-8, 2016.
- News articles in BiTE (part of BT): Danskudviklet robot er fremtidens sociale hjaelper: Se hvad den kan and Om fĆ Ćr kan du fĆ en robot, der kender dig sĆ godt som dine venner, August 2015.
- iSocioBot team offered 3 weeks summer school training course in Programming Social Robots for Human Interaction at Aalborg University, July 20-August 7, 2015.
- iSocioBots and the team were present at the People's Meeting (FolkemŅdet)
in Bornholm, June 2015. The People's Meeting, an annual event in
Denmark, featured with 2700 events and attacked ca. 100 000 guests this
- The second iSocioBot was built on the basis of Pioneer 3-DX, June 2015.
and talk by Research Scientist Najim Dehak, Computer Science and
Artificial Intelligence Laboratory, Massachusetts Institute of
Technology (MIT), U.S.A. Talk title: “I-vector representation based on GMM and DNN for audio classification", May 19, 2015 at 13.00.
- Nao and Pioneer 3-DX both joined the robot team in January 2015.
- iSocioBot was present at the Culture Night 2014
in Copenhagen at the Ministry of Higher Education and Science (with
about 4500 visitors) and interacted with Sofie Carsten Nielsen,
Minister for Higher Education and Science, October 10, 2014.
- iSocioBot inaugurated 'Safe 7' in Nibe, September 6, 2014.
- Visit and talk by Prof. Jen-Tzung Chien, National Chiao Tung University. Talk title: "Bayesian Nonparametric Information Processing", August 26, 2014 at 13:30.
iSocialBot and Presenter Ulla Essendrop presided over the official
opening of the Day of Research 2014 in Denmark (Forskningens DŅgn
2014), which Crown Princess Mary attended with 250 invited guests on 24
April 2014. Et dŅgn der varer tre dŅgn | Nordjyske.dk. A video clip of the opening.
- Visit and talk by Prof. John H.L. Hansen, The University of Texas at Dallas, U.S.A. Talk title: “Speaker & Noise Variability – Making Speech Systems Robust”, Jan. 29, 2014 at 13:00.
- News article in IngeniŅren: Aalborg-forskere: SĆdan bygger vi den menneskelige robot, 1 November 2013. Google English translation.
- Elizabeth Ann Jochum, Evgenios Vlachos, Sally Grindsted Nielsen,
Anja Christoffersen, Ibrahim Hameed and Zheng-Hua Tan, “”Using Theatre
to Study Interaction with Care Robots”,”accepted by International
Journal of Social Robotics. (Springer).
- Nicolai B. Thomsen, Xiaodong Duan, Zheng-Hua Tan, BŅrge Lindberg,
and SŅren Holdt Jensen, “Improving the Convergence of CO-training for
Audio-Visual Person Identification,” The International Workshop on
Sensing, Processing and Learning for Intelligent Machines (SPLINE2016),
July 6-8, 2016, Aalborg, Denmark.
- Ibrahim A. Hameed, Zheng-Hua Tan, Nicolai B. Thomsen and Xiaodong
Duan, “User Acceptance of Social Robots,” The 9th International
Conference on Advances in Computer-Human Interactions (ACHI 2016),
Venice, Italy, April 24-28, 2016.
- Xiaodong Duan and Zheng-Hua Tan, “Neighbors Based Discriminative
Feature Difference Learning for Kinship Verification,” The 11th
International Symposium on Visual Computing, December 14-16, 2015, Las
Vegas, Nevada, USA.
- Nicolai Thomsen, Zheng-Hua Tan, BŅrge Lindberg and SŅren Holdt
Jensen, “A Heuristic Approach for a Social Robot to Navigate to a
Person Based on Audio and Range Information,” 2015 IEEE/RSJ
International Conference on Intelligent Robots and Systems (iROS),
Hamburg, Germany, September 28 – October 02, 2015.
- Sally Grindsted Nielsen, Anja Christoffersen, Elizabeth Jochum
and Zheng-Hua Tan, “Robot Future: Using Theatre to Influence Acceptance
of Care Robots,” The New Friend 2015 Conference, Almere, The
Netherlands, October 22-23, 2015.
- Xiaodong Duan and Zheng-Hua Tan, “Local Feature Learning for Face
Recognition under Varying Poses,” IEEE International Conference on
Image Processing (ICIP 2015), 27-30 September 2015, Quebec City, Canada.
- Xiaodong Duan and Zheng-Hua Tan, “A Feature Subtraction Method
for Image Based Kinship Verification under Uncontrolled Environments,”
IEEE International Conference on Image Processing (ICIP 2015), 27-30
September 2015, Quebec City, Canada.
- Rasmus Lyngby Kristensen, Zheng-Hua Tan, Zhanyu Ma and Jun Guo,
“Binary Pattern Flavored Feature Extractors for Facial Expression
Recognition: An Overview,” CIS-MIPRO 2015, 25-29 May 2015, Opatija,
- Zheng-Hua Tan, Nicolai Bĺk Thomsen and Xiaodong Duan, “Designing
and Implementing an Interactive Social Robot from Off-the-shelf
Components,”The 3rd IFToMM Symposium on Mechanism Design for Robotics
(MEDER2015), June 2-4, 2015, Aalborg, Denmark.
- Nicolai Bĺk Thomsen, Zheng-Hua Tan, BŅrge Lindberg and SŅren
Holdt Jensen, “Learning Direction of Attention for a Social Robot in
Noisy Environments,” The 3rd AAU Workshop on Robotics (AAUROB2014),
- Nicolai B. Thomsen, Zheng-Hua Tan, BŅrge Lindberg and SŅren Holdt
Jensen, “Improving Robustness against Environmental Sounds for
Directing Attention of Social Robots,” The 2nd Workshop on Multimodal
Analyses Enabling Artificial Agents in Human-Machine Interaction,
September 14, 2014, Singapore.