Human in the loop design for intelligent interactive systems: A systematic review

Abstract

It is undeniable that modern computers are incredibly fast and accurate. However, computers cannot ‘think’ (act intelligently) as humans unless it is trained to learn from the past knowledge. Despite their intelligence, humans are comparatively slow in computational tasks. However, the combination of the computational capacity of computers and human intelligence could produce powerful systems beyond the imagination. This concept is called Human-in-the-Loop (HITL) where both human and machine intelligence support the creation of Machine Learning (ML) models. HITL design is an emerging technology which is used in many domains such as autonomous vehicle technology, health systems and interactive system implementations. In this research, we systematically reviewed past research of HITL systems with the objectives of identifying key benefits and limitations of the HITL design. This systematic review was conducted by analyzing 68 research papers published in top-ranked journals and conferences during the past decade. Moreover, the papers were selected using keyword-based searching and references of the most cited HITL research papers. The PRISMA model was used to exclude irrelevant papers, and keyword-based clustering was used to identify the frequent keywords in the selected papers. Although the HITL design often improves the performance of intelligent interactive systems, there are certain drawbacks of this concept when compared to fully manual or fully automated systems such as making decisions with emotional bias and being unable to take actions when demanded. Thus, we comprehensively discuss the approaches proposed by the recent researchers to overcome some of the issues of the existing HITL designs.