|
The ongoing pandemic caused by the COVID-19 virus is challenging many aspects of daily life such as restricting the conversation time. A vision-based face analyzing system is considerable for measuring and managing the person-wise speaking time, however, pointing a camera to people directly would be offensive and intrusive. In addition, privacy contents such as the identifiable face of the speakers should not be recorded during measuring. In this paper, we adopt a deep multimodal clustering method, called DMC, to perform unsupervised audiovisual learning for matching preprocessed audio with corresponding locations at videos. We set the camera above the speakers, and by feeding a pair of captured audio and visual data to a pre-trained DMC, a series of heatmaps that identify the location of the speaking people can be generated. Eventually, the speaking time measurement of each speaker can be achieved by accumulating the lasting speaking time of the corresponding heatmap.
|
