A life-size, vertical MRI (magnetic resonance imaging) movie display was created for an exhibiting focusing on the mechanisms of human speech production. First, MRI movies of three sections, namely head and neck, chest and abdomen were captured separately scanning the internal movements during speech utterance. Then, the three footages were combined to show the wholistic movement of the muscles and relevant organs. While MRI videos of only the neck and head are typically shown when explaining the mechanisms of speech, in the created display, the whole internal-body movement could be clearly seen, including the involvement of the diaphragm that changes the lung capacity to produce air stream that is necessary for speech-sound generation. Being part of the Special Exhibition “Homō loquēns ‘talking human’: Wonders of Language and Languages” that took place at the National Museum of Ethnology, Japan, in 2022, this display was one of the materialization the aim of the organizers, whose goal was to provide novel and persuasive information to the visitors, who included both spoken and signed language users, about the mechanisms of spoken language production.
A physical 3D figure model of human jaw was made using a 3D printer. The model is used for a research of human speech production mechanisms and for educational purposes of speech science. To design 3D geometrical data of jaws, 3D computer graphics software is used. The modeling data are stored in STL (Stereolithography) format and then, they are transferred to a 3D printer and figure models are made. The models are used to teach speech production mechanisms in a university class.
This study offers a comparative analysis of user experience and levels of immersion in regards to screen based solutions for virtual museums and virtual reality based (VR) solutions with Head mounted displays (HMD). The aim of this study is to better inform design and research and development of virtual museums using real-time 3DCG for achieving an experience of presence and immersion for viewer that comparable to viewing experiences in an actual museum. We have developed a virtual museum that can be experienced both on a large screen display and also via VR on an HMD. Users can experience the viewing space where paintings, picture scrolls, and 3D-objects are placed in the virtual museum, and the virtual space by freely walking through the virtual environment (VE). The development environment used for the study is the Unity’s game engine, created in-game objects, and introduced output content for outputting as VR in Oculus. Then, Compare the results of the comparison experiment of the virtual museum conducted in Sweden and the comparison experiment of the virtual museum conducted in Japan in the previous research, and evaluate what is close to actual viewing.
We propose an articulation awareness system with a three-dimensional (3D) tongue using virtual reality (VR). Human speech sounds are made through a combination of vocal fold vibration (voice source) and tongue or lip motion (articulation). Articulation is the movement of the jaws, tongue, and lips. A speaker should pay attention to the importance of these movements. In this study, the tongue shape is visualized to raise awareness about the speech organ. The subjects observed inside and outside of the mouth as if they were the size of the thumb. Models of the oral area were created from magnetic resonance imaging data collected during vowel production. The subjects reported that they were aware of the articulators after experiencing the 3D tongue using a VR system.
We propose a speech motion feedback system for improving articulation by showing 3D-CG jaw and abstracted lip. The subject's motion of the jaw is captured by a 3D position with rotation sensor. The lip motion is measured by four 3D position sensors using infrared emission. Subjects observe his/her own face on LCD screen, and 3D-CG jaw and abstracted lip motion on semi-transparent screen. Subjects reported that they noticed the importance of the motion of the speech organs after the experiment.
We have been researching and developing virtual museums that can enable a user to appreciate artworks with much reality and less stress as if one is experiencing in actual museums. For this time, as a method to further increase the reality of the virtual museum, we developed a virtual museum using Head Mounted Display. We used Oculus Rift and Unity Game Engine to develop this VR museum. We also performed a comparison of the developed VR museum and the screen-based virtual museum in order to maximize the user experience of the virtual museum. Four aspects of "Freedom of operation", "Immersion", "Comfort of play", and "Picture quality" are examined to clarify which type of virtual museum would be suitable for any specific user's needs.
We have been studying and developing the real-time Computer Graphics (CG) based virtual museum where a user can walk through to appreciate artworks digitized in high-resolution. Our virtual museum also has a function to automatically create TV program-like CG animations using 3D CG models in the virtual space as it is so that the user can learn about individual works by watching the art shows. The CG animation is produced with TVML (TV program Making Language) engine implemented on the virtual museum. However, the current problem is that it requires a lot of work for a developer to write the complicated TVML scripts manually. Therefore, this time we have developed a special tool to help the developer to prepare the TVML scripts easily. With this tool, the developer can produce the TVML-based art program simply by writing out a simple scenario on an ordinary text editor. In order to design this tool, TV art programs actually broadcasted are analyzed to determine the syntax of the simple scenario. Based on the analysis, we have developed the tool with TVML engine working on the Unity game Engine. We have also used this tool to imitate the broadcasted TV program to validate its usability.
Magnetic Resonance Imaging (MRI) was used for analyzing speech articulations. A synchronized sampling method, which is that a subject person repeats speaking a same pseudo word at a tempo by hearing click sounds, was used to enable capturing speech articulators' motion data from relatively slow imaging. For Japanese syllable sequences of /a-ei- u-e-o-a-o/, /ka-ke-ki-ku-ke-ko-ka-ko/, and /ga-ge-gi-gu-ge-go-ga-go/, positions and motions of speech articulators were compared for analyzing detailed differences of speech articulations depending on patterns of consonants and vowels. To analyze time series of narrowing positions of a vocal tract caused mainly by tongue motions, a kymograph method from MRI images is proposed. A thin slice is taken from MRI image in a vertical direction relative to a vocal tract. This thin sliced image has upper and lower boundaries of a vocal tract, that is, a hard or soft palate and an upper surface of a tongue. The same slices are taken from all frames and then they are put in a line in a sequence of time. This sequence of sliced image is looked like a kymograph. It will be a useful tool to observe time series of distance changes of one section of a vocal tract, caused by tongue motions.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.