1.

RELEVANT LITERATURE

Research indicates that the physical science workforce requires oral presentation skills (OPS)¹. 80% of survey respondents ranked OPS highly important to their daily work. OPS were found to be important in academia and industry. OPS were rated as more important than writing skills and were rated second only to critical thinking. Compared to ratings concerning the training of the same skills in graduate school, only 62% reported in-depth OPS training. Training regarding oral presentations was the third highest rated skill, indicating a gap between work requirements and graduate training.

Previous research collected surveys from 87 engineers². While 62% of those surveyed stated writing collaboratively as a job requirement, 78% of respondents indicated being required to give oral presentations. Multiple participants reported oral presentations as being required regularly. Out of thirteen areas suggested for improving engineering preparedness, OPS were mentioned most. Areas for improvement included keyboarding skills and training for preparing reports.

Previous research examined OPS in K-12 environments^{3, 4} focusing on eliminating problem behaviors or speaking anxiety. Research using college samples involved scientific and non-scientific presentation grading, courses for improving OPS, physiological responses while speaking, and public speaking anxiety^{5, 6, 7, 8, 9, 10, 12}. Recent research has acknowledged the scarcity of studies on OPS¹³.

The intervention in question is a year-round science and information technology program for 69 high school students, their parents and 59 teachers. Goals are to prepare underrepresented minority high school students for careers in science, technology, engineering and mathematics (STEM) directly and to aid parents and teachers in preparing students indirectly. The five program components are recruitment and retention activities; physics content; teacher professional development; parental engagement; and dissemination and evaluation. Each program component is operationalized through the principles of hands-on investigations; engagement in a supportive yet challenging environment; participation in leadership and professional development training; and interacting with outside professionals.

It was predicted that student demographic information and post-test scores for photonics and research methods would reveal a significant predictive model of student OPS scores. It was hypothesized that both post-tests would be uniquely predictive and positively correlated to student scores. Another hypothesis was that previous exposure to the same research group would add significant predictive ability to this model. Previous exposure was hypothesized to increase OPS scores due to increased formal experience discussing science. As novel skills, discussing scientific information or speaking in public would be considered controlled processes. Performing multiple controlled processes can be difficult and taxing. Automatizing a task aids an individual who is attempting to perform multiple tasks at once¹⁴. Public speaking is a common fear, and extra exposure to science might reduce a participant’s speaking anxiety. It was also hypothesized that a student’s total number of extracurricular activities would improve the predictive ability of a model including demographic information.

2.

METHOD

This paper is part of a larger study following students for three years.

3.

RESULTS

Multiple regression analysis was used to test hypothesis 1, that participant demographic and post-test information would produce a significantly predictive model of student OPS scores. The resulting model explained 56% of the variance in the dataset (F(8, 45)=5.98, p<.05). While a student’s cohort, age, weighted grade point average (G.P.A.), science course, extra quality points, and photonics knowledge post-test scores correlated positively with OPS grades, only initial G.P.A. (b=0.49, t=2.87, p<.05), science course (b=0.49, t=2.84, p<.05), and extra quality points (b=0.38, t=2.25, p<.05) were significantly predictive. While gender and research post-test scores correlated negatively with OPS grades, only research post-test scores (b=-0.42, t=-2.23, p<.05) were significantly predictive (see Table 3).

Multiple regression analysis was also used to test hypothesis 2, that both photonics and research knowledge post-test scores would be significant predictors of student oral presentation grades. While the model was significant, only research post-test scores (b=-0.42, t=-2.23, p<.05) were significantly predictive. Photonics post-test scores (b=0.13, t=0.86, p>.05) were not uniquely predictive of OPS scores.

Hierarchical regression testing investigated the hypothesis that previous exposure would be a unique predictor of OPS scores and would add a significant amount of variance to the model from hypothesis 1. Block 1 contained the variables of cohort, age, weighted G.P.A., gender, science course, extra quality points, photonics post test score, and research post test score. Block 1 was significantly predictive, explaining 56% of the variance in the data (F(8, 45)=5.98, p<.05). While cohort, age, G.P.A., science course, extra quality points, previous exposure, and photonics knowledge post test scores all correlated positively with OPS grades, only G.P.A. (b=0.49, t=2.87, p<.05), science course (b=0.49, t=2.84, p<.05), and extra quality points (b=0.38, t=2.25, p<.05) were uniquely predictive. Gender and research knowledge post test scores correlated negatively with OPS grades, but only research post test scores (b=-0.42, t=-2.23, p<.05) were uniquely predictive (see Table 2). Block 2 added previous exposure which correlated positively to the criterion, with a non-significant change in variance explained resulting (F(1, 36)= 5.18,p<.05). All unique predictors from Block 1 were predictive in Block 2.

Table 1

Results of Multiple Regression for Student Oral Presentation Scores using Demographic and Posttest Data

Note. *p<.05

Table 2

Results of Hierarchical Regression for Student Oral Presentation Scores using Demographic, Posttest Data, and Previous Intervention Exposure

Note. *p<.05

Hierarchical regression investigated unique predictability for student extracurricular activities. Block 1 consisted of cohort, age, G.P.A., gender, science course, extra quality points, photonics post-test score, and research post-test score. Block 1 was significantly predictive, explaining 56% of the variance in the dataset (F(8, 45)=5.98, p<.05). Cohort, age, G.P.A., science course, extra quality points, previous exposure, and photonics post test scores correlated positively with OPS grades, but only G.P.A. (b=0.49, t=2.87, p<.05), science course (b=0.49, t=2.84, p<.05), and extra quality points (b=0.38, t=2.25, p<.05) were uniquely predictive. Gender and research post-test scores correlated negatively with OPS grades, but only research scores (b=-0.42, t=-2.23, p<.05) were uniquely predictive (see Table 3). Block 2 added the number of student extracurricular activities, resulting in a non-significant change in the amount of variance explained(F(1, 36)= 5.38, p<.05). Extracurricular activities correlated positively to OPS grades without being uniquely predictive. Unique predictors in Block 1 remained significantly predictive in Block 2.

Table 3.

Results of Hierarchical Regression for Student Oral Presentation Scores using Demographic, Posttest Data, and Total Number of Extracurricular Activities

Note. *p<.05

4.

DISCUSSION

The hypothesis that demographics and post-tests data would result in a predictive model of OPS scores was supported. The hypothesis that both post-tests would be significantly predictive of OPS scores was partially supported. Previous exposure and number of extracurricular activities failed to explain significant amounts of variance. Thus, hypotheses 3 and 4 were not supported.

Lack of significant results for student cohort suggests that iterations in the multi-year program may not have significantly improved an individual’s OPS scores. Future testing should investigate this further. Age was non-significantly correlated with OPS grades. This may be due to the limited age range of participants. Future investigation should consider including a wider age range of students. Non-significance of photonics post-tests suggests that while increased general knowledge of photonics contributed to grades, it was not essential for discussing specific projects. Future test versions could include open-ended questions to mimic students’ expressing original thoughts. Gender correlated non-significantly and negatively with OPS scores. This should be further investigated using between groups testing. Previous exposure correlated positively to OPS scores but was not significant. This suggests although multiple programs by the same research group contribute to developing OPS, their benefit was not strong enough to be detected. The number of extracurricular activities correlated positively but non-significantly with OPS scores. Future investigation should examine the differing utility of total number of extracurricular activities versus only science-based extracurricular activities. The finding that the research posttest was uniquely predictive suggests research-based extracurricular activities may be beneficial for developing OPS.

Results suggest students taking science courses further along the NCSCOS were better equipped to present information. An explanation for this finding is in students’ improved mental models for science due to increased exposure to scientific topics. Results suggest taking a more rigorous course is related to effectiveness at delivering oral scientific information. This may be due to increased breadth or depth of topics covered, or it may be due to the nature of the assignments in such courses. Future research should examine how advanced courses contribute to improving OPS. Results suggest higher performance in overall coursework reflects an increased ability to present information. Explanations include that students with higher G.P.A.s have more developed OPS independent of coursework, or that students with higher G.P.A.s may be more likely to take more advanced courses which have an increased probability of using assignments that require public dissemination of information.

While the research test predicted OPS score, it was the only negatively correlated predictor. It is possible the research test focuses on declarative knowledge concerning scientific presentations instead of procedural knowledge. It is also possible that the research knowledge test focuses on procedural knowledge that students are unable to express due to automaticity. One of the difficulties in working with subject matter experts is their inability to verbalize reasons behind their actions or opinions. One of the issues in designing training meant to alter behavior is difficulty in helping participants translate declarative information into appropriate procedural modifications¹⁶. It is possible that students who knew more about proper research may have focused on elements of their study that were uninteresting or beyond the scientific understanding of those grading them. Finally, it may be that those with an increased knowledge of proper research are more likely to have deficits in their public speaking or other social skills.

Findings here may be of interest to those involved with interventions for high school students, but this research was not without limitations. Longitudinal efforts should attempt to determine if the predictive value of variables identified in this research remains over time with the same students. The current research paired demographic application information with student test scores after treatment. Future research should attempt to replicate findings with more current information as course load rigor may change over time.

While the photonics knowledge test was positively correlated with OPS grade and the research knowledge test was a unique predictor of presentation score, both tests were developed in-house and are in need of validation. Future research should investigate the reliability and validity of these tools. The 10 judges averaged only 2-3 grades per student. Future research into student presentations should attempt to obtain a larger number of scores for each student, minimizing the risk of bias or noise due to individual judges.

The National Science Foundation Innovative Technology Experiences for Students and Teachers under the division of Research on Learning in Formal and Informal Settings fully funded the Photonics Leaders II Program Award #0833615.