|
|
1.INTRODUCTIONSelf-regulated learning, which involves adapting one’s learning activities strategically and intentionally to achieve specific learning objectives, has been shown to improve student performance and expand depth of thinking [1-3]. Reflection and reflective writing is often used to encourage self-regulated and life-long learning skills. In recent years, there has been a growing recognition that engineering programs should provide greater opportunities for student reflection [4,5]. Typically, reflective is used in project-based and design courses, e.g., capstone design courses, especially to promote amongst students the use of reflection in professional practice [6,7]. However, there are fewer studies that examine the impact of reflection on student learning of conceptual and technical ideas in courses that are not focused on design [8,9]. In [8], Benson and Zhu use self-assessment (SA) in two core mechanical engineering courses, Introduction to Engineeirng and Solid Mechanics, to allow students to determine weaknesses and develop strategies to improve their performance on tests while in [9], Clark and Dickerson explore the impact of reflective activities in electrical and computer engineering courses on Digital and Analog Electronics. In this paper, we examine the impact of two types of reflective writing exercises in an introductory photonics course. Our study follows those conducted in [8] and [9] that focus on the use of reflection to enhance student learning of conceptual and technical issues. 2.COURSE DETAILSECSE 430 Photonic Devices and Systems is a final year technical elective course taken by electrical engineering students. During the Fall 2018 semester, ECSE 430 was co-taught with ECSE 540 Photonic Devices and Applications, a graduate version of the course (broadly speaking, both courses have the same learning outcomes). The course introduces photonic devices and their applications and specific topics include:
In terms of background, undergraduate students will have completed courses on electromagnetism and electromagnetic waves; they will also have seen some basic concepts of waves and physical optics in their freshman-level physics courses. On the other hand, the background of graduate students is highly variable; for some students, this is the first course that they took in photonics while others had familiarity with some or most of the topics The total enrolment was 30 students—17 undergraduates and 13 graduates. The course was taught using a flipped classroom approach. Students were expected to learn material on their own ahead of class through assigned readings in textbooks and/or course notes; class time was devoted to reviewing concepts, in-class polling questions, peer instruction, problem solving activities, and occasional content delivery. Formally, the class involved 3 hours of lecture and 2 hours of tutorial per week; however, there was no distinction between the lectures and tutorial. The 5 hours of contact time were run by the instructor with occasional assistance from the TA. Assessments (% of final grade) included reflective writing exercises (5%), pre-/post-class online quizzes (5%), 5 assignments (30%), and 4 in-class tests (60%). 3.REFLECTIVE WRITING EXERCISES AND EVALUATIONReflective writing exercises were used to provide students with an opportunity to engage in self-direction and self-evaluation, as well as a means to promote a better understanding of specific conceptual and technical topics. Both courses implemented two types of reflective writing exercises: exam wrappers (EWs) and SAs. 3.1Exam wrapperEWs are reflective exercises used to engage students in reflection following a test. In our implementation, the graded tests were returned to the students in the class immediately following, i.e., within 24. During class, a quick overview of the test was provided and then students were asked to complete a questionnaire to guide their reflection. The questions were designed to allow students to reflect on gaps in their knowledge of concepts and technical content. Specifically, students were asked to complete or respond to the following:
Students completed an EW after each in-class test. 3.2Self-assessmentIn this exercise, students responded to a series of questions that were again designed to guide their reflection. The specific points to address and questions to answer were the same for all reflective writing exercises. In particular, students were asked to describe the following:
There were 4 SAs that were assigned after the in-class tests; they were intended to supplement the reflections from the EWs and focused on specific technical issues that might not have related to questions on the tests. 3.3Implementation detailsThe EWs comprised a written questionnaire that students completed and uploaded via a learning management system (in our case, myCourses), along with a scanned copy of their marked test. While completing the EWs were not mandatory, each was worth 1 point out of the 30 of each test grade. No specific format was adopted for the SAs; students simply uploaded their responses to the questions on myCourses or via email to the instructor. Students received full credit towards the reflective writing component of their final grade if they completed 80% of the SAs, i.e., this was a participative exercise and the quality of reflection did not impact the students’ grade on the reflective writing component. 4.METHODS4.1Assessment of reflective writingThe quality of each reflective writing exercise was scored according to a set of rubrics that examined whether the students produced a coherent, relevant, and thoughtful reflection. Specifically, an ideal reflection was defined as one in which the student ‘clearly summarizes a thorough and honest effort of self-assessment relative to pre-defined learning objectives, technical concepts, or evaluation questions.’ Moreover, the reflection is ‘complete, focused and detailed, and identifies clearly problematic matters and specifies a strategy that can be adopted to address weaknesses in understanding or exam preparation.’ The length of a student’s reflection was not a factor in determining its quality. The reflections were scored on a scale of 0 to 3, depending on the extent to which the reflection satisfied the following statement, ‘the student’s reflection aligns well with the description of an ideal reflection’: The rubrics and scoring system were designed to align with the four-category scheme developed by Kember et al. [10]: (1) non-reflection (score of 0), (2) understanding (score of 1), (3) reflection (score of 2), and (4) critical reflection (score of 3). Each reflective writing exercise was scored by the instructor as well as a graduate research assistant who also served as a TA for the course. The scores were then added to obtain a total score out of 6. 4.2Assessment of student performanceFor each student, we determined their average score of the quality for all 8 reflective writing exercises (4 EWs and 4 SAs). We then grouped the students into 3 categories based on their average score: (1) Group A with an average score < 3 (associated with the categories of non-reflection and understanding), (2) Group B with an average score between 3 and 5 (associated with the category of reflection), and (3) Group C with an average score between 5 and 6 (associated with the category of critical reflection). First, we examined overall course grade as a function of average score of the quality of reflective writing. Next, for each group of students (i.e., Group A, Group B, and Group C), we examined (1) the average group grade relative to the average class grade for each in-class test and (2) the average group overall course grade relative to the average class overall course grade. 4.3Student surveysThe students were surveyed to gauge their opinion on the reflective writing exercises. The survey was conducted in-class using TurningPoint and the responses were recorded anonymously. Approximately 50% of the students responded to the poll questions. The questions are shown in Table 1. Table 1.Survey questions used in class to gauge student opinion on the reflective writing exercises.
5.RESULTSFigure 1 shows that students with a higher average score of the quality for the reflective writing exercises tended to perform better (this is generally expected and the results alone do not prove a causal relation). Figure 1.(a) Overall course grade as a function of the average score of the quality for all reflective writing exercises and (b) distribution of grades for students in Groups A, B, and C.  Figure 2 shows the difference between the average grades obtained by students in Groups A, B, and C and the average grade of the class for the in-class tests and overall course grade. Students in Group A, especially undergraduate students, performed below the class average on the class tests and had significantly lower overall grades. Undergraduate students in Group B also had grades below the class average on the tests; however, the group average overall course grade was only slightly lower than that of the class. It is clear that students in Group C had the best performance in the in-class tests and overall course grade. We observe that the reflective writing exercises seem to have a more variable impact on undergraduate students compared to graduate students who tended to perform close to or well above the class average. This may be attributed to the broader technical background and experience of the graduate students. Figure 2.Difference between group average and class average on in-class tests and overall course grade.  The results from the student surveys are shown in Figure 3. A significant percentage of students take more than 10 minutes to complete each reflection (Question 1). We infer that the students take the exercise seriously and do not simply write for the purposes of obtaining credit (especially for the SAs). Nearly half of the students found that the reflective writing exercises contributed to their learning (Question 4). At the same time, however, slightly more than one third never consulted their reflections (Question 2) or did not use their reflections in preparing for class and or in-class tests (Question 3). These latter results suggest that the objectives and benefits of reflection need to be communicated better to the students, and reinforce the notion that opportunities for reflection should be further integrated in engineering curricula. 6.CONCLUSIONSBased on our experience of implementing reflective writing exercises in the course, we observe a general correlation between a higher quality of reflection and better performance and/or improvement in grades on similar types of questions. Careful design and delivery of the reflective writing exercises are required in order to maximize gain. In particular, students should be guided in their reflection and it may be more beneficial to perform the SAs prior to assessments as the students can then take appropriate action, such as addressing gaps in knowledge and understanding. While reflection can be useful in all of the courses that a student is taking, we must ensure that they do not become overburdened with the process to the point that the quality of reflection degrades substantially. Further research is necessary to establish more concretely a causal relation between the quality of the reflection and student learning. REFERENCESNilson, L. B., Creating self-regulated learners: Strategies to strengthen students’ self-awareness and learning skills, Stylus Publishing, Sterling, VA
(2013). Google Scholar
Zimmerman, B. J.,
“Theories of self-regulated learning and academic achievement: An overview and analysis,”
Self-regulated learning and academic achievement, Routledge, New York, NY
(2013). https://doi.org/10.4324/9781410601032 Google Scholar
Winne, P. H. and Hadwin, A. F.,
“Self-regulated learning and socio-cognitive theory,”
International Encyclopedia of Education, 3Elsevier(2010). https://doi.org/10.1016/B978-0-08-044894-7.00470-X Google Scholar
Ambrose, S. A.,
“Undergraduate engineering curriculum: the ultimate design challenge,”
Bridge, 43
(2), 16
–23
(2013). Google Scholar
Turns, J. A., Sattler, B., Yasuhara, K., and Borgford-Parnell, J. L.,
“Integrating reflection into engineering education,”
in Proc. 121st ASEE Annual Conference & Exposition,
(2014). Google Scholar
Adams, R. S., Turns, J., and Atman, C. J.,
“Educating effective engineering designers: the role of reflective practice,”
Design Studies, 24
(3), 275
–294
(2003). https://doi.org/10.1016/S0142-694X(02)00056-X Google Scholar
Moon, J. A.,
“Learning journals: a handbook for reflective practice and professional development,”
2Routledge, New York, NY
(2006). https://doi.org/10.4324/9780203969212 Google Scholar
Benson D., and Zhu, H.,
“Student reflection, self-assessment and categorization of errors on exam questions as a tool to guide self-repair and profile student strengths and weaknesses in a course,”
in Proc. 122nd ASEE Annual Conference & Exposition,
(2015). https://doi.org/10.18260/p.24763 Google Scholar
Clark, R. M., and Dickerson, S. J.,
“Assessing the impact of reflective activities in digital and analog electronics courses,”
IEEE Transactions on Education, 2019). https://doi.org/10.1109/TE.13 Google Scholar
Kember, D., McKay, J., Sinclair, K., and Wong, F. K. Y.,
“A four-category scheme for coding and assessing the level of reflection in written work,”
Assessment and Evaluation in Higher Education, 33
(4), 369
–379
(2008). https://doi.org/10.1080/02602930701293355 Google Scholar
|
