Integrating Industry-based problems in Performance Testing and Optimization Course: an experience report

As part of SIT Infocomm Technology (ICT) Software Engineering programme curriculum, students are introduced to various software performance testing methodologies for software systems under varying load, stress, and scales in the Performance Testing and Optimization module (ICT3102). The learning outcomes of the module expect students to be able to establish requirements for the performance of a computer system, and develop and apply appropriate metrics to evaluate the performance of a system. To ensure that students achieve these learning outcomes, several authentic assessments [1] have been incorporated into the module’s curriculum. In contrast to traditional paper-based assessment which tend to be too small scale and unrealistic, authentic assessment evaluates if students can successfully transfer the knowledge and skills gained in the classroom to real—world complex problem [2,3,4]. A study was designed to determine whether integrating industry-based problems enhances student learning in the subject. A team-based assignment was designed to determine whether students can establish performance requirement and develop relevant performance metrics for a real industry application. The industry problem was derived from an on-going applied research project, in line with the vision of applied learning-research nexus at the university [5]. The industry application has been deployed as part of the SIT@NYP Living Lab, and faces real-life performance issues such as speed and concurrent access, commonly found in most real-life deployed solutions. Student solutions were evaluated based on criteria such as expected workload, performance requirements, architectural diagram and results of the performance tests. In line with the characteristics of authentic assessment, there is no single correct solution to the problem. Students are assessed on their overall thinking process and how they apply their classroom knowledge to the real-world problem. To supplement classroom knowledge, students were also exposed to current industry-relevant practices where industry practitioners were invited as guest lecturers and students get to experience hands-on lab session using the most current industry technology. Analysis was performed using data gathered from student feedback survey conducted online after the assignment deadline. Eight (8) students taking ICT3102 in Trimester 1 AY2021/22 responded to the survey. When asked to rate the difficulty of the given industry problem, ~57% indicated that the problem was hard, while the rest indicated the problem as medium (28.57%) or easy (14.29%). Majority of the respondents (71.43%) indicated that they spent between 5 to 10 hours to complete the team-based assignment, while 28.57% indicated that they spent more than 10 hours. Nevertheless, 100% of respondents agreed that using real-life industry problem has enhanced their learning of the subject. More than 70% of the response agreed or strongly agreed that they were overall satisfied with the incorporation of industry-based assignment in the module. Qualitative feedback from the students indicated that students appreciate the fact that the industry-based assignment allowed them to discover concepts that are not typically covered in classroom and better prepare them for the workforce. In summary, educators should not need to choose between authentic assessment and traditional assessments. A mixture of the two is probably the best combination to ensure that students have a good knowledge base about the subject, which are best assessed in traditional manner, and also able to apply the knowledge in real-context, assessed through authentic assessment. Industry-based problem are a perfect vehicle for authentic assessment benefitting all stakeholders. Students are exposed to real-life problems and gain the experience of applying classroom knowledge to more realistic scenarios. Faculties benefit by synergizing their applied research to their teaching, enriching module content, while industry collaborators benefit from various solutions presented by students, many of which highlight issues that are typically not found in regular tests.