Online Project-Based Learning for ESP: Determinants of Learning Outcomes during Covid-19

Evaluating the learning outcomes of a certain course implementation can improve the quality of education. This study is focused on Indonesian EFL undergraduates’ perceptions of the relationships between teacher performance, technological resources, online project-based learning (OPJBL)


INTRODUCTION
Course evaluation is a critical element of the learning system (Guba & States, 1981). Evaluation provides a strong platform for course execution decisions including comments on long-term changes and procedures. The evaluation process should mainly be concerned with the effectiveness and efficiency of putting educational policies into practice. What is to be evaluated includes contents, methods for local, national, and global concerns, and outcomes of educational program goals. Prior studies have advised academic stakeholders to keep a focus on the evaluation of classroom-based courses (Encandela et al., 2019;Karas, 2021;Villalba, 2022).
The advancement of information and communication technologies has undoubtedly altered the teaching and learning process. In fact, in the 21st century, using technology in the classroom and teaching and learning online through remote and open education systems may have become widely regarded as an effective method of teaching and learning. As a result, many educational researchers are keen to explore the benefits of integrating technology into classroom pedagogies to investigate the relationship between ICT (information and communications technology) and pedagogy. Technology, the internet, in particular, plays a crucial role in language teaching (Habibi et al., 2020b;Ulla & Perales, 2021). Various technologies in language education have transformed the classroom instructional environment and provided teachers and students with quick access to learning connections and rich online materials. Similarly, students can now build language learning connections with their friends and peers to participate in the continuous learning process. The application of technology in teaching processes has increasingly become widespread during Corona Virus Disease 2019  pandemic, particularly online learning.
While online education has been around for more than two decades, it still faces a number of obstacles in a variety of professions. Courses such as English for specific purposes (ESP) that rely heavily on hands-on skills can be harmed by the existence of additional obstacles posed during online learning. Students are unable to meet on a regular basis to converse in English about specific themes. Furthermore, due to the virtual setting, there may be a weaker tie between the students (Croft et al., 2010). Although systems like video conferencing allow teachers to organize project-based 987 | Studies in English Language and Education,9(3), [985][986][987][988][989][990][991][992][993][994][995][996][997][998][999][1000][1001]2022 learning groups, the students are physically separated. As a result, there is a chance of differing traits and dynamics among students (Dhawan, 2020), necessitating nesting variances at the personal level and conducting cross-level analysis for a more comprehensive view. Meanwhile, understanding the cognitive and epistemological foundations of the disciplines in ESP has enormous pedagogical potential for both students and teachers. It can offer more individualized guidance on the study and research methods and help students gain deeper insights into their chosen topic (Pennarola, 2019).
Limited studies, particularly in language learning, explored the implementation of certain techniques, such as online project-based learning (OPJBL) in the online environment during Covid-19 (Rahayu & Fauzi, 2020;Randazzo et al., 2021;Siska et al., 2022). In this study, OPJBL is defined as project-based learning conducted online, utilizing learning tools such as social media, video conferencing, and a learning management system. Evaluation studies on certain techniques are important to understand their impact on students as the main priority of teaching and learning activities. Therefore, this study was conducted to investigate the perceptions of ESP students, in the context of Indonesia, toward the effects of teacher performance and technological resources on OPJBL as an online learning technique during Covid-19. Besides, the influences of teacher performance, technological resources, and OPJBL were also explored toward ESP learning outcomes. Supporting the relationship assessment, and the difference tests of two variables (OPJBL and ESP learning outcomes) were also carried out. Three research questions are proposed in this study;  Is OPJBL affected by teacher performance and technological resources?  Are ESP learning outcomes affected by teacher performance, technological resources, and OPJBL?  Are there any significant differences regarding all proposed variables based on the respondents' location?

Project-Based Learning in Online Settings
Project-based learning (PJBL) is one of the greatest ways for teaching to develop crucial soft skills such as inventiveness and critical thinking. When students complete project tasks with proper decision-making, the analysis and resolution they have will enable them to acquire knowledge and problem-solving skills. PJBL allows students to learn about and reflect on real-world issues while developing 21st-century abilities such as cooperation, communication, problem-solving, and inventiveness (Guo et al, 2020). Besides, PJBL facilitates students' expression in their education and allows for a more accurate assessment of their abilities and skills. Its conversion to an online learning environment, however, necessitates thoughtful and deliberate preparation and should not be simplified to a scripted procedure of reading text, viewing videos, filling out virtual worksheets, and handling various quizzes (Awuor et al., 2022). In distance learning situations, the same degree of investigation, questioning, criticism, observation, scaffolding, and cooperation is required (Chen et al., 2019).
Four different ways to implement OPJBL into online learning are: being aware of the equipment that students utilize, using video conferencing systems for meaningful collaboration, employing program management tools for effective teamwork, supplying continuous feedback, and encouraging learning reflection (Awuor et al., 2022). Involving students in project-based assignments might be considered an efficient strategy to ensure that students in online classrooms use the knowledge learned in the course units (Chen et al., 2019). The OPJBL used in the ESP course within this study context were interconnected and subsequently integrated to form a bigger task; listening, speaking, reading, and writing should be integrated with various activities in PJBL related to the ESP.

Language Learning and Online Project-Based Learning
Online project-based learning (OPJBL) is a complete teaching technique in which students work together to study and solve an issue in an organized and supportive online environment. OPJBL is defined as a person or group action that takes place over time and results in production, presentation, or engagement via online instruction (Awuor et al., 2022). Project-based learning is reflected in this study via the creation of OPJBL. OPJBL typically has a timeline, milestones, and other features of formative review as the project progresses (Heo et al., 2010). This innovative instructional technique involves students in gaining information and skills through an extensive investigation process organized around complex, genuine inquiries and carefully constructed products and assignments. The preceding concepts are represented in constructivist learning theory, where students create knowledge by attempting to construct meaning using their previous and present knowledge.
In language learning, when students engage in meaningful debate and interaction with more capable classmates or professors, concepts emerge, and understanding occurs. This encourages students to model problem solving, and aid in the discovery of solutions, enabling them to track progress and assess success while participating in PJBL. PJBL is undeniably advantageous to students because they learn and practice skill sets and language skills while completing project activities. PJBL encourages students to distribute concepts learned in configurations that are similar to the real contexts into prospective scenarios, as students believe language learning is connected to their daily lives (Petersen & Nassaji, 2016). It solidifies students' diverse needs and abilities in the cooperative nature of the project task (Moss & van Duzer, 1998). In addition, PJBL provides students with project-related skills such as planning, organizing, negotiating, reaching consensus, accepting responsibility, conducting research, and presenting information (Petersen & Nassaji, 2016). The activities allow students to gain significant insight and interpretation from their peers and materials, beyond their current language knowledge (Kearney & Ellis, 1995). Incorporating PJBL into a professional communication skill unquestionably increases language learning while meeting employer needs. Project-based work is seen to be ideal for assessing both language and employability abilities (Moss & van Duzer, 1998).

The Study
The four constructs examined in this study are teacher performance, technological resources, OPJBL, and learning outcomes. To examine the disparities appearing in all variables, demographic information (location) was also incorporated (see Figure 1). Previous research has evaluated the quality of teacher performance in terms of educational, training, and research outcomes (Arimoto et al., 2012). Teacher performance focusing on the quality improvement of their teaching could increase the degree of student motivation and the quality of higher-order thinking skills based on the implemented course (Brown et al., 2011;Dehdashti et al., 2013;Fu & Sibert, 2017;Khalil & Kibble, 2014). The purpose of this study is to examine if teacher performance affects OPJBL and ESP learning outcomes to better understand the role of teacher performance in ensuring course implementation success and improving learning outcomes. Prior research has also found that instructional resources are essential influences in ensuring course implementation success (Cobb et al., 2015;Mehralizadeh et al., 2017). The term 'technology resource' is used in this study to refer to all items needed to implement the program. The impacts of resources such as the library, computers, classroom space, and laboratory facilities on the OPJBL and learning outcomes have been investigated. The course implementation is a multi-step process involving many different approaches. The totality of information, knowledge, abilities, and competencies that students should have after an ESP course is referred to as ESP learning outcomes. Aside from examining factors that influence the OPJBL and learning outcomes, demographic data were also included (Hammer, 2011). Demographic information provides data on research respondents and is required to establish whether respondents in a given study differ in their attitudes toward the topics at hand. To complement the structural model, the location of the respondents was included for the different tests related to OPJBL and learning outcomes.
The instrument was distributed to all ESP students after one-semester implementation of the OPJBL during school closure due to Covid-19. A 7-point Likert-type scale instrument was included to achieve the purposes of the study and it was discussed with four users and five experts. The main data were computed in the SmartPLS 3.3 to report the measurement and structural model of the study. Besides, the data were also computed through SPSS 23.0 for the tests of difference. A number of nine hypotheses were included in this study: H1. Teacher performance will significantly affect OPJBL. H2. Technological resources will significantly affect OPJBL. H3. Teacher performance will have a significant role in affecting ESP learning outcomes. H4. Technological resources will have a significant role in affecting ESP learning outcomes. H5. OPJBL will have a significant role in affecting ESP learning outcomes. H6. Teacher performance will be statistically different based on location. H7. Technological resources will be significantly different based on location. H8. OPJBL will be statistically different based on location. H9. ESP learning outcomes will be significantly different based on location.

Instrumentation and Data Collection
A self-administered questionnaire was developed for this study, which was adapted from previous research (Brown et al., 2011;Mehralizadeh et al., 2017;Watson et al., 2013). Furthermore, consultations with teachers having extensive expertise in ESP greatly contributed to the questionnaire development. The perceptions of students who have experience with the OPJBL, attending the course for one academic semester, were evaluated using four constructs: teacher performance, technological resources, OPJBL, and ESP learning outcomes (see Table 1). The scale included descriptive indicators for self-reporting on the OPJBL and the learning outcome, intending to improve the ESP course quality. Our survey instrument comprised thirty indicators in its first draft. The survey's initial validation included suggestions from users and professionals; the users were four ESP students having similar characteristics to the main samples of the study, while the professionals were five curriculum and educational technology experts with experience in research and teaching activities. In face validity, four ESP students contacted had previously attended the OPJBL. The extent to which an initial evaluation or assessment appears to assess the construct is defined as face validity. When an assessment appears to perform what it claims to do, it is tested for face validity (Nevo, 1985;Umanath & Coane, 2020). As a result, users' perceptions of the scale proposed in this research are critical. Five experts, including two academics in curriculum and instruction and university professors with experience in ESP, were invited to discuss the indicators as part of the content validity.
Through these two phases of initial validity procedures, five indicators were dropped, while five others were revised. Three indicators were dropped due to unsuitable context and setting, and two were eliminated because of repetition statements. Five indicators were revised to facilitate understanding of the indicator statements. As a result, twenty-five indicators were present in the final version distributed for the main data collection (see Table 1). Responses were given on a 7point Likert-type scale, and a back-translation procedure was implemented (Behr, 2017). The scale was translated from English to Indonesian language and vice versa. Two translation experts were invited to discuss the back-translation process. The degree to which respondents perceived enhancements of their cognitive knowledge, skills, and abilities after the implementation of OPJBL in ESP.

Respondents
The target subjects consisted of students who had attended ESP courses through the implementation of OPJBL. The survey was conducted as an evaluation tool for the course implementation. The inclusion criterion was having attended ESP courses. For sampling procedures, *G power 3 was used to examine multiple linear regression (Mukminin et al., 2020). *G power is a statistical analysis program designed to analyse different types and different statistical tests, such as the F, t, chi-square, and z tests. Issues relating to sample size can also be computed in *G power (Erdfelder et al., 1996). For four key predictors, the computation showed 146 minimal samples for this study. This study collected 347 measurable responses from 350 respondents or students who had passed ESP courses with OPJBL as one of the teaching strategies from two institutions.

Data Analysis
To be able to draw conclusions from the study, three phases were conducted: measurement model evaluation, structural model evaluation, and demographic difference level evaluation. The Partial Least Square Equation Modelling (SEM-PLS) procedure was used to assess the measurement model, including implicit or explicit models relating the latent variable to its indicators, by reporting reflective indicator loadings, internal consistency reliability, convergent validity, and discriminant validity (Habibi et al., 2021;Hair et al., 2019). Similarly, with the elaboration of path coefficient and R2, the structural model was evaluated using PLS-SEM techniques. A t-test was used to determine whether there were any changes in OPJBL and ESP learning outcomes based on respondent location

Measurement Model
As previously mentioned, the assessment of indicator loadings, internal consistency reliability, convergent validity, and discriminant validity were included to report the measurement model of the study. For the indicator loadings, according to the recommendation by Hair et al. (2014), the outer loadings should be >.708. However, indicators loading above .500 were still retained. Since some experts argue that if indicators loaded between 0.5 and 0.7 are to be included, the values of average variance extraction (AVE) must be between 0.5 and 0.7 (Hair et al., 2011;Noor et al., 2019). From the computation, two items' loadings were below 0.600. All loadings having values below the threshold should be subsequently dropped (Hair et al., 2014). The items dropped in this phase were OPJBL1 and OPJBL2. After cleaning the lowvalue of loading, 23 items remained for the next assessment phase of the measurement model (see Table 1).
Further, the internal consistency reliability was used to evaluate the consistency of results across items. Cronbach's alpha and composite reliability (CR) were tested (Habibi et al., 2020a), in which the values of the alpha and CR should be between 0 and 1. Greater values indicate a higher level of reliability. Both values must be higher than .700 (Hair et al., 2019). Table 1 presents the details of alpha and CR that are stable and have appropriate internal consistency reliability exceeding the recommended value of .700. For convergent validity, AVE values are recommended as the metric (Hair et al., 2019). The minimum acceptable AVE is >.500, which explains 50% or more of the items of the construct. From the computation, all constructs reached values higher than .500. Thus, convergent validity is not an issue in this measurement (see Table 1).  Language andEducation, 9(3), 985-1001, 2022  The discriminant validity is the extent to which a construct is empirically distinct from other constructs in the structural model (Hair et al., 2019). The discriminant validity was assessed through cross-loading and the heterotrait-monotrait (HTMT) ratio of correlations. The cross-loading could emerge if loading on a construct is greater than that of all its cross-loadings on the other constructs (Hair et al., 2019). Based on the computations, the outer loadings in Table 2 (in bold and italic) for every construct were greater than the whole cross-loadings on the other constructs. To sum up, the discriminant validity is established through the assessment of cross-loadings. Another assessment (the main consideration for the discriminant validity) is HTMT. The threshold for HTMT in this study follows the rules of Hair et al. (2019); HTMT should be below .900. As shown in Table 3, all HTMT values are below .900. Therefore, the discriminant validity of the active constructs is acceptable.

Structural Model
Assessment of the structural model for the course evaluation included an examination of the model's predictive capabilities. The process began with an examination of multicollinearity. The path coefficients (β) and the coefficient of determination (R 2 ) were examined. Multicollinearity issue emerges when two or more predictors in the model are correlated, which provides redundant information regarding the response. Multicollinearity in PLS-SEM should be measured by variance inflation factors (VIF). When VIF values exceed 4.0, there will be an issue with multicollinearity (Hair et al., 2010). Two sets of predictors were examined to obtain the VIF values, 1) Teacher performance and technological resources as the predictors of OPJBL; 2) Teacher performance, technological resources, and OPJBL as the predictors of learning outcomes (see Table 4). All VIF values were lower than 4.0; thus, multicollinearity is not an issue for the model of this study.
The assessment of the structural model is done to assess if the relationships are significant between exogenous and endogenous variables. The data were bootstrapped with a sub-sample of 5000, reporting that five coefficient correlations are significant with an assumption of a 5% significance level. The significances support H1 (teacher performance will significantly affect OPJBL), H2 (technological resources will significantly affect OPJBL), H3 (teacher performance will have a significant role in affecting ESP learning outcomes), H4 (technological resources will have a significant role in affecting ESP learning outcomes), and H5 (OPJBL will have a significant role in affecting ESP learning outcomes). Teacher performance has a significant contribution to OPJBL (β=.702; p<.01) that supports H1, the highest significant relationship. Similarly, technological resources have significant effects on OPJBL (β=.125; p<.01), supporting H2. Teacher performance significantly predicts OPJBL (β=.392; p<.01) that endorses H3. ESP learning outcomes are also significantly predicted by technological resources (β=.164; p<.01), confirming H4. Finally, OPJBL was found to be significant in determining ESP learning outcomes (β=.331; p<.01); supporting H5. The coefficient of determination (R 2 ) is reported to support the structural model. The R 2 is defined as the value measuring the model's predictive accuracy and is calculated as the square's correlation between a specific endogenous construct's actual and predicted values (Rigdon, 2012). R 2 value should be between 0 and 1. A greater value indicates a greater level of predictive accuracy. The R 2 value of .75 is considered substantial, .50 is moderate, and .25 is weak (Hair, et al., 2019). Figure 2 exhibits the result of R 2 ; OPJBL (R 2 =0.594, moderate), and ESP learning outcomes (R 2 =0.588, moderate). In conclusion, the data of this study is at an appropriate level of predictive accuracy or R 2 .

Differences Based on Respondent Location
In addition to the structural model reported in this study, demographic information (respondent location) is different regarding all variables; teacher performance, technological resources, OPJBL, and ESP learning outcomes. A number of 145 respondents were urban residents, while 202 were rural residents (see Table 5). The t-test findings reported that a significant difference emerged between locations regarding teacher performance (t=6.637; p<.01); the result confirms H6 (teacher performance will be statistically different based on location). Similarly, the technological resources are also reported to be significantly different based on location (t=2.079; p<.05), supporting H7 (technological resources will be significantly different based on location). OPJBL and ESP learning outcomes are also different based on the research participants' location; with the significance levels being below .01 with t values of 5.941 and 6.829, respectively. The results reveal that H8 (OPJBL will be statistically different based on location), and H9 (ESP learning outcomes will be significantly different based on location) are accepted. Table 6 presents the details of the t-test results.

DISCUSSION
Construct validity and reliability of the current study were assessed using face, content validity, and measurement model. Based on the computations, 23 valid and reliable indicators were reported. The valid and reliable criteria of the scale might be used to shape future evaluations of ESP courses with similar interests in diverse locations and scenarios; future researchers can adopt, adapt, or extend the instrument. Previously, certain studies have been conducted in the context of curriculum and educational technology with similar procedures to validate and assess instrument reliability (Hernández-Ramos et al., 2014;Mehralizadeh et al., 2017;Scherer et al., 2017), supporting the results of these validation procedures.
Furthermore, all hypotheses' significant correlations are reported in this study. Teacher performance had a direct impact on OPJBL (H1), with the structural model showing the strongest link. This influence was around five times that of the technological resources and OPJBL combined (H2). This means that the quality of the teacher's performance is critical in explaining the course integration in the ESP course after one semester of deployment. In other words, students' opinions of OPJBL integration will increase significantly if they receive more mentor encouragement in course integration (Gerlach, 2008).
According to a recent study, academic course assessments should be centered on the teaching performance activity (Koo et al., 2016). Studies have significantly informed that student-teacher interactions are crucial for successful course integration in education (Fu & Sibert, 2017;Khalil & Kibble, 2014). Similarly, the current study found that instructor performance and technological resources have a significant effect on ESP learning result prediction (H3 and H4). Academic departments with technology programs could increase teacher performance by taking into account their contacts with students, qualifications, knowledge, and effective teaching methods. Quality improvement strategies and technological resources could be applied to assist students to raise their standards for their future careers. Therefore, the results indicate that in the ESP setting, the two factors (teacher performance and technological resources) must be improved to increase learning outcomes.
The OPJBL course quality can be characterized as a determining factor in increasing ESP learning outcomes in this study (H5). Students who rated the OPJBL materials as appropriate reported an increase in their learning abilities, teamwork, and knowledge. According to the reports, OPJBL has a considerably favourable impact on student ESP achievement. The types of instructional methodologies used at the tertiary level play an important role in providing students with the knowledge and skills needed to succeed in their chosen profession (Watson et al., 2013;YuekMing & Manaf, 2014). Educational activities that engage students in a planned course may inspire situations that can improve educational outcomes and students' future professional preparedness regarding their English skills.
The differences in all variables (teacher performance, technological resources, OPJBL, and ESP learning outcomes) were computed to support the conclusions based on respondents' location. According to the t-test, respondents from urban and rural environments vary in all factors. Students living in cities benefit more from technology access (Silviyanti & Yusuf, 2014), which influences all types of opinions they had about OPJBL ESP in this study. Furthermore, school sites play an important role in the differences in OPJBL and ESP learning outcomes, alluding to the disparity between cities and villages in terms of digital technology infrastructure and training (Conrads et al., 2017).

CONCLUSION
The structural model, generated from this study, is beneficial for evaluating ESP courses and allowing all stakeholders to gain important data for internal program design and assessing the educational environment fit. One of the most successful ways to teach an ESP course is through project-based learning (Shaalan, 2020). The students not only study the fundamental concept of ESP theoretically but also build projects to demonstrate it.
The current study, however, does have some limitations. The research only analysed data from two academic institutions concentrating on ESP courses, and the variables developed may be limited to this study field. As a result, alternative contexts and study settings for course evaluation are suggested. Further data collection methods such as interviews, experiments, and research and development are recommended for future investigations. The suggestion can serve as a guide for different majors in using internet devices to perform project-based learning. It is critical to advance online teaching and learning, particularly during Covid-19, when schools are closed, and face-to-face learning is prohibited in several big cities to reduce the Covid-19 spread.