VEGA Institute periodically establishes review metrics to capture the performance of students and faculty as well as the effectiveness of its online learning offerings. Reviews reveal valuable information such as the number of times a student has attended a class, the time spent in a class, and the overall engagement in a course. Student evaluation surveys on self, course, and instructor performance as well as student progress tracking are common features of each course.
A suitable benchmark for a thorough review and evaluation of courses and programs is considered to be at least every other year. Faculty and administration regularly evaluate the university’s strategic plan while referring to enrollments, revenues, and learning outcomes measured against the predetermined goals. Faculty and administration, in addition, take time to analyze and evaluate the reviews, and they work together to make necessary adjustments to realize desired improvements. During such evaluation new opportunities need to be identified and considered, or necessary improvements be performed.
Defining Metrics
Typically, the metrics to be measured pertain to a specific course, and in many cases track the same results so to discover whether the intended teaching and skill-set were accomplished for each particular course. By knowing which metrics to track, online instructors are able to reveal their students’ standing in a course and to tell how their courses are perceived by students. Presented below are some examples of what can be tracked and how can they be tracked.
Student Satisfaction
How to track it: End of course survey
Electronic surveys are distributed to students who have completed a course, and they are asked to evaluate the course and the instructor. Surveys reveal valuable information in terms of students’ likes and dislikes about the course, allowing as such the university to benchmark against those when re-designing or improving a course and its accompanying arrangements.
Completion Rates
How to track it: Course records
Course records, those recorded through the LMS or instructor’s records, reveal the portion of the course that is completed by any student. Course completions are tracked over time, and reveal any outliers in terms of repeated periods during which students have been noticed to leave the course. The metrics allows tracking of those changes and analyzing their effect on the course completion rate.
Skills and Knowledge
How to track it: Course examination and coursework
Providing coursework and a final examination is crucial for online learning as it is an opportunity to put to test the knowledge of course attendees and evaluate their learning in the course.
What do metrics reveal?
1. Student Perception
This criterion is evaluated through surveys, formative course feedback, or one-on-one interactions. Potential questions that result in evaluating this criterion might include:
- What was students’ general perception or thought of the course?
- Which course aspect was the most-liked?
- Which course aspect was the least-liked?
- Would students recommend the course to their peers?
2. Student Learning
This criterion intends to find out if the students have acquired the knowledge and skills they were expecting to obtain. It can be assessed through final examination or student self-evaluative performance surveys. Potential questions that result in evaluating this criterion might include:
- Did students learn what they expected to learn?
- Has this course improved their knowledge, skills, outlook and confidence?
- Were all the necessary resources provided to succeed in this course?
3. Student Behavior
This criterion measures how well the knowledge and skills gained in a course prepare the student body for the workforce. It includes questions asked in forums, emails, placement surveys, social platforms, etc. Potential questions that result in evaluating this criterion might include:
- Could the knowledge and skills targeted in the course be applied and used in the workplace?
- Are students able to share the knowledge with others and function in a team spirit in the workplace?
- Have students noticed positive improvements in the workplace after taking the course?
4. Student Results
This criterion reveals if and to what extent planned course and program outcomes were accomplished as a result of taking the course. Potential questions that result in evaluating this criterion might include:
- Did the course increase student productivity?
- Did the course help improve student outcome results?
- Is student retention improving?
It is worth noting that such an approach where metrics are used to measure course effectiveness properly reveals the overall results of a course. Although it does not suggest corrective actions per se, it sets a proper base for planning the next steps in improving the quality of courses and programs. It is responsibility of faculty and administration to, upon evaluation of reviews, take appropriate actions to improve the results of course effectiveness.
Instructional Quality Assessment Guide
Instructors of online offerings have a responsibility to assess instructional quality throughout the duration of a course, not just at the end of the course. Depicted below are some criteria against which instructions can measure effectiveness and quality of the course, accompanied with the pertinent assessment methods.
| Criterion | Assessment |
| Do students demonstrate an understanding of course assignment/s? | Evaluate students’ emails, threaded discussions, chats, and office hours.Evaluate student work against marking criteria. |
| Do students demonstrate an understanding of the course content? | Evaluate students’ emails, threaded discussions, chats, and office hours.Evaluate student exams and coursework against marking criteria. |
| Are different learning practices and styles being utilized? | Compare instructional styles and strategies with those of other online offerings.Evaluate students’ emails, threaded discussions, chats, and office hours. |
| Are the course load and activities of the online course comparable to those of the traditional in-class format? | Analyze student concerns and if they are lacking behind with the deadlines.Evaluate students’ emails, threaded discussions, chats, and office hours.Compare student achievements with those of the class load. |
| What are student perceptions about the course? | Encourage and welcome continuous feedback.Suggest anonymous student feedback.Analyze email content, and student behaviors in threaded discussions, and chat communications. |
| How can it be ensured that students are attentive and participate in the class? | Require participation and group discussions.Structure assignments in a way that encourage discussions.Require an adequate number of assignments or activities. |
| Is there a group cohesiveness noticed in the virtual community? | Analyze and evaluate results of group projects.Evaluate quality of interactions (Do students appear to be interested about one another? Are they communicating regularly?)Evaluate students’ emails, threaded discussions, chats, and office hours. |
| Are the learning outcomes being met? | Evaluate student exams and coursework.Evaluate student questions, concerns or remarks.Compare grades on student work.Transmit the objectives clearly to students, and provide opportunity for discussions.Use student and instructor self-assessments. |
