Classroom
Assessment in Web-Based Instructional Environment: Instructors’ Experience
Xin Liang
University of Akron
Kim Creasy
Slippery Rock University
| While a great deal has been written on the advantage and
benefits of online teaching, little is known on how assessment is
implemented in online classrooms to monitor and inform performance and
progress. The purpose of this study is to investigate the dynamics
of WebCT
classroom assessment by analyzing the perceptions and experience of the
instructors. Grounded theory method was employed to generate a “process
theory”. The study included 10 faculties who taught WebCT classes, and 216
students in the College of Education in an urban university in the Mid west.
Interviews and classroom observations were undertaken on line. The findings
indicated that, performance-based assessment, writing skills, interactive
assessment and learner autonomy were major assessment aspects to inform
teaching and enhance learning. If one of the major roles of online instruction
is to increase self-directed learning, as part of the pedagogical mechanism,
web-based classroom assessment should be designed and practiced to impact
learner autonomy. |
The unique features, especially the
synchronous and asynchronous communication, web search, online resources and
technical support, allow teaching and learning to be place and time
independent. Although educators at all levels have embraced using online technology
as a teaching tool, the issue of assessment of student learning in an online
course has not been thoroughly addressed (Robles & Braathen, 2002). As an
instructional delivery method, online instruction should be designed to
facilitate teaching and promote learning. As (Meyer, 2002C) pointed out “It is
irrelevant to speak of the effects of using Web without understanding how it is
entwined with instructional design”. As instructors reflect upon online
learning as an instructional delivery mechanism, they must also examine their
assessment delivery method. They should ask questions about how assessment
practice as part of the instructional design is related to the quality of
online teaching.
Black and Wiliam (1998b) define
assessment broadly to include all activities that teachers and students
undertake to get information that can be used diagnostically to alter teaching
and learning. Under this definition, assessment is not limited to just
assigning grades to students in the form of paper-pencil exams. Assessment
should permeate many aspects of teaching and learning activities, encompassing
teacher observation, classroom discussion, group collaboration, and analysis of
student work. This form of assessment used as a regular element in classroom
work, holds the key to better learning (Broadfoot et al., 2001).
Web-based instruction takes place
on line, with different modes and resources for retrieving class content,
subject related information, and student-teacher interactions (Sherry, Bilig,
& Jesse, 2001). Without a teacher being physically present, web-based
teaching requires new instructional practices built on a unique relationship
between learners and instructors. As the mechanism of learning paradigms are
changed, so should the assessment delivery method. As a different
instructional practice, online teaching should practice a systematic assessment
that embraces and reflects the nature to this type of teaching and learning
environment. If we acknowledge that assessment impact student learning, it is
likely that assessment will be at the center of the curriculum design to ensure
the quality of online instruction. It is very important for educators to
examine the existing assessment practice, and seek guidelines applicable to the
design of assessment in online environment (Mcloughlin, & Luca, 2001, pp.
417). Penn State University in association with Lincoln University (Innovation
in Distance Education, 1999) provided some benchmarks for distance education
environment. Recommendations on assessment process are as follows:
- Enable students to self-monitor progress;
- Give regular feedback to students;
- Support peer learning and assessment;
- Design self-assessment practice.
These recommendations on assessment
process capitalized on the unique characteristics and situations of online
learners. Laurillard (1993) believed that one of the major roles for distance
learning is to promote self-directed learning and increased learner autonomy.
Collis and Moonen (2001) used the term pedagogical re-engineering to describe
the change in online pedagogy from one that is teacher centered to one that is
focused on learner activity. However, no research has been focused on the
interpretation of pedagogical reengineering to online classroom assessment.
The purpose of this study was to generate a theory that explains the “process”
of how assessment, as part of instructional practice in web-based environment
was implemented to promote learning from instructor’s perspectives. The
research questions that guided the exploration were:
- How assessment is designed and
implemented to reflect online instructional paradigm to facilitate teaching and
promote learning?
- What are the effective
assessment strategies available to support learning process on line?
- What is perceived to be the most
important assessment component (s) to obtain and process the information on
teaching and learning?
Assessment as Part of Instructional Practice
Findings from research on student
learning indicate that pedagogical techniques influence how well students learn
to apply concepts (Michlitsch, & Sidle, 2002). Achieving higher academic
standards for all students depends not only on a thorough knowledge of
pedagogical content, but also teachers’ ability to determine what students really
know and can do, and where the learning gaps are, so that they can target
instruction to fill the gaps. As Stiggins (1997) pointed out “good education
encompasses good assessment”. Rather than being an event at the end of a
course or period of learning, good assessment is an instructional event that
describes, and promotes students’ best performance across time and uses a range
of methods. Two experimental studies have shown that students who understand
the learning objectives and assessment criteria and have opportunities to
reflect on their own work show greater improvement than those who do not
(Frederikson & White, 1997). Elwood, & Klendowski (2002) believe there
is a distinction between “assessment of learning” (assessment for the
purposes of grading and reporting with its own established procedures) and “assessment
for learning” (assessment whose purpose is to enable students, through
effective feedback, to fully understand their own learning and the goals they
are aiming for). Assessment for learning requires teachers to never take a
student’s grade in and of itself as the only central goal of the assessment.
Rather, assessment performance is taken as a proxy for the student’s status
with respect to target instructional domain (Popham, 2002). No doubt, there is
a vital link between assessment, learning, and teaching. Instructors can build
in many opportunities to include students in the assessment of learning and
then use the information to make beneficial changes for both learning and
instruction. How to make it happen places a challenge for instructors,
especially for online instructors when the instructional environment and
communication devices are different. At the same time, the unique features of
web-based instructional environment also open up a new frontier for online
instructors to practice a more student-centered pedagogy.
Challenges and opportunities for online classroom assessment
Moving courses from the traditional
classroom to an online setting fundamentally shifts human interaction,
communication, and learning paradigms (Robles & Braathen, 2002). Jung
(2001) characterized three key components of Web-based instruction different
from traditional face-to-face classroom instruction as 1) content
expandability, 2) content adaptability, and 3) visual layouts. And also three
types of interaction were identified as essential for the success of online
teaching. They were: 1) academic interaction, 2) collaborative interaction,
and 3) interpersonal interaction (Jung, 2001). The characteristics of online
instruction present special challenges for assessment. As online instructors
can no longer monitor and react to student questions, comments, asides, body
language, and facial expressions, they must employ other techniques to acquire
the same information (Alessi & Trollip, 2001). The absence of low level of
social cues and emotions may minimize the richness of communication, limit and
impede a more interactive cyber learning community (Robels & Braathen,
2002). The verbal and written communication enjoyed in f2f classrooms has been
limited to written text with static images, slower because the student cannot
type as fast as he can talk and may be inexperienced in written communication
(The Institute for Higher Education Policy, 2000). It is often more difficult
to identify online cheating and student authentication as student has access to
various course materials, and impersonation is perceived as a greater risk
(Kerka & Wonacoot, 2000).
These same features in online
classes can offer a unique communication environment in which texts, pictures,
video and audio are integrated into one system. This system allows a much
easier access to huge database for students, and more flexible interactions
(Jung, 2001). The flexibility of communication to be time and place
independent increased the opportunities for dialogues, and more thoughtful
reflections. The unique features of online communication allows more
interactive assessment to not only accurately measure learning outcome, but
also to nurture peer feedbacks, and encourage participation. As more and more
educators and researchers realized that effective instruction with technology
must be driven by sound pedagogical principles (Daugherty & Funke, 1998),
it is very important to ask such questions as how this could be achieved, and
what aspects should be concerned for a more effective assessment to ensure the
quality of web-based instruction.
Method
WebCT learning environment
WebCT is a type of online teaching software that was
prevalent in the studied university. The WebCT classes included in the study
were web-based courses (over 50% of the course content delivered online)
offered by the college of education in a Midwestern university. There was no
restriction for students to register WebCT courses in the university.
Instructors were required to send in application and curriculum proposal to
move a course from traditional or web-enhanced (less than 50% online) to
web-based instruction. In the curriculum proposal, instructors were expected
to explain the rationales to move a course online, and a detailed teaching
plan. The proposal was reviewed by the department, the college council and,
approved by the provost office. The starting and ending dates of WebCT classes
were the same as f2f courses offered in campus. The course web page functioned
as the major setting for instructors to post class related materials. Students
log onto the class web to retrieve materials, interact with the instructors and
other classmates. Students and instructors usually did not meet f2f throughout
the whole semester. Some classes in the study did meet once or twice when
issues came up, and the instructors were needed to meet with students. Most
classes (13 out of 16) did not have a fixed “meeting” time (synchronous
communication). Each week, the instructor posted certain amount of content
materials for students to study by the form of class note, unit module or
studying materials of the week on the web. Instructors also posted
assignments, projects, cases analysis, and questions to help students
understand and exercise the content. Students were required to log onto WebCT
to retrieve materials, and to do self-study at their own convenience. Usually,
the class interactions between learners and instructors were asynchronous on
individual bases. Three classes required once a week (synchronous discussion
for two hours). Each member of a WebCT class was able to contact other members
privately or publicly via the WebCT mail list, or posting messages via threaded
or unthreaded discussions. Students were able to send in assignments, projects
or exams electronically to the instructors. Students were also able to obtain
both formal and informal assessment from the instructors on the web.
Participants
University faculties who teach
course content over 50 % on line were the target population. After sending out
consent forms, ten instructors (who taught 16 WebCT classes) agreed to
participate the study. The WebCT classes included in this study were both at
undergraduate and graduate levels in the College of Education. Among the ten
instructors, four were males and six were females. Two hundred and sixteen
students enrolled in the 16 WebCT classes were indirect participants, whose
classroom participation, discussion, and assignment were observed and recorded.
Data Collection Procedures
The major avenue for data resources
included in the study were 1) field notes of WebCT classroom observation, 2)
transcription of on-line interview with the instructors, 3) transcription of
threaded and unthreaded discussions, 4) record of classroom assessment
activities, and virtual artifacts (assignment, project, presentation, etc.).
The virtual classroom observation was carried on for nine months (three
semester, Spring, Summer, and Fall, 2003). The online observation data were
the base line to understand how assessment as an instructional process was
incorporated and operated in WebCT classes. Observation foci entailed 1)
general instructional procedures of web-based classroom, 2) classroom
interactions between learners and learners, between instructors and learners in
the web, 3) classroom assessment activities. The observation was recorded in
forms of field notes, journals, and memos for further analysis. Particular
attention was given to observe the dynamics of classroom interactions with a
cyclical process of content, input, procedures and product to inform and direct
teaching and learning. Two independent observers were present to conduct
online observation of the same WebCT classes to control the reliability and
accuracy of the data. Discussions were held routinely between the two
observers to exchange field notes, and observation findings. The discrepancy
between the two observers was resolved by reviewing the field notes, and by
comparing with online interview transcripts.
After the observation of WebCT
classrooms was finished, data was sorted, categorized, and compared. The ten
instructors were interviewed via e-mail with both structured and open-ended
questions. The purpose of online interview was to obtain the instructors’
reflections and experiences of their own teaching. Besides observation and
interview data, other data resources, such as syllabi, class notes, discussion
notes, individual projects, group projects, quizzes, tests and exams were used
with the observation and interview data to develop themes and build a chain of
evidence. The documentation started with the observation. As the observation
moved along, documentation began to accumulate, diverge by topics, categories
and themes. The different data resources were also merged and compared
together as a way to triangulate the truthfulness of the research outcome.
As the study focused on
understanding the process of implementing classroom assessment in relation to
the unique features of online instructional environment, it seemed appropriate
to use qualitative research method. It was decided to take a Straussian
approach to grounded theory, in which a “process theory” was generated to
explain an educational process of events, activities, actions, and interactions
that occurred over time (Creswell, 2002, p. 441). It is believed that the
“process theory” discovered during data collection “fit” the situation being
researched and will work when put into use (Glaser & Strauss, 1967, p.3).
The major characteristics of grounded theory study are the three stages of
coding procedures, which are open coding, axial coding, and selective coding
(Creswell, 2002, p. 441-442). In open coding, online observation, and class activity
documentation were sorted, categorized and compared. Concepts or themes with
similar properties were grouped together. During the interview transcription
analysis, data were considered in terms of their match to the existing
categories. This was a contrast to the initial data analysis. In order to
clarify the issues emerging from initial data analysis, another set of informal
interviews took place to discuss and obtain feedbacks from instructors about
the emerged themes. Three final themes derived from the open coding process.
In axial coding, we specifically compared each theme to explore the
interrelationships between one another. Selective coding whereby all data are
related to one single category for theory generation was not undertaken.
Findings
Descriptive of online assessment design and implementation
From all the classroom activities
and assignments we observed in the WebCT classes, instructors tend to have a
heavy focus on enhancing student’s ability to knowledge application. Among the
16 classes, 3 WebCT classes included quiz and exam to assess student learning.
Mostly, assessment activities were designed to help students to analyze and
demonstrate proficiency in solving real world problems. Collaboration and peer
learning were also clearly emphasized in the assessment design and practice.
Table 1 listed WebCT assessment design and implementation in the study.
|
Table 1: WebCT classroom assessment design and implementation |
Assessment type
|
Assessment Name
|
Assessment method
|
Test and exam
|
Module quiz
Exam
|
Numeric score
|
Written Assignment
|
Reaction paper
Evaluate an instructional software
Personal reflections
Description of a program
Journal writing
Lesson plan
|
Evaluation rubrics
Assessment guidelines
Work guideline
|
Proficiency demonstration
|
Electronic portfolio
Create slide show
Create a online brochure
Create a database
Create and deliver a multimedia presentation online
Submit a syllabus online
Create an evaluation instrument
Online mini teaching
Online case analysis
Electronic project presentation
|
Rubrics (Content coverage and showmanship.
Practicality and creativity)
|
Collaboration
|
Group research project,
Research method presentation
Peer reflection
Peer facilitation
Research scenario strategies
Leading panel discussion
|
The amount of time and contribution to learning
Rubrics
|
Participation
|
Threaded discussion (n times/week)
Unthreaded discussion (n times/week)
Questions posted
Answers posted
|
The amount of time and response quality
|
Integration of instructional objectives with performance-based
assessment
From the syllabus collected from
online observation, we found that all instructors claimed that they relied
heavily on “performance” assessment. The most commonly used assessment tasks
for the courses were writing projects, visual presentations, threaded or
unthreaded group discussions of a particular topic, and group work. The
instructors we interviewed told us they wanted to create a cyber arena for
their students to “show” their competency. One of the instructors, Jane said,
“All the assessments are directly tied to the course competencies
(objectives) to demonstrate proficiency”. David told us about his
assessment strategy as “Each of the product documents (Word product,
PowerPoint, Excel, Inspiration) were to access the learners ability to follow
structured instructional task-aids to develop product and master the ability to
utilize that particular software effectively and efficiently”. The
instructors felt that the unique features of online environment made it
possible for them to design performance oriented assessment model. They felt
they could preset very concrete performance assessments in the course content,
course objectives, and course activities. Ben commented, “I only assesses
what I teach, and try to make sure there is an explicit and direct link between
what I believe is important to assess, and the learning activities that I
provide”. According to the instructors we interviewed, one of the reasons
the online instructors relied so much on performance-based assessment is the
elimination of assessment bias. Becky told us “My grades are based strictly
on performance. The course and assessments are structured to allow learners to
authentically represent their proficiency.” Linda also expressed the same
feeling “The way they look, speak and socialize with me or other members of
the class did not affect me in grading student work in this type of class. The
only thing I need to make comments on was the answer to the quizzes, project
and visual display of the presentation. These factors allowed me to evaluate
student more objectively”.
Writing skill as a confounder in online assessment
A closer examination of the data
pointed out that the seemingly straightforward performance assessment model
applied by the instructors was far more complicated than it appeared to be.
One of the major issues that drew the instructors’ attention was that writing
and the assessment of one’s writing skill was confounded with performance-based
assessment. This made the seemingly objective assessment model problematic.
In an online environment, the most essential media to “demonstrated” competency
is writing competency. Angie admitted “I basically assess growth and work
production on students' own writing”. Tom agreed, “The student presence
is usually written which can influence student performance as well as student
evaluation”. Some instructors we interviewed felt that the nature of
relying heavily on writing to demonstrate competency allowed them to be more
objective to evaluate student work. David shared his experience with writing
process in online class “One big difference is that in an online course I
tend to remain unbiased in my informal assessments of students much longer than
in a classroom because of the writing process this type of course is required”.
But these instructors also felt that the absence of f2f interaction did seem to
problematic, especially for students with poor writing skills. Some
instructors felt they need to rethink students’ abilities to accurately demonstrate
in writing how actively engaged students were with assignments and discussions
since the major part assessment involved in evaluating how students wrote.
They were concerned that some students might not be accurately assessed because
the way they wrote. David admitted students with poor writing skills were put
in a disadvantaged position to demonstrate their performance. He said, “Students
who are poor writers may be at a greater disadvantage in an online classroom,
not because of their learning, but because of their writing skills”. Angie
also noticed that the writing skills intertwined made her objectively evaluate
students’ performance questionable, because “In an online class I can't as
easily identify students from those who are engaged, prepared, etc from those
who are engaged, but simply have poor writing skills. Sometimes it took me a
while to distinguish a student from lack of preparation or engagement and poor
writing skill. Sometimes, I never distinguished the two”. Obviously, the
essential role of writing skills in online communication has drawn the
instructor’s attention in their assessment design and instructional practice.
Interestingly, these
same instructors did not seemed to concern their own ability to accurately
express expectations and concerns to students in writing. There was an absence
in the faculty’s responses, of any level of concerns for the students’ need to
clearly understand instructors' assignments, directions, and writing style.
From our online observation, students would go off the topic originally posted
by the instructors for a chat room discussion. It would usually take half of
the discussion time before the instructor jumped in and pull the topic back on
track. Obviously, there was miscommunication between and among learners and
instructors. We suspect in many cases it might be because of the writing
process, or writing style that caused the miscommunication. Even though one
instructor did indicate, “The instructor needs to frame their comments in a
way that does not stifle the discussion”, the general sense was still not
there. Very few instructors recognized that the instructor’s writing and
writing style might have influenced interaction and learning.
Role
shift between learners and instructors
The observation of online class and discussions with
the instructors also revealed a role shift between instructor and learner’s in
web-based environment. The instructors tended to see themselves as a
facilitator, consultant and promoter to self-directed learning in online
classes. Jackie pointed out that her course is designed to be “extremely
interactive and requires learners to be independent and use peers as a resource”. As a result of this design, “Students do
“talk” to each other (online), help each other, and give direction to each
other. Through the shared environment students learn to constructively give
comment and reflect on their work” (Jackie).
Our nine months observation also indicated that there were many designed
activities that focused on the process of learning, and the student-student
interaction. Linda indicated in the interview that in her class “the
questions should be posed first to peers and the instructor is contacted when
peer interaction cannot resolve the question or issue”. She thought, “My primary role is a moderator and
"expert" in times when the discussion is going nowhere”. Tom allowed his students in charge of their own
learning. “Students select a primary study from a list provided, work
in collaborative groups to discuss the article, and may choose to work
collaboratively or independently to write a scholarly critique of the
article”. From the courses we observed,
instructors usually spent considerable amount of time for self-introduction,
and networking at the beginning of the classes. Most often the online dialogue
tended to be more in depth and personal than a regular first class
introduction. Almost all the online classes we observed spent the first and
second threaded discussions for just self-introduction. The instructors would
use first class introduction to learn and assess student’s learning style, and
personality. They also overtly told students to use the first and second
period introduction to familiarize with the basic computer skills essential for studying online, and to
reinforce the comfort level in cyber classrooms where there was not a classroom
to serve as a meeting place and a warm body to give guidance. From our
observation, we noticed that some instructors made a substantial move to
incorporate collaboration as part of the classroom assessment to encourage
teamwork and participation. These instructors recognized the importance of
collaborative learning and peer feedbacks. They also found that it was
relatively easier to document participation and involvement in online learning
environment. When Wendy expressed her experience of using assessment to
encourage students’ participation, she said, “In a f2f classroom I would
not consider grading participation. I believe there are too many constraints
in a f2f classroom to grade on participation. In an online classroom, many of
those constraints are lifted, and grading on participation or discussion is
more appropriate”.
Learner autonomy was a noticeable factor to practice
a more interactive assessment in online instructional environment. According
to Moor & Kearsley (1996), learner autonomy refers to the extent to which
learners make decisions regarding their own “learning”, and “construct” their
own knowledge based on their own experience. The observation data in the
present study indicated that in an online environment, learners were more often
put in charge to initiate the learning process. For example, students must be
responsible to read the material, explore the links, partake in the discussion,
ask questions, chose to learn the objectives, set aside the time to learn, and
select an layout for presenting learning outcome. Web provided access to
information, databases and course notes, but learners had a control on all
these sources. In this information sorting process, learner autonomy became
essential to ensure the quality of learning outcome. Learners also had much
more autonomy to choose the visual interface and screen layout in online
classes to fully express themselves, and to “show” what best present them as an
individual person. Vickie commented, “My role should be seen as the
facilitator, allowing the learners to construct their learning”. Tom also told us, “In the year that I have
taught online I see that I am putting students more “in charge” of directing
aspects of the discussion”. Our observation to
the threaded and unthreaded group discussion also repeated learner autonomy as
an important feature for online teaching and learning.
However, learner autonomy was also a learning process
for online students to adopt and develop as the course moved on. A lot of
these students from various educational backgrounds brought in different
knowledge base and life experience. To some extend, they all needed to take
some time to adjust their role to be in charge of their own learning.
Instructors also had to spent considerable amount of time to identify and
encourage such a shift of learning. Vickie told us that her students were “
very concerned about "doing it right" and "getting it the way I
want it." I had to spent some time to have them get used to the shift
that I do not have a specific way of “doing it right”. The assigned written papers, projects, and presentations we observed in
the WebCT classes repeatedly demonstrated the learners’ ways of “doing things right” when instructors recognized
that learner autonomy could be included as an important aspect of assessment.
It was obvious that the instructors sensed the
importance of learner autonomy, and learner control for an interactive learning
community. The performance based assessment model embraced the unique features
of web communication and interaction such as writing skills and learner
autonomy. From our observation, the instructors had a general sense of
satisfaction about the online teaching environment and assessment delivery
system. Wendy contended, “My expectations are so clear (in writing)
that I see no reason for noncompliance”.
Vickie admitted, “By the end of the course, I think the various
assessment activities worked pretty well to keep us on track and got my goal
accomplished”. However, when these instructors
indicated “Expectations”, “ Goal accomplishment”, they were more often
referring to “my” expectation, and “my” goal accomplishment. There was still a
gap for a shared ownership in goal setting, criterion setting for a more
interactive assessment to increase learner autonomy.
Conclusion
The study examined how assessments
were designed and implemented to reflect online instructional paradigm to
facilitate teaching and promote learning, identified assessment strategies
available to support on line instruction, and explored major components of
classroom assessment to obtain and process information in online learning
environment by studying WebCT classrooms and the instructor’s experiences. The
findings in the study indicated that as the mode of communication shifted,
assessment in online instructional environment should practice a different
assessment model to direct teaching and promote learning. The pedagogical
re-engineering process may indicate the greatest effort for innovation and
departure from traditional practices by modifying and developing effective and
reliable assessment to maximize online learning (Ryan, 2000). The unique
nature of web-based learning put learners up in the front to be responsible for
their own learning. Consequently, web-based classroom assessment should be
practiced to reflect this nature. Assessment should be centered on “assessment
for learning” to increase learner autonomy. The theory derived from the study
is reflected in this shifting “process” different from traditional assessment
in the following aspects:
1. Due to physical absence, test security, student
authenticity reasons, performance-based assessments were found to be prevalent
to describe learning and guide teaching for online instruction. The
traditional paper-pencil tests were rarely used in the 16 WebCT classes we
observed. The flexibility of space and time in online classroom environment
provides instructors with a choice of many different assessment designs for
students to demonstrate what they have learned. The absence of all the visual
elements to favor one student over another allowed online instructors to
evaluate student performance more objectively. The use of web expands the
range of channels available to learners to demonstrate proficiency. Instead of
using narrowly defined learning outcomes tested by examinations, technology
offers a rich environment where skills such as written communication,
collaboration, team work, reflective thinking can be assessed by giving
learners multiple channels, unlimited space of expression. Technology can be
used to create environments for assessment of learning.
2. The transformation of time and space in online learning
environment has changed the traditional role between learners and instructors.
The findings in the study indicated that the instructors were more of
facilitators than teachers in online classes. Without physically interact with
students, what the instructors actually did was to provide learners with
resources and information. Learners were responsible to make decisions to
when, where, what, how much, with whom they wanted to learn. Consequently,
online classroom assessment activities should reflect this aspect. The
experience of self- directed learning, and interactive assessment played an
important role to increase learner autonomy (Frederiksen, & White, 1997).
The data from our study revealed that the online instructors recognized the
shift of roles in online instructional environment. They put an effort to
practice a more student-centered pedagogy. The computer-mediated communication
tools also supplied online instructors with options to document, facilitate and
nurture active involvement, collaborative learning, and learner autonomy.
However, no observable strategies were found in the instructional practice to
include learners in the goal setting and assessment design process. Future
assessment should encompass not only measurement of learning objectives,
self-assessment, but also interactive assessment in a cyber learning community.
3. Instructors and learners relied heavily on writing
communication and visual layout to carry out the task to assess learning and
instructional effectiveness. One of the major issues that drew our attention
in the study was that writing and the assessment of writing skill was
confounded with performance-based assessment. This finding has several
implications relating to online classroom assessment. The absence of lower social
cues such as body language, physical appearance, voice, etc. allowed
instructors and learners to be more focused on writing, and visual layouts.
Without other social interference, instructors may feel they can evaluate
student “performance” more objectively. To put everything in writing also
encourages both learners and instructors to give more thoughtful reflection.
It’s much easier for instructors to document, thus give a more accurate
assessment to student participation and collaboration. The instructors and
learners also feel more comfortable and relaxed in writing up the questions,
and answers because no one is watching them, and they don’t see other people’s
physical response right away.
On the other hand, there exists
wide range of variability of writing skill among the students, and among the
instructors themselves. The lack of facial interaction also requires more time
for instructors to know their students, especially if the students are not good
writers. For the same reason, learners would have to spend more time on
self-exploring how the class is managed, how to communicate efficiently with
the instructors and other classmates simply because of the way each person
writes. Both instructors and learners must be aware that a very important
component of classroom interaction, and classroom assessment is heavily
dependant upon the assessment of writing. During the course of assessment
design and practice, it is very important for web-based instructors to
distinguish the assessment of the course objectives and the assessment of
writing. To better understand students in online learning environment depends
very much on the understanding of the writing styles, and this can only be
achieved by large amount of interactive writing. Instructors need to preset
the objectives and goals for a course explicitly and precisely in writing. The
assessment criterion and assessment procedures also need to be accurately
written to clarify the goals and objectives. Even though, web-based assessment
is heavily dependent on writing and the assessment of writing, very few
instructors have any background in assessment of writing. More empirical study
is needed to explore the impact of writing in web-based instruction and
web-based classroom assessment.
Angelo and Cross (1993) defined
assessment as the multidimensional process of appraising the learning that
occurs in the classroom before and after assignments are graded, with the
feedback used to improved teaching and, hence student learning. As such
assessment is not an end in itself but a vehicle for educational improvement
(Banta, Lund, Black, & Oblander, 1996), assessment is most effective when
it reflects an understanding of learning as multidimensional and integrated and
when it effects change in specific student performance outcomes. Brookhart
(1997) called for motivating student effort and achievement with the vehicle of
classroom assessment. Her theory is also applicable for online classroom
assessment. The findings in the study demonstrate that moving courses online
shifted the traditional meaning of teaching and learning paradigm, and
classroom assessment is no exception. As an important pedagogical component in
teaching and learning dynamics, classroom assessment in web-based learning
environment can be practiced to reflect the shift. Technology has provided
opportunities for online assessment to be more learner-centered to promote
self-directed learning, and to increase learner autonomy. Practicing
“assessment for learning” can cultivate student ownership, and will impact
effort and achievement eventually.
Acknowledgement
The research was generously funded by the University of Akron Faculty Research Grant (UA FRG 1568), Spring 2003.
References
Alessi, S. M.
& Trollip, S. R. (2001). Tests. Multimedia for learning: Methods and
development, 3rd. pp 334-368. Boston: Allyn &
Bacon.
Anglo, T.A.,
& Cross, K. P. (1993). Classroom assessment techniques: A handbook for
college teachers (2nd ed.). San Francisco: Jossey-Bass.
Banta, T. W., Lund, J. P., Black, K. E., & Oblander, F. W. (1996). Assessment in practice:
Putting principles to work on college campuses. San Francisco:
Jossey-Bass.
Black, P., &
William, D. (1998b). Inside the black box: Raising standards through
classroom assessment. Phi Delta Kappan, 80 (2): 139-148.
Broadfoot, P.,
Osborn, M., Sharper, K. & Planel, C. (2001). Pupil assessment and
classroom culture: a comparative study of language of assessment in England and France, in Scott, D. (Ed.): Curriculum and assessment. Westport, CT: Ablex Publishing.
Brookhart, M.,
S. (1997). A theoretical framework for the role of classroom assessment in
motivating student effort and achievement. Applied measurement in education,
10(2), 161-180.
Collis, B. &
Moonen, J. (2001). Flexible learning in digital world. London: Kogan
Creswell, W. J.
(2002). Educational Research- Planning, Conducting, and Evaluating
Quantitative and Qualitative Research. Merrill Prentice Hall, Upper Saddle River, New Jersey.
Daughterty, M.,
Funke, B. (1998). University faculty and student perceptions of web-based
instruction. Journal of Distance Education, 13 (1), pp. 21-39.
Elwood, J. &
Klendowski, V. (2002). Creating of shared practice: the challenges of
assessment use in learning and teaching. Assessment & Evaluation in
Higher Education, Vol, No.. 3, pp. 243-256
Frederiksen, J.
R., & White, B. J. (1997). Reflective assessment of students’ research
within an inquiry-based middle school science curriculum. Paper presented at
the annual meeting of the American Educational Research Association, Chicago, IL.
Glaser, B. &
Strauss, A. (1967). The Discovery of Grounded Theory. Chicago: Aldine.
Innovations in
distance education (1999). An emerging set of guiding principles for the
design and development of distance education. Pennsylvania: Pennsylvania Sate University. Retrieved from (September 24 2002): http://www.outreach.psu.edu/de/ide.
Jung, I.
(2001). Building a theoretical framework of web-based instruction in the
context of distance education. British Journal of Educational Technology,
Vol 32 No 5, 2001, pp. 525-534.
Kerka, S., &
Wonacoot, M. E. (2000). Assessing learners online: Practitioners file. Washington, DC: Office of Educational Research. (ERIC Document
Reproduction Service No. ED448285).
Laurillard, D.
(1993). Rethinking university teaching. London: Routledge.
Meyer, K. A.
(2002C). Quality in distance education: focus on on-line learning. San Francisco, Ca: Jossey-Bass.
Michlitsch, J.
F., & Sidle, M. W. (2002). Assessing Student Learning Outcomes: A
Comparative Study of Techniques Used in Business School Discipline. Journal
of Education for Business, January/February 2002.
Mcloughlin, C.,
& Luca, J. (2001). Quality in online delivery: What does it mean for
assessment in e-learning environment. (ERIC document, ED 467 959)
Moor, M.G., &
Kearsley, G. (1996). Distance education: a system view. Wadsworth, New York.
Popham, W. J.
(2002). Classroom Assessment – What Teachers Need to Know. Allyn and Bacon, Boston.
Robles, M., &
Braathen, S. (2002). Online Assessment Techniques. The Delta Pi Epsilon
Journal, Vol, XLIV No. 1 Winter 2002.
Ryan, R. C.
(2000) Student assessment comparison of lecture and online construction
equipment and methods classes. THE Journal, 27(6), 78-83.
Sherry, L.,
Bilig, S., Jesse, D., & Watson-Acosta, D. (2001). Instructional
technology on student achievement. THE Journal, Feb2001 Vol. 28 Issue
7, p. 40.
Stiggins, R.
(1997). Student-Centered Classroom Assessment (2nd edition).
Upper Saddle River, N.J. : Merrill.
Authors
Xin Laing
Assistant Professor
301 N, Zook Hall
Department of Educational
Foundations & Leadership
University of Akron
Akron, OH, 44325-4201
Phone: 330-972-6921
liang@uakron.edu
Kim Creasy
Assistant Professor
Elementary/Early
Childhood Department
Slippery Rock University
kim.creasy@sru.edu