Volume: 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1



A peer-reviewed electronic journal. ISSN 1531-7714 
Search:
Copyright 2004, PAREonline.net.

Permission is granted to distribute this article for nonprofit, educational purposes if it is copied in its entirety and the journal is credited. Please notify the editor if an article is to be used in a newsletter.


Liang, Xin & Kim Creasy (2004). Classroom assessment in web-based instructional environment: instructors’ experience. Practical Assessment, Research & Evaluation, 9(7). Retrieved July 31, 2010 from http://PAREonline.net/getvn.asp?v=9&n=7 . This paper has been viewed 47,157 times since 3/2/2004.

Classroom Assessment in Web-Based Instructional Environment:  Instructors’ Experience

Xin Liang
University of Akron

Kim Creasy
Slippery Rock University

While a great deal has been written on the advantage and benefits of online teaching, little is known on how assessment is implemented in online classrooms to monitor and inform performance and progress.  The purpose of this study is to investigate the dynamics of WebCT classroom assessment by analyzing the perceptions and experience of the instructors.  Grounded theory method was employed to generate a “process theory”.  The study included 10 faculties who taught WebCT classes, and 216 students in the College of Education in an urban university in the Mid west.  Interviews and classroom observations were undertaken on line.  The findings indicated that, performance-based assessment, writing skills, interactive assessment and learner autonomy were major assessment aspects to inform teaching and enhance learning.  If one of the major roles of online instruction is to increase self-directed learning, as part of the pedagogical mechanism, web-based classroom assessment should be designed and practiced to impact learner autonomy.

The unique features, especially the synchronous and asynchronous communication, web search, online resources and technical support, allow teaching and learning to be place and time independent.  Although educators at all levels have embraced using online technology as a teaching tool, the issue of assessment of student learning in an online course has not been thoroughly addressed (Robles & Braathen, 2002).  As an instructional delivery method, online instruction should be designed to facilitate teaching and promote learning.  As (Meyer, 2002C) pointed out “It is irrelevant to speak of the effects of using Web without understanding how it is entwined with instructional design”.  As instructors reflect upon online learning as an instructional delivery mechanism, they must also examine their assessment delivery method.  They should ask questions about how assessment practice as part of the instructional design is related to the quality of online teaching.

Black and Wiliam (1998b) define assessment broadly to include all activities that teachers and students undertake to get information that can be used diagnostically to alter teaching and learning.  Under this definition, assessment is not limited to just assigning grades to students in the form of paper-pencil exams.  Assessment should permeate many aspects of teaching and learning activities, encompassing teacher observation, classroom discussion, group collaboration, and analysis of student work.  This form of assessment used as a regular element in classroom work, holds the key to better learning (Broadfoot et al., 2001).

Web-based instruction takes place on line, with different modes and resources for retrieving class content, subject related information, and student-teacher interactions (Sherry, Bilig, & Jesse, 2001).  Without a teacher being physically present, web-based teaching requires new instructional practices built on a unique relationship between learners and instructors.  As the mechanism of learning paradigms are changed, so should the assessment delivery method.  As a different instructional practice, online teaching should practice a systematic assessment that embraces and reflects the nature to this type of teaching and learning environment.  If we acknowledge that assessment impact student learning, it is likely that assessment will be at the center of the curriculum design to ensure the quality of online instruction.  It is very important for educators to examine the existing assessment practice, and seek guidelines applicable to the design of assessment in online environment (Mcloughlin, & Luca, 2001, pp. 417).  Penn State University in association with Lincoln University (Innovation in Distance Education, 1999) provided some benchmarks for distance education environment.  Recommendations on assessment process are as follows:

  • Enable students to self-monitor progress;
  • Give regular feedback to students;
  • Support peer learning and assessment;
  • Design self-assessment practice.

These recommendations on assessment process capitalized on the unique characteristics and situations of online learners.  Laurillard (1993) believed that one of the major roles for distance learning is to promote self-directed learning and increased learner autonomy.  Collis and Moonen (2001) used the term pedagogical re-engineering to describe the change in online pedagogy from one that is teacher centered to one that is focused on learner activity.  However, no research has been focused on the interpretation of pedagogical reengineering to online classroom assessment.  The purpose of this study was to generate a theory that explains the “process” of how assessment, as part of instructional practice in web-based environment was implemented to promote learning from instructor’s perspectives.  The research questions that guided the exploration were:

  1. How assessment is designed and implemented to reflect online instructional paradigm to facilitate teaching and promote learning?
  2. What are the effective assessment strategies available to support learning process on line?
  3. What is perceived to be the most important assessment component (s) to obtain and process the information on teaching and learning?

Assessment as Part of Instructional Practice

Findings from research on student learning indicate that pedagogical techniques influence how well students learn to apply concepts (Michlitsch, & Sidle, 2002).  Achieving higher academic standards for all students depends not only on a thorough knowledge of pedagogical content, but also teachers’ ability to determine what students really know and can do, and where the learning gaps are, so that they can target instruction to fill the gaps.  As Stiggins (1997) pointed out “good education encompasses good assessment”.  Rather than being an event at the end of a course or period of learning, good assessment is an instructional event that describes, and promotes students’ best performance across time and uses a range of methods.  Two experimental studies have shown that students who understand the learning objectives and assessment criteria and have opportunities to reflect on their own work show greater improvement than those who do not (Frederikson & White, 1997).  Elwood, & Klendowski (2002) believe there is a distinction between “assessment of learning” (assessment for the purposes of grading and reporting with its own established procedures) and “assessment for learning” (assessment whose purpose is to enable students, through effective feedback, to fully understand their own learning and the goals they are aiming for).  Assessment for learning requires teachers to never take a student’s grade in and of itself as the only central goal of the assessment.  Rather, assessment performance is taken as a proxy for the student’s status with respect to target instructional domain (Popham, 2002).  No doubt, there is a vital link between assessment, learning, and teaching.  Instructors can build in many opportunities to include students in the assessment of learning and then use the information to make beneficial changes for both learning and instruction.  How to make it happen places a challenge for instructors, especially for online instructors when the instructional environment and communication devices are different.  At the same time, the unique features of web-based instructional environment also open up a new frontier for online instructors to practice a more student-centered pedagogy.

Challenges and opportunities for online classroom assessment

Moving courses from the traditional classroom to an online setting fundamentally shifts human interaction, communication, and learning paradigms (Robles & Braathen, 2002).  Jung (2001) characterized three key components of Web-based instruction different from traditional face-to-face classroom instruction as 1) content expandability, 2) content adaptability, and 3) visual layouts.  And also three types of interaction were identified as essential for the success of online teaching.  They were: 1) academic interaction, 2) collaborative interaction, and 3) interpersonal interaction (Jung, 2001).  The characteristics of online instruction present special challenges for assessment.  As online instructors can no longer monitor and react to student questions, comments, asides, body language, and facial expressions, they must employ other techniques to acquire the same information (Alessi & Trollip, 2001).  The absence of low level of social cues and emotions may minimize the richness of communication, limit and impede a more interactive cyber learning community (Robels & Braathen, 2002).  The verbal and written communication enjoyed in f2f classrooms has been limited to written text with static images, slower because the student cannot type as fast as he can talk and may be inexperienced in written communication (The Institute for Higher Education Policy, 2000).  It is often more difficult to identify online cheating and student authentication as student has access to various course materials, and impersonation is perceived as a greater risk (Kerka & Wonacoot, 2000).

These same features in online classes can offer a unique communication environment in which texts, pictures, video and audio are integrated into one system.  This system allows a much easier access to huge database for students, and more flexible interactions (Jung, 2001).  The flexibility of communication to be time and place independent increased the opportunities for dialogues, and more thoughtful reflections.  The unique features of online communication allows more interactive assessment to not only accurately measure learning outcome, but also to nurture peer feedbacks, and encourage participation.  As more and more educators and researchers realized that effective instruction with technology must be driven by sound pedagogical principles (Daugherty & Funke, 1998), it is very important to ask such questions as how this could be achieved, and what aspects should be concerned for a more effective assessment to ensure the quality of web-based instruction.

Method

WebCT learning environment

WebCT is a type of online teaching software that was prevalent in the studied university.  The WebCT classes included in the study were web-based courses (over 50% of the course content delivered online) offered by the college of education in a Midwestern university.  There was no restriction for students to register WebCT courses in the university.  Instructors were required to send in application and curriculum proposal to move a course from traditional or web-enhanced (less than 50% online) to web-based instruction.  In the curriculum proposal, instructors were expected to explain the rationales to move a course online, and a detailed teaching plan.  The proposal was reviewed by the department, the college council and, approved by the provost office.  The starting and ending dates of WebCT classes were the same as f2f courses offered in campus.  The course web page functioned as the major setting for instructors to post class related materials.  Students log onto the class web to retrieve materials, interact with the instructors and other classmates.  Students and instructors usually did not meet f2f throughout the whole semester.  Some classes in the study did meet once or twice when issues came up, and the instructors were needed to meet with students.  Most classes (13 out of 16) did not have a fixed “meeting” time (synchronous communication).  Each week, the instructor posted certain amount of content materials for students to study by the form of class note, unit module or studying materials of the week on the web.  Instructors also posted assignments, projects, cases analysis, and questions to help students understand and exercise the content.  Students were required to log onto WebCT to retrieve materials, and to do self-study at their own convenience.  Usually, the class interactions between learners and instructors were asynchronous on individual bases.  Three classes required once a week (synchronous discussion for two hours).  Each member of a WebCT class was able to contact other members privately or publicly via the WebCT mail list, or posting messages via threaded or unthreaded discussions.  Students were able to send in assignments, projects or exams electronically to the instructors.  Students were also able to obtain both formal and informal assessment from the instructors on the web.

Participants

University faculties who teach course content over 50 % on line were the target population.  After sending out consent forms, ten instructors (who taught 16 WebCT classes) agreed to participate the study.  The WebCT classes included in this study were both at undergraduate and graduate levels in the College of Education.  Among the ten instructors, four were males and six were females.  Two hundred and sixteen students enrolled in the 16 WebCT classes were indirect participants, whose classroom participation, discussion, and assignment were observed and recorded.

Data Collection Procedures

The major avenue for data resources included in the study were 1) field notes of WebCT classroom observation, 2) transcription of on-line interview with the instructors, 3) transcription of threaded and unthreaded discussions, 4) record of classroom assessment activities, and virtual artifacts (assignment, project, presentation, etc.).  The virtual classroom observation was carried on for nine months (three semester, Spring, Summer, and Fall, 2003).  The online observation data were the base line to understand how assessment as an instructional process was incorporated and operated in WebCT classes.  Observation foci entailed 1) general instructional procedures of web-based classroom, 2) classroom interactions between learners and learners, between instructors and learners in the web, 3) classroom assessment activities.  The observation was recorded in forms of field notes, journals, and memos for further analysis.  Particular attention was given to observe the dynamics of classroom interactions with a cyclical process of content, input, procedures and product to inform and direct teaching and learning.  Two independent observers were present to conduct online observation of the same WebCT classes to control the reliability and accuracy of the data.  Discussions were held routinely between the two observers to exchange field notes, and observation findings.  The discrepancy between the two observers was resolved by reviewing the field notes, and by comparing with online interview transcripts.

After the observation of WebCT classrooms was finished, data was sorted, categorized, and compared.  The ten instructors were interviewed via e-mail with both structured and open-ended questions.  The purpose of online interview was to obtain the instructors’ reflections and experiences of their own teaching.  Besides observation and interview data, other data resources, such as syllabi, class notes, discussion notes, individual projects, group projects, quizzes, tests and exams were used with the observation and interview data to develop themes and build a chain of evidence.  The documentation started with the observation.  As the observation moved along, documentation began to accumulate, diverge by topics, categories and themes.  The different data resources were also merged and compared together as a way to triangulate the truthfulness of the research outcome.

As the study focused on understanding the process of implementing classroom assessment in relation to the unique features of online instructional environment, it seemed appropriate to use qualitative research method.  It was decided to take a Straussian approach to grounded theory, in which a “process theory” was generated to explain an educational process of events, activities, actions, and interactions that occurred over time (Creswell, 2002, p. 441).  It is believed that the “process theory” discovered during data collection “fit” the situation being researched and will work when put into use (Glaser & Strauss, 1967, p.3).  The major characteristics of grounded theory study are the three stages of coding procedures, which are open coding, axial coding, and selective coding (Creswell, 2002, p. 441-442).  In open coding, online observation, and class activity documentation were sorted, categorized and compared.  Concepts or themes with similar properties were grouped together.  During the interview transcription analysis, data were considered in terms of their match to the existing categories.  This was a contrast to the initial data analysis.  In order to clarify the issues emerging from initial data analysis, another set of informal interviews took place to discuss and obtain feedbacks from instructors about the emerged themes.  Three final themes derived from the open coding process.  In axial coding, we specifically compared each theme to explore the interrelationships between one another.  Selective coding whereby all data are related to one single category for theory generation was not undertaken.

Findings

Descriptive of online assessment design and implementation

From all the classroom activities and assignments we observed in the WebCT classes, instructors tend to have a heavy focus on enhancing student’s ability to knowledge application.  Among the 16 classes, 3 WebCT classes included quiz and exam to assess student learning.  Mostly, assessment activities were designed to help students to analyze and demonstrate proficiency in solving real world problems.  Collaboration and peer learning were also clearly emphasized in the assessment design and practice.  Table 1 listed WebCT assessment design and implementation in the study.

 

Table 1: WebCT classroom assessment design and implementation
Assessment type
Assessment Name
Assessment method
 
Test and exam
Module quiz
Exam
Numeric score
 
Written Assignment
Reaction paper
Evaluate an instructional software
Personal reflections
Description of a program
Journal writing
Lesson plan
Evaluation rubrics
Assessment guidelines
Work guideline
Proficiency demonstration
 
Electronic portfolio
Create slide show
Create a online brochure
Create a database
Create and deliver a multimedia presentation online
Submit a syllabus online
Create an evaluation instrument
Online mini teaching
Online case analysis
Electronic project presentation
Rubrics (Content coverage and showmanship.
Practicality and creativity)
Collaboration
Group research project,
Research method presentation
Peer reflection
Peer facilitation
Research scenario strategies
Leading panel discussion
The amount of time and contribution to learning
Rubrics
Participation
Threaded discussion (n times/week)
Unthreaded discussion (n times/week)
Questions posted
Answers posted
The amount of time and response quality

 

Integration of instructional objectives with performance-based assessment

From the syllabus collected from online observation, we found that all instructors claimed that they relied heavily on “performance” assessment.  The most commonly used assessment tasks for the courses were writing projects, visual presentations, threaded or unthreaded group discussions of a particular topic, and group work.  The instructors we interviewed told us they wanted to create a cyber arena for their students to “show” their competency.  One of the instructors, Jane said, “All the assessments are directly tied to the course competencies (objectives) to demonstrate proficiency”.  David told us about his assessment strategy as “Each of the product documents (Word product, PowerPoint, Excel, Inspiration) were to access the learners ability to follow structured instructional task-aids to develop product and master the ability to utilize that particular software effectively and efficiently”.  The instructors felt that the unique features of online environment made it possible for them to design performance oriented assessment model.  They felt they could preset very concrete performance assessments in the course content, course objectives, and course activities.  Ben commented, “I only assesses what I teach, and try to make sure there is an explicit and direct link between what I believe is important to assess, and the learning activities that I provide”.  According to the instructors we interviewed, one of the reasons the online instructors relied so much on performance-based assessment is the elimination of assessment bias.  Becky told us “My grades are based strictly on performance.  The course and assessments are structured to allow learners to authentically represent their proficiency.”  Linda also expressed the same feeling “The way they look, speak and socialize with me or other members of the class did not affect me in grading student work in this type of class.  The only thing I need to make comments on was the answer to the quizzes, project and visual display of the presentation.  These factors allowed me to evaluate student more objectively”.

Writing skill as a confounder in online assessment

A closer examination of the data pointed out that the seemingly straightforward performance assessment model applied by the instructors was far more complicated than it appeared to be.  One of the major issues that drew the instructors’ attention was that writing and the assessment of one’s writing skill was confounded with performance-based assessment.  This made the seemingly objective assessment model problematic.  In an online environment, the most essential media to “demonstrated” competency is writing competency.  Angie admitted “I basically assess growth and work production on students' own writing”.  Tom agreed, “The student presence is usually written which can influence student performance as well as student evaluation”.  Some instructors we interviewed felt that the nature of relying heavily on writing to demonstrate competency allowed them to be more objective to evaluate student work.  David shared his experience with writing process in online class “One big difference is that in an online course I tend to remain unbiased in my informal assessments of students much longer than in a classroom because of the writing process this type of course is required”.  But these instructors also felt that the absence of f2f interaction did seem to problematic, especially for students with poor writing skills.  Some instructors felt they need to rethink students’ abilities to accurately demonstrate in writing how actively engaged students were with assignments and discussions since the major part assessment involved in evaluating how students wrote.  They were concerned that some students might not be accurately assessed because the way they wrote.  David admitted students with poor writing skills were put in a disadvantaged position to demonstrate their performance.  He said, “Students who are poor writers may be at a greater disadvantage in an online classroom, not because of their learning, but because of their writing skills”.  Angie also noticed that the writing skills intertwined made her objectively evaluate students’ performance questionable, because “In an online class I can't as easily identify students from those who are engaged, prepared, etc from those who are engaged, but simply have poor writing skills.  Sometimes it took me a while to distinguish a student from lack of preparation or engagement and poor writing skill.  Sometimes, I never distinguished the two”.  Obviously, the essential role of writing skills in online communication has drawn the instructor’s attention in their assessment design and instructional practice.

Interestingly, these same instructors did not seemed to concern their own ability to accurately express expectations and concerns to students in writing.  There was an absence in the faculty’s responses, of any level of concerns for the students’ need to clearly understand instructors' assignments, directions, and writing style.  From our online observation, students would go off the topic originally posted by the instructors for a chat room discussion.  It would usually take half of the discussion time before the instructor jumped in and pull the topic back on track.  Obviously, there was miscommunication between and among learners and instructors.  We suspect in many cases it might be because of the writing process, or writing style that caused the miscommunication.  Even though one instructor did indicate, “The instructor needs to frame their comments in a way that does not stifle the discussion”, the general sense was still not there.  Very few instructors recognized that the instructor’s writing and writing style might have influenced interaction and learning.

Role shift between learners and instructors

The observation of online class and discussions with the instructors also revealed a role shift between instructor and learner’s in web-based environment.  The instructors tended to see themselves as a facilitator, consultant and promoter to self-directed learning in online classes.  Jackie pointed out that her course is designed to be “extremely interactive and requires learners to be independent and use peers as a resource”.  As a result of this design, “Students do “talk” to each other (online), help each other, and give direction to each other.  Through the shared environment students learn to constructively give comment and reflect on their work” (Jackie).  Our nine months observation also indicated that there were many designed activities that focused on the process of learning, and the student-student interaction.  Linda indicated in the interview that in her class “the questions should be posed first to peers and the instructor is contacted when peer interaction cannot resolve the question or issue”.  She thought, “My primary role is a moderator and "expert" in times when the discussion is going nowhere”.  Tom allowed his students in charge of their own learning.  “Students select a primary study from a list provided, work in collaborative groups to discuss the article, and may choose to work collaboratively or independently to write a scholarly critique of the article”.  From the courses we observed, instructors usually spent considerable amount of time for self-introduction, and networking at the beginning of the classes.  Most often the online dialogue tended to be more in depth and personal than a regular first class introduction.  Almost all the online classes we observed spent the first and second threaded discussions for just self-introduction.  The instructors would use first class introduction to learn and assess student’s learning style, and personality.  They also overtly told students to use the first and second period introduction to familiarize with the basic computer skills essential for studying online, and to reinforce the comfort level in cyber classrooms where there was not a classroom to serve as a meeting place and a warm body to give guidance.  From our observation, we noticed that some instructors made a substantial move to incorporate collaboration as part of the classroom assessment to encourage teamwork and participation.  These instructors recognized the importance of collaborative learning and peer feedbacks.  They also found that it was relatively easier to document participation and involvement in online learning environment.  When Wendy expressed her experience of using assessment to encourage students’ participation, she said, “In a f2f classroom I would not consider grading participation.  I believe there are too many constraints in a f2f classroom to grade on participation.  In an online classroom, many of those constraints are lifted, and grading on participation or discussion is more appropriate”.

Learner autonomy was a noticeable factor to practice a more interactive assessment in online instructional environment.  According to Moor & Kearsley (1996), learner autonomy refers to the extent to which learners make decisions regarding their own “learning”, and “construct” their own knowledge based on their own experience.  The observation data in the present study indicated that in an online environment, learners were more often put in charge to initiate the learning process.  For example, students must be responsible to read the material, explore the links, partake in the discussion, ask questions, chose to learn the objectives, set aside the time to learn, and select an layout for presenting learning outcome.  Web provided access to information, databases and course notes, but learners had a control on all these sources.  In this information sorting process, learner autonomy became essential to ensure the quality of learning outcome.  Learners also had much more autonomy to choose the visual interface and screen layout in online classes to fully express themselves, and to “show” what best present them as an individual person.  Vickie commented, “My role should be seen as the facilitator, allowing the learners to construct their learning”.  Tom also told us, “In the year that I have taught online I see that I am putting students more “in charge” of directing aspects of the discussion”.  Our observation to the threaded and unthreaded group discussion also repeated learner autonomy as an important feature for online teaching and learning.

However, learner autonomy was also a learning process for online students to adopt and develop as the course moved on.  A lot of these students from various educational backgrounds brought in different knowledge base and life experience.  To some extend, they all needed to take some time to adjust their role to be in charge of their own learning.  Instructors also had to spent considerable amount of time to identify and encourage such a shift of learning.  Vickie told us that her students were “ very concerned about "doing it right" and "getting it the way I want it."  I had to spent some time to have them get used to the shift that I do not have a specific way of “doing it right”.  The assigned written papers, projects, and presentations we observed in the WebCT classes repeatedly demonstrated the learners’ ways of “doing things right” when instructors recognized that learner autonomy could be included as an important aspect of assessment.

It was obvious that the instructors sensed the importance of learner autonomy, and learner control for an interactive learning community.  The performance based assessment model embraced the unique features of web communication and interaction such as writing skills and learner autonomy.  From our observation, the instructors had a general sense of satisfaction about the online teaching environment and assessment delivery system.  Wendy contended, “My expectations are so clear (in writing) that I see no reason for noncompliance”.  Vickie admitted, “By the end of the course, I think the various assessment activities worked pretty well to keep us on track and got my goal accomplished”.  However, when these instructors indicated “Expectations”, “ Goal accomplishment”, they were more often referring to “my” expectation, and “my” goal accomplishment.  There was still a gap for a shared ownership in goal setting, criterion setting for a more interactive assessment to increase learner autonomy.

Conclusion

The study examined how assessments were designed and implemented to reflect online instructional paradigm to facilitate teaching and promote learning, identified assessment strategies available to support on line instruction, and explored major components of classroom assessment to obtain and process information in online learning environment by studying WebCT classrooms and the instructor’s experiences.  The findings in the study indicated that as the mode of communication shifted, assessment in online instructional environment should practice a different assessment model to direct teaching and promote learning.  The pedagogical re-engineering process may indicate the greatest effort for innovation and departure from traditional practices by modifying and developing effective and reliable assessment to maximize online learning (Ryan, 2000).  The unique nature of web-based learning put learners up in the front to be responsible for their own learning.  Consequently, web-based classroom assessment should be practiced to reflect this nature.  Assessment should be centered on “assessment for learning” to increase learner autonomy.  The theory derived from the study is reflected in this shifting “process” different from traditional assessment in the following aspects:

1. Due to physical absence, test security, student authenticity reasons, performance-based assessments were found to be prevalent to describe learning and guide teaching for online instruction.  The traditional paper-pencil tests were rarely used in the 16 WebCT classes we observed.  The flexibility of space and time in online classroom environment provides instructors with a choice of many different assessment designs for students to demonstrate what they have learned.  The absence of all the visual elements to favor one student over another allowed online instructors to evaluate student performance more objectively.  The use of web expands the range of channels available to learners to demonstrate proficiency.  Instead of using narrowly defined learning outcomes tested by examinations, technology offers a rich environment where skills such as written communication, collaboration, team work, reflective thinking can be assessed by giving learners multiple channels, unlimited space of expression.  Technology can be used to create environments for assessment of learning.

2. The transformation of time and space in online learning environment has changed the traditional role between learners and instructors.  The findings in the study indicated that the instructors were more of facilitators than teachers in online classes.  Without physically interact with students, what the instructors actually did was to provide learners with resources and information.  Learners were responsible to make decisions to when, where, what, how much, with whom they wanted to learn.  Consequently, online classroom assessment activities should reflect this aspect.  The experience of self- directed learning, and interactive assessment played an important role to increase learner autonomy (Frederiksen, & White, 1997).  The data from our study revealed that the online instructors recognized the shift of roles in online instructional environment.  They put an effort to practice a more student-centered pedagogy.  The computer-mediated communication tools also supplied online instructors with options to document, facilitate and nurture active involvement, collaborative learning, and learner autonomy.  However, no observable strategies were found in the instructional practice to include learners in the goal setting and assessment design process.  Future assessment should encompass not only measurement of learning objectives, self-assessment, but also interactive assessment in a cyber learning community.

3. Instructors and learners relied heavily on writing communication and visual layout to carry out the task to assess learning and instructional effectiveness.  One of the major issues that drew our attention in the study was that writing and the assessment of writing skill was confounded with performance-based assessment.  This finding has several implications relating to online classroom assessment.  The absence of lower social cues such as body language, physical appearance, voice, etc. allowed instructors and learners to be more focused on writing, and visual layouts.  Without other social interference, instructors may feel they can evaluate student “performance” more objectively.  To put everything in writing also encourages both learners and instructors to give more thoughtful reflection.  It’s much easier for instructors to document, thus give a more accurate assessment to student participation and collaboration.  The instructors and learners also feel more comfortable and relaxed in writing up the questions, and answers because no one is watching them, and they don’t see other people’s physical response right away.

On the other hand, there exists wide range of variability of writing skill among the students, and among the instructors themselves.  The lack of facial interaction also requires more time for instructors to know their students, especially if the students are not good writers.  For the same reason, learners would have to spend more time on self-exploring how the class is managed, how to communicate efficiently with the instructors and other classmates simply because of the way each person writes.  Both instructors and learners must be aware that a very important component of classroom interaction, and classroom assessment is heavily dependant upon the assessment of writing.  During the course of assessment design and practice, it is very important for web-based instructors to distinguish the assessment of the course objectives and the assessment of writing.  To better understand students in online learning environment depends very much on the understanding of the writing styles, and this can only be achieved by large amount of interactive writing.  Instructors need to preset the objectives and goals for a course explicitly and precisely in writing.  The assessment criterion and assessment procedures also need to be accurately written to clarify the goals and objectives.  Even though, web-based assessment is heavily dependent on writing and the assessment of writing, very few instructors have any background in assessment of writing.  More empirical study is needed to explore the impact of writing in web-based instruction and web-based classroom assessment.

Angelo and Cross (1993) defined assessment as the multidimensional process of appraising the learning that occurs in the classroom before and after assignments are graded, with the feedback used to improved teaching and, hence student learning.  As such assessment is not an end in itself but a vehicle for educational improvement (Banta, Lund, Black, & Oblander, 1996), assessment is most effective when it reflects an understanding of learning as multidimensional and integrated and when it effects change in specific student performance outcomes.  Brookhart (1997) called for motivating student effort and achievement with the vehicle of classroom assessment.  Her theory is also applicable for online classroom assessment.  The findings in the study demonstrate that moving courses online shifted the traditional meaning of teaching and learning paradigm, and classroom assessment is no exception.  As an important pedagogical component in teaching and learning dynamics, classroom assessment in web-based learning environment can be practiced to reflect the shift.  Technology has provided opportunities for online assessment to be more learner-centered to promote self-directed learning, and to increase learner autonomy.  Practicing “assessment for learning” can cultivate student ownership, and will impact effort and achievement eventually.

Acknowledgement

The research was generously funded by the University of Akron Faculty Research Grant (UA FRG 1568), Spring 2003.

References

Alessi, S. M. & Trollip, S. R.  (2001).  Tests.  Multimedia for learning: Methods and development, 3rd.  pp 334-368.  Boston: Allyn & Bacon.

Anglo, T.A., & Cross, K. P. (1993).  Classroom assessment techniques: A handbook for college teachers (2nd ed.).  San Francisco: Jossey-Bass.

Banta, T. W., Lund, J. P., Black, K. E., & Oblander, F. W.  (1996).  Assessment in practice: Putting principles to work on college campuses.  San Francisco:  Jossey-Bass.

Black, P., & William, D.  (1998b).  Inside the black box: Raising standards through classroom assessment.  Phi Delta Kappan, 80 (2): 139-148.

Broadfoot, P., Osborn, M., Sharper, K. & Planel, C.  (2001).  Pupil assessment and classroom culture: a comparative study of language of assessment in England and France, in Scott, D. (Ed.):  Curriculum and assessment.  Westport, CT: Ablex Publishing.

Brookhart, M., S.  (1997).  A theoretical framework for the role of classroom assessment in motivating student effort and achievement.  Applied measurement in education, 10(2), 161-180.

Collis, B. & Moonen, J.  (2001).  Flexible learning in digital world.  London: Kogan

Creswell, W. J.  (2002).  Educational Research- Planning, Conducting, and Evaluating Quantitative and Qualitative Research.  Merrill Prentice Hall, Upper Saddle River, New Jersey.

Daughterty, M., Funke, B.  (1998).  University faculty and student perceptions of web-based instruction.  Journal of Distance Education, 13 (1), pp. 21-39.

Elwood, J. & Klendowski, V.  (2002).  Creating of shared practice: the challenges of assessment use in learning and teaching.  Assessment & Evaluation in Higher Education, Vol, No.. 3, pp. 243-256

Frederiksen, J. R., & White, B. J.  (1997).  Reflective assessment of students’ research within an inquiry-based middle school science curriculum.  Paper presented at the annual meeting of the American Educational Research Association, Chicago, IL.

Glaser, B. & Strauss, A.  (1967).  The Discovery of Grounded Theory.  Chicago: Aldine.

Innovations in distance education (1999).  An emerging set of guiding principles for the design and development of distance education.  Pennsylvania: Pennsylvania Sate University.  Retrieved from (September 24 2002): http://www.outreach.psu.edu/de/ide.

Jung, I.  (2001).  Building a theoretical framework of web-based instruction in the context of distance education.  British Journal of Educational Technology, Vol 32 No 5, 2001, pp. 525-534.

Kerka, S., & Wonacoot, M. E.  (2000).  Assessing learners online: Practitioners file.  Washington, DC: Office of Educational Research.  (ERIC Document Reproduction Service No. ED448285).

Laurillard, D.  (1993).  Rethinking university teaching.  London: Routledge.

Meyer, K. A.  (2002C).  Quality in distance education: focus on on-line learning.  San Francisco, Ca: Jossey-Bass.

Michlitsch, J. F., & Sidle, M. W.  (2002).  Assessing Student Learning Outcomes:  A Comparative Study of Techniques Used in Business School Discipline.  Journal of Education for Business, January/February 2002.

Mcloughlin, C., & Luca, J.  (2001).  Quality in online delivery: What does it mean for assessment in e-learning environment.  (ERIC document, ED 467 959)

Moor, M.G., & Kearsley, G.  (1996).  Distance education: a system view.  Wadsworth, New York.

Popham, W. J.  (2002).  Classroom Assessment – What Teachers Need to Know.  Allyn and Bacon, Boston.

Robles, M., & Braathen, S.  (2002).  Online Assessment Techniques.  The Delta Pi Epsilon Journal, Vol, XLIV  No. 1  Winter 2002.

Ryan, R. C.  (2000)  Student assessment comparison of lecture and online construction equipment and methods classes.  THE Journal, 27(6), 78-83.

Sherry, L., Bilig, S., Jesse, D., & Watson-Acosta, D.  (2001).  Instructional technology on student achievement.  THE Journal, Feb2001 Vol. 28 Issue 7, p. 40.

Stiggins, R.  (1997).  Student-Centered Classroom Assessment (2nd edition).  Upper Saddle River, N.J. : Merrill.

Authors

Xin Laing
Assistant Professor
301 N, Zook Hall
Department of Educational Foundations & Leadership
University of Akron
Akron, OH, 44325-4201

Phone: 330-972-6921
liang@uakron.edu

Kim Creasy
Assistant Professor
Elementary/Early Childhood Department
Slippery Rock University

kim.creasy@sru.edu

Descriptors: Web Based Instruction; Formative Assessment; Performance Assessment; Grounded theory