Evaluating electronic literacy skills developed in project-based CALL

Background:  The potential of new technologies to transform language education is widely accepted. In developing countries, where the needs for improving English education are often even greater, there is often even more hope in the potential. But in practice, practical obstacles, such as a troubling lack of correlation between standard achievement tests and the complex of values requiring assessment and appreciation of technology-enhanced language learning, present barriers to getting the most relevant information about the effects of strong educational programs involving new technologies. This often causes a stuck of much-needed implementation of CALL programs.  

Research question:   Major Q: How can new electronic literacy skills developed in project-based CALL programs be best evaluated?   Sub-Qs include: What outcomes should be measured in evaluations of the effects of such programs? What are important knowledge, skills and other competencies that should be covered in such an evaluation? How can these outcomes be compared or related to those from standard achievement tests, especially in the context that only the latter is regarded as valid and reliable? What would a project-based assessment design look like? What are possible measurements that should and can be employed in the evaluation? How to account for problems of unreliability and generalizability? What are some economic, socio-cultural, educational and technical factors that should be considered in the assessment design and the process of implementation of such a design?

Suggested methodology/comments:  Becker & Lovitts (2000) proposed a good research-oriented project-based assessment model which follows a construct-centered approach (Messick, 1994). However, as he commented designing such complex outcome measures requires far more expertise in assessment that a CALL practitioner can provide. This is what I'm most puzzled with. I'm wondering what I should do and can do at the moment. I'm looking forward to hearing from other CALL researchers on the issue of CALL program evaluation.

Contact: Peiya Gu   pg2017@columbia.edu

Reader Comments: --