CALL Course Unit 6

Stanford

LINGUISTICS DEPARTMENT - STANFORD UNIVERSITY

An Invitation to CALL

Foundations of Computer-Assisted Language Learning

Home | Unit 1 | Unit 2 | Unit 3| Unit 4| Unit 5| Unit 6 | Unit 7 | Unit 8 | Supplement

An Invitation to CALL

Unit 6: CALL Research

OVERVIEW

So far, we have been going through this course with the implicit assumption that CALL works, that teaching language using computers in some way makes learning "better." But what exactly is "better?" Here are some possible interpretations:

learners pick up language knowledge or skills faster or with less effort (learning efficiency)
learners pick up what is targeted, retain language knowledge or skills longer, and/or learn more of what they need (effectiveness)
learners can get materials or experience interactions that would otherwise be difficult or impossible (access)
learners can learn with more or less equal effectiveness across a wider range of times/places (convenience)
learners enjoy the language learning process more or are willing to engage in it more (motivation)
learners require less space, less teacher time, or less expensive materials (institutional efficiency)

There are no doubt other ways of defining "better," but if we just consider these, a question arises: What are we comparing these to? Presumably, we are comparing a CALL activity to some corresponding non-CALL activity to see which gives us superior results for a given language learning target. This comparative approach has strong face value: language teachers and program administrators are reasonable in wanting evidence that CALL is worthwhile before putting time and expense into it, and comparative research seems the only way to provide definitive answers. Unfortunately, after two decades CALL researchers have not been able to provide those answers, and a number of influential researchers long ago came to the conclusion that in most cases the type of study that pitted CALL against non-CALL was a dead end, just as happened with "method comparison" (e.g., audiolingual vs. Total Physical Response) in the 1970s and 80s. Ultimately, the more interesting and answerable questions were not about the computer vs. its absence, but about specific applications, specific features of applications, specific types of activities, specific environments, and specific characteristics of learners.

As with other areas of second language learning, there are two ways for teachers to approach CALL research. One is as a research consumer; the other is as a classroom or action researcher. Each of these is briefly discussed below. As with other units, the objective here is to just give a taste of what is an enormous and constantly growing area. Those with more serious research interests are encouraged to consult the reference list. It should be mentioned before continuing that not all CALL research is aimed primarily at improving language teaching and learning with technology. In some cases, researchers may simply want to observe how the technology environment influences or changes the way humans interact with one another, without necessarily judging whether or not it's "better". Although such basic research is important to social scientists and may lead to more applied hypotheses, it does not directly impact teaching and learning and so will not be explicitly addressed here.

CALL RESEARCH TRENDS

As suggested above, most CAI (computer-assisted instruction) and early CALL research focused on comparing computer users with a control group typically using traditional methods. The results were mixed, often showing no significant difference, sometimes favoring the computer users, and occasionally favoring the traditional approaches (see Dunkel 1991).

Over time researchers began to argue against comparative research (see for example Chapelle and Jamieson 1989), stating that the number of variables was too great. There are now many areas being researched using a variety of quantitative and qualitative methods (although interest in comparative research remains, particularly in comparing face-to-face interaction with CMC). The results vary widely, and according to several research reviews (Felix 2005; Hubbard 2005; Huh and Hu 2005) the field is hampered by widespread problems with research designs and reporting.

Research has continued in all areas of CALL but recently has focused on several identifiable areas, such as:

Computer mediated communication; especially, interaction in synchronous chat settings and email in tandem settings
Visual, text and sound annotation to promote comprehension and vocabulary acquisition
Effectiveness of online collaborative and constructivist activities, including development of communities

Here are a few example studies. Keep in mind that these and other studies are generally conducted on small groups in specific settings and may suffer from methodological flaws of various sorts. As with all research, proceed with caution in connecting any findings to your own setting.

Study	Description	Some Results
Belz, J. (2001). Institutional and individual dimensions of transatlantic group work in network-based language teaching. ReCALL, 13 (2), 213-231.	Investigated tandem learning with German & US university students	Of 3 tandem groups studied, 1 failed & 2 succeeded, showing individual & group differences are important, not just tasks & technology
Darhower, M. (2002). Interactional features of synchronous computer-mediated communication in the intermediate L2 class: A sociocultural case study. CALICO Journal, 19 (2), 249-277: https://www.calico.org/html/article_426.pdf	Explores use of chat in 2 4th semester Spanish classes: analyzed weeks 1,4 & 8.	Off-task discussion occurred when instructor gone; limited use of L1, social cohesiveness
De Ridder, I. (2002). Visible or invisible links: Does the highlighting of hyperlinks affect incidental vocabulary learning, text comprehension, and the reading process? Language Learning & Technology, 6 (1), 123-146: http://llt.msu.edu/vol6num1/pdf/deridder.pdf	Explores whether visible or invisible links are more effective in getting students to check unknown vocabulary and the effect of each on reading process.	Highlighted links are clicked more often than hidden ones, but without affecting speed, comprehension or learning of vocab.
Sun, Y.-C. & Wang, L.-Y. (2003). Concordancers in the EFL classroom: Cognitive approaches and collocation difficulty. CALL Journal, 16 (1), 83-94.	Reports on a study of inductive vs. deductive approaches to learning collocations using concordance software: 2 easy, 2 hard ones	Inductive approach was significantly better for easy collocations and almost significant (p=.05) for hard ones
Thorne, S. (2003). Artifacts and cultures-of-use in intercultural communication. Language Learning & Technology, 7 (2), 38-67: http://llt.msu.edu/vol7num2/pdf/thorne.pdf	Reports on 3 case studies demonstrating artifact- mediated practices as cultural practices in US university students	Case 2: a dyad had an initial problem w/email but went to IM (instant messenger) on their own and started with a 6-hour session. Case 3: US students find email inappropriate for social interaction. Prefer IM.
Sengupta, S. (2001). Exchanging ideas with peers in network-based classrooms: an aid or a pain. Language Learning & Technology, 5 (1), 103-134: http://llt.msu.edu/vol5num1/sengupta/default.pdf	Reports on peer exchanges in two partly networked-based classes: observing patterns	Found two common discourse moves were agreeing & praising--helped build community. Accountability gave feeling of heavy workload. Some students were more active, and one small group was consistently lukewarm.
Stockwell, G.& Harrington, M. (2003). The incidental development of L2 proficiency in NS-NNS email interactions. CALICO Journal, 20 (2), 337-359. https://www.calico.org/html/article_326.pdf	Reports on a study of proficiency development via email messages: msgs 1, 5, 10 & 15 analyzed	Showed gains in error-free t-units. Recurring pattern was high performance on first msg, then drop on 5th followed by gradual increase to 15th.
Fernandez-Garcia, M. and Martinez-Arbelaiz, A. (2003). Learner interactions: A comparison of oral and computer assisted written conversations. ReCALL, 15 (1), 116-136.	Compared negotiation patterns/numbers across different pairings and different treatments.	NS-NNS group had most negotiation, especially in oral setting.
Schwienhorst, K. (2002). Evaluating tandem language learning in a MOO: Discourse repair strategies in a bilingual Internet project. CALL Journal, 15 (2), 135-145.	Reviewed MOO (an elaborated form of chat) logs for examples of repair strategies in a tandem setting.	Students said they used repetition requests a lot but logs disagreed; negotiation occurred and was more prevalent than avoidance or misunderstanding; Germans preferred paraphrases but their partners gave them translations
Kötter, M. (2003). Negotiation of meaning and codeswitching in online tandems. Language Learning & Technology, 7 (2), 145-172: http://llt.msu.edu/vol7num2/pdf/kotter.pdf	Reports on a tandem MOO project looking for examples/patterns of negotiation and codeswitching	Logs showed different patterns from self reports, especially with respect to repetition. Notable differences between MOO & previous face-to-face studies
Yoshii, M. and Flaitz, J. (2002). Second language incidental vocabulary retention: The effect of text and picture on annotation types. CALICO Journal, 20 (1), 33-58. https://www.calico.org/html/article_400.pdf	Compared 3 groups for vocabulary acquisition during reading: glosses with picture only, text only, or combination. Surprise vocabulary tests at end and 2 weeks later.	Combination group outperformed others overall; picture group was best on 2-week delayed posttest.
Green, A. & Youngs, B. (2001). Using the Web in elementary French and German courses: Quantitative and qualitative study results. CALICO Journal 19 (1), 89-123. https://www.calico.org/html/article_453.pdf	Reports on study of replacing one of four language class days with independent study on the web	No significant difference in performance after replacing 1 of 4 class days with web.

There are many areas of CALL that have been looked at, and we only cover a few of them here. Check the references at the end for resources to continue your CALL research review. It should be noted before continuing that CALL research has long been burdened by a problem which has not plagued most classroom-based SLA research: the technology adds a dimension of complexity and it is constantly changing; consequently definitive answers in any area don't seem to exist.

SURVEY OF UNANSWERED QUESTIONS IN CALL RESEARCH

One of the major concerns that scholars seem to have upon entering this field, particularly if they are trying to develop a project for a master's or doctoral thesis, is what sorts of research questions to study. To address that problem, in the summer of 2002, I sent a survey to 120 CALL professionals around the world asking them to articulate one research question in the field that they would like to see answered. I received 64 responses. A writeup of the results and the actual questions proposed by the contributors can be viewed at www.stanford.edu/~efs/callsurvey. You may submit your own question or comment on those there.

SUBJECT CHARACTERISTICS IN CALL RESEARCH

In 2004 I carried out a study of research articles found in four CALL journals over a 2-3 year period, focusing on subject characteristics. The overall conclusion was that "CALL research as a whole is unbalanced in the direction of the study of novices working on novel tasks or using novel applications" (Hubbard 2005: 363). Among other recommendations, I suggested that more studies be done using experienced and/or trained learners (see Unit 7) so that we can get a more complete idea of the potential effectiveness of specific CALL software and tasks. This should not be taken as a general criticism of more basic observational research (i.e., what do students do naturally when left on their own in a CALL environment), which is also quite important--the point is that the CALL research domain should be more balanced than it currently seems to be. Results from an unpublished followup study looking exclusively at CMC research reached a similar conclusion: see www.stanford.edu/~efs/pacslrf06.

DOING RESEARCH

As noted in the introduction, this course is largely aimed at classroom teachers interested in beginning or expanding their use of CALL, and teachers can take the role of researchers themselves. identifying a learning gap, creating a possible solution for it--in this case using technology--and then doing research on the effectiveness of that solution. There are several avenues available to teachers in the role of researchers of their own classroom or students.

Observation. When your students are using software or doing a computer-based task in a lab or other venue where you can--watch them. You can look over their shoulder, check their interactions, and make brief notes of what you notice. Interact with the students as they interact with the software. This can give you feedback on the effectiveness of a given piece of software, CALL exercise, or CALL task, and it can also help you determine student training needs.
Tracking. Some software has built-in tracking features. If you are using a discussion board, all student posts can be reviewed. Some chat programs also allow the sessions to be logged for later review.
Student surveys. Ask specific questions about usage--note that it's best to do this as soon as possible after a CALL session sense memories fade rapidly.
Pre- and post-testing to evaluate outcomes of the use of technology.
Student journals. Getting students to keep a reflective journal of their experiences with software or other CALL activities is useful both for them and to the teacher.

LOCATING RESEARCH STUDIES

A search through Google Scholar (http://scholar.google.com) using appropriate keywords is one way to find research materials on a CALL topic. However, a problem is that many of the sources discovered in this manner will not be freely available. Two useful sites to search are CALICO (www.calico.org), which has CALICO Journal articles over three years old freely available, and Language Learning & Technology (http://llt.msu.edu), where all the journal articles are freely available. Both sites have internal search features.

REFERENCES

Chapelle, C. & Jamieson, J. (1989). "Research Trends in Computer-Assisted Language Learning." in Pennington, M. (ed.) Teaching Language with Computers. La Jolla: Athelstan.

Dunkel, P. (1991). “The Effectiveness Research on Computer-Assisted Instruction and Computer-Assisted Language Learning." In Dunkel, P. (ed.) Computer-Assisted Language Learning and Testing: Research Issues and Practice. New York: Newbury House/Harper Collins.

Egbert, J. & Petrie, G. (eds.) (2005). CALL Research Perspectives. Mahwah NJ: Lawrence Erlbaum

Egbert, J. & Hanson-Smith, E. (eds.) (2007). CALL Environments: Research, Practice, and Critical Issues, Second Edition. Alexandria, VA: TESOL.

Felix, U. (2005). Analyzing recent CALL effectiveness research: Towards a common agenda. CALL Journal 18.1-2.

Hubbard, P. (2003). "A Survey of Unanswered Questions in CALL." CALL Journal 16.2-3.

Hubbard, P. (2005). "A Review of Subject Characteristics in CALL Research." CALL Journal 18.5

Huh, K and Hu, W. (2005). Criteria for effective CALL research. In J. Egbert & G. Petrie (eds.) (2005). CALL Research Perspectives. Mahwah NJ: Lawrence Erlbaum

Last modified: February 20, 2010, by Phil Hubbard