NCPI - Student Assessment Toolkit [=] Tutorial

The Undergraduate Student Survey

Use of Factor Scales and Individual Variables in Survey Data Analysis

National Center for Postsecondary Improvement (NCPI) research indicates that several variables factor particularly well into scales that can be used to analyze survey data. Researchers may want to consider the following list of variables and factor scales when designing survey instruments and/or analyzing survey data.

NCPI scale reliabilities are listed for those interested in using some of the variable sets as factors. The scale reliabilities are based on a sample of seven institutions (two community colleges, one liberal arts college, two comprehensive institutions, and one research university).

Variables and Factors extracted from the Undergraduate Student Survey

Student Expectations

Exposure to Assessment

Learning Styles

Innovative Approaches

Student Expectations

Category	Factor Scale – Variable	Survey Question – Variable Identification	Response Options
Classroom Experiences	Pedagogical Variety (alpha = .73)	How much do you agree with these statements about what you expect from your instructors at this institution? 1. Extensive use of audio/visual aids 2. Extensive one-on-one interaction 3. Outlines and other printed course aids 4. An entertaining lecture style 5. Frequent and extensive feedback on my work	1=disagree strongly 2=disagree somewhat 3=agree somewhat 4=agree strongly
	Challenging Learning (alpha = .76)	How much do you agree with these statements about what you expect from your instructors at this institution? 1. Support for exploring topics further on my own 2. Opportunities to synthesize what I am learning across different courses 3. Opportunities to pursue subject matter further 4. A challenging learning experience	1=disagree strongly 2=disagree somewhat 3=agree somewhat 4=agree strongly
	Course Learning Map from Faculty (alpha = .68)	How much do you agree with these statements about what you expect from your instructors at this institution? 1. Instructors to describe clearly all their expectations for coursework 2. Instructors to be available during office hours 3. A clear indication of what material will appear on course exams 4. Frequent summaries of key concepts	1=disagree strongly 2=disagree somewhat 3=agree somewhat 4=agree strongly

TOP

Exposure to Assessment

Category	Factor Scale – Variable	Survey Question – Variable Identification	Response Options
Classroom Experiences	Course Interaction (alpha = .86)	In the past year, how often have you experienced each of the following in your courses? 1. Received oral feedback 2. Received feedback more than three times 3. Discussed goals for learning with instructor 4. Received advice based on performance 5. Met with faculty outside formally scheduled time 6. Felt encouraged to ask question in class 7. Rated an instructor highly	1=never 2=once or twice 3=several times 4=regularly
Importance of Assessment Results	Centrality of Assessment (alpha = .76)	When it comes to succeeding in your intended major, how important is your performance on each of the following types of assessment? 1. State exams or requirements 2. Department exams 3. Professional exams (e.g. teaching certification, CPA) 4. Departmentally reviewed projects or performances 5. Exams determining course placement	1=not important at all 2=not very important 3=somewhat important 4=extremely important 5=not applicable
Perceptions of Assessment	Quality of Classroom Assessment (alpha = .77)	Please indicate your level of agreement on the following statements about student assessment at this institution. 1. I hardly ever get a chance to show what I have learned in a class at all 2. Most faculty regularly assess what their students are learning in the classroom 3. I usually have ample opportunity to express what I have learned on course exams 4. Most faculty do a good job of assessing what students are learning in the classroom 5. In general, the tests in my courses accurately measure what I have learned	1=disagree strongly 2=disagree somewhat 3=agree somewhat 4=agree strongly
	Efficacy of Departmental Assessment (alpha = .59)	Please indicate your level of agreement on the following statements about student assessment at this institution. 1. My performance on placement exams usually reflects my skill-level accurately 2. Departmental tests reflect what I learned in class 3. Student grades provide the best measure of what students learn	1=disagree strongly 2=disagree somewhat 3=agree somewhat 4=agree strongly
	Assessment for Institutional Improvement (alpha = .59)	Please indicate your level of agreement on the following statements about student assessment at this institution. 1. The main purpose of assessment is to help instructors improve their teaching 2. The main purpose of assessment is to help the institution improve 3. Student grades reflect the standards of this institution	1=disagree strongly 2=disagree somewhat 3=agree somewhat 4=agree strongly
	Assessment Techniques (alpha = .38)	Please indicate your level of agreement on the following statements about student assessment at this institution. 1. Multiple-choice or fill-in-the-blank type exams allow me to convey what I learned 2. Students here are assessed more than is really necessary 3. Essay-type exams allow me to convey what I learned	1=disagree strongly 2=disagree somewhat 3=agree somewhat 4=agree strongly

TOP

Learning Styles

Category	Factor Scale – Variable	Survey Question – Variable Identification	Response Options
Behaviors	Cross-Concept Learning (alpha = .82)	For each item, indicate how well it describes you: 1. Try to relate readings to what I already know 2. Discuss theories and ideas with students outside of class 3. Making connections between readings and lectures 4. Relate ideas of one class to materials of elsewhere 5. Go beyond required reading 6. Share my ideas and materials with other students 7. Try to determine which concepts I don’t understand well 8. It is good for students to help each other learn	1=not at all like me 2=a little bit like me 3=like me 4=very much like me
Beliefs	Preference of Group Learning (alpha = .68)	For each item, indicate how well it describes you: 1. Learn more in group setting than on my own 2. Think group project a waste of time 3. Find lecture-only class boring 4. Prefer lecture-based classes over students working on their own 5. Can learn important things with other students 6. It is good for students to help each other learn	1=not at all like me 2=a little bit like me 3=like me 4=very much like me
Satisfaction/ Expectation	Summarization	For each item, indicate how well it describes you: 1. Satisfied when instructors give clear summary on an upcoming exam 2. Expect instructors to provide a guide detailing how to succeed in their courses	1=not at all like me 2=a little bit like me 3=like me 4=very much like me

TOP

Innovative Approaches

Category	Factor Scale – Variable	Survey Question – Variable Identification	Response Options
Behaviors	Cross-Concept Learning (alpha = .82)	For each item, indicate how well it describes you: 1. Try to relate readings to what I already know 2. Discuss theories & ideas with students outside of class 3. Making connections b/w readings and lectures 4. Relate ideas of one class to materials of elsewhere 5. Go beyond required reading 6. Share my ideas and materials with other students 7. Try to determine which concepts I don’t understand well 8. It is good for students to help each other learn	1=not at all like me 2=a little bit like me 3=like me 4=very much like me
Beliefs	Preferences for Group Learning (alpha = .68)	For each item, indicate how well it describes you: 1. Learn more in group setting than on my own 2. Think group project a waste of time 3. Find lecture-only class boring 4. Prefer lecture-based classes over students working on their own 5. Can learn important things with other students 6. It is good for students to help each other learn	1=not at all like me 2=a little bit like me 3=like me 4=very much like me

TOP

Resources on this Page

Factors and Variables

Student Expectations

Exposure to Assessment

Learning Styles

Innovative Approaches

Related Resources

Learning Styles: Student Preferences vs. Faculty Perceptions a report on survey data

© 2003, National Center for Postsecondary Improvement, headquartered at the
Stanford Institute for Higher Education Research