Executive Summary of Assessment in General Education

~1995-2006

Several forms of assessment of the General Education Program have occurred since 1995.  The structure of the General Education program has not changed since that time, although some courses have been added, and specific programmatic learning outcomes have been approved in 2004. 

Direct Assessment of student learning:

1.   Scientific Understanding/Reasoning 1995-1996 and 2004-2005 using several different instruments. 

• 1995-1996. Sample of 99 students with some science; 100 no science. Students with at least one science course scored better on most questions than students with no university science.  However, responses across both groups and all questions on the different measures were marginal and not well-developed. Students had difficulty applying knowledge to novel problems (see http://www.uwlax.edu/provost/assessment/A_GEscience.html for a full report on this assessment).
• 2004-2005.  Two different measures were used.  Measures examined scientific reasoning as well as global understanding.   The instrument was piloted in the fall of 2004, revised, and administered in the spring of 2005 to 189 students who varied in age, year in school, major, and other academic experiences (e.g., study abroad).  Based on a rubric, scores could range from 0-2 for most questions pertaining to scientific knowledge and reasoning. Mean scores ranged from 1.73 to.60 with the majority of scores falling below 1.0. Correlations with demographic information indicated that, overall, SAH students performed better than students in other colleges, seniors performed better than freshmen, and performance was higher for students having taken more courses with a science focus.

2.  Global understanding.  2004 & 2005, Different instruments.

• Three different instruments have assessed the approved learning outcome, “able to explain the rationales for cultural behaviors different from one’s own.”  The first was successfully piloted in a modern language class in Fall 2003 with 17 students. It was then administered to 67 students in an upper level management class in Spring 2004, and then again to 189 students in spring of 2005 to students across age, year in school, and majors.  On a scale of 1-4, the mean scores ranged between 2.01 (upper level management students) and 2.14 (initial pilot group).
• The other global instruments were part of the Science assessments (described above), but were designed to assess the same outcome.  Mean scores ranged from .85 to 1.08 on a 0-2 point scale. There were no consistent correlations with age, year in school, major, or study abroad experience across all three instruments. However, in 2/3 instruments students reporting a “helpful experience” had statistically significant higher scores than students not reporting such an experience.

3.       Social Science understanding. 1995-1996.

·         A total of 108 students in two courses in the “Self and Society” category completed the pre-and post-assessment that asked students to answer two questions after reading a brief article on “welfare dependency.” Results indicated that slightly less than half the students wrote better answers on the post-test for question that examined students’ abilities to think about the welfare issues raised in the article and a sizable minority, 24%, actually wrote poorer answers on the post-test.  A similar pattern was found on the second question that examined students ability to understand methods of inquiry in the social sciences, although only 36% of the students’ improved their answers on the post-test. The "welfare dependency" test measures a kind of "social science reasoning," the ability to analyze and evaluate relationships among variables or factors that influence social behavior.  Overall, students' answers were underdeveloped, revealing only a very rudimentary understanding of the social context of human actions, attitudes and values.  These result are similar to the science assessment results described above that showed a relatively rudimentary understanding of science and scientific experimentation. (See http://www.uwlax.edu/provost/assessment/A_GEselfsoc.htm for a full report of this assessment).

4.       Art: The Aesthetic Experience. Spring 1997.

·         Students were asked to answer two questions as an assignment of courses in each area in this category (art, music, dance, theatre arts). The first assessed students valuing of the arts and the second focused on students’ ability to respond to a work of art. The majority of students were able to produce better responses on the post-test when compared with the pre-test responses. By the end of the semester, 71% of the students produced at least a marginal response to the question.  However, while students could articulate a reason why art is of value to a society-- most did not provide a very compelling argument.  In addition, although about half the students improved their answers, about an equal number either wrote poorer answers on the post-test than the pre-test or  the quality of answers did not change appreciably.

• On the second question, a majority of students improved their ability to discuss a work of art--painting, musical composition, dance or theatre performance.  Although nearly 20% still produced poor responses on the post-test, 61% of the students improved their answers.
• The quality of responses tend to reflect the same concern as those found in the science and social science assessments, that a sizable number of students’ were considered marginal, and barely half were considered well-developed responses. 

5.  Writing. Will be forthcoming, but overall, students seem to be OK with basics, but do not perform as well when asked to write a convincing argument to support a position or handle more complex writing assignments.  Students’ are not as strong as we would like in tailoring their writing to an audience other than the instructor of a course.

Indirect assessment: student perceptions of program

Focus Groups 1995-96. A total number of 21 students in several small focus groups of 3-4 students discussed perceptions of GE program.    

Summary of findings. Students' comments covered a broad range of topics, but several themes recurred throughout the focus groups:

• Students have two predominant views of general education. First, they see it as an opportunity to find areas of interest for students undecided about an academic major. Second, it is a broad curriculum intended to broaden students' perspectives and outlook.
• Students did not describe a cumulative effect of general education. Instead they focused on experiences in individual courses that had influenced them.
• What students did not say about general education was an important finding. They did not say that the program had been a compelling educational experience.
• Students view the instructor as the key to general education. They described the best instructors as those enthusiastic about their subjects and about teaching, and as individuals who care about whether students learn. They criticized instructors who appeared disinterested or who merely presented information to be memorized for tests.
• Students believe the quality of teaching in their major is better overall than in general education courses because class size is smaller, the classes are more personal and instructors care more about the subject and about them as learners.
• Students experienced poor advising related to general education, which they believe has added to the length of their education.
• Students urged the university to do something about advising, and to use the student evaluation of instruction process to improve teaching in the program.

Graduating senior survey, 1996.  A total of 197 graduating seniors (31% of total surveys sent out) completed the survey. Basic findings include:

1. A majority of students believe that general education contributed substantively to their skills in writing, speaking, critical thinking and working independently and collaboratively.
2. A majority of students believe that general education did not contribute substantively to their understanding in most liberal studies areas--giving the impression that the program teaches students a little about a lot of different things.
3. Critical thinking is the overarching goal of the program, and two questions asked students about it. Fifty-seven percent of the students said that the program contributed substantively to their ability to think critically. On the second question, only 44% agreed that general education developed their ability to think critically.
4. A majority of students (65%) believe that general education is a valuable part of their education. Only 44% believe that it prepared them to understand complex issues and problems in life outside the university. And, only 39% believe that it prepared them for lifelong learning.
5. A majority of students believe that general education courses are not intellectually challenging, and that much of general education repeats their high school curriculum. A majority of students see the program as consisting of unrelated courses that have little to do with one another.
6. A majority of students believe that the teaching in their academic major was better than the teaching in their general education courses.
7. A majority of students believe that the advising they received for general education was not helpful.
8. When asked about general education experiences that had a positive effect on their learning, students singled out individual instructors, stimulating and engaging teaching approaches, and personal changes that happened due to general education.
9. When asked about general education experiences that had a negative effect on their learning students also singled out individual instructors, classes in which students are expected to be passive recipients of information, lack of challenge, lack of connection and relevance of the curriculum, poor advising, and the size of the program.
10. In order to improve the general education program students recommend that the university should do something to improve advising and teaching, reduce the number of required credits, increase course selection and student choice. Students also advocated making the program more meaningful" and relevant" by connecting courses to one another and to issues and problems related to life outside the university.

Senior survey 2003-2004.

Approximately 440 students, mostly seniors, completed a survey in the fall of 2004 and spring of 2005.  Students were asked a variety of questions related to their perceptions of the program and their view of possible changes in the program.  Several questions were multiple choice format and two questions were open-ended.  In terms of the skills category, students identified the communication courses (ENG 110 and CST 110 as well as WE/WIMP requirements) as particularly helpful or valuable but were equivocal about Math, Computer Science and Modern Languages and most had not taken Philosophy. 

Related to the Liberal Studies categories, in response to the question, “How well have your General Education courses helped you develop the following abilities and perspectives?” students rated Gen Ed favorably in the following areas

§         Communicate effectively ** 

§         Define and solve problems

§         Integrate knowledge across different disciplines**

§         Analyze contemporary complex issues

§         Understand human diversity

Students were equivocal regarding Gen Ed’s impact on:

§         Understanding of scientific methods/applications

§         Providing a global perspective 

§         Providing a basis for ethical decision making

Students rated Gen Ed less favorably for

§         Appreciation for the arts

§         Engage in responsible citizenship

§         Live a healthy lifestyle (Spring only)

In response to questions about possible changes to the program, students generally supported:

§         a required 2-3 credit freshmen seminar on current topics (as well as college success)

§         300/400 level courses as options in Gen Ed

§         a long list of course choices within each category**

§         some kind of off-campus multicultural experience beyond UWL

§         at least one interdisciplinary course

But students were equivocal or non-supportive of

§         a required 1-credit “student success” freshmen seminar

§         comprehensive freshmen year experience

§         a shorter list of courses, most students taking the same course

§         a required capstone experience as part of GE

§         requiring students to take a set of linked courses

§         requiring some kind of international learning experience outside the US

§         requiring some kind of community experience

The open-ended comments to a large extent mimicked the results of the 1996 focus groups and graduating senior survey.  Students wanted courses to be more connected to their majors, to real life, and/or to each other.  Several students seemed to think that GE was a way to find a major, and thus, if they already knew what their majors will be, GE is a waste of time and/or a repeat of high school.  Students’ comments suggested that they did not see the purpose of the general educations courses, nor did they see the courses as part of a coherent program.  . Students also suggested, often indirectly, that courses that open their eyes to new ideas, to global issues, to issues of diversity, or that helped “think beyond the box” were particularly valuable, but less available.  A sizeable number of students suggested that courses needed to be more interrelated, whether specifically linked or not, and that having more upper level classes was a good idea.  Reasons for this latter suggestion included:  Students felt they already knew the material in 100 level classes and they were not challenged; others had to wait to take some GE until junior or senior year and then found the 100 level courses “ridiculous and a waste of time.”  Many remarked that it was the instructor who made the class interesting, enjoyable and valuable.

Indirect assessment: faculty and staff perceptions, feedback, and work on outcomes

Departmental review of outcomes,  Fall 2004.

• Math/logical systems and languages:  This is one category where the outcomes selected as most important different significantly across courses in this category.  The outcomes selected for the Language courses, in particular, did not match the math or computer science outcomes.  There was a little more overlap with PHL 101 and the math/computer science, but still not a lot of overlap.

• Outcomes in what is now the personal, social and global responsibility were largely not identified as priority outcomes by any courses in the program.

Campus Survey, Spring 2005 .  Results are equivocal in terms of answers to each questions, but these patterns emerged:

1. Too much variability among courses in terms of academic rigor, even within same course/different sections; too many mundane courses--Gen ed should be our “perk” not a requirement. Ensure instructor quality
2. Skill courses should be integrated into liberal studies.
3. Too loose, too complicated, too many courses that are intro’s to majors, lacks cohesiveness; program needs more integration, program needs to be simplified.
4. RE: content--need more science, where is ethics?, senior integrative capstone maybe desirable,  information literacy needs to be included in some course like CST 110 or ENG 110. Require basic courses only. Stress cultural and environmental awareness-critical for the world we live in today.
5. Consider making diversity or international awareness more like WE
6. Seems to emphasize quantity over quality, students graduate without adequate knowledge of history, the world, or moral, ethical, civic formation
7. Keep everything the same, just change the college core(s).  Some want smaller program, some say keep the same.
8. Why is Gen Ed always the “whipping boy?” SCH is the big problem
9. Seniors in 100 level courses doesn’t make sense