Self and Society:  Understanding the Social World Assessment Report
October 1997

As part of general education assessment, four instructors, Bill  Cerbin, Joel Lazinger, Betsy Morgan, and Pam Rodgers, assessed student learning outcomes in the general education category.  Self and Society:  Understanding Oneself and the Social World.   The assessment took place Spring semester 1997 with students in Sociology 110:  The Social World and Psychology 100: General Psychology.

The purpose of the assessment was to determine the extent to which students achieve the intended learning outcomes of the Self and Society:  Understanding the Social World category, which are to:

    1.    understand and describe the use of elementary methods of social science inquiry
           (observation, experiments, surveys, etc.).

    2.    distinguish between personal experience/personal values and systematically tested
           scientific conclusions.

    3.    articulate the impact of societal institutions on the experience of individuals, groups,
           organizations, and categories of people.

    4.    demonstrate comprehension of basic processes used by human societies in their social
           and technological adaptations to the natural environment.

    5.    compare at least two major societal institutions across two or more societies, reflecting
           an appreciation of human diversity.

    6.    articulate the meaning of basic concepts from one of the social sciences and indicate
           their relevance to one's conduct as a citizen, consumer, worker or parent.

    7.    apply perspectives originating in any two social sciences to the analysis of a current
           event of the student's choice.

The Self and Society category contains courses from seven disciplines -- anthropology, archaeology, economics, history, political science, psychology and sociology.  We decided to assess outcomes 1, 2 and 3 which seem to represent core outcomes of  the category, i.e., the kinds of things every self and society course would address.  Other outcomes appear to be course specific, and we did not know how to adequately assess all of them.  In addition, we note that outcome 7 should be deleted from the category since students are not required to complete two courses in the category.

The Self and Society Test.  The group selected a topic, "welfare dependency," that has been studied from a variety of social science perspectives.  Students read a brief newspaper article, "Working Her Way Back," an account of a young mother trying to get off welfare (the appendix contains the article), and then wrote answers to the following questions:

1.    Social scientists believe that a number of economic, political, societal and psychological factors determine whether people develop welfare dependency.  Identify one factor (economic, political, societal or psychological) that might be affecting Tracy's situation.  Explain how it leads to or helps prolong welfare dependence.  Try to be thorough and specific.

2.    Tracy's story is incomplete.  In order to better understand welfare dependency we need more comprehensive, systematic information.  For the factor your identified in Question #1, describe a method you would use to study or ascertain how it contributes to welfare dependency.

Test Administration and Scoring.   Students were pre-tested in January, 1997 and took the test again at the end of Spring semester.  A total of 108 students from sections of Psychology 100:  Psychology and Sociology 110:  The Social World completed both the pre- and post-tests.

We developed the following rubrics to evaluate students' answers, and each instructor scored a subset of the tests in the sample.

Question 1:  Factors affecting welfare dependency.

Question 2:  Method for investigating welfare dependency.

Results and discussion.  Due to differences in the ways instructors scored the answers, we do not report the distribution of total scores on the items.  However, Table 1 reports the changes in scores from pre-test to post-test for both questions.

Table 1:  Qualitative Differences Between Pre-test and Post-test Answers

The table indicates that slightly less than half the students wrote better answers on the post-test for question 1, and a sizable minority, 24%, actually wrote poorer answers on the post-test.  It appears that the self and society courses had a marginal effect on students' abilities to think about the welfare dependency issue depicted in the article.  There was a similar pattern of results for question 2, but in this case only 36% of the students improved their answers on the post-test.

We found few exemplary answers for both questions, but a large percentage of the responses merely repeated information stated in the newspaper article.  On the other hand, fewer students than expected invoked popular stereotypes of welfare recipients as lazy, immoral, freeloading, or about welfare as a wasteful system that encourages these kinds of behavior and attitudes, etc.

The "welfare dependency" test measures a kind of "social science reasoning," the ability to analyze and evaluate relationships among variables or factors that influence social behavior.  Overall, students' answers were underdeveloped, revealing only a very rudimentary understanding of the social context of human actions, attitudes and values.  The results of the Self and Society test are similar to those of the General Education Science Test conducted in 1996, which also showed a relatively rudimentary understanding of science and scientific experimentation.

General education tries to prepare students to think critically.  We hope that students are able to analyze information in the popular media and to develop well-reasoned, informed positions.  Why don't students do better?

One possibility is that skillful thinking is difficult to acquire, and that a single social science course is unlikely to have a profound influence on social science reasoning.  Perhaps, the test should be given to students at the end of their undergraduate education after they have been able to integrate concepts from a variety of disciplinary areas.

A second possibility is that self and society classes do not prepare students for broad, real-life problem solving.  For example, students are "exposed" to social science methodologies in most self and society courses, but there is a big difference between "exposure" and actually being able to use those methods to analyze and evaluate "real-life" situations.  We wonder how often students are asked to grapple with complex, "real-life" dilemmas in self and society classes.  Our hunch is that the courses focus on specific content of the respective disciplines, and do not engage students in problem solving.  If this is true, perhaps we should not expect students to generalize information learned in a very specific context to an open-ended real-life dilemma.  In this sense, the test may lack content validity,  in that students are not really "taught to do" what the test measures.

A third reason for the relatively low performance and lack of  improvement lies in the assessment methodology.  Students had little incentive to do well, and may not have put forward their best effort.  In addition, the test is an academic exercise; perhaps it fails to engage students fully in thinking about the topic.  We suspect that the quality of students' reasoning might be significantly better if the task was more authentic, richer and really mattered.

Some additional issues arise from the assessment.  Is "social science reasoning" a collective aim of the Self and Society category?  We do not know the answer to this question, but it seems important in the course review process to examine the extent to which each course tries to foster social science reasoning.

If social science reasoning is a collective aim, what can we do to improve the level of students' social science reasoning?  There is no simple answer to this question.  However, if we want students to be able to critically evaluate "real-life dilemmas" then they should have opportunities to grapple with them in general education courses.  This suggests a more problem-based focus in these classes.

Are outcomes of the Self and Society category shared outcomes addressed by all the courses?  We suspect they are not.  If there really are no core content and skills for the Self and Society category, then future assessment should focus on what students learn in individual courses.  This would provide instructors with feedback they could use to improve student learning.  The disadvantage of course-based assessment is that we lose a sense of students' cumulative learning -- what they acquire as a result of the General Education Program.

Perhaps future assessment should try to assess student learning in individual courses as well as across all the courses in the Self and Society category.  One way to accomplish this is to ask departments to conduct a systematic assessment of general education outcomes in their classes, and to include some assessment tasks (e.g., problems, dilemmas, cases) that measure social science reasoning.  The tasks would be developed by a group of instructors representing the different Self and Society disciplines.