I would like to thank Professor Fan <and others> for the fine hospitality you have shown my wife, Suzanne, and myself. You have already made us feel as though we are a part of the Chinese sociological and survey research families.
For me, this event has a very special meaning. About thirty years ago, I had the honor of meeting Professor Fei Xiao Tung, when he and some colleagues were making a tour of the United States. Professor Fei had been given the task of reinstating sociology in the People's Republic after a thirty-year hiatus. He visited several American universities to establish ties and to get a some ideas that might be useful to his efforts at home.
I was his last American stop, at the University of Hawaii. Whereas his previous contacts had shown much interest in joint research projects and exchange programs, I think I was the first to hear another question Professor Fei had asked everywhere he visited: "What do you teach in Introductory Sociology?" Not sure I had heard him correctly, I cautiously showed him the introductory sociology textbook I had written, and his face lit up. I was encouraged by that reaction to show him the textbook I had written in research methods, and Professor Fei left Hawaii with both books in hand. I believe he used those books in his early courses in 1979 and after, and I am delighted that a Chinese edition of the research methods book is currently published in Beijing.
Although I will devote most of my remarks to contemporary survey research, I want to start by noting it is an ancient research technique. The ancient Egyptians, for example, conducted surveys to help them plan the future need for crops and other resources. And Christians believe Jesus was born in Bethlehem because his parents had to report there for a regular Roman census. (Those were the old days, when survey researchers could order respondents to report to a location to be surveyed.)
Today, I have been asked to briefly address two topics: (1) developments in survey research in the United States and (2) ideas about teaching social research methods. Let me begin with some current developments in survey research.
When I wrote my first textbook in survey research in 1973, I suggested there were two well-accepted ways of conducting surveys: in face-to-face interviews and in self-administered questionnaires, typically through the mail. I also indicated that researchers sometimes conducted surveys over the telephone, but this was known as a "quick and dirty" technique and was not very well respected. My textbook addressed all the problems inherent in telephone surveys.
As you know, over the years since then, the methodology of telephone surveys has much improved, and they have become much more acceptable by researchers and by the public. Researchers have loved them because they are so much easier and cheaper than face-to-face household surveys. To some extent, I think it became more acceptable to a public that had grown wary of strangers appearing on their doorsteps asking to come into their houses to conduct an interview.
One of the chief problems with telephone surveys initially was that the most obvious sampling frame--telephone directories--did not contain everyone with a telephone and those same directories contained numbers for businesses, government offices, and other agencies inappropriate for the survey. The development of Random Digit Dialing was a major advance, allowing the selection of random numbers within the ranges of active telephone numbers, whether they were listed in the published directories or not. Omitting the ranges of numbers set aside for government and some others inappropriate to the survey made the process more efficient.
Computer Assisted Telephone Interviewing (CATI) allows interviewers to call randomly-selected numbers and read questions displayed on the computer screen, using the keyboard to enter responses directly into the growing data base. Laptop computers are sometimes used by interviewers in a similar fashion in face-to-face interviews. Going a step farther, sometimes respondents are given a laptop computer for a self-administered survey, reading the questions and entering their own answers. This is especially good in surveys of sensitive issues which respondents might not feel comfortable discussing with a stranger.
Robo Surveys involve computers even more deeply in survey research. When the respondent answers the telephone, a recorded voice introduces the survey, and asks a question. When the respondent answers, voice recognition software identifies and records the response. Then the recorded voice asks the next question. While this is obviously a very inexpensive way of conducting surveys, it has not always been well received by the public, at least according to the comments I have heard. The negative reception has been worsened by the use of Robo Calls to advertise products or political candidates.
Over the past thirty years, telephone interviewing has become a main workhorse of survey research in the United States. However, as this method has become a well-honed professional tool, it has begun experiencing new problems. Initially, respondents were not accustomed to researchers calling them to conduct a survey, but over time it became more familiar, and some respondents undoubtedly have felt important to have been selected to express their opinions. In recent years, however, such calls have become too familiar, and completion rates have declined. Refusals to participate in telephone surveys have been worsened by telephone advertising campaigns, especially those that masquerade as opinion surveys. Similarly, some unscrupulous political campaigns engage in “push-polls” that pretend to poll opinions but are really intended to spread negative images of opponents. For example, “Would it affect your vote if you learned that Mr. Smith had been accused of sexually molesting children?”
One of the most serious problems complicating telephone surveys at present involves the role of cell phones. Initially in the United States, cell phone users had to pay for each call made--including those received from others. As a consequence, researchers and advertisers were prohibited by law from calling cell-phone numbers. At first, this was not considered a significant problem, since there were relatively few cells phones and those who had cell phones also had regular land-line telephones which would be available for selection in survey samples.
In recent years, however, the number of cell phones has increased a great deal, and many people use their cell phone as their only telephone. (I realize this is much more common in many other countries, including China, I imagine.) This creates a difficult problem for survey sampling. It is now legal to call cell phones for surveys, and to ignore that population would risk skewing the sample selected--since we know that cell-phone users tend to be younger and cell phones are more common among some ethnic groups than among others.
At the same time, including cell phones in survey samples runs the risk of giving some people too great a chance of selection: those who have both a land lines and cell phones. At the very least, it is important to ask respondents how many telephone numbers they have and take that information into account in weighting respondents.
I’m sure that telephone surveys will remain a common research technique in the future, but some methodological adjustments will have to be made.
I want to say a word about another survey technique that is gaining popularity at present: online surveys. Ironically, I now find my textbooks saying the same things about online surveys that I said about telephone surveys thirty years ago. First, they are quicker and cheaper than the established methods. Initially, online surveys were a novelty, unfamiliar to potential respondents. Some were no doubt wary about participating. Only short and reasonably simple surveys seemed possible. And, the chief problem was representativeness. In the case of online surveys, clearly not everyone has access to the internet, and those people who do not--a non-random sample of the population--will be unrepresented in an online survey.
There are, however, some surveys for which the online respondents are the perfect people to hear from. For example, if you are running a website and want to know how well it works for those who visit it, you are directly in touch with precisely the population you wish to survey. Asking questions of a sample of those who visit your website will give you exactly the information you want. Similarly, when I call a help line, I am frequently asked if I would be willing to participate in a survey afterward, to learn how satisfied I was with the service.
Some populations are sufficiently present on the internet to make online surveys quite appropriate. College students would be an example. Perhaps student email addresses could be sampled and questionnaires sent to them. Or they could be directed to a website containing the questionnaire.
There are now programs available for conducting online surveys. The best known is Survey Monkey (at surveymonkey.com). This service is free of charge for limited surveys, and, of course, you can pay for the Professional version with more capabilities. I have found this a useful device for teaching students about online surveys.
In addition to these specialized uses of online surveys, American pollsters are attempting to use them for more general purposes: such as political polls. The Harris company, for example, has been very involved in developing this usage. While they have shown some impressive results at times, they have been very secretive about their techniques.
In addition to these and other developments in the conduct of surveys, the secondary analysis of survey data has become easier and easier. Since the 1970s, the National Opinion Research Center in Chicago has been conducting a large-scale survey of the American public: The General Social Survey. The explicit purpose of this activity has been to provide a large data base available for analysis by students and faculty everywhere. Initially, it was necessary to obtain a physical copy of the data set to be analyzed with SPSS or a similar program, but today there are effective online analysis programs that can be used to analyze the GSS data without ever downloading a copy of the data file.
So, in sum, survey research is alive and well in the Unites States, as elsewhere. It serves as a vital source of information for scientific, governmental, and other purposes. Moreover, survey research is an evolving methodology, and I am excited to see what new forms it takes.
Finally, I will say a little about the teaching of survey and other social research methods. Though I began my career as an active researcher, the writing of research-methods textbooks has made me more of a teacher than a researcher. So let me mention a few of my observations as a classroom teacher and as a textbook author.
One of the decisions teachers must face is whether to focus on the fundamentals of research or attempt to include as much advanced materials as possible. I have mostly taken the first of these options. My feeling is that advanced techniques will keep evolving, and what you teach today may be outdated by the time students need to use it. If they are firmly grounded in the fundamentals of social research, students will be able to learn and employ new techniques as they come along.
Some of the fundamental concepts of social research can be difficult to grasp. Please realize that I am speaking from my experience with American students, and I know that cultural differences can affect these. I’ll mention three ideas that seem especially hard for my students and perhaps for yours as well.
Probabalistic causation. Everyday notions of cause and effect sometimes conflict with the notion of causal relationships in social research. When I report to students that women are more religious than men in the United States (and elsewhere), they often object that they know of contradictions to that causal relationship: very religious men and very irreligious women. They need to learn that one variable can “cause” another even when it does not do so in every case. This is all the harder when percentages on the dependent variables are low. For example, I report that children from broken homes (e.g., parents divorced) are more likely to become delinquent than those brought up in intact homes: even though only a small percentage of those in broken homes become delinquent--but a higher percentage than for others.
Units of Analysis. This is a critical concept since it resolves many miscommunications about research results. I’ll give you an example from my own university. Students were concerned that the administration was using more and more part-time faculty members (cheaper for the university) than full-time, tenured and tenure-track faculty. They did some research and published their complaints in the school newspaper, indicating that half the faculty were part-time. The administration responded by reporting that not half but three-fourths of the courses were taught by full-time faculty members. You may have noticed the shift of units of analysis: the students spoke of faculty members, the administration spoke of courses.
Both assertions could be right. If the average full-time faculty member taught 3 courses and the average part-time faculty member taught one course, and if the students were right about equal numbers of faculty in the two types, then the administration would also be right. You can see this easily if you think of a pair of faculty members: one full-time and one part-time. Half of the pair is full-time, but that full-time faculty member teaches three of their four courses.
The shifting of units of analysis explains a lot of the seemingly contradictory claims made in political arguments: whether unemployment, the cost of living, crimes, etc. have gone up or down.
Nominal definitions. Social researchers study things that ordinary people talk about and think they understand: political orientations, social class, prejudice, etc. Students must be taught that all such concepts are socially defined, none are “real” the way physical objects are real. What is social class? In the United States, it can be defined in a number of ways, involving income and wealth, education, occupation, etc. Different definitions are likely to produce different research results. I often tell my students that while I may teach them some new things, it is also my job to take away many things they think they already know. They are seldom happy with that prospect.
I hope these comments will provide some useful views of the evolution, use, and teaching of survey research and social research in the United States. I will be very interested in learning from the others at this conference, from China and elsewhere. Over the years, I have tried to make my textbooks and teaching more and more global, and I look forward to learning about some research techniques and research examples from the other participants at this conference.
Finally, I would like to thank the organizers again for their hospitality and for organizing this opportunity for this survey research family gathering. Survey research is more than a data-collection method for academic researchers. It is also a voice for the people of a society, and I look forward to a continuing and expanding conversation among the many societies represented at this conference. I am deeply honored to be a small part of this great conversation.