Introduction to Research Methods in Political Science:
I. POLITICAL SCIENCE AS A SOCIAL SCIENCE
Political Science is in part a social science, and in part a humanity. Both are important. In this topic, we'll look at the basics of social science inquiry, and then proceed to show how this differs from, on the one hand, inquiry in the natural sciences and, on the other, inquiry in the humanities.
Social science inquiry seeks to develop empirical theory. "Empirical" refers to things that can be experienced through the five senses of seeing, hearing, touching, tasting, or (in the case of political corruption) smelling. “Theory” basically means explanation. An empirical theory of politics, then, is an explanation of why people behave the way they do politically.
While this approach is only part of political science, it has become a very important part. In 2001, for example, almost three-quarters of the articles in arguably the top three scholarly journals in the discipline (the American Political Science Review, the American Journal of Political Science, and the Journal of Politics) included analysis of empirical data.
If a social scientist (or anyone else) observes people engaging in political behavior, he or she will need to focus on certain characteristics of the people being observed. The observer may wonder why some people differ from others in their political characteristics. Why, for example, are some people Democrats while others are Republicans?
A characteristic that differs from one individual (or "aggregate," such as state, country, etc.) to another is called a variable. One that does not is called a constant. Constants are generally less interesting than variables. There is not much point in trying to explain voting choices in a country in which only one party appears on the ballot. Of course, we might then ask why some countries have only one party whereas others have multi-party systems, but now we are treating “number of parties” as a variable.
Variables take on different values. These may or may not be mathematical values. If we are comparing party systems of different countries, the values of the variable may be the number of parties the country has. On the other hand, if we are studying individual party identification, the values of our variable might be “Democrat,” “Republican,” and so on.
The observer may notice that the values that a variable takes on are not random, but are related to the values of another variable. For example, one-party political systems may be more common in countries with low levels of literacy.
A statement positing a relationship between two variables is called a hypothesis. Hypotheses have three elements:
The terms “dependent variable” and “independent variable” are similar to the terms “effect” and “cause” respectively. The fact that two variables are related, however, does not necessarily mean that one causes the other, even indirectly.
Everyday language is full of what are, in effect, hypotheses about political behavior. For example, talk about a “gender gap” in voting hypothesizes that vote (the dependent variable) is in part a function of gender (the independent variable), with women more likely than men to vote for Democrats and men more likely than women to vote Republican.
Social science research differs in two ways from everyday discussion that attempts to explain politics. The first is where hypotheses come from. Anyone who follows politics will likely carry around in his or her head a lot of ideas about what explains political behavior. Such ideas may come from personal experience, from conversations with others, or from following politics through the mass media. This is true as well for the ways social scientists think about politics. In addition, however, social scientists develop hypotheses more systematically by studying the scholarly literature for the results of previous research. This is important for at least a couple of reasons.
For one thing, it is usually the case that the more you learn what is already known about a subject, the more new questions you are likely to have. A review of the literature helps generate new hypotheses. For another thing, and even more importantly, social science seeks not merely to describe raw facts, but to explain why people behave the way that they do. To accomplish this, we need to put our ideas into a broader theoretical context that offers such an explanation. It is a fact that, from 1936 through 2000, the incumbent party had always won the presidency whenever the Washington Redskins (who were the Boston Redskins in 1936) won their last home game before the election, and lost whenever the Redskins lost. However, since there is no reasonable explanation for why this should be the case, it is merely an interesting bit of trivia, and no serious observer of politics would rely on it in analyzing the next presidential contest.  (The pattern did not hold in 2004 or 2012, when Presidents Bush and Obama won reelection despite Redskin losses at home on the Sundays preceding election day.)
A second difference is that, for many people, ideas about patterns of political behavior remain merely assumptions. Social science insists that the validity of assumptions must be tested against data.
Testing a hypothesis requires, among other things, defining its terms. This needs to be done at two different levels.
We strive for a consistent one-to-one correspondence between our conceptual definitions and our measurements (operational definitions) of them. If we succeed, then our measurements have validity and reliability.
Data needed to provide operational definitions of our variables come from a wide variety of sources. We may gather the data ourselves. Analysis of data that we gather in order to test hypotheses that we have formulated is called primary analysis. Often, however, this approach would be totally beyond our resources of time, money, and expertise. A nationwide survey of public opinion, for example, would take months to design and carry out, would cost many thousands of dollars, and would require the services of a large survey research organization. Often, secondary analysis of data (that is, analysis of data originally gathered for other purposes) will suit our needs far better. Indeed, a number of very important databases are used almost exclusively for secondary analysis. The United States Census is a good example. (Its primary purpose is to provide the basis for apportioning seats in the U.S. House of Representatives among the states.) The General Social Survey was created for the express purpose of providing quality survey data for secondary analysis. The bibliography for the American National Election Study includes thousands of entries, the bulk of them employing secondary analysis.
To facilitate secondary analysis, the Inter-university Consortium for Political and Social Research (ICPSR) was established in 1962, providing an archive for social science data. Today, there are approximately 700 member institutions, mostly colleges and universities, from all over the world. Students and faculty at these institutions obtain datasets that provide the basis for numerous scholarly books, articles, and conference papers, graduate theses and dissertations, and undergraduate term papers.
We also often distinguish between individual data (for example, a survey of prospective voters) and aggregate data (for example, information about states or nations). POWERMUTT includes individual data files (the subsets of the American National Election Study and the General Social Survey), aggregate data files (the files on American states and on countries), and files containing both type of data (the file on the U.S. Senate, which includes data about both states as aggregates and about members as individuals).
The Social Sciences and the Natural Sciences
What we have described as the social science method – the effort to explain empirical phenomena by developing and testing hypotheses – could as easily be called simply “the scientific method,” without the “social” qualifier. There are, however, important differences between the social sciences, including political science, and the natural sciences.
One difference is that the natural sciences rely much more heavily on experimental design, in which subjects are assigned randomly to groups and in which the researcher is able to manipulate the independent variable in order to measure its impact on the dependent variable. Often, when people think about the scientific method, what they have in mind are these sorts of controlled experiments. In political science, we for the most part are not able to carry out experimental designs. If, for example, we wish to study the impact of party affiliation on decisions by judges, we cannot very well assign judges to different parties, but rather have to take the data as they come to us from observing judges in their natural setting.
Experimental design, however, does not define the natural sciences, nor does its absence define the social sciences. Astronomy, for example, must of necessity rely on observation of things that cannot be manipulated. “Epidemiological” medical research relies on non-experimental data. Conversely, the social science discipline of social psychology has been built in large part from experiments in small group laboratories. In political science, a great deal of laboratory research on the impact of campaign commercials has been carried out in recent years. Field experiments are also common, as when survey researchers test the impact of alternative question wordings by splitting their sample and administering different questionnaire forms to different subsets of respondents. Nevertheless, it is fair to say that experimental designs are much less common in the social sciences, including political science, than in the natural sciences. Most of our research design is, in effect, an effort to approximate the logic of experimental design as closely as possible.
Other differences, also differences in degree, have to do with lower levels of consensus in the social sciences.
It bears repeating that these differences are ones of degree. In the natural sciences there are also disputes at the frontiers of the various disciplines about what concepts are important, what they mean, and how they should be measured. In the social sciences, consensus is likely to break down from the start.
Even if we can agree that a particular concept is important, on what it means, and on how it should be measured, we will usually encounter far larger problems of measurement error than those in the natural sciences, where measurement is not without error, but is typically much more precise.
Finally, remember that we are involved in trying to explain human behavior. People do not seem to behave as well as molecules. It may be that human behavior is inherently less predictable.
All of these things mean that our theories are less rigorous and complete than many that have been developed in the natural sciences. Instead of laws (that is, statements that predict with great accuracy what will happen under certain given conditions, such as Newton’s laws of dynamics or Einstein's theory of relativity), we have tendencies. The absence of laws greatly limits our ability to develop theories. The fact, for example, that the outcomes of past presidential elections have been closely correlated with the state of the economy (or, as Bill Clinton's 1992 campaign manager James Carville famously put it, “it’s the economy, stupid”), does not mean that the same will necessarily hold in the next election.
The fact that we deal with tendencies rather than with laws means that, for the most part (and despite impressive work by “rational choice” theorists to develop formal mathematical models of political behavior), political science makes relatively little use of geometry, with its elegant systems of deduction, but considerable use of statistics, "the science of uncertainty," which provides us with tools for dealing with probabilities.
Despite its unavoidable limitations, political science as a social science has produced an explosion in our knowledge about politics. This has had important practical consequences. For example, no serious aspirant for a major elected office in an economically developed democracy would consider embarking on a campaign without consulting experts in survey research, a signature social science method.
The Social Sciences and the Humanities
In addition to being, in part, a social science, political science is also in part a humanity. Political science as a humanity means at least a couple of different things.
Normative theory. Because social science is limited to studying that which can be tested empirically, there are many vital questions about politics that are beyond its scope. Also central to political science, therefore, is what is called “political philosophy” or “normative theory.” Whereas empirical social theory seeks to explain why people behave the way that they do, normative theory seeks standards for judging how we ought to behave. Examples include just war theory and theories about the equitable distribution of wealth, power, or other resources.
Empirical and normative theorists have often squabbled about the relative value and validity of their respective parts of the discipline, a dispute manifested in recent years in the arguments for and against the “Perestroika” movement. Both approaches, however, are needed for a comprehensive study of politics. Well over two thousand years ago, Aristotle managed to combine both approaches in his study of Greek city-states, classifying them in terms of whether their regimes were just (a concern of normative theory) and whether power was in the hands of the one, the few, or the many (an empirical question).
Descriptive analysis. Not all empirical study of politics involves the methodology of social science. While Aristotle's classification of city-states had an empirical component, he did not develop or test hypotheses. This is hardly surprising, since the scientific method as we know it was not developed until about 400 years ago though the writings of Francis Bacon (1561-1626) and others, did not spread to the social sciences until the 1800s, and did not become a major part of the political science mainstream until well into the Twentieth Century. What, for want of a better term, might be called “descriptive analysis” remains an essential part of the discipline. It differs from political science as a social science in several important ways.
Some areas of political science lend themselves more to social science inquiry than others. At one end of the spectrum, social scientific approaches dominate studies of voting, which involve analyzing patterns of behavior within entire electorates that may consist of millions of people. On the other hand, President Lyndon Johnson once responded to critics by noting that “I’m the only president you’ve got.” Because most countries have only one president (or one prime minister, chancellor, dictator, or monarch) at a time, studies of chief executives tend with notable exceptions to take a more holistic, humanities-oriented approach. Studies of legislative bodies fall someplace in between – biographies of legislators and case histories of individual bills will combine with roll-call analyses that seek to find patterns in voting alignments. Studies of industrialized nations generally lend themselves more to social science analysis than those of developing nations because of the relative lack of reliable data in the latter.
As with empirical theory and normative theory, there need be no quarrel between social science theorists and those doing descriptive analysis. Rigorous efforts that develop valid generalizations about political behavior though analysis of large databases are complemented by the rich context and detail found in studies of unique individuals and events.
ExercisesThese are group exercises for the class as a whole, or for small groups.
1. Develop a rough and ready
theory of party identification. Taking the case of the
2. Repeat the first exercise, but with a different dependent variable.
For Further Study
“Elementary Concepts in Statistics,” StatSoft. http://www.statsoft.com/textbook/esc.html.
John L. Korey, “Political Science,” in Kimberley Kempf-Leonard (ed.). Encyclopedia of Social Measurement. San Diego: Academic Press, 2004: Vol. II, 99-108. This number does not include articles using "analytical theory," most of which employed highly mathematical formal models of politics but which contained no actual data. These made up almost half of the remaining articles.
David Juran,“Continuous Distributions and Portfolio
Analysis,” Managerial Statistics http://www.columbia.edu/~dj114/part3.doc, 105. Accessed
The NES Bibliography .http://www.electionstudies.org/resources/papers/reference_library.htm. Accessed
D.W.Miller, “Storming the Palace in Political Science,” The Chronicle of Higher Education; Stephen Earl Bennett, “‘Perestroika’ Lost: Why the Latest ‘Reform’ Movement in Political Science Should Fail,” PS. June, 2002; David Little. "The Perestroika Debate in Political Science," Understanding Society. April 19, 2008. http://understandingsociety.blogspot.com/2008/04/perestroika-debate-in-political-science.html. Accessed April 22, 2013.
April 28, 2013 .
© 2003---2013 John L. Korey. Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.