Molly McPup

Introduction to Research Methods in Political Science:
The POWERMUTT* Project
(for use with SPSS)

*Politically-Oriented Web-Enhanced Research Methods for Undergraduates — Topics & Tools
Resources for introductory research methods courses in political science and related disciplines

SITE
MAP

II. RESEARCH DESIGNS AND REPORTS
Revised version of essays by Charles McCall, California State University, Bakersfield [1]

Subtopics


The Research Design

Good research is not accidental.  It requires careful planning as well as careful execution.  Before launching a research project, social scientists prepare a research design, in which they set forth their plans for the research they intend to undertake.  Often, such designs are formal in nature, as when a scholar prepares a research grant request to a funding agency, or when a student submits a proposal for a dissertation, thesis, or other research project.

Even when a research design is not a formal requirement, it is a step in the research process that should not be bypassed.  With a good design the student will be able to foresee and avoid numerous obstacles and pitfalls, save time in the long run, and produce a superior finished product.

The length and complexity of research designs obviously vary considerably, but any sound design, be it for a massive, government or foundation supported study, or for an undergraduate term paper, will do the following things:

1.      Identify the problem clearly and justify its selection.

2.      Review previously published literature dealing with the problem area.

3.      Clearly and explicitly specify hypotheses central to the problem selected.

4.      Clearly describe the data which will be necessary for an adequate test of the hypotheses and explain how such data will be obtained.

5.      Describe the methods of analysis which will be applied to the data in determining whether or not the hypotheses are false.

1.         As you can see, step 1 requires you to state the problem and justify its choice; that is, you should explain your reasons for selecting it.  It is correct to say that the choice of a topic depends principally on an investigator's interest and values.  Nonetheless, there are some general criteria which we can use to evaluate the relative worth of topics.  For example, how important is the topic for development of knowledge?  How would the answer to questions which you pose for yourself affect other parts of the knowledge structure within political science?  Would it extend theories to areas not yet explored?  Would it replicate earlier findings in a different setting?  Would it resolve present inconsistencies in our interpretation of evidence or create inconsistencies by calling into doubt currently accepted interpretations?  A research project which adds to our knowledge in such ways is bound to be considered important.

While the promise of acquisition of knowledge is an important criterion in evaluating research proposals, it is not the only standard we apply.  If additional research on the problem will improve the social, political, or economic well-being of people, that fact helps justify the choice of a research topic.  Would the answers to the questions that you pose for yourself affect the achievement of accepted social values?  Might such answers cause changes in accepted social values?  Positive answers to such questions suggest that the research proposal is one which is important.

A third criterion which we use in evaluating a choice of research problems is the criterion of economy.  How easily can the topic be studied?  We are not likely to establish a moon colony simply to discover whether or not the relationship between sense of political efficacy and political participation which we've discovered repeatedly in Western nations persists among those living on our permanent satellite.  No one is likely to support a proposal to expend great amounts of time, skill, and money on a project likely to add little to our knowledge or to human welfare.

2.         Before beginning any research project, you should familiarize yourself with the work that has already been done in your area, so that your project can build on and extend (or, if necessary, correct) the findings of those who have preceded you.  A summary of this review of the literature should be included in your research design. Such a review is helpful not only in systematically gathering and organizing existing substantive knowledge about the topic but also in sensitizing scholars to the methods and procedures which have been used by others investigating the same general area.

You are probably used to using Wikipedia and similar Websites to find information.  While these are great resources for casual inquiry, they are not the best places to go to find scholarly research, which is most commonly found in professional journals like the American Political Science Review, the Journal of Politics, or the American Journal of Political Science

One way to get started on a “lit review” of journal articles and other scholarly publications is to find a journal article on your topic; the list of “references” or “works cited” in this article will in turn lead you to other relevant material.  Your campus library has holdings, hard copy and/or electronic, of many scholarly journals.  In addition, the library may subscribe to JSTOR (http://www.jstor.org) (short for “journal storage”), which allows you to access full text of articles from a large and growing list of publications.  Visit your library’s Website, or talk to a reference librarian to find out whether your institution is a JSTOR subscriber and, if so, how to access it at your campus.  Even if your campus doesn’t subscribe to JSTOR, you can still obtain citations and article abstracts, but will then need to go to other sources for the full text of the article. If your campus library doesn't have what you are looking for, it may be able to obtain it for you through interlibrary loan (also known as "document delivery"). Caution: articles from many journals are subject to a 3 - 5 year embargo before becoming available through JSTOR.

Another good source for finding citations of scholarly articles is Google Scholar (http://scholar.google.com/; not the general Google search engine with which you are probably already familiar).   Once you’ve located an article, you can also click on “Cited by” for a list of later publications referencing that article.  While freely available, Google Scholar will generally provide only citations and abstracts of articles, not full text.

Yet another place to find citations of data-driven research is the Interuniversity Consortium for Political and Social Research (ICPSR) (http://www.icpsr.org).   You can search the “Bibliography of Data-Related Literature” for publications using data in the ICPSR archives.  Alternatively, you can go to “Find and Analyze Data,” locate a dataset that you think might be of interest, and click on “View related literature.”  All of this information is freely available, but to actually obtain the data used by the author(s), you need in many cases to be at a campus that is an ICPSR member (see below).

3.         The third major step in a research design is to clearly state the hypotheses to be tested.  As used here, a hypothesis is a declarative statement positing a relationship between variables. In its simplest form a hypothesis contains these elements: an independent variable; a dependent variable; the nature of the relationship between the two.

The development of hypotheses belongs to that area of inquiry which has been called the logic of discovery.  The logic of discovery deals with the identification of good reasons for suggesting a hypothesis — reasons for putting it forward or testing it, not reasons for accepting it.  One good reason for testing a hypothesis is that it is commonly believed.  Gerald Pomper in his book, Elections in America, noted that the literature of political science was replete with claims that party platforms were unrelated to legislative enactments.  Pomper also noticed that there was very little systematic evidence in the literature supporting the hypothesis; thus he decided to test it.  His research falsified the hypothesis, which had been a standard one in political science, and therefore has been of great importance to the discipline.

A second way in which hypotheses are developed is through the use of analogy.  Hypotheses are often based on metaphors drawn from our environment.  Darwin explained that it was his perception of the analogy between Malthus' theory of human population and processes in the animal kingdom which led him to develop the theory of evolution.  The analogy between chess and international politics is an old one in political science, and we have discovered that a careful study of games can be quite suggestive for the development of hypotheses regarding the ways in which nations treat each other.

Another successful use of analogy in the social sciences is evident in the application of formulae developed to describe the spread of communicable diseases as formulae to describe the spread of rumors in society.  Not all analogies are good ones.  The test of a good analogy is that it suggests numerous and significant hypotheses and that those hypotheses are not refuted by systematic testing.

A third way of developing hypotheses is retroduction.  That method which Arthur Goldburg has defined "as an imaginative leap, a flight of fancy, taken to account for observed phenomena" is insufficiently understood.  Many of you will know the tale regarding the discovery of Archimedes' Law that a solid body will, when immersed in water, suffer a loss in weight equal to the weight of the displaced water.  As the story goes, Archimedes had long been perplexed by a problem set for him by the tyrant of Syracuse.  He had commanded that a votive crown of pure gold be prepared for placement in one of the temples, but gossip concerning the goldsmith suggested that some silver had been mixed with the gold in the manufacture of the crown.  Archimedes was charged to discover without injuring the crown whether or not that was the case.

The story continues that Archimedes, while taking a bath noticed that his limbs were unusually light when in the water and that in proportion as his body was immersed in the tub, the water spilled over its edges.  That perception led to instantaneous realization that he could solve the problem.  But if the myth has any basis, the fact that Archimedes after long study was hit so quickly by the solution to the problem that he was moved to leap out of the tub and run home naked shouting "eureka!" does not mean that the solution was not a result of his long study.

Regardless of the way in which hypotheses have been discovered or developed, it is crucial that they be stated clearly.  If they are clearly stated, the distinction between independent and dependent variables should be obvious.  Naturally, careful statement of hypotheses also entails clear, conceptual definition of the major variables involved.

4.         After you have stated your hypotheses clearly, you are ready to proceed to the fourth major stage in the research design, the description of the data required to check the hypotheses and the explanation of how the data are to be obtained.  Your hypotheses determine in large part the kind of data required and suggest methods for collecting or obtaining them.

Let us assume that you plan to test the assertion that conservatism is related to social class such that the higher peoples' social class, the more likely they are to be conservative.  Your first task is to operationalize the variables.  How will you identify "conservatives" and "liberals" in terms of operations which others could repeat?  How will you identify in similar terms a person's social class?

Suppose you decide to operationalize conservatism on the basis of a scale based on a series of questions about political issues.  You then need to specify the questions to be included in the scale and describe the method by which responses to those questions will be scored and summarized.  Now what about social class?  Suppose you decide to operationalize social class on the basis of income.  Suppose further that you decide to group your data into the following categories: less than $40,000, $40,000 to $99,999, and $100,000 and over.

Your next step is to explain how the necessary data are to be obtained.  In general, this might involve either primary analysis of data which you will collect especially for this project, or secondary analysis of data that has already been collected for some other purpose. "Secondary" doesn't mean second rate.  In fact, a number of sources provide data of the highest quality that are far beyond what you could hope to produce yourself (assuming that you lack unlimited time, money, and organizational resources).

Some of these sources are available by institutional subscription only. Check with your instructor or reference librarian to find out what's available at your campus. These include:

There are a number of other sources of data that are freely available. For a list of just a few of these, see http://www.ssric.org/data/other.

Wherever they come from, the data that are to be analyzed must be described fully and accurately.  The goal is to give readers sufficient information to allow them to successfully evaluate your proposed project and the analysis and conclusions which you will present later.

The main point to remember is that in principle the data you plan to analyze must be capable of falsifying the hypothesis if the hypothesis is in fact untrue.  What will count against it?  That is probably the most important question in building theory.  Indeed, the principle which should govern the selection of evidence to test the hypotheses is the principle that the chances of discovering decisive negative evidence should be maximized.  The best design is one which would most clearly and quickly expose the error in a working hypothesis.  That means that we go out of our way to look for negative evidence.

5.         You are now ready to turn to the fifth major element in a good research design, the identification of the methods of analysis appropriate for treating your data.  Be as specific as possible in describing the approach you will use, but do not propose to employ a technique that you do not really understand in the hope that this will somehow give your design an aura of erudition.


The Research Report

A research report contains all the elements of a research design, and in addition presents, interprets, and evaluates the findings of the research.   If you have written a sufficiently careful research design (and have been rather lucky), you may be able to incorporate your design in its entirety in your research report.  In practice, however, few research projects go precisely as planned, and the original design will likely have undergone at least some modification before the finished report is prepared.

In presenting your findings, the necessary evidence should be set forth and the conclusions based upon it should be clearly stated.  Many beginning scholars fall victim to the temptation of presenting too much evidence and analysis in the central portion of the paper.  Remember the problem you set out to attack. Interesting findings which are tangential to the problem should be presented in footnotes or in an appendix.   If you include them in the main body of the text, you may only succeed in directing the reader's attention away from your principal focus.

It is in this central portion of the paper that you will present most of your tables and graphs.  Throughout the report, your main objective must be to communicate clearly, and this applies just as much to your tables as it does to your sentences.  Tables and graphs should be placed as close as possible to the discussion of the data which they contain.  They should be so structured and so titled that the reader can understand the figures presented in them without reference to the text surrounding them.  Proofread your tables and graphs with extra care for it is very easy to make an error in copying figures.

The concluding section of the paper, like the opening one, performs several functions.  You should begin by summarizing the findings clearly and concisely, showing how they confirm or disconfirm the hypotheses with which you started.  This section also includes your own critique of your research indicating the shortcomings of your work and suggesting how they might be overcome with sufficient resources.  Finally, the concluding section of the report should identify further questions raised by the findings of your study which demand future research.


Exercises

These exercises use the search term “voter turnout,” but feel free to try another term of your choosing.

  1. Go to JSTOR at http://www.jstor.org, or access JSTOR through your library web site.  Search on “voter turnout,” sorting by “newest to oldest.”  Note: if you include the quotation marks, you’ll retrieve entries about voting turnout; if you leave out the quotes, you’ll retrieve a much longer list of entries, but the list will cover a broader area of inquiry. Try narrowing your focus by searching on “voter turnout” and “presidential elections” (both terms in quotation marks).
  2. Try a similar search at Google Scholar (http://scholar.google.com/).  You won’t be able to sort results by date, but can limit your search to only works published in recent years.
  3. Go to the ICPSR site (http://www.icpsr.org), or access ICPSR through your library web site.  Click on “Find and Analyze Data” and on “Find Publications.”  Again search on “voter turnout.”  Sort by “Pub date (newest).”  Click on any citation to find out what data in the ICPSR archives were used in this publication. Now click on “Find and Analyze Data.”  Search on “voter turnout .”  Sort by “Time period (newest).” Scroll down to “Latino National Survey (LNS), 2006 (ICPSR 20862)" by Luis R. Fraga et al., and click on it.  From the menu on the left, click on “Related Publications.”
  4. You have decided to write a senior thesis on recent changes in patterns of voter turnout. Having reviewed the literature, you are now ready to obtain data necessary to test the hypotheses you have formulated. You obviously do not have the resources necessary to carrying out a series of nationwide public opinion surveys, and so wish to obtain data that have been archived. Using the resources described above, find and download a survey that you think might be useful. Save the survey for later use.

[1] This essay is a revised version of “How to Write Research Designs and Research Reports,” © Charles McCall and the CSU Social Science Research and Instructional Council (SSRIC), 1998. Used with permission.

 


Last updated April 28, 2013 .
© 2003---2013  John L. Korey.  Licensed under a  Description: Creative Commons License Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.