|
|
The
Number of Cases
Comparative politics often appears afflicted with a variety of analytic
greed, in wliicli 'more is better'. This greed is driven in part by the
dominance of statistical analysis in contemporary political science, with
[he consequent need to have relatively large 'samples' of countries in order
to be able to meet the assumptions and requirements of statistical analyses.
In some ways, however, this concentration on the demands of particular statistical
techniques reverses [he appropriate logic of research design. The statistical
techniques, or other methodological tools, that researchers select for use
should be employed in the service oflhe analytic and theoretical questions
we arc pursuing, not vice versa. Certainly, there are some research questions
that can only be addressed with large and almost comprehensive data-sets,
but they are also some questions that are best addressed with small and
focused selections of cases, or even a single case. Indeed, (lie use of
larger 'samples' of cases may make some styles of research less useful rather
than more.
In addition to meeting (he requirements of formal explanation of phenomena,
the statistical method is also attempting to achieve another of the fundamenlal
goals of scientific enquiry ~ parsimony. One problem with cross-national
research based on a limited number of cases is tliat it is dilncull to ekclude
many of the extremely numerous possible causes for the observed differences
in the dependent variable. Kadi case bring-, witli it so manv interrelated
variables that it is impossible to exclude very many independent variables
with such a limited niiinlK'r of cases. As llic si/.c of the .A1'in ihc
sample increases, the possibility of reliably excluding possible cau.s<\s
increases with il, and greater parsimony of explanation is also achicv.iblr.
Even more fundamentally, the statistical and comparative methods are fundamentally
different orientations to social science research (Frcndcris, 1983; Ragin,
1987). The comparative depends upon the careful a firiori selection of
[lie cases. If we return to the fundamental litany of research design
- maximise experimental variance, minimise error variance and control
extraneous variance - then tlic comparative method depends upon the initial
selection of cases to aciiicve those goals of research design. The cases
must be selected in such a way as to control known sources of extraneous
variance. This research method is therefore often inaccurate and incomplete;
if we knew all the sources of extraneous variance in advance tlien there
might be tittle need to do tlie research. Still, if care is taken in the
selection of cases then there is some greater hope of achieving control.
Statistical analysis, on the other hand, generally is wilting and even
anxious to include all available cases, and (lien use a variety ofex-posi
statistical techniques to attempt to determine tlic impact of extraneous
variables. This is a slight overstatement, but not by much. The canons
of statistical methods seem to always push for more cases, given that
tlicy can be seen as part of tlic same population of cases. Statistical
iccliniques have tlie virtue of being more independent of the selection
of the cases, but even so, if tlie appropriate data arc not collected
then there is no way in which to determine the sources of tlie extraneous
variance. Further, if some countries are systematically excluded because
of poor quality of data then another source of extraneous variance may
be introduced. Looking at tlie outliers in a regression analysis may enable
the identification of the geographical source of the variance, but often
not the variables producing the deviations from the neatly devised theory
of the researcher. This potential problem also points to (lie need for
substantial descriptive knowledge of llic cases being studied, even in
a statistical analysis. Without dial sort of substantive knowledge, identifying
why country X is an outlier, and choosing variables to use for a subsequent
test for the fundamental source of extraneous variance, can become little
more than an academic lishing expedition. Of course, no researcher can
know about all relevant countries, but as comparativists we abandon our
descriptive studies in favour of being 'high lecli' at great peril to
any real understanding of countries and political dynamics.
Both tlic comparative method and tlie statistical method depend upon tlic
ability to identify, ex ante, (lie potential sources of extraneous variance
in tlie research question. If a comparative analysis does not identify
tliose potential sources of confounding variance when making
the initial selection ofihe
cases the researcher will not likely be able to recover and create adequate
research later. Tlic consequences will be results lhai are confounded
by the unmeasured characteristics of the cases selected for the 'sample'.
Th? statistical method, likewise, may have its results confounded in an
o^^ irretrievable manner by ihe failure to identify, and measure, itnport.uit
correlalcd variables- A statistical researcher does have clx-slight advantage
of being able to rerun the analysis with the previously unidentified variable
included. The identification of [lie crucial ^measured factors may, however,
require waiting for new thcoreticil development and a substantial passage
of time. This delay may noibe as serious in the social sciences as it
might be in medicine or the na^al sciences, but it may still lead
researchers and even practitioners down false and/or unproductive lines
of enquiry.
In addition to the theoretical considerations that may argue for a smaller
and more focused analysis, t^ere are always practical research questions.
Unless there is aggregate yata - for example, budgets or voting data -
readily at hand, Klanvely few social scientists can afford to collect
extensive data over 3 large range of political systems. There have been
a number of stu^s of this type historically, but fewer have been'completed
recently. There are some exceptions to that generalisation, for example,
die huge and detailed data-sets compiled by Barry Amcs, first on Laiin
American budgets (1987) and then on Brazilian public expenditure (1996),
and the massive study of public opinion in Western cou^ies titled Beliefs
in Government (Kaase and Newton, 1995), but they have been few. Further,
even if extensive data collection in a nuinbtr of countries is feasible
economically, it may still not be the best usc^f scarce resources "
temporal as well as financial - for the rcsearcf^. The fundamental questions
must always be, What does a researcher gain from adding another doy.en
cases, or even adding a single case, lo the sample? And, Does that case
add anything to (lie argument and the ability to reject
theoretical propositions, or is it simpiy just another data point for
the analysis?
This chapter addresses the import 'Goldilocks question' of just how many
cases arc sufncient for endive comparative analysis. The previous chapter
addressed one asp^ of the case selection question by asking whether (he
cases selected should be most similar or most different, and further,
on what du^^io,.^ i^y .should be most similar or diHerent. This chapter
will extend that set of questions by looking at the quantitative dmiensio,^
j, iyi;iy appear excessive to
The lumber of Cases 61
have iwo chapters concerned with die selection of cases, but in some
ways case selection is (lie crucial question in the design of comparative
research design - first, how many cases, and then which ones? It may be
that more cases arc, on average, better, but more cases may merely be
a compensatory mechanism for tlie poor selection of a smaller number of
cases. As willi so many things in this field, there is no single correct
answer as to wliat is the right number of cases - the only real answer
is that 'It depends.'
Strategies with Different Numbers of Cases
The remainder of this cliaptcr is an examination of strategies available
to the researcher wlio is faced with the possibility of using different
numbers of cases. This choice could also be expressed in oilier ways,
given that we have been arguing throughout that theoretical questions
should guide the selection of cases, including both the number and the
particular cases. The theoretical issues can lead to a choice of the appropriate
number of cases, but there remain questions oljust how those cases are
to he investigated, and what can be done to maximise the tlieorctical
utility of the research. Further, there are some instances in which there
is simply a possibility of only using a very small number of cases. If,
for example, a scholar wants to examine successful democratic governments
in Africa, the range of possibilities is unfortunately very limited at
present (Picard and Oiirrity, 1994; Molutsi and Holm, 1990).
One Case
It is indeed possible to do comparative research of a sort with a single
case. This statement appears to be almost a contradiction in terms, but
the seemingly impossible can be done successfully in several ways, and
for several reasons. The reasons for choosing this particular route for
comparative research are more important tlian llie particular manner in
which the research will be done, although the two issues niay lie closely
related in practice. We should also be cognizant that another of the social
sciences, anthropology, lias developed primarily through [lie accretion
of case after individual case, each one developed and considered individually.
Although perhaps less directly concerned with the process of comparison
llian either political science or sociology (liul -see Naroll and Coln-ii,
l']73), anthropology is
certainly a comparative
social science and has developed most of its theory through (lie accumulation
of case evidence. Although often seen as alhcorctical ;md purely descriptive,
if it is well done then case-analysis can, paradoxically be one of the
more theory-driven forms of
comparative analysis.
One primary reason for attempting to do comparative research
with only a single case is 10 utilise a very particular case to characterise
a phenomenon lliat appears to be especially apparent in tliat one case.
The single case therefore becomes a pre-lheorcticai exercise, leading,
il is hoped, to a general si.iteiiifin about (lie phenomenon; tins strategy
is what Richard Rose (1991: 454) calls ihe 'extrovertcd case-study'. The
researcher lias identified, or believes that lie or slie has identified,
an important exception to the prevailing theory, or a case which demonstrates
a phenomenon tliat previously had been excluded from the literature. Tocqucvillc
was following this strategy quite explicitly when lie did his research
on the USA. Tocqueviile states: 'It is not, then, merely to satisfy a
curiosity, however legitimate that 1 have examined America ... I confess
than in America I saw more tliat America; I sought tliere the image of
democracy itself, in order to learn wtial we have to fear or hope from
its progress' (1946: 14).
TIie purpose of the extroverted case-study then becomes to explore futly
this one case with the existing theory in mind, with the expectation of
elaborating or expanding tliat body of theory with the resulting data.
Take, for example, Stein Rokkan's investigations of corporate pluralism
in Norway (1966; sec Key Text 3.1 and also Olscn. 1987). These studies
were being developed at. or even before, llic time in which there was
a great deal ofthcorclk-al attention directed at the development of corporatism
as a more general concept describing state-society interactions (Schmitter,
1974; 1989; VViarda, 1997). This work can also be seen as a reaction to
the prevailing pluralist model of American political science. Indeed,
one of Rok-k;m's most important arliclcs on tin; subject appears in a
volume edited by Roben Dalil (1966), the leading figure in pluralist political
science. The more genera! corporalist model implies a formalised and structured
relationship between interest groups and the stale. The standard corporan.st
model further implies a singular rclation.sllip between (lie state and
each interest group, with government choosing a single group (or peak
association) to represent each sector of the economy.
Key Text 3.1 Stein Rokkan and the Study of Centre-Periphery Politics
The Norwegian Stein Rokkan was one of the founders of contemporary comparative
politics. He made a number of significant contributions lo the discipline,
including developing the idea of 'corporate pluralism' (1966) that bridged
the gap between corporatism and pluralism (see Key Text 2.2). He also
developed some of the earliest concepts of data archives for the social
sciences, and was instrumental in compiling such an archive at the University
of Bergen.
One o( the most important of his contributions was the construction of
a model derived from historical sociology to explain the development of
centre-periphery politics in Western Europe (and see Lipset, 1967). This
model argued that contemporary structures of political cleavage in Europe
developed through successive overlays of three sources of cleavage- The
first was the national revolution, as feudal decentralisation is threatened
by centralising elites. The second was the Protestant reformation, and
the continuing divisions between religions in countries such as Switzerland,
Germany and especially Northern Ireland. Finally there was the industrial
revolution with the separation between labour and capital that has been
the dominant source of political conflict in European countries (or most
of the twentieth century-Despite the significance of economic cleavages
in Europe Rokkan pointed to continuing importance of the other sources
of cleavage. This was to some extent a function of his observations of
politics in Norway. with the political importance of peripheral groups
such as fishermen and farmers. Further, he could not help but notice the
importance of religious cleavages in other smalt democracies in Europe.
Rokkan's background as a sociologist led him to emphasise these social
bases of politics.
It might be argued thai Rokkan's theories were to some extent an attempt
to generalise from Norway to the rest of Europe. Further, political. developments
since the time of his book have pointed to some of the ways in which economic
cleavages can be overcome in the management of stales. Still, Rokkan's
contributions remain one of the foundations for the understanding of contemporary
politics in Western Europe, and to some extent in the rest of the world.
The model developed hv Rokkan identifies a somewhat different relationship
between state and society, in which llic state consults a wide variety
of societal actors over policies, and expects them to bargain among themselves,
as well as with the public sector actor or actors involved, until a consciTiiis
is attained. This concept is similar to corporatism, in thai il requires
[lie state to accord a legitimate
status lo interest groups, but diners in the number of groups involved
and tlieir relationship among themselves- This fundamental observation
and conceptualisation of' the nature of government in Norway was then
expanded into a much longer treatment (Otsen, 1987). Corporate pluralism
has also been the subject of critiques arguing that even if the model
did succeed in building a theoretical basis for understanding politics
in Norway, that model coutd not be extended to analyse other systems (Hcisler,
1979).
Tile second reason for using a single case is that tlie case may be the
hardest one, so that if the theory appears to work in tills setting it
should work in all otiiers. This strategy can be seen at work in many
coniemporary analyses of Japan. The mystique of Japan as a distinctive
political system lias grown to the point that it is being used as the
most difncult test case for a variety of political theories, with the
assumption that they will not work in that setting- For example, McCubbins
and Noble (1995) undertook a study ofbudget-ary behaviour in Japan, testing
the common assumption that this political system was dominated by bureaucracy
and that there would be minimal political influence. Even in tliiscase,
however, they found tliat there was a pronounced influence of political
parties over budgetary choices, thus putting the usual characterisation
of Japanese politics into question.
Finally, there is tlie 'Mount Everest' reason for doing studies with just
a single case - it is there. That having been said, a careful and insightful
examination of a single case, even one already very well known, may be
proto-llieoretical. For example, it is not at all clear that Samuel Beer
set out to develop general social science theory when lie wrote his second
major study of British politics - Britain Against Itself (1982). That
may not have been the intention, but his analysis of the blockages produced
by the seemingly excessively pluralistic nature of British politics parallels
closely Mancur Olson's more theoretical analysis of the Rise and Decline
of ration', (1982), as well as some discussions of hyper-pluralism in
the United States (see Peters, 1995b). In short, although it is not the
usually recommended strategy for comparative research, an insightful analysis
of a single country can produce large dividends, even if the benefits
are unanticipated, or not even recognised, at the time.
Whether or nol ii is the bcsl strategy for llicory development for corn
piir;t live politics, tlie single country study is likely to remain a
cenlr;il feature of comparative politics fur a very long lime. Language
and geographical restrictions are likely to persuade individual scho-
The Number of Cases 65
lars to focus ihcir attention on single cases, or at most a limited range
of cases. Furthermore, we might well think lhat, given llie continuing
dearth of resources available for extensive multi-country research, there
will be more rather than less attention paid to monographic studies of
single countries. Tlie real task for comparativists, consequently, becomes
to develop those individual case studies (sec below, pp. 139-54) so thai
they have as much theoretical content as possible, while still retaining
their descriptive richness. Tlie second part of that challenge is to work
to compile the individual case studies into theoretically meaningful groupings
and to attempt to draw some conclusions from them. (
Two, or a Few, Cases
We will now double the number of instances to be considered and think
about llie logic of doing comparative analysis with only two cases. Using
two cases appears to be a more sensible way to do comparative politics
- then- is now something to compare. With properly selected 'focused comparisons'
(Hague et al., 1992: 39-40) some comparative understanding can be generated
from two cases. Despite that, this strategy is fraught with many of the
same dangers encountered with single country analyses. Therefore, using
two cases requires the same caution in design and execution as do single
country studies. Indeed, the two country comparison may be even more dangerous,
simply because it appears that a genuine comparison is being (lone, and
then-lore llie logic of analysis must .surely be acceptable. Thai may
1101 be llie situation, however, and absence of care in ihe selection
and analysis of the two cases may produce misleading results (De Winter
el a!., 1996).
I'he problem with tilis mode ofanalysis is not smalt N per se, hut tlie
mismatch between a rather small number of cases and a large number of
variables (Lijphart, 1971: 686). When a country is selected for a comparative
analysis, it brings with it a large bundle of variables. Tlie history,
culture, economy and society all come along with the particular political
dimensions in which the researcher is interested primarily. Therefore,
lliere arc a liuge number of sources of extraneous variance, but only
a few cases in which to aiicmpt to discover [he miinin;r in which all
those variables operate. In statistical language, the dependent variable
is over-determined, with too many possible fxpt.in.uions, so llial no
real choice can lie made (Lupe-/.,
1993), Tlic researcher
may focus on a few variables he or slie thinks are particularly importani,
but unless the cases are selected carefully to falsify a hypothesis, any
number of explanations will still be
acccptiiblc.
We must return to some of llie logic ofTeune and Przcworski in thinking
aboul this form of comparison. Which two cases, and wliy? -these are the
doniin.inl questions. In some instances, the logic of most dinercnt systems
will direct us lo alicrnpt lo find two systems that are as different as
possible on a number of variables, thereby maximising the variance with
which to explore (lie differences among countries, The (rouble may be
that with only the two cases llicre are so many variables that il is difficult
to determine how to explain the diuer-ences. As Sartori (1991: 244) argues,
(lie fundamental aspect of comparison is control, and the capacity to
remove as many alternative explanations as possible from tlie 'equation',
i.e. tlic implicit regression equation explaining the dependent variable.
Any two cases are in some ways comparable. Any two countries (or provinces,
or whatever) that we select for analysis will have some features that
arc comparable - they all have politics, they all have political institutions
of some sort, they all make policies of some sort -but the question is
whether the comparison is the best one for the particular item of research
under consideration. The problem for comparison is tliat cases often are
selected for reasons tliat are not scientific, but rather are based on
familiarity, funding, convenience, or even commercial considerations (most
two coumry studies seek to include the USA, given the size of the US market
for books). If there is to be only a pair of cases then [liose cases should
be as similar as possible on as many variables as possible, but (lie dominance
of familiarity and convenience usually prevent dial from occurring.
One way to understand the logic of comparison for (wo cases is tliat this
is itihcrcnttv a ^must different systems' design, and llie two cases must
be conceptualised as almost totally different. Given tliaE a researcher
lias no oilier information available when looking at the two cases, he
01" she must assume that tlic cases are substantially different.
Tlic logic of comparative analysis, therefore, becomes to attempt to identify
similarities between the cases (sec Chapter 2) that can then be used to
explore common themes, rather than looking at their all-too-apparcm dilierences.
The dangers of binary research in compannive politics can be illustrated
by Seymour Martin Lipsci's (1994) interesting, but
basically Hawed, example of (lie comparison of the United States and Japan
as two 'exceptional' political economics. Tlic model lie uses is, in effect,
a version of tlic Teunc and Prxewurski 'most diiTcrcnt system design'.
The question Lipsct poses is, VVIi.il, if anything, can explain the economic
success of these two very disparaic conn tries? Lipsel examines both on
a variety of variables, and finds llicv differ fund amen tally on virtually
all of tbeni, vet have both been vcrv successful. Fukuyama (1995) provides
a more extensive examination of tlic economic success of systems, using
some of tlic same variables used by Lipsel, but focusing on llie existence
of trust in tlic c<Jmmcr-cial system rather than on the social and
political variables that arc more central to Lipset's analysis. In particular,
he points to the differences in the roots of economic success in Japan
and Korea.
The research design flaw here is selecting cases on tlic dependent variable,
combined with the extreme differences in llie systems, with the consequence
that every difference appears equally invalid as an explanation. If, however,
the cases had been selected with some variance on the dependent variable
then liicre might have been some belter chance of generating more useful
results. As we pointed out above, however, tlicre are disputes over the
appropriatcness of selecting cases on the dependent variable, willi qualitative
researchers arguing tliat tills is essential, and quantitative researchers
arguing
tliat it invalidates most findings.
The problems with binary comparison can be reduced by focusing on a single
institution, policy or process. For example, a researcher may want 10
compare the Britisli prime minister and the German chancellor (Helms,
1996). Given that these two oHiccs perform many of the same (unctions
in government, some of tlie potential for extraneous variance is reduced-
Further, because for most of recent history both of thesf ofnccs have
been dominated by a single, conservative party, llie extraneous variance
may be reduced further. These factors are still however no a guarantee
that the research will make the contribution a scholar might like. Why
these two countries? Is there anylliing theoretically interesting about
(tie pairing of (wo cases, or have they been selected merely for convenience:*
Roy Pierce (1995), for example, focuses on (lie presidency in France ;ind
the USA as tlie two major examples of tills ollicc in developed democracies.
as a me;ms ol understanding how electing llie chief executive ;tll'c(
is voting, l)ui is slill able to make inieresUng tlieorcticsil arguments
abuul tin- dillerences between tlie two sy.slems.
\ Smalt M Research in General
The same logic of comparability discussed above is applicable if researchers
extend the analysis to a number of cases greater limn two, albeit still
a small number. If 'the units arc very disparate, then (lie comparison
in;iy have to be at such a high level of generality that the results are
not very meaningful or useful for llieory development. Of course, (lie
strategy of most different systems designs (pp. 37^H) argues for a great
deal of diversity of cases, but that argument is more readily applicable
to the tradition of statistical analysis rather than to the more constrained
number of cases used for comparative analysis. With only a limited number
of cases, the most different systems design usually generates comparisons
of the type common when systems analysis and structural-functionalism
were the dominant approaches to the comparative politics (Wiseman, 1966;
Almond and Coleman, 1960; Jones, 1967). These functionalist studies managed
to compare extremely disparate systems, but also managed to do so in such
an equivocal manner that the results actually could say very little about
what happened in those varied systems. In such a research outcome, there
are simply too many factors that differentiate the systems to be
able to say what the more important independent variable or variables
might be.
For a strictly comparative analysis, careful selection of cases is essential-
Even then, it is not clear how much leverage [lie researcher can get for
isolating cause and effect in the analysis. Lieberson (1991) argues that,
for small J\' studies to be able to make causal claims, a number ol stringcnl
conditions must be met, most of which simply do not exist in the social
sciences. For example, he argues (pp. 309-12) that (he cause must be deterministic
rather than probabilistic. That is, for a study with only a Few cases,
to say tliat A' causes ?, the linkage must be necessary and sufficient,
and }" will not occur without X. Very few relationships in political
science would meet that criterion, and even more 'advanced' social sciences
depend heavily upon the cetens fxinbus disclaimer. Most of the relationships
with whidi we work are probabilistic, and with a small sample it is not
possible to rule out a chance finding.
Furllier, Lieberson also argues that for small-A" studies lo demonstrate
causaiion, llierc c;in only tie a single cause, and no interaction among
independent variables. This statement assumes that [he mcihods for determining
causation developed by Mill (see above, pp. 29 30) are being applied,
but llie ;irgumcni would liold even
without that assumption. With a small number of cases there are rarely
sufficient cases to have all possible combinations of several independent
variables, so that the research runs the risk of falsely imputing causation
when changes in the dependent variable may well be a function of interactions
among variables. Given the complexity of most social processes, those
interactions are almost inevitable. Lieberson also argues that for these
mctliods to work there must be no measurement errors. In other words,
for small-JV" studies to produce valid causal results, [here is no
room for either extraneous or error variance, both of which are almost
unavoidable in tlic social sciences.
There have been several critiques of the rather rigid claims made by Lieberson.
For example, Savolainen (1994) argues that Lieberson has confused comparative
methods used to prove causal assumptions with those intended to eliminate
possible causes. Establishing that X causes }" is difficult; it is
much easier to argue that X does not cause Y (see also Skocpol, 1984).
Eliminating possible causes is in itself a valuable service to comparative
politics, however, given the number of contending theories of cause available.
Similarly, Savolainen argues that probabilistic explanations can be used
with small-jV studies, assuming that the researcher is conservative in
the claims made for the findings- The same would be true for the presence
of interaction effects in the explanation. Even if we grant [he critics
their points, imputing causality with a small A is difficult, however
carefully [he cases are selected in advance. Tlie complexity of the social
and political world is such that eliminating other possible causes is
difficult. The Boolean approach (sec below, pp. 162-73) begins to address
these problems, lint the findings then still depend upon the nature and
size of the sample used.
Tims, very much like the single country case studies, small JV studies
appear more useful for generating hypotheses than for testing them. The
only viable exception to tilis statement appears to be in carefully crafied,
most similar systems designs, wlien [lie similarity of the countries selected
can be demonstrated very clearly and convincingly. Without that knowledge
of die countries, llie capacity of small-A studies for making statements
about causation appears a very limited indeed. Even in 'comparable' cases,
however, Prxeworski and Tcune (1970) would argue lhal llie dependent variable
would most likely Ix- ovcr-dclcrmim'd because of the number ol" rival
hypotheses [lial could nol be rejected on (lie basis of any small mimbcr
of cases.
Reducing Variables
An alternative method for coping with (lie small-X, many variables* problem
is to reduce tlic number of variables included in the ;in;itysis (Collier,
1993). This reduction can be done ill one of two manners. Tlic first way
is to employ a more elegant theory, with lower presumed explanatory variables
tti;>ii oilier contending approaches-The argument here is that if the
researcher goes into the lield with a loosely constructed theory, then
the data from ;t small number of cases can support any number of possible
theoretical explanations. If, on the other hand, the researcher works
witli a clearer and more parsimonious theorv, then that theory can be
tested with the limited number of cases. This is especially true if (lie
cases are selected to make Falsification more likely, for example, if
the researcher looks (or llie hardest cases. Collier, for instance, uses
(he example of rational choice theory, with a very few variables and relatively
simple hypotheses.
The danger of the variable reduction strategy is that the researcher imposes
a premature closure on the possible explanations for the phenomenon. Just
because a single hypothesis can be tested witli a limited number of cases
does not mean that there are not otiicr possible explanations. Further,
a strong theory such as rational choice may contain a number of untested
assumptions that arc, in many ways, also hypotheses. Failure to test tlic
assumptions empirically may lead (tie researcher to accept twhat is in
essence a spurious relationship. For example, in the case used an example
by Collier (Geddes, 1991), a game theoretic analysis was applied to explain
administrative reforms in Latin America. This appears to be an elegant
treatment of those reforms witli few variables, bin also appears lo avoid
a number of factors lliai scholars coming from other traditions might
consider crucial components of any investigation of administrative reforms.
Most scholars of public administration, for CNample, might consider the
exclusive focus on legislative decision-making for studying administration
risible.
The alternative means of reducing the number of variables is to perform
that task empirically rather tlian theoretically. Many "fthe variables
that researchers use in comparative research are closely allied to oilier
variables, so that not only arc tliere statistical problems In using them
together, l)ul lliere may also lie conceptual and tlicorrlical confusion.
The statistical problem i** iniilluolliiic.irny. or the confounding cdi-cis
of highly iiitemirrelaird independent vari-
ables in regrcsaipn equations. This tends both to reduce the reliability
oCthe estimates and also to reverse signs for the variables that are not
the mostly strongly correlated with the dependent variable. The
theoretical problems arc more dillicult to solve.
A variety "1 statistical techniques such as factor analysis can be
used to combine possible independent variables, and, even without these
statistical techniques, indexes can be created to combine variables that
are conceptually related. These aggregation strategies do not permit tlic
researcher to answer questions about tlic relative cnccts of (lie ctoscly
related components of llicsc 'syndromes' of variables, but they do permit
tlic testing of theories in rather broad ways, with the possibility of
later refinements of the analysis witli the
individual variables.
For example, a number of micro-level comparative research projects have
used various data aggregation techniques to produce more parsimonious
explanations. The typical method is to collapse a number of separate independent
variables into composite explanations using methods such as factor analysis
(Lewis-Beck, 1994). These aggregation techniques select a number of candidate
independent variables, and identify underlying dimensions that link those
variables. Thus, one variable can be used to represent a number. Further,
these techniques produce dimensions lliat are themselves uncorre-laled,
so eliminating any further statistical difficulties.
One potential pitfall in doing small-./V research is to select cases on
ihe basis of values of the dependent variable rather than on those of
the independent variables. This is an all too human tendency. Researchers
identify several cases that are interesting because of their success or
failure, and then proceed to study them alone, for example, cases of successful
democratisauon. Tlic problem is that even if there arc independent variables
that arc highly correlated with those dependent variables, there is no
real variance to explain. How do we know tliat the same pattern would
not emerge for cases tliat were unsuccessful, or that did not attempt
to democratise at all?
Research based on a common value of the dependent 'variable' can eHcctively
eliminate viable hypotheses - if none of llie cases in the sample exhibit
a certain trait [lien it is rallicr certain that this variable is not
a concomitant of thai dependent variable. This statement again ]X)inis
to tlic essential role of theory in tlic selection and analysis of comparative
cases, anci in guiding tlic collection of data. If we do not initiate
research with some ihcorclic.il i^iifl.mcc then we will not know winch
hypolhe.sfs in consider aiii-l which to
reject. Further, withoul (lie theory we will not even recognise what data
to look for lo be able to reject relevant hypotheses. There is no shortage
of interesting research questions, but the real consideration is how to
relate those specific questions to more genera! theories.
Courtier/actuals
If nature supplies the researcher with too few usable cases, or if funding
agencies supply the researcher with too tittle money to investigate .ill
the potentially usable cases, there may still be a way to increase the
A of the study, and thus satisfy the usual scientific canons. One of the
more interesting ways is to develop counterfactual arguments that are,
in effect, additional cases. Max Weber (1949; see also Polsby, 1982) actually
discussed this as one strategy for dealing willi the problem of too few
relevant cases for many issues in social research. Fearon (1990) also
has argued for using this methodology for comparative research, and its
logic lias been examined relatively thoroughly in several recent methodological
studies (Tetlock and Belkin, 1996; Kahneman, 1995). Fearon argues that
this style of analysis is at least a partial antidote to the research
problem already mentioned several times - having a collection of cases
without variance on the dependent variable. With counterfactuals, researchers
can always generate such variance, even if it does not occur in the 'real'
world.
Arguably, any small-A" study is based on counterfactual arguments.
If causation is imputed to a cause in the case-study, or in a study with
only a few cases, tlie assumption is that were there cases lliat had the
other values on a variable present then the outcomes would have been different.
A case researcher tells a causal story, and must attempt thereby to dismiss
other causes and to show that liad things bcc-n different then the outcome
would have been different. In [lie example tfiai Fcaron uses, Alfred Stepan
(1988) argues why a military coup in Brazil was not inevitable but, given
particular events and decisions by [he government, did eventually occur.
In the criteria for counterfaclual arguments advanced by Tetlock and Belkin
(1996:
19-22), one of tlic guiding principles is that arguments that require
fewer deviations from the 'real world' and the specification of fewer
steps are more realistic than more elaborate departures from observations.
Przcworski (1995) argues ihilt counierlactual argument is essential
to comparative political aii.tly.sis. The limited number of cases *
The Number of Cases 73
available for any interesting research questions means that the researcher
inevitably must use counterfactuats, especially to fill in the cases that
are more extreme. Nature rarely provides the researcher with llie full
range of cases that might he needed to test some hypotheses, so that tlie
only option is to ask 'What if . . .?', and attempt to fill in [hose missing
cases in as unbiased a manner as possible. These 'thought experiments'
are not substitutes for field research, in Przeworski's view, but they
are crucial (or developing theories after the range of real, available
cases has been exhausted.
Analogies
It is also easy to move from 'what if to 'as iP. That is, analogies can
be an important source of comparative case material and counterfactuals.
The use of analogies lias been developed more in the study of international
relations than in comparative politics, but the logical basis of their
use is very much llie same in the two fields. Khong (1992; see also Houghton,
1996), for example, examined the extent to which various analogies were
used tojustify decisions in foreign policy debates, and tlic extent to
which the analogies selected could explain decisions. One way to employ
analogical reasoning for comparative politics is to think about time more
creatively and to utilise historical cases that are functional analogies
of more contemporary events (see Bartolini, 1993: 139ff.). This style
of analysis may be especially important for studying political development
as countries today go through processes that are not dissimilar from processes
that others went through decades ago.
For example, Halpcrin (1997) discusses asynchronic comparisons of weak
slates in contemporary mid-East and medieval Europe. The logic of this
analysis is clear. In cadi case, there were states thai, by contfinporary
standards, were wcak'and liad diulcully in enforcing their authority throughout
their territory and dominated by informal politics. By using (lie two
sets of countries, the researcher had increased her A' significantly.
If tlic processes in tlie two sets of systems appear similar, despite
their separation in time and space, then there is greater hope that a
general explanation of development can resull from the analysis. This
strategy of asynchronous comparison appears potentially extremely fruitful,
but also raises questions, iriosi notably just what sort of evidence would
be needed to c-siablisb tlic functional equivalence ofdillercnt eras and
political CVCIHS- "I'lierc appear ID be no clear i-uk"i yet
for this strategy, the tcsl being what
survives when it is exposed to ttif scrutiny of llic remainder of the
scholarly community.
In summary, the countcrfactu.il siraicgy of coping with confirmation of
hypotheses is the .mtithcsi'i of the statistical method (see below), Fcaron
argues that this strategy is to be preferred in many ways, because i[
docs not give ihc researcher a false sense of security. Tlic largc-A slucly
assumes iliiit all the threats to validity arc taken care of by the size
of the sample, but in social science tliat may not be the case. Thai assumption
mav be especially problematic in comparative politics, given the lack
of independence of tlif units of measurement and the problems of diffusion
and compensatory cnects in the dependent variable. There may well be a
number of systematic retail onships and a systematic dispersion of political
practice, which make the independence of observations very suspect.
Comparisons of'Natural Groups'
Another strategy for comparison is to begin with relatively small groups
for comparison, with those groups being defined as composed of relatively
homogeneous 'families ol nations' (Castles, 1993; Esping-Andcrson, 1990
- see Key Text 10.1). For example, there lias been a great deal of interesting
and useful research done comparing the Scandinavian countries (Elder el
a/., 1988; Lundquist and Stahlberg, 1983; Laegreid and Pedcrsen, 1994).
These countries (three to live, depending upon how one counts) have a
number of political, economic and social characteristics in common. Further,
they countries arc in almost constant contaci witli one another on issues
of policy and administration, so tin" researcher can determine relatively
clearly how ideas liliered through national political settings can be
implemented. Thus, while this research may liave tlie advantage of liolding
constant a number of socio-economic and even political factors, it has
(lie disadvantage of a severe case of Gallon's problem, that is, sorting
oui tlic eni'cls ofditl'usion from (lie effects of indigenous developments
(see pp. 41-3).
Anotlicr of these natural groups of countries for comparative analysis
is the 'Old Commonwealth', or the Westminster democracies, meaning those
industrialised democracies derivative of the-British tradition, including
Britain, ;ind sometimes tin' United .Slates to make the 'Angle American
democr.icies' (Lijphari, 1984; 1990;
we also Aucoin/ 199.'"). ll is a.ssunn'd that llii-s ci"nnmKi
heritage
makes these countries readily 'comparable'. Ycl, if we look at some relevant
data on this group of countries (Tables 2.1 and 2.2), we find that in
many ways they are quilc i i etc rogen ecus socially, economically and
politically. Therefore, any comparison among these countries lends to
be arguing implicitly that political heritage and political culture are
more important features than arc the social and economic fabric of a country.
Further, it lends to he arguing that for comparison the constitutional
heritage is more significant than even some important structural features
of constitutions such as federalism.
That argument would be particularly interesting were llic comparison extended
to cover the members of the 'New Coinmomnwcalth' (see Hraibanti, 1966).
In these countries, there was also a long period of British rule, and
a direct implementation of British governmental institutions before and
after independence. Thai connection was substantially longer in some cases
(India, Pakistan) than it was in tlic United Slates, and more recent than
in most of (he Old Commonwealth countries. The failure to include llicse
countries in most comparative analyses looking at tlic impact oF the Westminster
model therefore seems to argue that cultural and social factors arc more
important than constitutional inheritance. So lliese two implicit arguments
are inconsistent, but the contradiction does point to the way in which
conventions and indeed prejudices can creep into comparative analysis-
Allhough the comparison of these families of nations may face some apparent
or not so apparent problems, there are also some potential benefits. Joseph
LaPaIombara (1968), for example, argued in favour of developing 'middle-range
theories' of comparative politics, rather than tlie grand theories such
as .systems analysis or siructural-lunclionalisin that tended to dominate
comparative analysis when LaPaIombara was most concerned with theory development.
These middle-range theories would address the politics of a geographical
region, or perhaps a government institution, ralhcr than providing the
global explanations sometimes thought necessary to qualify as a theory.
By using this mid-range strategy, LaPaIombara argued, the discipline could
develop building blocks for tlic more comprehensive theories while at
the same lime providing more useful and convincing explanations for politics
within (hose more limited domains. Roy Macricii.s (1986; 22) supported
lli.it view, and argued that 'clustering [geographically], a.s opposed
to (lie search for grand ihcory, has g^vcn to tlie field a new vitalilv,
and may ultimately pave tlie way to llic development of some unifying
models and priorities'. We will point out (sec Chapter 5) some problems
in defining the clusters, but this docs appear to be tlie current direction
ol' theory development.
Sometimes, without explicitly acknowledging the strategy, comparative
politics has been proceeding in this direction. Any number of books and
articles have been published ofTering theories for some parts of the world,
or for particular institutions. Take for example, Hyden and Bratton's
(1992) work on governance in Africa. The theoretical chapters of this
book address primarily questions of African politics and the breakdown
of governance in that continent, but the general considerations about
governance raised there could be applied anywhere. Thus, by developing
a theory about governance in Africa, the authors have actually made a
good beginning on a
more genera) theory about governing, and especially about governing in
less-developed political systems.
One subsidiary question that arises is how to determine the appropriate
focus for these middle-range theories. As pointed out above, the natural
groups that we might begin with do not always have the degree of homogeneity
assumed. Although Hyden and Rratton's analysis of African states is interesting,
the degree of variance in political form, political tradition, social
structure and the like among those nearly fifty sub-Saharan states may
make attempting to piece together one approach to fit them virtually liopeiess.
Indeed, such an approach may have to be so general that it would become
capable of being applied almost anywhere, regardless of (lie geograpliical
location. On the oilier hand, schemes that have been put forth as theories
of European politics tend to do belter
at describing and explaining a part of Europe rather llian the diverse
politics in that continent.
One approacli to die problem of identifying an appropriate focus for a
middle-range theory is to make that determination empirically. As discussed
above, some approaches to comparative politics (perhaps more appropriately
referred to as statistical analyses of sets of countries) have employed
factor analysis and other dimensional techniques to identify meaningful
clusters of countries (Rummcl, 1972; Casiies and Mitchell, 1993; Peters,
1993). This body of research has tlie virtue of being able to demonstrate
that (lie countries in question do constilute a statistically meaningful
group, but is potentially suspect on other ground;). In particular, the
clusters of countries tlial emerge arc a function ofihc data ilia! are
put into the clustering programme. Iftin-'re was inadfqu.ne theory guiding
selection of the concepts, or even inadequate measurement of (lie concepts
used, then
the results wilt not necessarily
reflect the groupings of countries that actually exist.
Further, having grouped countries in this way, it may be difficult to
employ the groupings to test comparative propositions anyhow other than
through tlie most similar systems design. If anything further is done
witli tlie data within a grouping, the results may be almost predetermined.
As Teune (1990: 43-4) points out, 'Selecting a type to reduce variance
made it impossible to find out what diiTerence the category of inclusion
made. Thus, if being wealthy and democratic has consequences, it is necessary
to look al some cases of being non-democratic and poor.' What therefore
may be most interesting approach to comparison is a two-stage process.
At the first stage, some dimensional technique can be used to identify
groups of countries, while in the second stage comparisons are made across
the groups, rallier than within tlicm. This strategy does not meet all
the stipulations of LaPalombara's mid-range theories, but it does enable
the creation of theories that dinerenliate among groups of
cases.
Global Comparisons
Finally, we come to what might be considered to be the dream of the statistical
version of comparative analysis - global comparisons of virtually all
political systems. We have noted already that there have been several
major research studies of this type, attempting to develop measures and
concepts that would travel, and then to classify the countries of tlie
world on those variables. In many ways, these global studies represented
tlie early, euphoric days of the behavioural revolution in political science
theory and tlie computer revolution in the managing ofdata. The assumption
was that if we could just get enough data from enough political systems
then we could answer all the relevant questions of political science.
fhe euphoria concerning this style of research quickly wore ofT, and lias
been replaced witli some healthy scepticism. First, as we point out below
in Chapter 4 on measurement, developing measures that can travel from
(lie United Kingdom to South Korea to Vanutu often implies that they become
so general that they mean very little, and actually are able to contribute
very little in (lie development of social science theory. For example,
jean Blondel's study (1990) of political leaders and institutions around
the world lias been criticised for iis generality and its failure to make
useful interpretative stale-
ments (sec also Key Text
G.2). For Blonciel, there were so Few indicators that could 'travel' across
ihis range of systems that [here was rclalivcly little that could be .said
to compare the systems, despite the importance simply or amassing comparative
information on political executives that was not previously available.
Similarly, the idea of the country as ;i unit of analysis and as a meaningful
entity tends to become lost in studies of this nature. A country simply
becomes a data point, and (lie possibilities of understanding why and
how Vanutu or Bulgaria functions the way it does may be sacrificed on
a statistical attar. This research may be comprehensive statistical analysis,
but often it is not really comparative politics. Comparative politics
is constantly forced to trade off the particular for the general. Individual
scholars must decide if they are experts in one country, or even one region
of one country, or in one region of the world, or in comparative political
theory that spans the globe - it is dinicult to do all those things- Also,
we must decide whether the real questions to be answered are about real
countries or about distributions of data.
Summary and Conclusion
Comparative politics requires the selection of particular cases in the
pursuit of genera! theory. At the extreme, a wide range of cases should
be used, but that is a luxury available only for aggregate data analysis
or a few extremely well-funded studies. Further, involving a large number
of cases may mean that cadi one is not addressed carefully. If there is
to be any primary data collection tlicn there must be some selection of
cases, and witli tliat (tie development of criteria for how many arc sullicient.
This chapter should have demonstrated that ttierc is no single answer
to questions about [lie appropriate size or composition of samples for
comparative research.
What is crucial in this discussion is lliat the size and nature of the
sample be determined by the questions tliat are being addressed, rather
than some predetermined idea of how research should lie done-In some instances
a single case is all that is required, and indeed in some instances more
tii;"n one case might nniddy the water rattier than clarify the issues
aJ hand. If a single c.isc is selected wisely, and the research is done
thoroughly, it may say a good deal more than any statistical s[iid\. On
the oilier h;ui<l, there arc some questions that may rc<[uire a
large sample and rail for including as many cases ;is possible.
This chapter has pointed out that an iniermediate strategy is often a
good option for comparative research. Developing approaches that capture
some of the complexity of individual c.iscs in a geographical reeil*",
but still pel-mil covering some portion of the possible cases, mav provide
something approaching the best of both worlds. Despite its potential virtues,
however, this approach can be taken too far. It can produce research that
takes as given what may be important hypotheses, for example, tliat the
countries within the regional classification arc similar in theoretically
relevant ways. Fiii-tbcr, this strategy can virtually eliminate the testing
of some hypotheses by limiting the range of variance available. Like almost
everything else about comparative politics that we have discussed here
there is a traclc-olf, and there is no magic solution to substitute for
the good judgment of the scholar in making decisions.
|
|