RESEARCH METHODS NATIONAL RESEARCH UNIVERSITY HSE ECONOMICS, PH.

RESEARCH METHODS NATIONAL RESEARCH UNIVERSITY HSE ECONOMICS, PH.

RESEARCH METHODS NATIONAL RESEARCH UNIVERSITY HSE ECONOMICS, PH. D PROGRAM DR C S LEONARD JUNE 2011 Lecture 1 What is a good question? OUTLINE OF LECTURE 2 Knowledge Claims Classical Approaches

Post-positivism Strategies of inquiry Data mining Design-based research Good questions emerge from good research designs RESEARCH METHODS 8/06/2011 KNOWLEDGE CLAIMS RESEARCH METHODS 3 8/06/2011

KNOWLEDGE CLAIMS 4 What warrants knowledge? How is scientific method applied? RESEARCH METHODS 8/06/2011 TWO POSITIONS 5 Two scientific positions, inductivism and deductivism From which method emerges

Deductive (logic) argumentation If premises are true and no fallacies in the argument, then conclusion will be true Not concerned with truth or falsity Inductive arguments may have true premises, but we cannot be certain that conclusions will also be true (ampliative reasoning) LEONARD GSOM PH.D. RESEARCH METHODS 2010 4/02/20010 INDUCTIVE, DEDUCTIVE

6 Deductive, from general to particular, and inductive, from particulars to general Inductive: Frances Bacon vs the medieval Church: purging ourselves of idols Problem (Hume) Can we predict the future? No Positivism is descended from Bacon Research becomes historical, truth confined to a systematic empirical study, that might obtain general laws

Empirical findings worthless to some deductivists LEONARD GSOM PH.D. RESEARCH METHODS 2010 4/02/20010 FALSIFICATION: KARL POPPER (1902-1994) 7 Karl Poppers critical rationalism has generated much debate since the 1930s Intellectual autobiography, Unended Quest Background, early Marxist, training with Adler and Freudian theories convinced them that the theories were too broad

Later rejected psychologism Favored theory of relativity, Einstein, could be tested, verified, falsified LEONARD GSOM PH.D. RESEARCH METHODS 2010 4/02/20010 LOGIC OF FALSIFICATIONISM 8 Scientific theories are abstract can be tested only indirectly, by reference to their implications.

Scientific theory, and human knowledge generally, is irreducibly conjectural or hypothetical to solve problems that have arisen in specific historio-cultural settings Logically, no number of positive outcomes at the level of experimental testing can confirm a scientific theory A single counterexample is logically decisive: it shows the theory, from which the implication is derived, to be false. RESEARCH METHODS 8/06/2011 VS INDUCTIVISM 9

Direct antithesis of inductivism Growth of knowledge requires overturning previous beliefs Have a theory, test it, falsify it and move on Many economists seek to prove theories correct; the job of science is to disprove them Dont strive for certainty (verificationism) Bans ad hoc adjustments to a theory to prevent it from being falsified A new theory will possess greater empirical content than its

predecessors LEONARD GSOM PH.D. RESEARCH METHODS 2010 4/02/20010 CRITICS 10 This hinders not promotes science Cant reject theories so easily, some theories are better at some things than others Marxism, accommodates business cycles and disequilibrium better then orthodox Keynsianism, but the Monetarists better understand inflationary processes than the Keynsians, who argue that their theories and policies are more effective against unemployment

Should not critique a new theory too rigorously, because it may have something good in it LEONARD GSOM PH.D. RESEARCH METHODS 2010 4/02/20010 DEFENDER: BLAUG 11 Blaug (be taken seriously) Have a prediction about the future Require a formal model

Falsification is essential Be scientific, or not Falsificationism: much tougher Lay down restrictions on what Popper calls immunizing strategems RESEARCH METHODS 8/06/2011 ENORMOUS INFLUENCE 12

Mark Blaug Even econometricians, however, econometric results difficult to falsify Plain fact Most economists tend to verify.. LEONARD GSOM PH.D. RESEARCH METHODS 2010 4/02/20010 THOMAS KUHN 13 Structure of Scientific Revolutions

Science is not good science unless it is working under the framework of a theory, makes no progress It adopts a particular view of the world, and all subsequent research adds to that Pre sciencelots of theories hoping to explain the same thing The paradigm: an achievement so important that it attracts an enduring group of adherents Commitment and consensus are prerequisites for normal science LEONARD GSOM PH.D. RESEARCH METHODS 2010 4/02/20010

PARADIGMS 14 Older generations stay with their paradigms, new ones acquire new paradigms Releases scientists from the necessity of debating fundamentals They can then concentrate on subtle, esoteric aspects of their subject LEONARD GSOM PH.D. RESEARCH METHODS 2010 4/02/20010 KUHN: NORMAL SCIENCE

15 Mopping up Determination of facts Setting the facts within theory Articulating the theory Then, anomalies, followed by crisis, followed by fundamental change LEONARD GSOM PH.D. RESEARCH METHODS 2010 4/02/20010

APPLIED ECONOMETRICS 16 Middle ground Use theory, provide initial specification Data exploration techniques to extend or refine it Bridge theory and empirical data analysis How do we know a theory is correct?

Different users have different tastes and beliefs Complications with computation: large data sets numerous models possible LEONARD GSOM PH.D. RESEARCH METHODS 2010 4/02/20010 PARADIGMS 17 How researchers will learn/what they will learn, assumptions Philosophical assumptions, epistemologies (how we know something), ontologies (what is knowledge), axiology (what values go into

knowledge), methodology (process for studying) LEONARD GSOM PH.D. RESEARCH METHODS 2010 4/02/20010 EXTENSIONS 18 These debates shaped much social science theory about Innovations Science Path dependence

Historical legacies Nature of change Nature of reform and timing LEONARD GSOM PH.D. RESEARCH METHODS 2010 4/02/20010 POSITIVISM 19 Can we be positive about our claims of knowledge when studying behavior and actions (Comte, Mill, Durkheim, Newton and Locke)?

Causes probably determine effects? Reductionism: reduce ideas into small discrete sets to be tested RESEARCH METHODS 8/06/2011 POSITIVISM VS INTERPRETIVISM 20 Interpretivism: Weber (Verstehen) RESEARCH METHODS 8/06/2011

21 POSITIVISM AND ITS OPPONENTS Quantitative, positivist, post-positivist research, empirical science Challenge to positivism: against the traditional notion of the absolute truth of knowledge; playing tennis with the net down RESEARCH METHODS 8/06/2011 METHODOLOGICAL PLURALISM? 22 Bruce Caldwell (let 100 flowers bloom)

Little economics will survive if we take this seriously Confirmationism Verification Falsificationism is never practiced because it is unpracticeable RESEARCH METHODS 8/06/2011 BAYSIAN METHODS 23 Test, verify and refine the laws and theories governing behavior

Baysian methods: from theory, to test, to revision RESEARCH METHODS 8/06/2011 (1) POST POSITIVISM 24 Knowledge is conjectural Research is to make claims and refine or abandon them Data, evidence and rational considerations shape knowledge

Being objective is key RESEARCH METHODS 8/06/2011 (2) SOCIAL CONSTRUCTION 25 Social construction Mannheim, Burger, Luckmann, Neuman Look at participants views Judgments are subjective, meanings are varied and multiple

Interviews: open ended questioning, the more the interviewee talks spontaneously, the better Participants allowed to construct meaning (rather than responding to concrete situations) Process of interaction, context of work RESEARCH METHODS 8/06/2011 (3) PRAGMATISM 26 Pierce, James and Dewey

Knowledge claims arise out of actions, situations, consequences, rather than ex ante conditions Concern with what works Pluralistic approach Mixed methods, qualitative, quantitative Research always occurs in social contexts Stop asking questions about the laws of nature RESEARCH METHODS 8/06/2011

LOGIC OF RESEARCH DESIGN Influence Research Design RESEARCH METHODS 27 8/06/2011 28 ACROSS MOST FIELDS APPLIED ECONOMISTS ARE NOW LESS LIKELY TO PIN A CAUSAL INTERPRETATION OF THE RESULTS ON ECONOMETRIC METHODOLOGY ALONE. DESIGN-BASED STUDIES ARE DISTINGUISHED BY THEIR PRIMA FACIE CREDIBILITY AND BY THE AT TENTION INVESTIGATORS DEVOTE TO MAKING BOTH AN INSTITUTIONAL AND A DATADRIVEN CASE FOR CAUSALITY. JOSHUA D. ANGRIST AND JRNSTEFFEN PISCHKE RESEARCH METHODS

8/06/2011 29 GENERAL TO SPECIFIC METHODOLOGY LSE tradition of time-series econometrics that began in the 1960s at the London School of Economics Mizon (1995) a brief history The practitioners of LSE econometrics are now widely dispersed among academic institutions throughout Britain and the world. The LSE approach is described sympathetically in Gilbert (1986), Hendry (1987,1995, esp. chs. 9-15), Pagan (1987), Phillips (1988), Ericsson, Campos and Tran(1990), and Mizon (1995). For more sceptical accounts, see Hansen (1996) and Faust and Whiteman (1995, 1997) RESEARCH METHODS 8/06/2011 GENERAL TO SPECIFIC 30

Context: Linear (cross-country growth) Use: Time-Series Step 1. General regression will include every possible variable -- all the information about the true determinants. Step 2. The information content is then sharpened by a more parsimonious regression the specific regression it is statistically well specified (for example, it has white noise errors); (b) that it is a valid restriction of the general regression, and (c) that it encompasses every other parsimonious regression that is a valid restriction of the general regression (a) Criticism: data-mining, RESEARCH METHODS 8/06/2011 31 EXTREME BOUNDS ANALYSIS: THE NEW CRITIQUE

Edward Leamers extreme-bounds analysis (1983, 1985). A coefficient of interest is robust only to the degree that it displays a small variation to the presence or absence of other regressors. Leamer and Leonard (1983) define the extremebounds for the coefficient of a particular variable within a search universe as ranging between the lowest estimate of its value minus two times its standard error to the highest estimate of its value plus two times its standard error, where the extreme values are drawn from the set of every possible subset of regressors that include the variable of interest. A variable is said to be robust if its extreme bounds lie strictly to one side or the other of zero. RESEARCH METHODS 8/06/2011 32 JUDGMENT CALLS The main difference between structural and experimental (or ``atheoretic'') approaches is not in the number of assumptions but the extent to which they are made explicit. (Michael Keane)

RESEARCH METHODS 8/06/2011 SUMMARY 33 Quantitative (numbers) Experimental design--controls Non experimental design, surveys Qualitative (words) Narratives, phenomenologies, ethnographies,

grounded theory, case studies Mixed methods Sequential, concurrent, transformative RESEARCH METHODS 8/06/2011 QUANTITATIVE 34 Random assignment of subjects to treatment Quasi experiments: non random designs Surveys cross sectional and longitudinal, generalize from sample to population

RESEARCH METHODS 8/06/2011 QUALITATIVE 35 Ethnographies: researcher studies an intact cultural group in its setting over time (responses) Grounded theory Derive an abstract theory of a process, action or interaction, grounded in views of participants Case Studies

Exploring in depth a program, event, activity, process, or individuals, bound in time, variety of procedures Phenomenological: lived experiences Narrative research: lives, stories, retellings RESEARCH METHODS 8/06/2011 MIXED 36 Gets around biases in any one method used exclusively Progress from one method to another

Illustrate Determine what the concept is Test assumptions on one case RESEARCH METHODS 8/06/2011 LOGIC 37 Qualitative:

Wider range of methods, non-numerical by definition Small n, intensive interviews, depth analysis, discursive, account of event or unit Focus on event, decision, institution, location, issue or legislation Incident important in its own right (war, election, change in leadership, marketing strategy, community decision, etc) Against bifurcation? mixed methods Systematic, scientific research of all kinds Most research does not neatly fit one or other category

RESEARCH METHODS 8/06/2011 GOAL OF SOCIAL SCIENCE RESEARCH 38 Inference (descriptive, explanatory) Attempting to infer beyond immediate facts to something broader Learning about causal effects from data Public procedures (explicit, codified)

Replication Conclusions are uncertain Observes rules of inference RESEARCH METHODS 8/06/2011 DESCRIPTIVE INFERENCE 39 Distinguish systematic from non-systematic features Systematic from stochastic

Counter-factuals (what would have happened, had meters not struck the earth 65 million yrs ago) RESEARCH METHODS 8/06/2011 RULES FOR RESEARCH DESIGN 40 Intuition: Choice of better topics is idiosyncratic Two ways to test if it is a good topic Is it about something important in the real world

A research topic should make a specific contribution to an identifiable scholarly literature by increasing our ability to construct verified scientific explanations ie: locating it within the framework of existing social science literature This is the subject of the second lecture todaywhat makes a theory or theoretical contributions valuable to the community of editors of journals RESEARCH METHODS 8/06/2011 CAUTION 41

There may be reasons a theory is practicable, even though its long term scientific value has been questioned Theoretically incoherent models used to forecast the US economydiversion of macroeconomic theory and applied macroeconomics (see Mankiw 1990) New theories, however, remain speculative RESEARCH METHODS 8/06/2011 DESIGN BASED RESEARCH 42 Leamer 1983 highlighted the benefits of

sensitivity analysis, a procedure in which researchers show how their results change with changes in specification or functional form. Sensitivity analysis has had a salutary but not a revolutionary effect on econometric practice. As we see it, the credibility revolution in empirical work can be traced to the rise of a design-based approach that emphasizes the identification of causal effects. RESEARCH METHODS 8/06/2011 WOW-FACTOR 43 Design-based studies typically feature either real or natural experiments and are distinguished by their prima facie credibility and by the attention investigators devote to making the case for a

causal interpretation of the findings their designs generate. Design-based studies are most often found in the microeconomic fields of Development, Education, Environment, Labor, Health, and Public Finance, but are still rare in Industrial Organization and Macroeconomics. RESEARCH METHODS 8/06/2011 LITERATURE REVIEW RESEARCH METHODS 44 8/06/2011 LITERATURE REVIEW 45

Classical Systematic review Meta-analysis Narrative review Search issues Presentation RESEARCH METHODS

8/06/2011 CLASSICAL LITERATURE REVIEW 46 Large Disciplinary Differences Sociology, Psychology, Business Economics RESEARCH METHODS 8/06/2011 SYSTEMATIC REVIEW 47

A systematic review aims to provide an exhaustive summary of literature relevant to a research questions The first step of a systematic review is a thorough search of the literature for relevant papers. The Methodology section of the review will list the databases and citation indexes searched as well as any individual journals. Next, the titles and the abstracts of the identified articles are checked against predetermined criteria for eligibility and relevance. RESEARCH METHODS 8/06/2011 MORE 48

Ability to control for between-study variation Including moderators to explain variation Deal with information overload: the high number of articles published each year. It combines several studies and will therefore be less influenced by local findings than single studies will be. Makes it possible to show if a bias for published works exists. RESEARCH METHODS 8/06/2011

SYSTEMATIC ANALYSIS 49 Meta-Analysis Meta-analysis leads to a shift of emphasis from single studies to multiple studies. It emphasizes the practical importance of the effect size instead of the statistical significance of individual studies. This shift in thinking has been termed "metaanalytic thinking" RESEARCH METHODS 8/06/2011 50 META-ANALYSIS ( AV E R A G E E F F E C T S I Z E , F O R E S T P LO T )

RESEARCH METHODS 8/06/2011 META-ANALYSIS: CRIT 51 About science, but not science Statistical examination of scientific studies Cannot propose ways to falsify a theory RESEARCH METHODS 8/06/2011 META-NARRATIVE REVIEW

52 Example: Connected Communities, Meta-narrative review shows the diversity in the meaning of communities -various conceptualisations and meanings of community across disciplines, over time, and within different cultures and contexts a research programme of the Institute of Health and Human Development (IHHD) to investigate the meanings of community within and across research disciplines by adopting an innovative methodology based on a meta-narrative systematic review approach. Policy and academic interest in the concept of community is longstanding and such interest has become central to policy making in the last two decades. (Greenhalgh et al, 2005) is a type of systematic review rather than a traditional expert driven literature review A focus on identifying the storylines of research within and across disciplinary boundaries. Identifies the meta-narratives of each discipline and

analyse the different discourses and languages of community. RESEARCH METHODS 8/06/2011 THE END Until Next Hour

Recently Viewed Presentations

  • Dark Matter Mathematics - uwyo.edu

    Dark Matter Mathematics - uwyo.edu

    Velocities do not drop off Result: Dark Matter mass is about 10x Luminous Matter mass What is Dark Matter? Baryonic (Normal) Matter: Low mass stars, brown dwarfs (likely), large planets, meteoroids, black holes, neutron stars, white dwarfs, hydrogen snowballs, clouds...
  • Lecture 21 Forensic Entomology Guest Lecturer: Dr. Richard

    Lecture 21 Forensic Entomology Guest Lecturer: Dr. Richard

    VII. Stages of Human Decomposition and Associated Anthropods Decay Stage - Black Putrefaction (Days 5-11) -- Decay stage begins when the abdominal wall is broken, allowing gasses to escape and carcass deflates. This process is facilitated by feeding activities of...
  • Orange circles design template - Ed W. Clark High School

    Orange circles design template - Ed W. Clark High School

    Define: OPEC, Chernobyl. ... Vocabulary - Dissident, glasnost, perestroika, default. Focus/Review - What ideas guided Soviet political, economic, and foreign policy? Why did the Soviet Union collapse? What problems have Russia and the other republics faced since the fall of...
  • L'interaction - Michael McGuffin

    L'interaction - Michael McGuffin

    Fitts' law: a predictive model of motor capability (note: the word "motor" here refers to something that creates movement) Calculator on the Samsung Galaxy Note II (5.5 inch screen) Option to reduce size of keyboard Thanko Thumb Extender (from Japan)...
  • The University of Glasgow

    The University of Glasgow

    Arial Rockwell Wingdings 2 Calibri Foundry 1_Foundry 2_Foundry 3_Foundry 4_Foundry 5_Foundry 6_Foundry 7_Foundry 8_Foundry 9_Foundry The University of Glasgow Principia Consortium Application Process Housing Courses Courses Courses
  • UV-Curved Nano Imprint Lithography - 123seminarsonly.com

    UV-Curved Nano Imprint Lithography - 123seminarsonly.com

    This was first invented by Prof. Stephen Chou and his students in 1995. This is a method of fabricating nanometer scale patterns. It creates patterns by mechanical deformation of imprint resist and subsequent processes.
  • Away from Supervision - dhhr.wv.gov

    Away from Supervision - dhhr.wv.gov

    What is Away From Supervision/Runaway Monthly Report? Residential Facilities must fill out a monthly report about children who are away from supervision. This report will include the following information: the number of runaway episodes, the number of episodes per gender,...
  • Presentation Title Here Presentation Subtitle Here

    Presentation Title Here Presentation Subtitle Here

    Mayo Clinic Radio marked 25 years and celebrated the occasion in the Plummer Library reading room. Also a huge shout out to Pat for all her work on the invites and other work behind the scenes. A great job by...