Questions to Guide Critical Analysis of Research Studies

(For ECI 803, Dr. Spires)
Use questions below to prompt your analysis that are appropriate for specific research designs. Feel free to add to these prompts as needed based on the readings
RATIONALE FOR THE STUDY
What is the rationale for the research? Is previous research adequately rationalized?
How is the research review organized?

What is the theoretical framework guiding the research?

What are the research questions? Does the review logically lead to the research questions?

CRITIQUE OF RESEARCH METHODS
What research method(s) is used?

Who are the participants? How were they selected? What are the demographics?

What are the independent variables?

What are the dependent measures?
--Are the measures operationalized?
--Are they reliable?
--Is there treatment fidelity?

How are internal and external validity addressed?
--Random assignment?
--Control group intervention?
--Counterbalancing of instructors?
--Treatment conditions clearly described?
--Equivalent instructional time and mortality rates?
--Task transfer and effect maintenance?

If qualitative methods are used, are they clearly explained?

CRITIQUE OF DATA ANALYSIS
How is the data treated?

What analytical approaches were used and were they appropriate to answer the research questions?

If the data is analyzed qualitatively, are the analytical approaches clearly described? Is data triangulated? Are themes substantiated, etc?

CRITIQUE OF DISCUSSION
Are the results clearly rationalized?

Were unexpected outcomes explained?

Were results tied to previous research?

Was theory generated or extended?

Were limitations clearly addressed?

Was future research suggested based on findings of the study?

How would you assess the significance of the research contribution?

ADDITIONAL OBSERVATIONS
What questions do you have?
What are you confused about?
__

Elements to be included in Literacy Research Synthesis