Cogprints

Are Null Results Becoming an Endangered Species in Marketing?

Hubbard, Raymond and Armstrong, J. Scott (1992) Are Null Results Becoming an Endangered Species in Marketing? [Journal (Paginated)]

Full text available as:

[img]
Preview
 PDF
31Kb

Abstract

Editorial procedures in the social and biomedical sciences are said to promote studies that falsely reject the null hypothesis. This problem may also exist in major marketing journals. Of 692 papers using statistical significance tests sampled from the Journal of Marketing, Journal of Marketing Research, and Journal of Consumer Research between 1974 and 1989, only 7.8% failed to reject the null hypothesis. The percentage of null results declined by one-half from the 1970s to the 1980s. The JM and the JMR registered marked decreases. The small percentage of insignificant results could not be explained as being due to inadequate statistical power. Various scholars have claimed that editorial policies in the social and medical sciences are biased against studies reporting null results, and thus encourage the proliferation of Type 1 errors (erroneous rejection of the null hypothesis). Greenwald (1975, p. 15) maintains that Type I publication errors are underestimated to the extent that they are: “. . . frightening, even calling into question the scientific basis for much published literature.” Our paper examines the publication frequency of null results in marketing. First, we discuss how editorial policies might foster an atmosphere receptive to Type I error proliferation. Second, we review the evidence on the publication of null results in the social and biomedical sciences. Third, we report on an empirical investigation of the publication frequency of null results in the marketing literature. Fourth, we examine power levels for statistically insignificant findings in marketing to see if they are underpowered and thus less deserving of publication. Finally, we provide suggestions to facilitate the publication of null results.

Item Type:Journal (Paginated)
Keywords:File Drawer Problem, Null Results, Publication Bias, Statistical Power Analysis, Statistical Significance
Subjects:Psychology > Behavioral Analysis
ID Code:5178
Deposited By: Armstrong, J. Scott
Deposited On:25 Sep 2006
Last Modified:11 Mar 2011 08:56

References in Article

Select the SEEK icon to attempt to find the referenced article. If it does not appear to be in cogprints you will be forwarded to the paracite service. Poorly formated references will probably not work.

Angell, Marcia. (1989). “Negative Studies,” New England Journal of Medicine, 321, 464 466.

Atkinson, Donald R., Michael J. Furlong, and Bruce E. Wampold. (1982). “Statistical Significance, Reviewer Evaluations, and the Scientific Process: Is There a (Statistically) Significant Relationship?” Journal of Counseling Psychology, 29, 189-194.

Bakan, David. (1966). “The Test of Significance in Psychological Research,” Psychological Bulletin, 77, 423 437.

Bozarth, Jerold D., and Ralph R. Roberts, Jr. (1972). “Signifying Significant Significance,” American Psychologist, 27, 774-775.

Cohen, Jacob. (1962). “The Statistical Power of Abnormal-Social Psychological Research: A Review,” Journal of Abnormal and Social Psychology, 65, 145-153.

Cohen, Jacob. (1988). Statistical Power Analysis for the Behavioral Sciences. Hillsdale, N.J.: Lawrence Erlbaum Associates.

Cohen, Jacob. (1990). “Things I Have Learned (So Far),” American Psychologist, 45, 1304-1312.

Coursol, Allan, and Edwin E. Wagner. (1986). “Effect of Positive Findings on Submission and Acceptance Rates: A Note on Meta-Analysis Bias,” Professional Psychology: Research and Practice, 17, 136-137.

Dickersin, K., S. Chan, T. C. Chalmers, H. S. Sacks, and H. Smith, Jr. (1987). “Publication Bias and Clinical Trials,” Controlled Clinical Trials, 8, 343-353.

Fagley, N. S. (1985). “Applied Statistical Power Analysis and the Interpretation of Nonsignificant Results by Research Consumers,” Journal of Counseling Psychology, 32,

391-396.

Feige, Edgar L. (1975). “The Consequences of Journal Editorial Policies and a Suggestion for Revision,” Journal of Political Economy, 83, 1291-1295.

Greenwald, Anthony G. (1975). “Consequences of Prejudice Against the Null Hypothesis,” Psychological Bulletin, 82, I-20.

Kerr, Steven, James Tolliver, and Doretta Petree. (1977). “Manuscript Characteristics Which Influence Acceptance for Management and Social Science Journals,” Academy of Management Journal, 20, 132-141.

Kupfersmid, Joel, and Michael Fiala. (1991). “A Survey of Attitudes and Behaviors of Authors Who Publish in Psychology and Education Journals,” American Psychologist, 46, 249-250.

Rosenthal, Robert. (1979). “The ‘File Drawer Problem’ and Tolerance for Null Results,” Psychological Bulletin, 86, 638-641.

Rosnow, Ralph L., and Robert Rosenthal. (1989). “Statistical Procedures and the Justification of Knowledge in Psychological Science,” American Psychologist, 44, 1276-1284.

Rourke, Byron P., and Louis Costa. (1979). “Editorial Policy 11,” Journal of Clinical Neuropsychology, 1, 93-95.

Rowney, Julie A., and Thomas J. Zenisek. (1980). “Manuscript Characteristics Influencing Reviewers' Decisions,” Canadian Psychology, 21, 17-21.

Rust, Roland T., Donald R. Lehmann, and John U. Farley. (1990). “Estimating Publication Bias in Meta-Analysis,” Journal of Marketing Research, 27, 220-226.

Salsburg, David S. (1985). “The Religion of Statistics as Practiced in Medical Journals,” American Statistician, 39, 220-223.

Sawyer, Alan G., and A. Dwayne Ball. (1981). “Statistical Power and Effect Size in Marketing Research,” Journal of Marketing Research, 18, 275-290.

Shaddish, William R., Maria Doherty, and Linda M. Montgomery. (1989). “How Many Studies Are in the File Drawer? An Estimate from the Family/Marital Psychotherapy Literature,” Clinical Psychology Review, 9, 589-603.

Simes, Robert J. (1986). “Publication Bias: The Case for an International Registry of Clinical Trials,” Journal of Clinical Oncology, 4, 1529-1541.

Smart, Reginald G. (1964). “The Importance of Negative Results in Psychological Research,” Canadian Psychologist, 5, 225-232.

Smith, Mary L. (1980). “Publication Bias and Meta-Analysis,” Evaluation in Education, 4, 22-24.

Sommer, Barbara. (1987). “The File Drawer Effect and Publication Rates in Menstrual Cycle Research,” Psychology of Women Quarterly, l l, 233-241.

Sterling, Theodore D. (1959). “Publication Decisions and their Possible Effects on Inferences Drawn from Tests of Significance – or Vice Versa,” Journal of the American Statistical Association, 54, 30-34.

Metadata

Repository Staff Only: item control page

Cogprints is powered by EPrints 3 which is developed by the School of Electronics and Computer Science at the University of Southampton. More information and software credits.