"Blacklists" and "whitelists" to tackle predatory publishing : A cross-sectional comparison and thematic analysis

Abstract

Background. Despite growing awareness of predatory publishing and research on its market characteristics, the defining attributes of fraudulent journals remain controversial. We aimed to develop a better understanding of quality criteria for scholarly journals by analysing journals and publishers indexed in blacklists of predatory journals and whitelists of legitimate journals and the lists’ inclusion criteria. Methods. We searched for blacklists and whitelists in early 2018. Lists that included journals across disciplines were eligible. We used a mixed methods approach, combining quantitative and qualitative analyses. To quantify overlaps between lists in terms of indexed journals and publishers we employed the Jaro-Winkler string metric and Venn diagrams. To identify topics addressed by the lists’ inclusion criteria and to derive their broader conceptual categories, we used a qualitative coding approach. Results. Two blacklists (Beall’s and Cabell’s) and two whitelists (DOAJ and Cabell’s) were eligible. The number of journals per list ranged from 1404 to 12357 and the number of publishers from 473 to 5638. Seventy-three journals and 42 publishers were included both in a blacklist and whitelist. A total of 198 inclusion criteria were examined. Seven thematic themes were identified: (i) peer review, (ii) editorial services, (iii) policy, (iv) business practices, (v) publishing, archiving and access, (vi) website and (vii) indexing and metrics. Business practices accounted for almost half of blacklists’ criteria, whereas whitelists gave more emphasis to criteria related to policy and guidelines. Criteria were grouped into four broad concepts: (i) transparency, (ii) ethics, (iii) professional standards and (iv) peer review and other services. Whitelists gave more weight to transparency whereas blacklists focused on ethics and professional standards. The criteria included in whitelists were easier to verify than those used in blacklists. Both types of list gave relatively little emphasis to the quality of peer review. Conclusions. There is overlap between journals and publishers included in blacklists and whitelists. Blacklists and whitelists differ in their criteria for quality and the weight given to different dimensions of quality. Aspects that are central but difficult to verify receive insufficient attention.