Discussion Paper

Evaluating
Information
Resources in the
Era of Deceptive
Information
The Librarian’s Expanding Role

Presented by ASME as a Service to Our Librarian Community

The American Society of Mechanical Engineers®

ASME®
Contents

3 Introduction
Evaluating information resources in the era of deceptive information: A new set
of challenges.

4 Information Literacy Education

Librarians: Helping users navigate a complex information landscape.
Framework for Information Literacy for Higher Education – A Snapshot

5 Tips for Evaluating Specific Web Resources

Librarians: Media and information literacy educators and advocators.
IFLA recommendations for evaluating resources
The CRAAP Test
Final word of advice: Beware of content farms

6 Tips for Identifying Predatory Publishers and Questionable

Conference Organizers
Librarians: Helping researchers avoid dubious journals and conferences.
Some warning signs
What can librarians do to avoid pitfalls?

7 References, Resources, and Useful Links

2
Introduction

Evaluating information resources in the era of deceptive

information: A new set of challenges.

This document provides a snapshot of scholarly librarians’ expanding role in differentiating

legitimate information resources from deceptive ones in order to guide their user communities
towards adopting best practices.

Librarians, as trained information professionals, are increasingly called upon to apply their expertise to many different aspects of
their users’ professional activities in part because of the proliferation of information on the web and its ease of access.

Librarians are often active participants in the dialog between publishers and information users surrounding scholarly publishing
and publication practices. They are particularly well positioned to share their collective expertise with researchers in academic,
corporate, and governmental settings regarding how to identify reliable, trusted information resources and avoid those that are
questionable. Information literacy education organized by the library plays a significant role, for example, in creating awareness of
how scholarly content is produced and the real value of legitimate open access initiatives. It can also provide deeper understanding
of the value of rigorous peer review, prestigious editorial boards, inclusion in reputable abstracting and indexing services, etc.

Not only are librarians instrumental in helping researchers access scientific, technical, and medical (STM) content, but they can also
offer sound advice about how to identify and avoid predatory journals. Librarians can use their expertise and apply best practices to
identify journals that should be avoided or supported and contribute to other initiatives aimed at deterring predatory publishers.

Concurrently, there has been an increase in the number of predatory conferences. Although not traditionally within the role of the
librarian to alert researchers to potential questionable scientific conferences, it is increasingly falling within the scope of avoiding
deceptive information.

Drawing on guidelines from professional societies such as the American Library Association (ALA), the Association of College and
Research Libraries (ACRL), the International Federation of Library Associations and Institutions (IFLA), and other leading institutions,
this document presents strategies and tactics for evaluating information resources, a framework for designing information literacy
programs, useful checklists to evaluate specific web resources, and steps for identifying predatory publisher and conference
organizers.

3
Information Literacy
Education

Librarians: Helping users navigate a complex information landscape.

Guiding librarians in instilling the benefits of information literacy throughout their user communities is the American Library Association
(ALA), Association of College and Research Libraries’ (ACRL) Framework for Information Literacy for Higher Education.1 This cluster of
six interconnected core concepts form the basis of instruction of information literacy to individuals throughout their academic careers
and beyond.

The Framework is intended to provide recommendations and guidance rather than being prescriptive. It enables librarians, faculty,
and other stakeholders to design programs that are best suited to their own institution. Within this context, information literacy is
defined as “the set of integrated abilities encompassing the reflective discovery of information, the understanding of how information
is produced and valued, and the use of information in creating new knowledge and participating ethically in communities of learning.”

Framework for Information Literacy for Higher Education – A Snapshot

The six pillars presented within this Framework can be adopted to develop an information literacy program in their entirety or in
part, depending on what is deemed most applicable for a specific institution. They are summarized below in alphabetical order
since these concepts are not intended to be implemented in a specific order. Select knowledge practices are included for each
pillar within the Framework.

AUTHORITY IS CONSTRUCTED AND CONTEXTUAL RESEARCH AS INQUIRY

Information resources reflect their creators’ expertise Research is iterative and depends upon asking
and credibility increasingly complex or new questions
Knowledge Practices: Knowledge Practices:
• Define different types of authority, such as subject • Determine an appropriate scope of investigation.
expertise, societal position, or special experience. • Deal with complex research by breaking complex
• Use research tools and indicators of authority to questions into simple ones, limiting the scope of
determine the credibility of sources, understanding the investigations.
elements that might temper this credibility. • Use various research methods based on need,
• Recognize that information may be perceived circumstance, and type of inquiry.
differently based on the format in which it is packaged.
SCHOLARSHIP AS CONVERSATION
INFORMATION CREATION AS A PROCESS Communities of scholars, researchers, or professionals
Information in any format is produced to convey a engage in sustained discourse
message and is shared through various delivery methods
Knowledge Practices:
Knowledge Practices: • Cite the contributing work of others in your own
• Articulate the capabilities and constraints of information production.
information developed through various creation • Contribute to scholarly conversation at an appropriate
processes. level.
• Assess the fit between an information product’s • Critically evaluate contributions made by others in
creation process and the information application need. participatory information environments.
• Articulate the traditional and emerging processes
of information creation and dissemination within a SEARCHING AS STRATEGIC EXPLORATION
discipline. Searching for information is often nonlinear and iterative,
requiring the evaluation of a range of information
INFORMATION HAS VALUE sources and the mental flexibility to pursue alternate
As a commodity, as a means of education, and as a avenues as new understanding develops
means to influence
Knowledge Practices:
Knowledge Practices: • Determine the initial scope of the task required to
• Give credit to the original ideas of others through meet information needs.
proper attribution and citation. • Identify interested parties who might produce
• Understand that intellectual property is a legal and information about a topic.
social construct that varies by culture. • Define and refine needs and search strategies as
• Articulate the purpose and distinguishing necessary based on search results.
characteristics of copyright, fair use, open access, and
the public domain.

4
1
Adapted from “Framework for Information Literacy for Higher Education,” filed by the ACRL Board on February 2, 2015, adopted by the
ACRL Board, January 11, 2016, accessed March 1, 2019, http://www.ala.org/acrl/standards/ilframework
Tips for Evaluating Specific
Web Resources

Librarians: Media and information literacy educators and advocators.

Librarians routinely evaluate web resources and help users navigate the complexities of determining which sources are reliable and
which are not. Media and information literacy education offered by the library plays a key role in guiding users.

The two organizations listed below provide valuable checklists for evaluating web resources. There are many other similar resources that can
guide users to reliable information.

IFLA recommendations for The CRAAP Test

evaluating resources
Another widely acknowledged yardstick for evaluating resources was developed in
The global voice of the library and information the United States by librarians at the California State University Meriam Library (Chico,
profession, the International Federation of CA, USA). Commonly known by its acronym, the CRAAP Test3 establishes a valuable
Library Associations and Institutions (IFLA) checklist for determining whether a source can be trusted.
is the leading international organization
representing the interests of library and CRAAP is an acronym for Currency, Relevance, Authority, Accuracy, and Purpose. The
information services and their users. In 2016, user should apply the following questions as part of the resource evaluation process:
IFLA issued a set of simple steps entitled CURRENCY: THE TIMELINESS OF THE INFORMATION
“How to Spot Fake News”2 to help detect • When was the information published or posted?
deceptive information resources. These • Has the information been revised or updated?
recommendations have been extensively • Does your topic require current information, or will older sources work as well?
shared by libraries around the world. • Are the links functional?
CONSIDER THE SOURCE RELEVANCE: THE IMPORTANCE OF THE INFORMATION FOR YOUR NEEDS
Click away from the story to investigate the • Does the information relate to your topic or answer your question?
site, its mission and its contact information. • Who is the intended audience?
CHECK THE AUTHOR • Is the information at an appropriate level (i.e. not too elementary or advanced for your needs)?
Do a quick search on the author(s). Are they • Have you looked at a variety of sources before determining this is one that you will use?
credible? Are they real? • Would you be comfortable citing this source in your research paper?

CHECK THE DATE AUTHORITY: THE SOURCE OF THE INFORMATION

• Who is the author/publisher/source/sponsor?
Reposting old news stories doesn’t mean
• What are the author’s credentials or organizational affiliations?
they’re relevant to current events.
• Is the author qualified to write on the topic?
CHECK YOUR BIASES • Is there contact information, such as a publisher or email address?
Consider if your own beliefs could affect your • Does the URL reveal anything about the author or source?
judgement. Examples: .com .edu .gov .org .net

READ BEYOND ACCURACY: THE RELIABILITY, TRUTHFULNESS, AND CORRECTNESS OF THE CONTENT
Headlines can be outrageous in an effort to • Where does the information come from?
get clicks. What’s the whole story? • Is the information supported by evidence?
• Has the information been reviewed or refereed?
SUPPORTING SOURCES • Can you verify any of the information in another source or from personal knowledge?
Click on those lines. Determine if the • Does the language or tone seem unbiased and free of emotion?
information given actually supports the story. • Are there spelling, grammatical, or typographical errors?
IS IT A JOKE? PURPOSE: THE REASON THE INFORMATION EXISTS
If it is too outlandish, it might be satire. • What is the purpose of the information? Is it to inform, teach, sell, entertain, or persuade?
Research the site and author to be sure. • Do the authors/sponsors make their intentions or purpose clear?
• Is the information fact, opinion, or propaganda?
ASK THE EXPERTS
• Does the point of view appear objective and impartial?
Users should ask a librarian or consult a fact- • Are there political, ideological, cultural, religious, institutional, or personal biases?
checking site.

Final word of advice: Beware of content farms

A content farm is a website that offers a large quantity of low-quality, short articles on general topics with no citation (and
sometimes copies from other websites), largely generated by freelance writers. Students and early career researchers (ECRs) in
particular may be susceptible to be lured to these sites, which are strategically written with embedded keywords to maximize
retrieval via search engines. Some search engines continue to refine their algorithms so that more reliable sites will be ranked
more highly.

2
IFLA, “How to Spot Fake News,” last modified January 7, 2019, accessed March 1, 2019, https://www.ifla.org/publications/node/11174 5
3
Meriam Library, California State University, Chico, CA, “Evaluating Information – Applying the CRAAP Test,” last modified September
17, 2010, accessed March 1, 2019, https://www.csuchico.edu/lins/handouts/eval_websites.pdf
Tips for Identifying Predatory
Publishers and Questionable
Conference Organizers
Librarians: Helping researchers avoid dubious journals and conferences.

In recent years, in abuse of the spirit of the open access movement, there has been a proliferation of predatory journals portraying themselves
as legitimate publications within the scientific, technical, and medical (STM) publishing landscape. Some predatory publishers have recently
moved into multimedia, charging high fees for posting videos. Another growing concern is the increasing number of dubious conferences
organized by predatory publishers and other less than scrupulous organizations.

Individuals in the STM community are frequently approached by these publishers and conference organizers, and it is not uncommon for
people from many countries to fall prey to their tactics, contributing articles to fraudulent journals or participating at questionable conferences.

“If in doubt, check it out”

Librarians should teach their users about • Don’t assume all journals are trustworthy!
predatory publishers and conference • Don’t assume all scientific conferences are legitimate!
organizers so that they are not fooled. • Don’t submit papers to journals or conferences that
appear questionable!

Some warning signs

It can be difficult to differentiate between authentic journals or conferences and ones that are suspicious. Scammers are becoming
more adept at mimicking professional and credible solicitations. Librarians should educate their community about deceptive
practices and potential warning signs in order to judge legitimacy.

PREDATORY JOURNALS PREDATORY CONFERENCES

• Vague or poorly stated/conceptualized aims & scope • Conference has an overly ambitious title and the program is too broad
• A reputable journal with a very similar title exists • Renowned organizations are purportedly sponsoring a low profile
• Falsified editorial board members, reviewers, or endorsements conference
• Website containing spelling and grammatical errors • A reputable conference with a suspiciously similar conference name
• Distorted or fuzzy images (possibly unauthorized usage) • Conference is unusually frequent
• Overly flattering and/or hard sell manuscript solicitation emails • Partnership with a recognized predatory publisher or unknown
• Extremely rapid manuscript acceptance (which could be institution or commercial entity
indicative of no peer review) • Speakers who are unknown in the field
• Poorly documented manuscript submission procedures • Website containing spelling and grammatical errors
• Exploitation of the open access model • Distorted or fuzzy images (possibly unauthorized usage)
• Submission or hidden fees requested upfront • Overly flattering and/or hard sell solicitation emails to present or attend
• Author charges for non-existent services • Extremely rapid acceptance of conference papers (which could be
• No retraction, copyright, or digital preservation policies indicative of no peer review)
• Contact names or information unclear or not available • Organizers charging higher than normal fees
• Public email addresses (without publisher or institution name), • Unclear information about the publication of conference proceedings
e.g. gmail.com or yahoo.com • Contact names or information unclear; email addresses without
• Journal not listed in reputable abstracting and indexing conference organizer affiliation (e.g. gmail.com or yahoo.com)
databases like Web of Science, Scopus, and others

What can librarians do to avoid pitfalls?

Here are a few tactics that librarians can employ to steer their user community away from predatory journals and disreputable
conferences:

• Create specific webpages and/or LibGuides or PubGuides that address these with links to helpful resources like Think Check
Submit (thinkchecksubmit.org) and Think Check Attend (thinkcheckattend.org)
• Advocate for and include instruction modules in information literacy training programs
• Investigate journal and/or publisher using reputable abstracting and indexing databases like Web of Science, Scopus, and others
• Consult Journal Impact Factors, journal h-indexes, and other recognized journal evaluation metrics
• Search for journal citation data

6
References, Resources,
and Useful Links

References Useful Links

1
Adapted from “Framework for Information Literacy for Higher Clarivate Analytics Master Journal List: http://mjl.clarivate.com
Education,” filed by the ACRL Board on February 2, 2015, Committee on Publication Ethics: COPE: https://
adopted by the ACRL Board, January 11, 2016, accessed March publicationethics.org/core-practices
1, 2019, http://www.ala.org/acrl/standards/ilframework Directory of Open Access Journals: DOAJ: https://doaj.org
Scopus Content: https://www.elsevier.com/solutions/scopus/
2
IFLA, “How to Spot Fake News,” last modified January 7, 2019, how-scopus-works/content
accessed March 1, 2019, https://www.ifla.org/publications/ Stop Predatory Journals: https://predatoryjournals.com/about
node/11174
Think. Check. Submit.: https://thinkchecksubmit.org
Think. Check. Attend.: https://thinkcheckattend.org
3
Meriam Library, California State University, Chico, CA,
“Evaluating Information – Applying the CRAAP Test,” last
modified September 17, 2010, accessed March 1, 2019, https://
www.csuchico.edu/lins/handouts/eval_websites.pdf

Resources
Austin Community College, “Low Quality Websites: Content
Farms: Avoid Content Farms,” last modified January 22, 2019,
accessed March 1, 2019, http://researchguides.austincc.edu/c.
php?g=435086&p=2965634

Berger, M., 2017, “Everything You Ever Wanted to

Know About Predatory Publishing but Were Afraid
to Ask,” ACRL Conference, Baltimore, MD, March 22-
25, 2017, http://www.ala.org/acrl/sites/ala.org.acrl/
files/content/conferences/confsandpreconfs/2017/
EverythingYouEverWantedtoKnowAboutPredatoryPublishing.
pdf

Berger, M., and Cirasella, J., 2015, “Beyond Beall’s List: Better
understanding of predatory publishers,” College & Research
Libraries, 76(3), https://crln.acrl.org/index.php/crlnews/article/
view/9277/10342

Campbell, B., “9 Signs a conference is fake,” last modified

January 25, 2018, accessed March 1, 2019, https://www.exordo.
com/blog/9-signs-this-is-a-fake-conference

Lewis, A. B., “Tips from the Experts: What Does Bad Information
Look Like? Using the CRAAP Test for Evaluating Substandard
Resources,” 2018, Issues in Science and Technology
Librarianship, Science & Technology Section, Association of
College & Research Libraries, http://www.istl.org/18-winter/
tips2.html

Shamseer, L. et al, 2017, “Potential predatory and legitimate

biomedical journals: Can you tell the difference? A cross-
sectional comparison,” BMC Medicine, 15(28), https://doi.
org/10.1186/s12916-017-0785-9