![Overview:
Illustrative interactive theme
Elaine Toomey [1] wants to collect published knowledge on
“Childhood obesity prevention”
How can we use the current literature to find answers?
(PEOPLE) ECRs have fought for the right to search, and have built tools.
(INFRASTRUCTURE) We’ll
• search/download the literature (getpapers + EuropePMC).
• Add semantics with Wikipedia and Wikidata
Let’s use these to get a sample, scoping review within minutes
[1] http://www.nuigalway.ie/colleges-and-schools/arts-social-sciences-and-celtic-studies/staffprofiles/elainetoomey/
(Many slides are primarily intended for reference)](https://image.slidesharecdn.com/galway2-190411113606/85/Early-Career-Reseachers-and-Open-Healthcare-1-320.jpg)
![I am a statistician interested in detecting potentially problematic research such as data fabrication,
which results in unreliable findings and can harm policy-making, confound funding decisions, and
hampers research progress.
To this end, I am content mining results reported in the psychology literature. Content mining the
literature is a valuable avenue of investigating research questions with innovative methods. For
example, our research group has written an automated program to mine research papers for errors in
the reported results and found that 1/8 papers (of 30,000) contains at least one result that could
directly influence the substantive conclusion [1].
In new research, I am trying to extract test results, figures, tables, and other information reported in
papers throughout the majority of the psychology literature. As such, I need the research papers
published in psychology that I can mine for these data. To this end, I started ‘bulk’ downloading research
papers from, for instance, Sciencedirect. I was doing this for scholarly purposes and took into account
potential server load by limiting the amount of papers I downloaded per minute to 9. I had no intention
to redistribute the downloaded materials, had legal access to them because my university pays a
subscription, and I only wanted to extract facts from these papers.
Full disclosure, I downloaded approximately 30GB of data from Sciencedirect in approximately 10 days.
This boils down to a server load of 0.0021GB/[min], 0.125GB/h, 3GB/day.
Approximately two weeks after I started downloading psychology research papers, Elsevier notified my
university that this was a violation of the access contract, that this could be considered stealing of
content, and that they wanted it to stop. My librarian explicitly instructed me to stop downloading
(which I did immediately), otherwise Elsevier would cut all access to Sciencedirect for my university.
I am now not able to mine a substantial part of the literature, and because of this Elsevier is directly
hampering me in my research.
[1] Nuijten, M. B., Hartgerink, C. H. J., van Assen, M. A. L. M., Epskamp, S., & Wicherts, J. M. (2015). The
prevalence of statistical reporting errors in psychology (1985–2013). Behavior Research Methods, 1–22.
doi: 10.3758/s13428-015-0664-2
Chris Hartgerink’s blog post](https://image.slidesharecdn.com/galway2-190411113606/85/Early-Career-Reseachers-and-Open-Healthcare-19-320.jpg)
![OpenKnowledgeMaps
• Peter Kraker[1] and colleagues, Austria
• Open; protected against takeover by Austrian Law
• Unsupervised machine learning on Open Access
• Easy to use
• Vibrant community
• http://openknowledgemaps.org
• [1] Panton Fellow](https://image.slidesharecdn.com/galway2-190411113606/85/Early-Career-Reseachers-and-Open-Healthcare-31-320.jpg)
![ContentMine software
• Open Source, platform-independent, free
• Tutorial material [1]
• getpapers search and download [2]
• ami dictionary-based search and analysis [3]
• Explicit use of Wikipedia and Wikidata
[1] (http://github.com/petermr/tigr2ess ) example shows
crops in India but easily extensible to most disciplines
[2] Rik Smith-Unna;
http://github.com/ContentMine/getpapers, Node.js
[2] Peter Murray-Rust;
http://github.com/ContentMine/normami , Java7](https://image.slidesharecdn.com/galway2-190411113606/85/Early-Career-Reseachers-and-Open-Healthcare-33-320.jpg)
Early Career Reseachers and Open Healthcare
- 1. Overview: Illustrative interactive theme Elaine Toomey [1] wants to collect published knowledge on “Childhood obesity prevention” How can we use the current literature to find answers? (PEOPLE) ECRs have fought for the right to search, and have built tools. (INFRASTRUCTURE) We’ll • search/download the literature (getpapers + EuropePMC). • Add semantics with Wikipedia and Wikidata Let’s use these to get a sample, scoping review within minutes [1] http://www.nuigalway.ie/colleges-and-schools/arts-social-sciences-and-celtic-studies/staffprofiles/elainetoomey/ (Many slides are primarily intended for reference)
- 2. (2x digital music industry!) ContentMine is an OpenLocked Non-Profit company Mining millions of Open facts every week The Right to Read is the Right to Mine
- 3. Oxford 2013 Berlin 2014 Delhi 2014 Jenny Molloy with mascot AMI Gita Yadav + colleagues ContentMine-ing Community
- 4. Why Open? • Better Science (in every aspect) • Quicker • Flexible • Inclusive • Preservable • Principled
- 5. Opening Knowledge overcomes injustice • Open comes from the heart. • Closed Access Means People Die. • Megacorporations are pwning the knowledge infrastructure/ • The Right to Read is the Right to Mine • Young people change the world. Give them the chance. • ACT: Advocacy , Community, Tools
- 6. (ContentMine) software can do a simple triage in a few minutes Ellie: “there were 10,000 abstracts and due to time pressures, we split this between 6 researchers. It took about 2-3 days of work (working only on this) to get through ~1,600 papers each. So, at a minimum this equates to 12 days of full-time work (and would normally be done over several weeks under normal time pressures).” The Problem…
- 7. http://chemicaltagger.ch.cam.ac.uk/ • Typical Typical scientific language Online demo of Natural Language Processing (NLP) Text => Science instantaneously … machines can help
- 8. … machines can “understand” every phrase … can be configured for drugs, diseases, statistics
- 9. Community
- 10. Panton Authors and Fellows Jenny Molloy Rufus Pollock Panton Arms Pub Cambridge 2009 Panton Principles For Open Science Data Ross Mounce Sophie Kay-Kershaw Sam Smith Open publishing Peter Kraker Open Knowledge Maps Panton Fellows
- 11. Ross Mounce Bioscientist and palaeontologist Tireless champion of Open and gadfly of publishers. When still a grad student he championed “The Right to Mine” in The European Commission, defeating the publishers’ “Licences for Europe”
- 12. PMR’s Shuttleworth Flash Grantees • Erin McKiernan (Open Access) • Daniel Mietchen (Wikimedian) • Joe McArthur (Right to Research, SPARC) • Chris Hartgerink (Text mining) • Pinkie Chan* (Fighting land grabs in Cambodia w OpenData) • Heather Piwowar (Impact Story, Unpaywall) • Sophie Kay-Kershaw (OSTI, reproducibility) • Corina Logan (Bullied into Bad Science) • Jon Tennant (Open Science MOOC) • Lorraine Chuen (Opencon)
- 13. Unpaywall Jason Priem Heather Piwowar https://unpaywall.org/products/extension A gamechanger. Professionally run and hopefully reaching the stage where publishers can’t afford NOT to be in unpaywall. Expect more APIs and discovery over the next year.
- 14. KNOWLEDGE IS A PUBLIC GOOD
- 16. ECRs often feel pressured into taking actions against our ethics to pursue an academic career (e.g., publishing in particular journals) ECRs: Sign the petition to help us change academic culture Non-ECRs: Join the supporters by valuing open practices, especially when making decisions about hiring, promotion, and grants Leading individuals and institutions in adopting open practices to improve research rigor We won’t be… Corina Logan & Laurent Gatto BIBS Leading Team: Dr. Laurent Gatto, Dr. Ross Mounce, Dr. Stephen Eglen, Dr. Adrian Currie, Dr. Lauren Maggio www.BulliedIntoBadScience.org | Twitter: #BulliedIntoBadScience
- 17. Julia Reda, Pirate MEP, running ContentMine software to liberate science 2016-04-16
- 18. @Senficon (Julia Reda) :Text & Data mining in times of #copyright maximalism: "Elsevier stopped me doing my research" http://onsnetwork.org/chartgerink/2015/11/16/elsevi er-stopped-me-doing-my-research/ … #opencon #TDM Elsevier stopped me doing my research Chris Hartgerink
- 19. I am a statistician interested in detecting potentially problematic research such as data fabrication, which results in unreliable findings and can harm policy-making, confound funding decisions, and hampers research progress. To this end, I am content mining results reported in the psychology literature. Content mining the literature is a valuable avenue of investigating research questions with innovative methods. For example, our research group has written an automated program to mine research papers for errors in the reported results and found that 1/8 papers (of 30,000) contains at least one result that could directly influence the substantive conclusion [1]. In new research, I am trying to extract test results, figures, tables, and other information reported in papers throughout the majority of the psychology literature. As such, I need the research papers published in psychology that I can mine for these data. To this end, I started ‘bulk’ downloading research papers from, for instance, Sciencedirect. I was doing this for scholarly purposes and took into account potential server load by limiting the amount of papers I downloaded per minute to 9. I had no intention to redistribute the downloaded materials, had legal access to them because my university pays a subscription, and I only wanted to extract facts from these papers. Full disclosure, I downloaded approximately 30GB of data from Sciencedirect in approximately 10 days. This boils down to a server load of 0.0021GB/[min], 0.125GB/h, 3GB/day. Approximately two weeks after I started downloading psychology research papers, Elsevier notified my university that this was a violation of the access contract, that this could be considered stealing of content, and that they wanted it to stop. My librarian explicitly instructed me to stop downloading (which I did immediately), otherwise Elsevier would cut all access to Sciencedirect for my university. I am now not able to mine a substantial part of the literature, and because of this Elsevier is directly hampering me in my research. [1] Nuijten, M. B., Hartgerink, C. H. J., van Assen, M. A. L. M., Epskamp, S., & Wicherts, J. M. (2015). The prevalence of statistical reporting errors in psychology (1985–2013). Behavior Research Methods, 1–22. doi: 10.3758/s13428-015-0664-2 Chris Hartgerink’s blog post
- 20. [email protected] @chartgerink Open (to me) is... * redistributing resources * improving (e)quality Advice * All problems nested in a bigger problem; what’s the bigger problem for what you’re trying to address? * Personal, empathetic growth is key to growing more open (email if you’d like to chat) Working on… * creating an *open* scholarly communication infrastructure that makes research fun * disregarding stakeholder engagement (promotes the already powerful players)
- 21. 6 ContentMine Fellows for 6 months
- 22. Alexandra Bannach-Brown Edinburgh, Neuroscience Problem: huge body of works in animal studies about depressions. systematic review is the main approach for getting insight. Wants: identify papers in systematic review of depressive behaviour in animals. What drugs, what methods, what outcomes and signs/phenotypes. Use outcomes for document clustering. and expedite scientific advances." Corpus: 70.000 Papers
- 23. Alexandra Bannach-Brown To me, Open Science is… • ensuring your work transparent & reproducible • ensuring the knowledge you create is open & accessible Research Fellow in Automation of Evidence Synthesis Centre for Research in Evidence-Based Practice, Bond University ContentMine Fellow 2016: Animal Models of Depression AMI Dictionaries: Anti-depressants Automation Tools for Medical & Biomedical Reviews • SyRF (CAMARADES, Edinburgh) – SyRF.org.uk • Systematic Review Accelerator (CREBP, Bond Uni) CREBP-SRA.com & Github Contact: Twitter: @ABannachBrown Email: [email protected]
- 24. Lars Willighagen 15 years old NL Wants: extract data about conifers (relations to chemicals, height etc.) Outcome: database with webpage containing conifer properties Table Facts Visualiser DEMO Card DEMO Word Cloud „ I applied to this fellowship to learn new things and combine the ContentMine with two previous projects I never got to finish, and I got really excited by the idea and the ContentMine at large.“
- 25. Daniel Mietchen, Wikimedia https://en.wikipedia.org/wiki/Wikipedia:Wikipedia_Signpost/2011-07-25/Wikimedian_in_Residence_interview https://en.wikipedia.org/wiki/User:Daniel_Mietchen https://commons.wikipedia.org/wiki/User:Daniel_Mietchen Thesupermat - Own work Intervention de Daniel Mietchen aux Rencontres Wikimedia France. CC BY-SA 3.0 File:Rencontres Wikimedia France - Intervention de Daniel Mietchen.JPG Created: 4 December 2010
- 27. Wikidata links to 100+ Wikipedias en https://www.wikidata.org/wiki/Q3241451 childhood obesit https://www.wikidata.org/wiki/Q12174 obesity https://en.wikipedia.org/wiki/Childhood_obesity CC BY-SA Wikidata.org
- 28. Wikidata has over 50 million items of public metadata
- 29. “childhood Obesity” dictionary created from Wikipedia page • <dictionary title="childobese" name="childhobese"> • <desc>hand-created from Wikipedia https://en.wikipedia.org/wiki/Childhood_obesity PeterMR 2019- 03</desc> • <entry wikipedia=” https://en.wikipedia.org/wiki/Abdominal_surgery " term="Abdominal surgery" name="Abdominal surgery"/> • <entry wikipedia=”https://en.wikipedia.org/wiki/Adipose_tissue " term="Adipose tissue" name="Adipose tissue"/> • <entry wikipedia=”https://en.wikipedia.org/wiki/American_Obesity_Associ ation " term="American Obesity Association" name="American Obesity Association"/> … 100 more …
- 30. #DontLeaveItToGoogle Peter Kraker An Open Infrastructure is essential Or… ...Mendeley …WebOfScience …Scopus … Elsevier …Springer-Nature https://elephantinthelab.org/google-and-research-data/ Repositories: BASE , EuropePMC Free articles: Unpaywall Preprints: arXiv, bioRxiv, medRxiv Indexing: Crossref Scraping: getpapers, quickscrape Text analysis: OpenKnowledgeMaps, ami
- 31. OpenKnowledgeMaps • Peter Kraker[1] and colleagues, Austria • Open; protected against takeover by Austrian Law • Unsupervised machine learning on Open Access • Easy to use • Vibrant community • http://openknowledgemaps.org • [1] Panton Fellow
- 32. Peter Kraker and team http://openknowledgemaps.org Let’s ask for “childhood obesity prevention”
- 33. ContentMine software • Open Source, platform-independent, free • Tutorial material [1] • getpapers search and download [2] • ami dictionary-based search and analysis [3] • Explicit use of Wikipedia and Wikidata [1] (http://github.com/petermr/tigr2ess ) example shows crops in India but easily extensible to most disciplines [2] Rik Smith-Unna; http://github.com/ContentMine/getpapers, Node.js [2] Peter Murray-Rust; http://github.com/ContentMine/normami , Java7
- 34. catalogue getpapers query Daily Crawl EuPMC, arXiv CORE , HAL, (UNIV repos) ToC services PDF HTML DOC ePUB TeX XML PNG EPS CSV XLSURLs DOIs crawl quickscrape norma Normalizer Structurer Semantic Tagger Text Data Figures ami UNIV Repos search Lookup CONTENT MINING Chem Phylo Trials Crystal Plants COMMUNITY plugins Visualization and Analysis PloSONE, BMC, peerJ… Nature, IEEE, Elsevier… Publisher Sites scrapers queries taggers abstract methods references Captioned Figures Fig. 1 HTML tables 30, 000 pages/day Semantic ScholarlyHTML Facts CONTENTMINE Complete OPEN Platform for Mining Scientific Literature
- 35. getpapers Rik Smith-Unna To install the software see http://github.com/petermr/tigr2ess
- 36. ami for ”childhood obesity prevention” • Builtin dictionaries for: – Country – Funders – Drugs – Disease • Bespoke dictionary for Wikipedia “childhood obesity” (see aboveO • Word frequency (cloud) Components at http://github.com/petermr/galwayecr
- 37. Wikipedia +Wikidata for “childhood obesity”
- 38. ami-search -p . -–dictionary country disease drugs $PMR/childobese
- 39. Ami finds: embedded words and phrases All blue links are to active documents <- original papers. Wikipedia / wikidata ->
- 40. Co-occurrence of phrases in the document (not necessarily semantically linked)
- 41. Chemicals (INN) and obesity terms Mentioned In paper (not necessarily semantically linked)
- 42. Open Thoughts for ECRs An Open presence: Blog, reviews, software, data resources, advocacy, activism. Think globally. Redalyc (Latam), africarxiv Collaborate. Go to Open Science meetings, hacks Imagine futures. Have fun Other options than becoming an academic? Believe in yourself! Resources : http://sparcopen.org is probably the best starting place. Also the ECRs mentioned are very approachable.