COS goes FOSS
The sorry state of scientific publishing and how we could move to an open and resilient infrastructure

• reproducibility crisis
• only a small fraction of primary data available
• even smaller fraction of code
• open access, if exists, is very expensive, maintains the profit of legacy publishers
• antiquated, dysfunctional system that rewards prestige/hype over quality/integrity
• scholarly workflows use non-professional, closed-source software (MS, Adobe, Prism etc.)
• sharing, integration, automation and collaboration difficult (who can use Git?)
• final product of years of research: pdf file (1990s tech) behind a paywall
• data, code, text not searchable, reuseable, discoverable

• study to assess the effectiveness of code sharing policy
• random sample of 204 Science papers
• artifacts from 44%
• reproduce the findings for 26%

• Typical responses:

• “The data files remains our property and are not deposited for free access.”

• “When you approach a PI for the source codes and raw data, you better explain who you are, whom you work for, why you need the data and what you are going to do with it.”

• “I have to say that this is a very unusual request without any explanation! Please ask your supervisor to send me an email with a detailed, and I mean detailed, explanation.”

• “We do not typically share our internal data or code with people outside our collaboration.”

The structures of MsbA (purple) and Sav1866 (green) overlap little (left) until MsbA is inverted (right).

• buggy non-published program flipped two columns, inverting electron density
• program was inherited from another lab
• mistake repeated in several papers
• led to five retractions (three in Science)

• The cumulative impact of reporting and citation biases on the evidence base for antidepressants

• 50% of randomized controlled trials have never been published
• trials with statistically significant findings are more likely to be published
• citation bias -> studies with positive results receive more citations than negative studies

• The Reproducibility Project: Cancer Biology (RP:CB)
• failure to replicate 30 of 53 papers published by Science, Nature, and Cell from 2010 to 2012
• credibility of preclinical cancer biology?
• need for authors to share more details of their experiments
• vague protocols and uncooperative authors
• one-third of contacted authors declined or did not respond

• steep increase in retractions
• monitoring retractions: http://retractionwatch.com
• majority of all retractions is due to misconduct

Brembs

• chasing ‘stories’ and IF instead of integrity, hypothesis-testing, rigour, openness
• under the spell of glamour journals
• “If I get this result, this will be a Nature paper!”
• reporting bias (only positive results are reported)
• low statistical power (a p-value of 0.05 brings only 50% reproducibility)
• in worst cases data are fabricated

• highly skewed distributions
• distorted by outliers (see Nature)
• journal IF comparisons: comparison of means of two populations
• allowed only if distributions follow a normal distributions!
• simple ranking by mean is incorrect
• median would be better or a more complex statistical test (e.g. Kruskal–Wallis test)

• fitting a more complex exponential function to the citation data
• a journal impact factor can be calculated from the parameters of the fit
• Science JIF = 25.3 instead of the reported 34
• Nature JIF = 26.8 instead of the reported 37
• a few highly cited papers have a substantial effect on the mean, but less on the exponential function

• no association between statistical power and journal IF

• ‘journal rank’ is a strong predictor of the rate of retractions
• (also of Excel errors :))

Robert Maxwell in 1985. Photograph: Terry O’Neill/Hulton/Getty

• worldwide sales > USD 19 billion
• dominated by five large publishing houses: Elsevier, Black & Wiley, Taylor & Francis, Springer Nature and SAGE
• Elsevier has a profit margin around 40 % (higher than Microsoft, Google and Coca Cola)
• about USD 6 billion per year goes to profits = 2 CERNs/year
• APCs can be as high as $12,000

• an oligopoly of legacy publishers
• Elsevierstan
• Wileystan
• Taylorfrancistan
• Springerstan
• …

• workflow monopoly
• tools to cover the entire academic workflow (e.g. Elsevier)
• high risk of vendor lock-in
• totalizing, homogenising workflows, extractive of research communities

• scientists provide content for free
• scientist peer review for free
• scientists buy over-priced product as APC, subscription or ‘transformative’ deals
• publisher (now ‘data analysis company’) sells entire workflows to scientists
• publishers tracks scientists on its platforms
• sells the data to the employers (e.g. quality assessment) or third parties

Watchthem ‘Data gathering is an essential process, and most companies use it for their success.’

• tracking site visits via authentication systems
• detailed real-time data on the information behaviour of individuals and institution
• page visits, accesses, clicks, downloads, etc.
• assembly of granular profiles of academic behaviour
• without user consent
• selling the data, e.g. RELX – the parent company of Elsevier – establishes PURE at universities around the world
• to provide ‘insights’ into the entire research cycle
• RELX now also sells data to ICE…

• journal publishing system is fundamentally broken
• a legacy system that prevents science from meeting its true potential for society
• about 40,000 journals
• public trust problem
• science publishing must be built anew
• illusion of truth and finality
• artificial scarcity
• narrow formats
• incomplete information
• prestige and journal-rank fallacies

• a comprehensive resource for protein sequence and annotation data
• entries uniquely identified by a stable URL
• rich metadata that is both human-readable and machine-readable
• shared vocabularies and ontologies
• interlinking with more than 150 different databases
• © 2002 – 2026 UniProt consortium

- https://www.budapestopenaccessinitiative.org/
February 14, 2002
Budapest, Hungary

• removing barriers to literature
• free and unrestricted online availability = open access
• the costs of providing OA to literature are far lower than the costs of traditional publishing (printed press)
• opportunity to save money and expand the scope of dissemination
• recommendations: self-archiving (I.) and a new generation of open-access journals (II.)

• in 2011, 24 y old internet hacktivist Aaron Swartz was arrested at MIT
• he downloaded several million articles from an online archive (JSTOR)
• legal troubles
• Swartz committed suicide in 2013
• the internet was created so that scientists could communicate their research results with each other
• billions of videos of cats for free, research results behind paywalls
• (GPT-4o showed an 82% recognition rate for paywalled content)

• Diamond: OA journal without an APC
• Green: not openly accessible from the publisher website but a free copy is accessible via a repository
• Gold: OA journal with APC (profits can remain high!)
• Hybrid: some papers OA others not (profits can remain high!)
• Bronze: free to read, no identifiable licence
• ‘transformative agreements’

• Public institutions (universities, libraries, funders etc.) should take back control of the digital scholarly infrastructure
• create conditions of open competition for private sector (not oligopoly of a few publishers)
• control data, text, code, citation metrics, scholarly workflows, databases, standards etc.
• cancel all subscriptions and use money to fund databases, libraries, publishing etc.
• support initiatives like OpenAIRE
• build community, the commons -> publishing is community, care

• the San Francisco Declaration on Research Assessment
• eliminate the use of journal-based metrics (IF) in funding, appointment, and promotion decisions
• assess research on its own merits rather than based on the journal
• capitalize on the opportunities provided by online publication (e.g. relax limits on the number of words, figures, and references)

Harold Varnus
“We need to get away from false metrics and return to the task of looking at our colleagues’ work closely.”

• we believe the most important work is published in so-called ‘high-impact’ journals
• ceding judgments to journal editors
• we have to eliminate the current situation in which the fate of researcher and their trainees depends on publishing in certain journals

• funded by HHMI, Wellcome Trust, MPS, K&A Wallenberg Foundation

• https://elifesciences.org/
• The eLife process has five steps:
  • Submission or transfer of a preprint from bioRxiv
  • Peer review (eLife editors - who are all active researchers - discuss new submissions and decide which will be peer reviewed)
  • Publication of Reviewed Preprint
  • Publication of revised version
  • Publication of Version of Record
  • papers published together with eLife Assessment
  • eLife has no IF! (good!!)

• Institutions should host their GitLab server for code
• (GitLab is a database-backed web application running git)
• (git is a Distributed Version Control Systems)
• servers should be federated
• European (-> world-wide) network of research/education institutions and libraries
• code shared upon publication in a permanent repo with DOI

• Permanent repository for data, text and code
• integration with GitHub
• version control
• Safe — your research is stored safely for the future in CERN’s Data Centre for as long as CERN exists
• citeable
• usage statistics

• a federated infrastructure
• run by public institutions (universities, libraries etc.)
• for communication (microblogging = Mastodon)
• for code (GitLab), data (e.g. Omero), text (preprint servers) etc.
• taking back control of scholarly infrastructure

• plan for a federated scholarly information network
• a system that cannot be taken over by corporations
• designed redundantly
• open standards
• “a decentralized, resilient, evolvable network that is interconnected by open standards and open-source norms under the governance of the scholarly community”

• open access publishing venue for EC-funded researchers
• no author or reader fees
• Diamond (but authors need to be EC funded)
• maintained by the European Commission
• Wellcome Open Research (https://wellcomeopenresearch.org/) is similar, maintained by the Wellcome Trust
• but too centralised and no community behind it

• open up access methods, results, publications, data, software, materials, tools and peer reviews
• standard tender process held regularly
• no lock-in with a single publisher
• regular procurement processes, no monopoly, fair prices

https://open-science-cloud.ec.europa.eu/

• Available Services:
  • File Sync & Share
  • Interactive Notebooks
  • Large File Transfer
  • Virtual Machines
  • Cloud Container Platform
  • Bulk Data Transfer

• individual labs can change behaviour
• my lab has completely switched to publishing preprints and in OA-only not-for-profit journals
• raise your voice in hiring/promotion committees for DORA principles