Ad
More Related Content
What's hot (20)
Similar to Tutorial contributing to nf-core (20)
Ad
Recently uploaded (20)
Ad
Tutorial contributing to nf-core
- 1. Phil Ewels, Gisela Gabernet, Rike Hanssen Contributing to nf-core Last updated March 2022
- 2. Documentation Slack workspace Hackathons Twitter updates Starter template Code guidelines Helper tools CI code linting and tests Stable pipelines Centralised configs Download for offline use List and update pipelines What is nf-core? Participate Develop Deploy
- 3. Contribution guidelines Cooperate, don’t duplicate One pipeline per analysis type, contribute by adding new tools, new features… Develop with the community Join slack, communicate and contribute together to a pipeline
- 4. https://nf-co.re/usage/tutorials/nf_core_contributing_overview 2 Creating 3 Testing 4 Modules 5 Releasing 1 Installation Tutorial
- 5. Installation
- 6. Helper tools pip install nf-core conda install -c bioconda nf-core
- 10. nf-core create
- 12. Template
- 13. Template
- 14. Template ├── CHANGELOG.md ├── CITATIONS.md ├── CODE_OF_CONDUCT.md ├── LICENSE ├── README.md ├── assets │ ├── ... ├── bin │ └── ... ├── conf │ ├── ... ├── docs │ ├── ... ├── lib │ ├── ... ├── main.nf ├── modules │ ├── local │ └── nf-core ├── modules.json ├── nextflow.config ├── nextflow_schema.json ├── subworkflows │ └── local └── workflows └── pipeline.nf • 12 directories • 32 files • + > 40 hidden files Template
- 15. ├── CHANGELOG.md ├── CITATIONS.md ├── CODE_OF_CONDUCT.md ├── LICENSE ├── README.md ├── assets │ ├── ... ├── bin │ └── ... ├── conf │ ├── ... ├── docs │ ├── ... ├── lib │ ├── ... ├── main.nf ├── modules │ ├── local │ └── nf-core ├── modules.json ├── nextflow.config ├── nextflow_schema.json ├── subworkflows │ └── local └── workflows └── pipeline.nf Nextflow code DSL2 brings modularity A pipeline can contain: • modules • subworkflows • workflows Template
- 16. ├── CHANGELOG.md ├── CITATIONS.md ├── CODE_OF_CONDUCT.md ├── LICENSE ├── README.md ├── assets │ ├── ... ├── bin │ └── ... ├── conf │ ├── ... ├── docs │ ├── ... ├── lib │ ├── NfcoreSchema.groovy │ ├── NfcoreTemplate.groovy │ ├── Utils.groovy │ ├── pipeline.groovy │ ├── WorkflowMain.groovy │ └── nfcore_external_java_deps.jar ├── main.nf ├── modules ├── modules.json ├── nextflow.config ├── nextflow_schema.json ├── subworkflows └── workflows Groovy classes and functions used by the pipeline Template
- 17. ├── CHANGELOG.md ├── CITATIONS.md ├── CODE_OF_CONDUCT.md ├── LICENSE ├── README.md ├── assets │ ├── email_template.html │ ├── email_template.txt │ ├── multiqc_config.yaml │ ├── nf-core-pipeline_logo.png │ ├── samplesheet.csv │ ├── schema_input.json │ └── sendmail_template.txt ├── bin │ └── check_samplesheet.py ├── conf │ ├── ... ├── docs │ ├── ... ├── lib ├── main.nf ├── modules ├── modules.json ├── nextflow.config ├── nextflow_schema.json ├── subworkflows └── workflows bin/ • Executable scripts • Added to the path by Nextflow assets/ • Email template • MultiQC config • Pipeline logo • Input file schema Template
- 18. ├── CHANGELOG.md ├── CITATIONS.md ├── CODE_OF_CONDUCT.md ├── LICENSE ├── README.md ├── assets │ ├── ... ├── bin │ └── ... ├── conf │ ├── ... ├── docs │ ├── README.md │ ├── images │ ├── output.md │ └── usage.md ├── lib ├── main.nf ├── modules ├── modules.json ├── nextflow.config ├── nextflow_schema.json ├── subworkflows └── workflows Documentation • rendered on the website Template
- 19. ├── CHANGELOG.md ├── CITATIONS.md ├── CODE_OF_CONDUCT.md ├── LICENSE ├── README.md ├── assets │ ├── ... ├── bin │ └── ... ├── conf │ ├── ... ├── docs │ ├── README.md │ ├── images │ ├── output.md │ └── usage.md ├── lib ├── main.nf ├── modules ├── modules.json ├── nextflow.config ├── nextflow_schema.json ├── subworkflows └── workflows Parameter documentation • rendered from json nf-core schema build Template
- 20. ├── CHANGELOG.md ├── CITATIONS.md ├── CODE_OF_CONDUCT.md ├── LICENSE ├── README.md ├── assets │ ├── ... ├── bin │ └── ... ├── conf │ ├── base.config │ ├── igenomes.config │ ├── modules.config │ ├── test.config │ └── test_full.config ├── docs │ ├── ... ├── lib ├── main.nf ├── modules ├── modules.json ├── nextflow.config ├── nextflow_schema.json ├── subworkflows └── workflows Configuration nextflow.config imports: + Template
- 21. ├── CHANGELOG.md ├── CITATIONS.md ├── CODE_OF_CONDUCT.md ├── LICENSE ├── README.md ├── assets │ ├── ... ├── bin │ └── ... ├── conf │ ├── base.config │ ├── igenomes.config │ ├── modules.config │ ├── test.config │ └── test_full.config ├── docs │ ├── ... ├── lib ├── main.nf ├── modules ├── modules.json ├── nextflow.config ├── nextflow_schema.json ├── subworkflows └── workflows Configuration nextflow.config imports: Test profile Full test profile Run with GitHub actions: Template
- 22. ├── .editorconfig ├── .github │ ├── .dockstore.yml │ ├── CONTRIBUTING.md │ ├── ISSUE_TEMPLATE │ │ ├── bug_report.yml │ │ ├── config.yml │ │ └── feature_request.yml │ ├── PULL_REQUEST_TEMPLATE.md │ └── workflows │ ├── awsfulltest.yml │ ├── awstest.yml │ ├── branch.yml │ ├── ci.yml │ ├── linting.yml │ └── linting_comment.yml ├── .gitignore ├── .markdownlint.yml the hidden files: GitHub actions workflows • AWS full pipeline tests • AWS pipeline tests • Branch protection • CI tests • Linting tests • Linting comment Template
- 23. Forks & branches
- 24. Forks & branches Branch Merge
- 26. Forks & branches Pull Request Merge
- 28. dev master TEMPLATE Forks & branches Stable releases only
- 29. dev master TEMPLATE Forks & branches Used by @nf-core-bot to keep in sync
- 30. dev master TEMPLATE Forks & branches All development code (and all pull requests)
- 31. Forks & branches PRs must be against the dev branch Changelog must be updated PRs need to be reviewed REMEMBER!
- 34. Lint tests nf-core lint . https://nf-co.re/errors
- 35. nextflow run nf-core/rnaseq -profile test,docker Test data
- 36. Test data Make a new branch for your pipeline Add a really (really!) tiny dataset Set up the test config to use raw URLs
- 37. Exercises 2, 3 Exercise 2 (creating pipelines) • Make a new pipeline using the template • Update the readme file to fill in the TODO statements • Add a new process to the pipeline in main.nf • Add the new software dependencies from this process in to environment.yaml Exercise 3 (testing pipelines) • Run nf-core lint on your pipeline and make note of any test warnings / failures • Read up on one or two of the linting rules on the nf-core website and see if you can fix some. • Take a look at conf/test.config and switch the test data for another dataset on nf-core/test_data
- 38. nf-core modules
- 41. modules
- 44. Exercise 4 Exercise 4 (modules) • Use the pipeline you created in Exercise 2 and add an already existing nf-core module (e.g. trimgalore). • Connect the module to the main pipeline workflow.
- 46. First release https://nf-co.re/developers/adding_pipelines Tell us about it: #new-pipelines Join the @nf-core organisation Fork your repo to nf-core Set up Zenodo
- 47. First release https://nf-co.re/developers/adding_pipelines Community review: "Pseudo PR" Open PR from dev to master Make sure that all tests are passing Create release on GitHub
- 48. Version numbers Semantic versioning: 1.4.3 Major - results not backwards compatible Minor - new feature(s) Patch - bug fix nf-core bump-version 1.4.3 nf-core bump-version --nextflow 19.07.0
- 50. Template updates
- 51. Exercise 5 Exercise 5 (releasing pipelines) • Use nf-core bump-version to update the required version of Nextflow in your pipeline • Bump your pipeline's version to 1.0, ready for its first release! • Make sure that you're signed up to the nf-core slack (https://nf-co.re/join) and drop us a line about your latest and greatest pipeline plans! • Ask to be a member of the nf-core GitHub organisation by commenting on this GitHub issue: https://github.com/nf-core/nf-co.re/issues/3 • If you're a twitter user, make sure to follow the @nf_core twitter account