Skip to content

Conversation

@pinin4fjords
Copy link
Collaborator

@pinin4fjords pinin4fjords commented Jan 9, 2026

Summary

Convert the plugin development side quest into a full Hello Plugins training module with 7 progressive parts:

  • Part 1: Plugin Basics - discovering and using existing plugins
  • Part 2: Create Project - scaffolding with nextflow plugin create
  • Part 3: Custom Functions - implementing @function methods
  • Part 4: Build and Test - Gradle workflow and Spock testing
  • Part 5: Observers - TraceObserver for workflow lifecycle events
  • Part 6: Configuration - session.config.navigate() and ConfigScope
  • Part 7: Distribution - public registry and internal hosting

Audience

  • Parts 1-2: All Nextflow users (using existing plugins)
  • Parts 3-7: Developers (creating custom plugins)

Contents

Documentation (docs/hello_plugins/):

  • 7 lesson files (Parts 1-7)
  • Orientation, index, summary, next steps, survey
  • Supporting images

Exercise files (hello-plugins/):

  • Starting files for hands-on exercises
  • Progressive solutions for each part (1-6)

Updates for Nextflow 25.10.2

  • Use Greeting* class naming (not NfGreeting*)
  • Use io.nextflow.nextflow-plugin version 1.0.0-beta.10
  • Set nextflowVersion = '25.10.0' (required for config.spec.* classes)
  • Use nextflow.config.spec.* imports (not nextflow.config.dsl.*)
  • Use @ScopeName + implements ConfigScope (not @ConfigScope annotation)

Test plan

  • Preview locally with MkDocs
  • Heading numbering passes validation
  • Tutorial walkthrough tested for Parts 3-7 with Nextflow 25.10.2
  • All solutions build and run correctly

🤖 Generated with Claude Code

Add comprehensive training material for Nextflow plugin development covering:
- Using existing plugins (discovery, installation, configuration)
- Creating plugins with custom functions, operators, and trace observers
- Building, testing, and distributing plugins

Includes:
- Full tutorial documentation with Before/After code comparisons
- Starting files for hands-on exercises
- Solution files including complete nf-greeting plugin
- Supporting images (registry screenshot, test report)

Co-Authored-By: Claude Opus 4.5 <[email protected]>
@netlify
Copy link

netlify bot commented Jan 9, 2026

Deploy Preview for nextflow-training ready!

Name Link
🔨 Latest commit dba78e0
🔍 Latest deploy log https://app.netlify.com/projects/nextflow-training/deploys/69806ffa75eeae0008747097
😎 Deploy Preview https://deploy-preview-761--nextflow-training.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@github-actions
Copy link
Contributor

github-actions bot commented Jan 9, 2026

Nextflow linting complete!

❌ 1 files had 1 errors
✅ 182 files had no errors
🔧 132 files would be changed by auto-formatting

💡 Tip: Click filename locations to go directly to that code.

View all 1 issues
Type Location Message
Error hello-nf-core/solutions/core-hello-part2/nextflow.config:154:17 Invalid include source: '/home/runner/work/training/training/hello-nf-core/solutions/core-hello-part2/conf/test_full.config'
View formatting changes
FileDiff
hello-nextflow/solutions/1-hello-world/hello-world-2.nf
View
@@ -4,7 +4,6 @@
  * Use echo to print 'Hello World!' to a file
  */
 process sayHello {
-
     output:
     path 'output.txt'
 
hello-nextflow/solutions/1-hello-world/hello-world-3.nf
View
@@ -4,7 +4,6 @@
  * Use echo to print 'Hello World!' to a file
  */
 process sayHello {
-
     input:
     val greeting
 
hello-nextflow/solutions/2-hello-channels/hello-channels-1.nf
View
@@ -4,7 +4,6 @@
  * Use echo to print 'Hello World!' to a file
  */
 process sayHello {
-
     input:
     val greeting
 
@@ -29,7 +28,7 @@ workflow {
     main:
     // create a channel for inputs
     greeting_ch = channel.of('Hello Channels!')
-                        .view()
+        .view()
     // emit a greeting
... (truncated)
hello-nextflow/solutions/2-hello-channels/hello-channels-2.nf
View
@@ -4,7 +4,6 @@
  * Use echo to print 'Hello World!' to a file
  */
 process sayHello {
-
     input:
     val greeting
 
@@ -29,7 +28,7 @@ workflow {
     main:
     // create a channel for inputs
     greeting_ch = channel.of('Hello', 'Bonjour', 'Holà')
-                        .view()
+        .view()
     // emit a greeting
... (truncated)
hello-nextflow/solutions/2-hello-channels/hello-channels-3.nf
View
@@ -4,7 +4,6 @@
  * Use echo to print 'Hello World!' to a file
  */
 process sayHello {
-
     input:
     val greeting
 
@@ -28,12 +27,12 @@ workflow {
 
     main:
     // declare an array of input greetings
-    greetings_array = ['Hello','Bonjour','Holà']
+    greetings_array = ['Hello', 'Bonjour', 'Holà']
     // create a channel for inputs
... (truncated)
hello-nextflow/solutions/2-hello-channels/hello-channels-4.nf
View
@@ -4,7 +4,6 @@
  * Use echo to print 'Hello World!' to a file
  */
 process sayHello {
-
     input:
     val greeting
 
@@ -29,11 +28,11 @@ workflow {
     main:
     // create a channel for inputs from a CSV file
     greeting_ch = channel.fromPath(params.input)
-                        .view { csv -> "Before splitCsv: $csv" }
-                        .splitCsv()
-                        .view { csv -> "After splitCsv: $csv" }
... (truncated)
hello-nextflow/solutions/3-hello-workflow/hello-workflow-1.nf
View
@@ -4,7 +4,6 @@
  * Use echo to print 'Hello World!' to a file
  */
 process sayHello {
-
     input:
     val greeting
 
@@ -21,7 +20,6 @@ process sayHello {
  * Use a text replacement tool to convert the greeting to uppercase
  */
 process convertToUpper {
-
     input:
     path input_file
... (truncated)
hello-nextflow/solutions/3-hello-workflow/hello-workflow-2.nf
View
@@ -4,7 +4,6 @@
  * Use echo to print 'Hello World!' to a file
  */
 process sayHello {
-
     input:
     val greeting
 
@@ -21,7 +20,6 @@ process sayHello {
  * Use a text replacement tool to convert the greeting to uppercase
  */
 process convertToUpper {
-
     input:
     path input_file
... (truncated)
hello-nextflow/solutions/3-hello-workflow/hello-workflow-3.nf
View
@@ -4,7 +4,6 @@
  * Use echo to print 'Hello World!' to a file
  */
 process sayHello {
-
     input:
     val greeting
 
@@ -21,7 +20,6 @@ process sayHello {
  * Use a text replacement tool to convert the greeting to uppercase
  */
 process convertToUpper {
-
     input:
     path input_file
... (truncated)
hello-nextflow/solutions/3-hello-workflow/hello-workflow-4.nf
View
@@ -4,7 +4,6 @@
  * Use echo to print 'Hello World!' to a file
  */
 process sayHello {
-
     input:
     val greeting
 
@@ -21,7 +20,6 @@ process sayHello {
  * Use a text replacement tool to convert the greeting to uppercase
  */
 process convertToUpper {
-
     input:
     path input_file
... (truncated)
hello-nextflow/solutions/4-hello-modules/hello-modules-2.nf
View
@@ -7,7 +7,6 @@ include { sayHello } from './modules/sayHello.nf'
  * Use a text replacement tool to convert the greeting to uppercase
  */
 process convertToUpper {
-
     input:
     path input_file
 
@@ -24,7 +23,6 @@ process convertToUpper {
  * Collect uppercase greetings into a single output file
  */
 process collectGreetings {
-
     input:
     path input_files
... (truncated)
hello-nextflow/solutions/4-hello-modules/hello-modules-3.nf
View
@@ -8,7 +8,6 @@ include { convertToUpper } from './modules/convertToUpper.nf'
  * Collect uppercase greetings into a single output file
  */
 process collectGreetings {
-
     input:
     path input_files
     val batch_name
@@ -38,8 +37,8 @@ workflow {
     main:
     // create a channel for inputs from a CSV file
     greeting_ch = channel.fromPath(params.input)
-                        .splitCsv()
-                        .map { line -> line[0] }
+        .splitCsv()
... (truncated)
hello-nextflow/solutions/4-hello-modules/hello-modules-4.nf
View
@@ -18,8 +18,8 @@ workflow {
     main:
     // create a channel for inputs from a CSV file
     greeting_ch = channel.fromPath(params.input)
-                        .splitCsv()
-                        .map { line -> line[0] }
+        .splitCsv()
+        .map { line -> line[0] }
     // emit a greeting
     sayHello(greeting_ch)
     // convert the greeting to uppercase
hello-nextflow/solutions/4-hello-modules/modules/collectGreetings.nf
View
@@ -4,7 +4,6 @@
  * Collect uppercase greetings into a single output file
  */
 process collectGreetings {
-
     input:
     path input_files
     val batch_name
hello-nextflow/solutions/4-hello-modules/modules/convertToUpper.nf
View
@@ -4,7 +4,6 @@
  * Use a text replacement tool to convert the greeting to uppercase
  */
 process convertToUpper {
-
     input:
     path input_file
 
hello-nextflow/solutions/4-hello-modules/modules/sayHello.nf
View
@@ -4,7 +4,6 @@
  * Use echo to print 'Hello World!' to a file
  */
 process sayHello {
-
     input:
     val greeting
 
hello-nextflow/solutions/5-hello-containers/hello-containers-2.nf
View
@@ -20,8 +20,8 @@ workflow {
     main:
     // create a channel for inputs from a CSV file
     greeting_ch = channel.fromPath(params.input)
-                        .splitCsv()
-                        .map { line -> line[0] }
+        .splitCsv()
+        .map { line -> line[0] }
     // emit a greeting
     sayHello(greeting_ch)
     // convert the greeting to uppercase
hello-nextflow/solutions/5-hello-containers/modules/collectGreetings.nf
View
@@ -2,7 +2,6 @@
  * Collect uppercase greetings into a single output file
  */
 process collectGreetings {
-
     input:
     path input_files
     val batch_name
hello-nextflow/solutions/5-hello-containers/modules/convertToUpper.nf
View
@@ -4,7 +4,6 @@
  * Use a text replacement tool to convert the greeting to uppercase
  */
 process convertToUpper {
-
     input:
     path input_file
 
hello-nextflow/solutions/5-hello-containers/modules/sayHello.nf
View
@@ -4,7 +4,6 @@
  * Use echo to print 'Hello World!' to a file
  */
 process sayHello {
-
     input:
     val greeting
 
hello-nextflow/solutions/6-hello-config/hello-config.nf
View
@@ -20,8 +20,8 @@ workflow {
     main:
     // create a channel for inputs from a CSV file
     greeting_ch = channel.fromPath(params.input)
-                        .splitCsv()
-                        .map { line -> line[0] }
+        .splitCsv()
+        .map { line -> line[0] }
     // emit a greeting
     sayHello(greeting_ch)
     // convert the greeting to uppercase
hello-nextflow/solutions/6-hello-config/modules/collectGreetings.nf
View
@@ -2,7 +2,6 @@
  * Collect uppercase greetings into a single output file
  */
 process collectGreetings {
-
     input:
     path input_files
     val batch_name
hello-nextflow/solutions/6-hello-config/modules/convertToUpper.nf
View
@@ -4,7 +4,6 @@
  * Use a text replacement tool to convert the greeting to uppercase
  */
 process convertToUpper {
-
     input:
     path input_file
 
hello-nextflow/solutions/6-hello-config/modules/sayHello.nf
View
@@ -4,7 +4,6 @@
  * Use echo to print 'Hello World!' to a file
  */
 process sayHello {
-
     input:
     val greeting
 
hello-nextflow/solutions/6-hello-config/nextflow.config
View
@@ -6,7 +6,7 @@ conda.enabled = true
 */
 process {
     memory = 1.GB
-    withName: 'cowpy' {
+    withName: cowpy {
         memory = 2.GB
         cpus = 2
     }
@@ -35,7 +35,7 @@ profiles {
         process.resourceLimits = [
             memory: 750.GB,
             cpus: 200,
-            time: 30.d
+            time: 30.d,
... (truncated)
hello-nf-core/solutions/composable-hello/hello.nf
View
@@ -14,9 +14,7 @@ include { collectGreetings } from './modules/collectGreetings.nf'
 include { cowpy } from './modules/cowpy.nf'
 
 workflow HELLO {
-
     take:
-    // channel of greetings
     greeting_ch
 
     main:
hello-nf-core/solutions/composable-hello/main.nf
View
@@ -9,12 +9,12 @@ params.greeting = 'greetings.csv'
 workflow {
     // create a channel for inputs from a CSV file
     greeting_ch = channel.fromPath(params.greeting)
-                        .splitCsv()
-                        .map { line -> line[0] }
+        .splitCsv()
+        .map { line -> line[0] }
 
     // call the imported workflow on the channel of greetings
     HELLO(greeting_ch)
 
     // view the outputs emitted by the workflow
-    HELLO.out.view { output -> "Output: $output" }
+    HELLO.out.view { output -> "Output: ${output}" }
... (truncated)
hello-nf-core/solutions/composable-hello/modules/collectGreetings.nf
View
@@ -10,8 +10,8 @@ process collectGreetings {
     val batch_name
 
     output:
-    path "COLLECTED-${batch_name}-output.txt" , emit: outfile
-    val count_greetings , emit: count
+    path "COLLECTED-${batch_name}-output.txt", emit: outfile
+    val count_greetings, emit: count
 
     script:
     count_greetings = input_files.size()
hello-nf-core/solutions/composable-hello/modules/convertToUpper.nf
View
@@ -15,6 +15,6 @@ process convertToUpper {
 
     script:
     """
-    cat '$input_file' | tr '[a-z]' '[A-Z]' > 'UPPER-${input_file}'
+    cat '${input_file}' | tr '[a-z]' '[A-Z]' > 'UPPER-${input_file}'
     """
 }
hello-nf-core/solutions/composable-hello/modules/cowpy.nf
View
@@ -17,6 +17,6 @@ process cowpy {
 
     script:
     """
-    cat $input_file | cowpy -c "$character" > cowpy-${input_file}
+    cat ${input_file} | cowpy -c "${character}" > cowpy-${input_file}
     """
 }
hello-nf-core/solutions/composable-hello/modules/sayHello.nf
View
@@ -15,6 +15,6 @@ process sayHello {
 
     script:
     """
-    echo '$greeting' > '$greeting-output.txt'
+    echo '${greeting}' > '${greeting}-output.txt'
     """
 }
hello-nf-core/solutions/core-hello-part2/conf/base.config
View
@@ -11,13 +11,13 @@
 process {
 
     // TODO nf-core: Check the defaults for all processes
-    cpus   = { 1      * task.attempt }
-    memory = { 6.GB   * task.attempt }
-    time   = { 4.h    * task.attempt }
+    cpus = { 1 * task.attempt }
+    memory = { 6.GB * task.attempt }
+    time = { 4.h * task.attempt }
 
     errorStrategy = { task.exitStatus in ((130..145) + 104 + 175) ? 'retry' : 'finish' }
-    maxRetries    = 1
-    maxErrors     = '-1'
+    maxRetries = 1
... (truncated)
hello-nf-core/solutions/core-hello-part2/conf/modules.config
View
@@ -15,7 +15,6 @@ process {
     publishDir = [
         path: { "${params.outdir}/${task.process.tokenize(':')[-1].tokenize('_')[0].toLowerCase()}" },
         mode: params.publish_dir_mode,
-        saveAs: { filename -> filename.equals('versions.yml') ? null : filename }
+        saveAs: { filename -> filename.equals('versions.yml') ? null : filename },
     ]
-
 }
hello-nf-core/solutions/core-hello-part2/conf/test.config
View
@@ -14,18 +14,18 @@ process {
     resourceLimits = [
         cpus: 2,
         memory: '4.GB',
-        time: '1.h'
+        time: '1.h',
     ]
 }
 
 params {
-    config_profile_name        = 'Test profile'
+    config_profile_name = 'Test profile'
     config_profile_description = 'Minimal test dataset to check pipeline function'
 
     // Input data
... (truncated)
hello-nf-core/solutions/core-hello-part2/main.nf
View
@@ -13,9 +13,9 @@
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 */
 
-include { HELLO  } from './workflows/hello'
+include { HELLO } from './workflows/hello'
 include { PIPELINE_INITIALISATION } from './subworkflows/local/utils_nfcore_hello_pipeline'
-include { PIPELINE_COMPLETION     } from './subworkflows/local/utils_nfcore_hello_pipeline'
+include { PIPELINE_COMPLETION } from './subworkflows/local/utils_nfcore_hello_pipeline'
 /*
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     NAMED WORKFLOWS FOR PIPELINE
@@ -26,7 +26,6 @@ include { PIPELINE_COMPLETION     } from './subworkflows/local/utils_nfcore_hell
 // WORKFLOW: Run main analysis pipeline depending on type of input
 //
... (truncated)
hello-nf-core/solutions/core-hello-part2/modules/local/collectGreetings.nf
View
@@ -10,8 +10,8 @@ process collectGreetings {
     val batch_name
 
     output:
-    path "COLLECTED-${batch_name}-output.txt" , emit: outfile
-    val count_greetings , emit: count
+    path "COLLECTED-${batch_name}-output.txt", emit: outfile
+    val count_greetings, emit: count
 
     script:
     count_greetings = input_files.size()
hello-nf-core/solutions/core-hello-part2/modules/local/convertToUpper.nf
View
@@ -15,6 +15,6 @@ process convertToUpper {
 
     script:
     """
-    cat '$input_file' | tr '[a-z]' '[A-Z]' > 'UPPER-${input_file}'
+    cat '${input_file}' | tr '[a-z]' '[A-Z]' > 'UPPER-${input_file}'
     """
 }
hello-nf-core/solutions/core-hello-part2/modules/local/cowpy.nf
View
@@ -17,6 +17,6 @@ process cowpy {
 
     script:
     """
-    cat $input_file | cowpy -c "$character" > cowpy-${input_file}
+    cat ${input_file} | cowpy -c "${character}" > cowpy-${input_file}
     """
 }
hello-nf-core/solutions/core-hello-part2/modules/local/sayHello.nf
View
@@ -15,6 +15,6 @@ process sayHello {
 
     script:
     """
-    echo '$greeting' > '$greeting-output.txt'
+    echo '${greeting}' > '${greeting}-output.txt'
     """
 }
hello-nf-core/solutions/core-hello-part2/subworkflows/local/utils_nfcore_hello_pipeline/main.nf
View
@@ -8,13 +8,13 @@
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 */
 
-include { UTILS_NFSCHEMA_PLUGIN     } from '../../nf-core/utils_nfschema_plugin'
-include { paramsSummaryMap          } from 'plugin/nf-schema'
-include { samplesheetToList         } from 'plugin/nf-schema'
-include { paramsHelp                } from 'plugin/nf-schema'
-include { completionSummary         } from '../../nf-core/utils_nfcore_pipeline'
-include { UTILS_NFCORE_PIPELINE     } from '../../nf-core/utils_nfcore_pipeline'
-include { UTILS_NEXTFLOW_PIPELINE   } from '../../nf-core/utils_nextflow_pipeline'
+include { UTILS_NFSCHEMA_PLUGIN } from '../../nf-core/utils_nfschema_plugin'
+include { paramsSummaryMap } from 'plugin/nf-schema'
+include { samplesheetToList } from 'plugin/nf-schema'
+include { paramsHelp } from 'plugin/nf-schema'
... (truncated)
hello-nf-core/solutions/core-hello-part2/subworkflows/nf-core/utils_nextflow_pipeline/main.nf
View
@@ -10,9 +10,9 @@
 
 workflow UTILS_NEXTFLOW_PIPELINE {
     take:
-    print_version        // boolean: print version
-    dump_parameters      // boolean: dump parameters
-    outdir               //    path: base directory used to publish pipeline results
+    print_version // boolean: print version
+    dump_parameters // boolean: dump parameters
+    outdir //    path: base directory used to publish pipeline results
     check_conda_channels // boolean: check conda channels
 
     main:
@@ -72,10 +72,10 @@ def getWorkflowVersion() {
 //
... (truncated)
hello-nf-core/solutions/core-hello-part2/subworkflows/nf-core/utils_nextflow_pipeline/tests/nextflow.config
View
@@ -1,9 +1,9 @@
 manifest {
-    name            = 'nextflow_workflow'
-    author          = """nf-core"""
-    homePage        = 'https://127.0.0.1'
-    description     = """Dummy pipeline"""
+    name = 'nextflow_workflow'
+    author = """nf-core"""
+    homePage = 'https://127.0.0.1'
+    description = """Dummy pipeline"""
     nextflowVersion = '!>=23.04.0'
-    version         = '9.9.9'
-    doi             = 'https://doi.org/10.5281/zenodo.5070524'
+    version = '9.9.9'
+    doi = 'https://doi.org/10.5281/zenodo.5070524'
... (truncated)
hello-nf-core/solutions/core-hello-part2/subworkflows/nf-core/utils_nfcore_pipeline/main.nf
View
@@ -125,12 +125,12 @@ def paramsSummaryMultiqc(summary_params) {
         }
 
     def yaml_file_text = "id: '${workflow.manifest.name.replace('/', '-')}-summary'\n" as String
-    yaml_file_text     += "description: ' - this information is collected when the pipeline is started.'\n"
-    yaml_file_text     += "section_name: '${workflow.manifest.name} Workflow Summary'\n"
-    yaml_file_text     += "section_href: 'https://github.com/${workflow.manifest.name}'\n"
-    yaml_file_text     += "plot_type: 'html'\n"
-    yaml_file_text     += "data: |\n"
-    yaml_file_text     += "${summary_section}"
+    yaml_file_text += "description: ' - this information is collected when the pipeline is started.'\n"
+    yaml_file_text += "section_name: '${workflow.manifest.name} Workflow Summary'\n"
+    yaml_file_text += "section_href: 'https://github.com/${workflow.manifest.name}'\n"
+    yaml_file_text += "plot_type: 'html'\n"
+    yaml_file_text += "data: |\n"
... (truncated)
hello-nf-core/solutions/core-hello-part2/subworkflows/nf-core/utils_nfcore_pipeline/tests/nextflow.config
View
@@ -1,9 +1,9 @@
 manifest {
-    name            = 'nextflow_workflow'
-    author          = """nf-core"""
-    homePage        = 'https://127.0.0.1'
-    description     = """Dummy pipeline"""
-    nextflowVersion  = '!>=23.04.0'
-    version         = '9.9.9'
-    doi             = 'https://doi.org/10.5281/zenodo.5070524'
+    name = 'nextflow_workflow'
+    author = """nf-core"""
+    homePage = 'https://127.0.0.1'
+    description = """Dummy pipeline"""
+    nextflowVersion = '!>=23.04.0'
+    version = '9.9.9'
... (truncated)
hello-nf-core/solutions/core-hello-part2/subworkflows/nf-core/utils_nfschema_plugin/main.nf(truncated)
hello-nf-core/solutions/core-hello-part2/workflows/hello.nf(truncated)
hello-nf-core/solutions/core-hello-part3/conf/base.config(truncated)
hello-nf-core/solutions/core-hello-part3/conf/modules.config(truncated)
hello-nf-core/solutions/core-hello-part3/conf/test.config(truncated)
hello-nf-core/solutions/core-hello-part3/conf/test_full.config(truncated)
hello-nf-core/solutions/core-hello-part3/main.nf(truncated)
hello-nf-core/solutions/core-hello-part3/modules/local/convertToUpper.nf(truncated)
hello-nf-core/solutions/core-hello-part3/modules/local/cowpy.nf(truncated)
hello-nf-core/solutions/core-hello-part3/modules/local/sayHello.nf(truncated)
hello-nf-core/solutions/core-hello-part3/modules/nf-core/cat/cat/main.nf(truncated)
hello-nf-core/solutions/core-hello-part3/modules/nf-core/cat/cat/tests/nextflow_unzipped_zipped.config(truncated)
hello-nf-core/solutions/core-hello-part3/modules/nf-core/cat/cat/tests/nextflow_zipped_unzipped.config(truncated)
hello-nf-core/solutions/core-hello-part3/nextflow.config(truncated)
hello-nf-core/solutions/core-hello-part3/subworkflows/local/utils_nfcore_hello_pipeline/main.nf(truncated)
hello-nf-core/solutions/core-hello-part3/subworkflows/nf-core/utils_nextflow_pipeline/main.nf(truncated)
hello-nf-core/solutions/core-hello-part3/subworkflows/nf-core/utils_nextflow_pipeline/tests/nextflow.config(truncated)
hello-nf-core/solutions/core-hello-part3/subworkflows/nf-core/utils_nfcore_pipeline/main.nf(truncated)
hello-nf-core/solutions/core-hello-part3/subworkflows/nf-core/utils_nfcore_pipeline/tests/nextflow.config(truncated)
hello-nf-core/solutions/core-hello-part3/subworkflows/nf-core/utils_nfschema_plugin/main.nf(truncated)
hello-nf-core/solutions/core-hello-part3/workflows/hello.nf(truncated)
hello-nf-core/solutions/core-hello-part4/conf/base.config(truncated)
hello-nf-core/solutions/core-hello-part4/conf/modules.config(truncated)
hello-nf-core/solutions/core-hello-part4/conf/test.config(truncated)
hello-nf-core/solutions/core-hello-part4/conf/test_full.config(truncated)
hello-nf-core/solutions/core-hello-part4/main.nf(truncated)
hello-nf-core/solutions/core-hello-part4/modules/local/convertToUpper.nf(truncated)
hello-nf-core/solutions/core-hello-part4/modules/local/cowpy.nf(truncated)
hello-nf-core/solutions/core-hello-part4/modules/local/cowpy/main.nf(truncated)
hello-nf-core/solutions/core-hello-part4/modules/local/sayHello.nf(truncated)
hello-nf-core/solutions/core-hello-part4/modules/nf-core/cat/cat/main.nf(truncated)
hello-nf-core/solutions/core-hello-part4/modules/nf-core/cat/cat/tests/nextflow_unzipped_zipped.config(truncated)
hello-nf-core/solutions/core-hello-part4/modules/nf-core/cat/cat/tests/nextflow_zipped_unzipped.config(truncated)
hello-nf-core/solutions/core-hello-part4/nextflow.config(truncated)
hello-nf-core/solutions/core-hello-part4/subworkflows/local/utils_nfcore_hello_pipeline/main.nf(truncated)
hello-nf-core/solutions/core-hello-part4/subworkflows/nf-core/utils_nextflow_pipeline/main.nf(truncated)
hello-nf-core/solutions/core-hello-part4/subworkflows/nf-core/utils_nextflow_pipeline/tests/nextflow.config(truncated)
hello-nf-core/solutions/core-hello-part4/subworkflows/nf-core/utils_nfcore_pipeline/main.nf(truncated)
hello-nf-core/solutions/core-hello-part4/subworkflows/nf-core/utils_nfcore_pipeline/tests/nextflow.config(truncated)
hello-nf-core/solutions/core-hello-part4/subworkflows/nf-core/utils_nfschema_plugin/main.nf(truncated)
hello-nf-core/solutions/core-hello-part4/workflows/hello.nf(truncated)
hello-nf-core/solutions/core-hello-part5/conf/base.config(truncated)
hello-nf-core/solutions/core-hello-part5/conf/modules.config(truncated)
hello-nf-core/solutions/core-hello-part5/conf/test.config(truncated)
hello-nf-core/solutions/core-hello-part5/conf/test_full.config(truncated)
hello-nf-core/solutions/core-hello-part5/main.nf(truncated)
hello-nf-core/solutions/core-hello-part5/modules/local/convertToUpper.nf(truncated)
hello-nf-core/solutions/core-hello-part5/modules/local/cowpy.nf(truncated)
hello-nf-core/solutions/core-hello-part5/modules/local/cowpy/main.nf(truncated)
hello-nf-core/solutions/core-hello-part5/modules/local/sayHello.nf(truncated)
hello-nf-core/solutions/core-hello-part5/modules/nf-core/cat/cat/main.nf(truncated)
hello-nf-core/solutions/core-hello-part5/modules/nf-core/cat/cat/tests/nextflow_unzipped_zipped.config(truncated)
hello-nf-core/solutions/core-hello-part5/modules/nf-core/cat/cat/tests/nextflow_zipped_unzipped.config(truncated)
hello-nf-core/solutions/core-hello-part5/nextflow.config(truncated)
hello-nf-core/solutions/core-hello-part5/subworkflows/local/utils_nfcore_hello_pipeline/main.nf(truncated)
hello-nf-core/solutions/core-hello-part5/subworkflows/nf-core/utils_nextflow_pipeline/main.nf(truncated)
hello-nf-core/solutions/core-hello-part5/subworkflows/nf-core/utils_nextflow_pipeline/tests/nextflow.config(truncated)
hello-nf-core/solutions/core-hello-part5/subworkflows/nf-core/utils_nfcore_pipeline/main.nf(truncated)
hello-nf-core/solutions/core-hello-part5/subworkflows/nf-core/utils_nfcore_pipeline/tests/nextflow.config(truncated)
hello-nf-core/solutions/core-hello-part5/subworkflows/nf-core/utils_nfschema_plugin/main.nf(truncated)
hello-nf-core/solutions/core-hello-part5/workflows/hello.nf(truncated)
hello-nf-core/solutions/core-hello-start/conf/base.config(truncated)
hello-nf-core/solutions/core-hello-start/conf/modules.config(truncated)
hello-nf-core/solutions/core-hello-start/conf/test.config(truncated)
hello-nf-core/solutions/core-hello-start/conf/test_full.config(truncated)
hello-nf-core/solutions/core-hello-start/main.nf(truncated)
hello-nf-core/solutions/core-hello-start/nextflow.config(truncated)
hello-nf-core/solutions/core-hello-start/subworkflows/local/utils_nfcore_hello_pipeline/main.nf(truncated)
hello-nf-core/solutions/core-hello-start/subworkflows/nf-core/utils_nextflow_pipeline/main.nf(truncated)
hello-nf-core/solutions/core-hello-start/subworkflows/nf-core/utils_nextflow_pipeline/tests/nextflow.config(truncated)
hello-nf-core/solutions/core-hello-start/subworkflows/nf-core/utils_nfcore_pipeline/main.nf(truncated)
hello-nf-core/solutions/core-hello-start/subworkflows/nf-core/utils_nfcore_pipeline/tests/nextflow.config(truncated)
hello-nf-core/solutions/core-hello-start/subworkflows/nf-core/utils_nfschema_plugin/main.nf(truncated)
hello-nf-core/solutions/core-hello-start/workflows/hello.nf(truncated)
hello-plugins/solutions/1-plugin-basics/random_id_example.nf(truncated)
hello-plugins/solutions/3-custom-functions/main.nf(truncated)
hello-plugins/solutions/4-build-and-test/main.nf(truncated)
hello-plugins/solutions/5-observers/main.nf(truncated)
hello-plugins/solutions/6-configuration/main.nf(truncated)
nextflow-run/solutions/3-main.nf(truncated)
nextflow-run/solutions/modules/collectGreetings.nf(truncated)
nextflow-run/solutions/modules/convertToUpper.nf(truncated)
nextflow-run/solutions/modules/sayHello.nf(truncated)
nextflow-run/solutions/nextflow.config(truncated)
side-quests/solutions/debugging/buggy_workflow.nf(truncated)
side-quests/solutions/essential_scripting_patterns/main.nf(truncated)
side-quests/solutions/plugin_development/main.nf(truncated)
side-quests/solutions/plugin_development/random_id_example.nf(truncated)

pinin4fjords and others added 2 commits January 9, 2026 13:49
Makefiles require tabs for indentation per POSIX standard.

Co-Authored-By: Claude Opus 4.5 <[email protected]>
@bentsherman
Copy link
Member

@pinin4fjords I would completely remove any mention of creating custom factories / operators. Nearly every example of a custom factory / operator that I see in the wild would have been better as a custom function, and they are currently in a precarious position in terms of future type checking support.

Otherwise, it looks like the right idea. Custom functions and trace observers are by far the most important to cover. Custom executors and filesystems are also powerful but way more complicated -- if you try to cover them at all, I would put them in a separate tutorial.

@pinin4fjords
Copy link
Collaborator Author

@pinin4fjords I would completely remove any mention of creating custom factories / operators. Nearly every example of a custom factory / operator that I see in the wild would have been better as a custom function, and they are currently in a precarious position in terms of future type checking support.

Otherwise, it looks like the right idea. Custom functions and trace observers are by far the most important to cover. Custom executors and filesystems are also powerful but way more complicated -- if you try to cover them at all, I would put them in a separate tutorial.

Thanks @bentsherman !

pinin4fjords and others added 4 commits January 9, 2026 18:55
Based on feedback from @bentsherman: custom operators/factories are in a
precarious position for future type checking support, and nearly every
example would have been better as a custom function.

Changes:
- Remove Section 8 (Custom operators and factories) entirely
- Remove Section 10.1 (Channel factories)
- Renumber sections 9-11 to 8-10
- Update all references to operators/factories throughout
- Update code examples to use functions instead of shoutAll operator
- Keep focus on functions and trace observers as the key extension types

Co-Authored-By: Claude Opus 4.5 <[email protected]>
Remove shoutAll operator from main.nf solution, use reverseGreeting
function instead to match the updated tutorial content.

Co-Authored-By: Claude Opus 4.5 <[email protected]>
Change "functions, factories, etc." to "functions, observers, etc." to
avoid confusion with channel factories (which were removed).

Co-Authored-By: Claude Opus 4.5 <[email protected]>
Remove extra features from solution files that aren't covered in the
tutorial:
- Remove unused prefix/suffix config options from nextflow.config
- Simplify output message in main.nf to match tutorial
- Remove extra comment block from random_id_example.nf

Co-Authored-By: Claude Opus 4.5 <[email protected]>
Copy link
Collaborator

@adamrtalbot adamrtalbot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just reviewed the "using plugins" section.

  • Let's make it shorter and tighter, it's verbose and wordy.
  • Focus on how plugins extend Nextflow functionality rather than sharing code
  • Keep it simple and explain it one way and only one way.

pinin4fjords and others added 5 commits January 14, 2026 11:13
- Remove make comparison for Gradle (advanced concept)
- Replace reverseString() with samplesheetToList() as example (more recognizable)
- Reduce hyperbole throughout, get to the point faster
- Add Slack as example for extending pipelines
- Clarify that community can add features without modifying Nextflow core
- Simplify tip about using plugins section
- Add notes about latest version default and runtime download
- Restructure "Try it" section: remove local function comparison,
  use config-only approach, remove CLI flag diversion
- Add note about development burden being on plugin developer
  and Nextflow handling plugin management

Co-Authored-By: Claude Opus 4.5 <[email protected]>
The plugin registry already lists available plugins, so this section
was redundant.

Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Replace em-dash elaborations with periods or semicolons
- Use colons instead of hyphens for list item explanations
- Remove unnecessary trailing explanations
- Simplify sentences to be more direct

Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Fix section 2.7 -> 2.6 after removing Popular plugins section
- Run prettier to fix table formatting

Co-Authored-By: Claude Opus 4.5 <[email protected]>
pinin4fjords added a commit that referenced this pull request Jan 14, 2026
Removing plugin files so they can be cleanly merged from the
plugin-development-side-quest branch, which has the reviewed
and updated version.
pinin4fjords added a commit that referenced this pull request Jan 14, 2026
@bentsherman
Copy link
Member

For section 8 on custom plugin configuration, it might be worth mentioning that you can define your custom config as a proper class with some annotations, so that the Nextflow linter and language server can recognize it.

But this tutorial is already pretty long and I've already documented it here, so you could just add a blurb with a link to these docs.

pinin4fjords and others added 2 commits January 14, 2026 14:33
Links to official docs for defining custom config as a proper class
with annotations for IDE and linter support.

Addresses review feedback from @adamrtalbot.

Co-Authored-By: Claude Opus 4.5 <[email protected]>
Changes based on reviewing transcript of live delivery:

- Update Java version requirement from 17 to 21 (matches training env)
- Fix error output line numbers in Section 8.3 (was "line 7", now "line 30")
- Make Section 4 progressive: add functions one at a time instead of all
  three at once, reinforcing the @function pattern incrementally

Co-Authored-By: Claude Opus 4.5 <[email protected]>
@pinin4fjords
Copy link
Collaborator Author

Complete list of changes from transcript analysis

After reviewing the transcript of the live delivery session, the following improvements were made across multiple commits:

Structural improvements

  • Progressive Section 4: Now adds functions one at a time (reverseGreeting → decorateGreeting → friendlyGreeting) instead of all three at once
  • Progressive Section 7.2: Observer built in 3 steps (minimal observer → register it → add counting logic)
  • Progressive Section 8.3: Intentionally leads learners through the "variable not declared" error as a teaching moment, then shows the fix
  • Class relationship diagram: Added mermaid diagram in Section 3.2 showing how plugin classes relate

"Run it first" pedagogy

  • Added step to run random_id_example.nf before examining code (Section 2.6)
  • Added step to run main.nf before modifying it (Section 6.2)
  • This matches the teaching pattern of "see it work, then understand how"

Clearer transitions

  • Section 6.1: Fixed awkward transition from nf-hello to nf-greeting config - now explicitly shows replacing one with the other
  • Added note explaining random_id_example.nf won't work after config change (and that's OK)

Additional context and reassurance

  • Local vs published plugins: Added info box explaining where local plugins are stored ($NXF_HOME/plugins/)
  • Registry browsing exercise: Added exercise to explore plugins.nextflow.io
  • Java expertise reassurance: Enhanced prerequisites to reassure learners they don't need Java expertise
  • Common runtime issues: Added warning box with troubleshooting tips
  • -ansi-log false tip: Enhanced explanation of why this flag is needed for observer output

Configuration section enhancements

  • Type-safe config example: Added collapsible advanced example showing @ConfigScope annotations
  • Expected output blocks: Added more output examples so learners know what to expect

Accuracy fixes (this commit)

  • Java version: 17 → 21 (matches training environment)
  • Error line numbers: "line 7" → "line 30" (matches actual compiler output)

All changes verified by running complete tutorial walkthrough in Docker container.

pinin4fjords and others added 4 commits January 14, 2026 18:55
Convert from pipe syntax to dot syntax for consistency:
- `| map` → `.map`
- `| view` → `.view()`

Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Replace 14 "Let's" patterns with direct statements per style guide
- Change "Don't worry about the warnings!" to "The warnings are expected."
- Fix hl_lines at line 1801 to highlight actual code changes (7-8 18)
  instead of method signatures (5-6 17)
- Update solution files to match final tutorial state (section 8.3):
  - main.nf: add script comment, update output message
  - nextflow.config: add prefix/suffix settings
  - random_id_example.nf: use dot notation for consistency

Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Adjust table column alignment
- Use underscore for italics instead of asterisks

Co-Authored-By: Claude Opus 4.5 <[email protected]>
@pinin4fjords pinin4fjords changed the title Add plugin development side quest Add plugin development course Jan 15, 2026
@pinin4fjords pinin4fjords changed the title Add plugin development course Add Hello Plugins training module Jan 15, 2026
@pinin4fjords pinin4fjords force-pushed the plugin-development-side-quest branch 3 times, most recently from 943b1e1 to d82ed8e Compare January 15, 2026 13:13
Restructure the plugin development content from a single side quest into
a full training module with 7 progressive parts:

- Part 1: Plugin Basics - discovering and using existing plugins
- Part 2: Create Project - scaffolding with `nextflow plugin create`
- Part 3: Custom Functions - implementing @function methods
- Part 4: Build and Test - Gradle workflow and Spock testing
- Part 5: Observers - TraceObserver for workflow lifecycle events
- Part 6: Configuration - session.config.navigate() and ConfigScope
- Part 7: Distribution - public registry and internal hosting

Updates for Nextflow 25.10.2 plugin scaffolding:
- Use Greeting* class naming (not NfGreeting*)
- Use io.nextflow.nextflow-plugin version 1.0.0-beta.10
- Set nextflowVersion = '25.10.0' (required for config.spec.* classes)
- Use nextflow.config.spec.* imports (not nextflow.config.dsl.*)
- Use @ScopeName + implements ConfigScope (not @ConfigScope annotation)

Documentation fixes:
- Fix hl_lines in 06_configuration.md (line 4 not 5 for build.gradle)
- Remove exclamation marks from summary.md and next_steps.md
- Replace "let's" with direct phrasing in 01_plugin_basics.md

Co-Authored-By: Claude Opus 4.5 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

improve New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants