In a channel that is directed to a publish target, Files listed inside a Map object are not published.

[hillan141]

Bug report

If I direct a channel that contains Files inside a map structure to a publish target, the files are not published.

Expected behavior and actual behavior

Expected: Per the documentation here , if I publish a channel with elements like this, I had expected the files listed in the files element to be published:

[ 
 resource_uri:https://foo.com/resource-api/..., 
 files:[/home/xxx/repos/technical-documentation-pages/.../work/03/16dad87b8f89f823f5e1cedd9f939b/my_file.txt]
]

Is that not the case?

Actual behavior: files listed inside a map key are not published.

Steps to reproduce the problem

Workflow:

nextflow.preview.output = true

process create_one_file_process {
    output:
        tuple val(my_tuple), path("my_file.txt"), emit: file_outputs
    script:
    my_tuple = ["resource_uri": "https://foo.com/resource-api/bar/sX8G8W/glom/5sQ1BZ"]
    """
    echo "my_file" > my_file.txt
    """
}

workflow {
    main:
        create_one_file_process()
        create_one_file_process.out.file_outputs.map {
            it -> ["resource_uri": it[0].resource_uri, "files": it[1..-1].flatten()]
        }.view().set {ch_publish_one_file}

    publish:
        ch_publish_one_file >> 'publish_one_file'
}

output {
    publish_one_file {
        mode 'copy'
        overwrite true
    }
}

Program output

The execution output from the above workflow:

$ nextflow run output_file_manifest_explore_manifest_format.nf 

 N E X T F L O W   ~  version 24.10.5

Launching `output_file_manifest_explore_manifest_format.nf` [jolly_easley] DSL2 - revision: 64451e7155

WARN: WORKFLOW OUTPUT DSL IS A PREVIEW FEATURE - SYNTAX AND FUNCTIONALITY CAN CHANGE IN FUTURE RELEASES
executor >  local (1)
executor >  local (1)
[03/16dad8] process > create_one_file_process [100%] 1 of 1 ✔
[resource_uri:https://foo.com/resource-api/bar/sX8G8W/glom/5sQ1BZ, files:[/home/xxx/repos/technical-documentation-pages/docs/Technical-Proposals/nextflow-publish-manifest/work/03/16dad87b8f89f823f5e1cedd9f939b/my_file.txt]]

The workflow succeeds, but the file is not published:

$ ls results
ls: cannot access 'results': No such file or directory

If I change my output channel structure to be a simple List, then the files are published as expected (not shown here). So I'm presuming the issue is that my file list is inside a map?

Environment

• Nextflow version: 24.10.5
• Java version: 21.0.6
• Operating system: Linux
• Bash version: 5.0.17(1)-release

Additional context

My understanding from docs is "By default, all files emitted by a published channel will be published into the specified directory. Lists and maps are recursively scanned for nested files.". In this case, when my file list is inside a Map, I'm not seeing them being published.

[bentsherman]

The map scanning was added in 25.01.0-edge. At the time I thought it would be backported to 24.10 but it wasn't. It will be fixed in #5909

[hillan141]

Thanks for the explanation. Appreciate all the effort to maintain and develop nextflow.