Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for network tags in Google Batch executor #5950

Open
ejseqera opened this issue Apr 7, 2025 · 0 comments
Open

Add support for network tags in Google Batch executor #5950

ejseqera opened this issue Apr 7, 2025 · 0 comments

Comments

@ejseqera
Copy link

ejseqera commented Apr 7, 2025

New feature

Add support for network tags when running Nextflow pipelines on Google Batch. This would allow users to specify network tags in their Nextflow configuration that will be applied to the Google Batch job's allocation policy.

Use case

Network tags are used in GCP to apply firewall rules and manage network access for compute instances. When running Nextflow pipelines on Google Batch, users might:

  • Apply specific firewall rules to their compute instances
  • Control ingress/egress traffic based on tags
  • Manage network security at scale across multiple pipeline executions
  • Ensure compliance with organizational network security policies

Suggested implementation

The implementation would involve:

  1. Add networkTags property to the BatchConfig class to store the configuration
  2. Update GoogleBatchTaskHandler to add the tags to the AllocationPolicy when creating Google Batch jobs
  3. Add appropriate warning(s) when tags are specified with instance templates for example

Example config:

google {
    project = 'tower-cloud-testing'
    location = 'europe-north1'
    batch.spot = true
    batch.networkTags = ['nextflow-test']
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant