Intergrate conf snippets in include_sections from migrid PR244.

[jonasbardino]

Reuse the strategy from external-certificates folder for the new MiGserver.conf include_sections support. Copies any files in the external-conf-sections folder into the resulting container /home/mig/mig/server/MiGserver.d/ folder where they will be included by default.

[jonasbardino]

Tested to work in practical deployment, so mainly left as a decision if this is indeed the way we want to support configuration snippets or if a bind mounted volume with snippets is better, as we briefly discussed off-list. Rebuilding is of course tedious but updates are supposedly rare and typically will require at least service restarts, anyway.
In short, I'm slightly undecided so input is very welcome.

[Martin-Rehr]

Tested to work in practical deployment, so mainly left as a decision if this is indeed the way we want to support configuration snippets or if a bind mounted volume with snippets is better, as we briefly discussed off-list. Rebuilding is of course tedious but updates are supposedly rare and typically will require at least service restarts, anyway. In short, I'm slightly undecided so input is very welcome.

I foresee that we will be generating the configuration snippets as a part of the build process such as the global conf ?

That is, I don't think neither the volume bind nor the cert-like copy will suit us?

[jonasbardino]

Tested to work in practical deployment, so mainly left as a decision if this is indeed the way we want to support configuration snippets or if a bind mounted volume with snippets is better, as we briefly discussed off-list. Rebuilding is of course tedious but updates are supposedly rare and typically will require at least service restarts, anyway. In short, I'm slightly undecided so input is very welcome.

I foresee that we will be generating the configuration snippets as a part of the build process such as the global conf ?

That is, I don't think neither the volume bind nor the cert-like copy will suit us?

We will not be generating e.g. the cloud conf as pulling that out of the generator loop was the main motivation for introducing conf snippet support.
I believe we will want to push 'static' overrides to SITE and GLOBAL conf snippets for a number of required options not supported in generateconfs and/or exposed in docker-migrid. Please refer to my recent follow-up in PR ucphhpc/migrid-sync#249 for details on that idea. We can still e.g. do that with ansible combined with either copy or volume.

[Martin-Rehr]

Tested to work in practical deployment, so mainly left as a decision if this is indeed the way we want to support configuration snippets or if a bind mounted volume with snippets is better, as we briefly discussed off-list. Rebuilding is of course tedious but updates are supposedly rare and typically will require at least service restarts, anyway. In short, I'm slightly undecided so input is very welcome.

I foresee that we will be generating the configuration snippets as a part of the build process such as the global conf ?
That is, I don't think neither the volume bind nor the cert-like copy will suit us?

We will not be generating e.g. the cloud conf as pulling that out of the generator loop was the main motivation for introducing conf snippet support. I believe we will want to push 'static' overrides to SITE and GLOBAL conf snippets for a number of required options not supported in generateconfs and/or exposed in docker-migrid. Please refer to my recent follow-up in PR ucphhpc/migrid-sync#249 for details on that idea. We can still e.g. do that with ansible combined with either copy or volume.

Fine with pulling the non-core functionality out of generateconfs but I think that eg. the CLOUD confs should be build/generated as a part of the image like the rest of our external dependencies.

[jonasbardino]

We will not be generating e.g. the cloud conf as pulling that out of the generator loop was the main motivation for introducing conf snippet support. I believe we will want to push 'static' overrides to SITE and GLOBAL conf snippets for a number of required options not supported in generateconfs and/or exposed in docker-migrid. Please refer to my recent follow-up in PR ucphhpc/migrid-sync#249 for details on that idea. We can still e.g. do that with ansible combined with either copy or volume.

Fine with pulling the non-core functionality out of generateconfs but I think that eg. the CLOUD confs should be build/generated as a part of the image like the rest of our external dependencies.

Well, the COPY method implemented here will insert any available conf snippets including CLOUD into the image during build.
In particular the numerous user specific parts of the CLOUD conf (instance limits, image maps, etc.) are ill-suited for generation (as in generateconf). Even docker-migrid rebuild/redeploy on each such change is inconvenient, but at least that step can be skipped with a one-liner manual injection of the updated conf snippet. That will just leave the change volatile until next rebuild in contrast to the alternative bind-mount solution.

[Martin-Rehr]

We will not be generating e.g. the cloud conf as pulling that out of the generator loop was the main motivation for introducing conf snippet support. I believe we will want to push 'static' overrides to SITE and GLOBAL conf snippets for a number of required options not supported in generateconfs and/or exposed in docker-migrid. Please refer to my recent follow-up in PR ucphhpc/migrid-sync#249 for details on that idea. We can still e.g. do that with ansible combined with either copy or volume.

Fine with pulling the non-core functionality out of generateconfs but I think that eg. the CLOUD confs should be build/generated as a part of the image like the rest of our external dependencies.

Well, the COPY method implemented here will insert any available conf snippets including CLOUD into the image during build. In particular the numerous user specific parts of the CLOUD conf (instance limits, image maps, etc.) are ill-suited for generation (as in generateconf). Even docker-migrid rebuild/redeploy on each such change is inconvenient, but at least that step can be skipped with a one-liner manual injection of the updated conf snippet. That will just leave the change volatile until next rebuild in contrast to the alternative bind-mount solution.

Then I would prefer MiGserver.d as a bind mounted volume.

[jonasbardino]

...

Well, the COPY method implemented here will insert any available conf snippets including CLOUD into the image during build. In particular the numerous user specific parts of the CLOUD conf (instance limits, image maps, etc.) are ill-suited for generation (as in generateconf). Even docker-migrid rebuild/redeploy on each such change is inconvenient, but at least that step can be skipped with a one-liner manual injection of the updated conf snippet. That will just leave the change volatile until next rebuild in contrast to the alternative bind-mount solution.

Then I would prefer MiGserver.d as a bind mounted volume.

Alright, thanks for the input both of you (partially off-list). I'll take a look at a bind mount solution.

[jonasbardino]

Obsoleted by PR #106 .