[JENKINS-56063] (Expansion of variables in refspec in case of honor refspec on initial clone)

[rishabhBudhouliya]

JENKINS-56063 - Addition of variable expansion in refspec in case of Honoring refspec on initial clone

Checklist

• I have read the CONTRIBUTING doc
• I have referenced the Jira issue related to my changes in one or more commit messages
• I have added tests that verify my changes
• Unit tests pass locally with my changes
• No Javadoc warnings were introduced with my changes
• No spotbugs warnings were introduced with my changes
• I have interactively tested my changes
• Any dependent changes have been merged and published in upstream modules (like git-client-plugin)

Types of changes

• Breaking change (fix or feature that would cause existing functionality to not work as expected)

[rishabhBudhouliya]

I haven't added tests yet, would add them if this is the right approach to fix this issue.

[rishabhBudhouliya]

@proski Thanks for reviewing my PR, after going through your suggestions I have changed the implementation which is more coherent to how variables are expanded in GitSCM.

[proski]

Much better now, and also simpler!

[MarkEWaite]

This looks great! Thanks very much.

It needs several automated tests to help us assure that we can duplicate the original problem from an automated test. It also helps us avoid the functionality being lost due to a regression.

[MarkEWaite]

Needs automated tests to confirm:

• Operating system environment variables (like PATH) are expanded as expected (PATH will generate a nonsense refspec, but it has the benefit of being defined on all supported platforms)
• Jenkins variables (like GIT_BRANCH) are expanded as expected in a Freestyle project
• Jenkins token macros (like GIT_REVISION) are expanded as expected in a Freestyle project

[rishabhBudhouliya]

I have added a test file which is running successful tests for "BUILD_ID", "BUILD_NO" and "GIT_COMMIT".
I have two issues here:

• I can't make a freestyle build with ${PATH} added in the refspec, it fails the build with a "can't file the commitFile in the working directory". Although I have tested this interactively where you can clearly see that the refspec is expanded but the build fails, I still would want your opinion on this.

• By Jenkins token macros do you mean running a build with parameters? Is that how tokens are used? If yes then how does one set a parameter for a job? Should I put the variable in the EnvVars store?

[rishabhBudhouliya]

I've seen the Token Macro API but I don't think that serves our purpose here
assertThat(TokenMacro.expandAll(build2, listener, "${GIT_REVISION}"), is(sha1String));

[rishabhBudhouliya]

After doing some RCA I found out that adding ${PATH} to refspec would make it illegal, hence the job will fail before doing anything else.

[fcojfernandez]

I can't make a freestyle build with ${PATH} added in the refspec, it fails the build with a "can't file the commitFile in the working directory"

How did you tried? Just adding ${PATH} to permuteRefSpecVariable?

I'd also like to know how this PR behaves with job parameters (https://wiki.jenkins.io/display/JENKINS/Define+Parameters)

[MarkEWaite]

Changes from this pull request were also included in #841 but were then reverted from #841. This pull request still needs to be reviewed and merged before the change will be available to users.

[jetersen]

What is blocking this issue? We recently updated to v4 and found jobs that no longer worked.

[MarkEWaite]

Needs review and merge by me or @rsandell or @fcojfernandez . Needs interactive testing to confirm that it is behaves correctly with:

• operating system environment variables
• variables set as parameters by job definition
• environment variables defined by env-inject plugin

[jetersen]

After digging further into the problem our problem was actually related to #772 #898

[MarkEWaite]

@rishabhBudhouliya I restarted my investigation of the issue this is describing. I have pushed a rebased version of this pull request as rebased-JENKINS-56063-Fix. When master is merged to this branch or when this is rebased on the master branch, test failures are visible, just as they are visible in the CI job.

If you have time to investigate further, I can leave this pull request open.

If you don't have time to investigate further, I'll close this pull request and keep the branch on my private fork for future work.

I've also added test cases to my docker-lfs repository, docker-lfs with jgit and my jenkins-bugs repository so that the bug will be regularly assessed during automated testing and exploratory.

[MarkEWaite]

Test failures need to be fixed. My interactive testing confirmed that (as described in the original report), command line git behaves OK because the shell expands the environment variables even though the git plugin does not expand them. It also confirmed that JGit fails because it does not expand the variables and does not invoke a shell.

[MarkEWaite]

Closing until someone has time to fix the test failures.