KAFKA-18500 Build PRs at HEAD commit (apache#18449)

The default checkout behavior for GitHub Actions is to use a special
merge ref which is equivalent to the base branch with the PR merged into
it. While this is crucial for checking compilation issues against trunk,
it significantly diminishes our ability to use any build caching.

This patch changes the JUnit test jobs to checkout the HEAD commit of the PR 
when building. The "Compile and Check" step still checks out the merge commit
so we can keep that level of validation.

Reviewers: Ismael Juma <[email protected]>, Chia-Ping Tsai <[email protected]>

Author: mumrah

.github/workflows/README.md

@@ -133,4 +133,47 @@ Composite actions are a convenient way to reuse build logic, but they have
 some limitations. 
 
 - Cannot run more than one step in a composite action (see `workflow_call` instead)
-- Inputs can only be strings, no support for typed parameters. See: https://github.com/actions/runner/issues/2238
+- Inputs can only be strings, no support for typed parameters. See: https://github.com/actions/runner/issues/2238
+
+## Troubleshooting
+
+### Gradle Cache Misses
+
+If your PR is running for longer than you would expect due to cache misses, there are a
+few things to check. 
+
+First, find the cache that was loaded into your PR build. This is found in the Setup Gradle
+output. Look for a line starting with "Restored Gradle User Home from cache key". 
+For example,
+
+```
+Restored Gradle User Home from cache key: gradle-home-v1|Linux-X64|test[188616818c9a3165053ef8704c27b28e]-5c20aa187aa8f51af4270d7d1b0db4963b0cd10b
+```
+
+The last part of the cache key is the SHA of the commit on trunk where the cache
+was created. If that commit is not on your branch, it means your build loaded a 
+cache that includes changes your PR does not yet have. This is a common way to
+have cache misses. To resolve this, update your PR with the latest cached trunk commit:
+
+```commandline
+git fetch origin
+./committer-tools/update-cache.sh
+git merge trunk-cached
+```
+
+then push your branch.
+
+If your build seems to be using the correct cache, the next thing to check is for
+changes to task inputs. You can find this by locating the trunk Build Scan from 
+the cache commit on trunk and comparing it with the build scan of your PR build.
+This is done in the Develocity UI using the two overlapping circles like `(A()B)`. 
+This will show you differences in the task inputs for the two builds.
+
+Finally, you can run your PR with extra cache debugging. Add this to the gradle invocation in
+[run-gradle/action.yml](../actions/run-gradle/action.yml). 
+
+```
+-Dorg.gradle.caching.debug=true
+```
+
+This will dump out a lot of output, so you may also reduce the test target to one module.

.github/workflows/build.yml

@@ -36,7 +36,8 @@ jobs:
     name: Configure Workflow
     outputs:
       is-draft: ${{ steps.check-draft-pr.outputs.is-draft }}
-      test-catalog-days: ${{ steps.test-catalog.outputs.days }}
+      test-catalog-days: ${{ steps.configure-outputs.outputs.days }}
+      sha: ${{ steps.configure-outputs.outputs.sha }}
     steps:
       - name: Env
         run: printenv
@@ -48,13 +49,15 @@ jobs:
           github.event_name == 'pull_request' && 
           github.event.pull_request.draft
         run: echo "is-draft=true" >> "$GITHUB_OUTPUT"
-      - name: Test Catalog Age
-        id: test-catalog
+      - name: Configure Outputs
+        id: configure-outputs
         run: |
           if [ "${{ github.event_name }}" = "pull_request" ]; then
             echo "days=0" >> "$GITHUB_OUTPUT"
+            echo "sha=${{ github.event.pull_request.head.sha }}" >> "$GITHUB_OUTPUT"
           else
             echo "days=7" >> "$GITHUB_OUTPUT"
+            echo "sha=${{ github.sha }}" >> "$GITHUB_OUTPUT"
           fi
 
   load-catalog:
@@ -108,7 +111,7 @@ jobs:
   validate:
     needs: [configure]
     runs-on: ubuntu-latest
-    name: Compile and Check Java
+    name: Compile and Check (Merge Ref)
     steps:
       - name: Env
         run: printenv
@@ -118,6 +121,7 @@ jobs:
         uses: actions/checkout@v4
         with:
           persist-credentials: false
+          ref: ${{ github.sha }}  # this is the default, just being explicit.
       - name: Setup Python
         uses: ./.github/actions/setup-python
       - name: Setup Gradle
@@ -189,6 +193,7 @@ jobs:
         uses: actions/checkout@v4
         with:
           persist-credentials: false
+          ref: ${{ needs.configure.outputs.sha }}
       - name: Setup Python
         uses: ./.github/actions/setup-python
       - name: Setup Gradle