Skip to content

Stress test Offheap with large arrays #21596

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
amicic opened this issue Apr 8, 2025 · 4 comments
Open

Stress test Offheap with large arrays #21596

amicic opened this issue Apr 8, 2025 · 4 comments
Assignees
@amicic
Labels
comp:gc

Comments

@amicic
Copy link
Contributor

amicic commented Apr 8, 2025

Create a test case and demonstrate that Offheap can cope with large arrays where otherwise contiguous in-heap representation can struggle (require larger heap and require significant compaction) due to external fragmentation.
@amicic amicic added the comp:gc label Apr 8, 2025
@amicic
Copy link
Contributor Author

amicic commented Apr 8, 2025 •
edited
Loading 

class LargeArrays {
        static byte[][] backbone; // just a container to keep alive all large arrays that will be allocated
        static int backboneSize = 1024;
        static int oneMB = 1024 * 1024;

        static public void fragmentAndAllocateLarger(int power) {
                int sizeInMB = (1 << power);
                System.out.println("sizeInMB " + sizeInMB);

                // free up 1/2 of each allocate group
                long freedSize = 0;
                for (int i = 0; i < backboneSize; i+= sizeInMB) {
                        for (int j = 0; j < (sizeInMB / 2); j+= 1) {
                                if (null != backbone[i + j]) {
                                        int length = backbone[i + j].length;
                                        freedSize += length;
                                        backbone[i + j] = null;
                                        System.out.println("freed backbone[" + i + " " + j + "] " + " length " + length + " total freedSize " + freedSize);
                                }
                        }
                }

                // fill up 1/2 of heap that we just freed up, but with larger sizes
                long allocCount = 0;
                for (int i = 0; i < backboneSize / 2; i+= sizeInMB) {
                        allocCount += 1;
                        backbone[i] = new byte[sizeInMB * oneMB];
                        System.out.println("allocated backbone[" + i + "] " + " length " + backbone[i].length + " allocCount " + allocCount + " total allocSize " + (sizeInMB * oneMB * allocCount));
                }

        }

        static public void main(String[] args) {

                backbone = new byte[backboneSize][];

                // fill up heap with 1024 arrays of 1MB, in total of 1GB
                for (int i = 0; i < backboneSize; i+= 1) {
                        backbone[i] = new byte[oneMB];
                        System.out.println("allocated backbone[" + i + "] " + " length " + backbone[i].length);
                }

                // fragment memory by freeing up existing smaller sizes and allocating double the size (2MB, 4MB,... 512MB) that cannot fit in the created holes
                for (int power = 1; power <= 9; power += 1) {
                        fragmentAndAllocateLarger(power);
                }

                System.out.println("backbone " + backbone);

                backbone = null;

                // check that GC coalesced (or compacted) the freed memory, by allocating one 1GB array
                byte[] largeArray = new byte[1024 * oneMB];

                System.out.println("largeArray " + largeArray);
        }
}

@amicic
Copy link
Contributor Author

amicic commented Apr 8, 2025 •
edited
Loading

The above test allocates objects with sizes from 1MB to 512MB with total of 1GB (hence requiring just a bit above 1G heap size), but incrementally creates holes, so that new larger allocation may require compaction (in a simple contiguous array representation and simple flat/contiguous heap organization).

The test passes with just a couple of MBs extra with both optthruput and balanced offheap. Optthruput needs a few longish global compactions (~200ms) to resolve the holes created by the test.

Balanced also compacts, due to very tight overall space, but only to move small objects, but never those larger than 1MB (that are all allocated in offheap).

Optthruput:

root@off1:/home/openj9-openjdk-jdk21.offheap/build/linux-x86_64-server-release/jdk/bin# time ./java -Xint -Xnoloa   -Xmx1026M -Xms1026M  -verbose:gc -Xgcpolicy:optthruput LargeArrays &> LargeArrays_optthruput.out

real	0m4.271s
user	0m7.553s
sys	0m0.700s
root@off1:/home/openj9-openjdk-jdk21.offheap/build/linux-x86_64-server-release/jdk/bin# grep '<gc-op.*compact' LargeArrays_optthruput.out
<gc-op id="11" type="compact" timems="10.465" contextid="4" timestamp="2025-04-08T14:15:18.137">
<gc-op id="26" type="compact" timems="4.533" contextid="20" timestamp="2025-04-08T14:15:18.147">
<gc-op id="37" type="compact" timems="231.865" contextid="30" timestamp="2025-04-08T14:15:18.383">
<gc-op id="53" type="compact" timems="32.699" contextid="46" timestamp="2025-04-08T14:15:18.475">
<gc-op id="69" type="compact" timems="5.696" contextid="62" timestamp="2025-04-08T14:15:18.589">
<gc-op id="80" type="compact" timems="200.765" contextid="73" timestamp="2025-04-08T14:15:18.794">
<gc-op id="108" type="compact" timems="33.793" contextid="102" timestamp="2025-04-08T14:15:18.879">
<gc-op id="123" type="compact" timems="15.905" contextid="117" timestamp="2025-04-08T14:15:18.936">
<gc-op id="138" type="compact" timems="15.641" contextid="132" timestamp="2025-04-08T14:15:18.959">
<gc-op id="153" type="compact" timems="14.642" contextid="147" timestamp="2025-04-08T14:15:18.979">
<gc-op id="168" type="compact" timems="14.826" contextid="162" timestamp="2025-04-08T14:15:18.998">
<gc-op id="179" type="compact" timems="182.782" contextid="172" timestamp="2025-04-08T14:15:19.186">
<gc-op id="207" type="compact" timems="30.753" contextid="201" timestamp="2025-04-08T14:15:19.285">
<gc-op id="222" type="compact" timems="14.720" contextid="216" timestamp="2025-04-08T14:15:19.328">
<gc-op id="237" type="compact" timems="13.696" contextid="231" timestamp="2025-04-08T14:15:19.349">
<gc-op id="252" type="compact" timems="14.779" contextid="246" timestamp="2025-04-08T14:15:19.371">
<gc-op id="263" type="compact" timems="178.708" contextid="256" timestamp="2025-04-08T14:15:19.555">
<gc-op id="291" type="compact" timems="18.974" contextid="285" timestamp="2025-04-08T14:15:19.652">
<gc-op id="306" type="compact" timems="13.206" contextid="300" timestamp="2025-04-08T14:15:19.685">
<gc-op id="321" type="compact" timems="11.532" contextid="315" timestamp="2025-04-08T14:15:19.709">
<gc-op id="336" type="compact" timems="8.667" contextid="330" timestamp="2025-04-08T14:15:19.724">
<gc-op id="347" type="compact" timems="174.209" contextid="340" timestamp="2025-04-08T14:15:19.903">
<gc-op id="375" type="compact" timems="20.148" contextid="369" timestamp="2025-04-08T14:15:19.982">
<gc-op id="390" type="compact" timems="13.340" contextid="384" timestamp="2025-04-08T14:15:20.022">
<gc-op id="405" type="compact" timems="12.122" contextid="399" timestamp="2025-04-08T14:15:20.055">
<gc-op id="420" type="compact" timems="5.788" contextid="414" timestamp="2025-04-08T14:15:20.080">
<gc-op id="431" type="compact" timems="184.658" contextid="424" timestamp="2025-04-08T14:15:20.270">
<gc-op id="459" type="compact" timems="10.802" contextid="453" timestamp="2025-04-08T14:15:20.343">
<gc-op id="474" type="compact" timems="6.697" contextid="468" timestamp="2025-04-08T14:15:20.365">
<gc-op id="485" type="compact" timems="150.759" contextid="478" timestamp="2025-04-08T14:15:20.520">
<gc-op id="513" type="compact" timems="16.874" contextid="507" timestamp="2025-04-08T14:15:20.661">
<gc-op id="528" type="compact" timems="10.175" contextid="522" timestamp="2025-04-08T14:15:20.694">
<gc-op id="539" type="compact" timems="168.426" contextid="532" timestamp="2025-04-08T14:15:20.868">
<gc-op id="554" type="compact" timems="16.314" contextid="548" timestamp="2025-04-08T14:15:20.932">
<gc-op id="565" type="compact" timems="115.761" contextid="558" timestamp="2025-04-08T14:15:21.054">
<gc-op id="580" type="compact" timems="11.853" contextid="574" timestamp="2025-04-08T14:15:21.152">
<gc-op id="591" type="compact" timems="119.376" contextid="584" timestamp="2025-04-08T14:15:21.277">

Balanced offheap:

root@off1:/home/openj9-openjdk-jdk21.offheap/build/linux-x86_64-server-release/jdk/bin# time ./java -Xint  -Xmx1026M -Xms1026M  -verbose:gc -Xgcpolicy:balanced LargeArrays &> LargeArrays_balanced.out

real	0m1.773s
user	0m3.262s
sys	0m0.863s
root@off1:/home/openj9-openjdk-jdk21.offheap/build/linux-x86_64-server-release/jdk/bin# grep '<gc-op.*compact' LargeArrays_balanced.out
<gc-op id="10" type="compact" timems="5.579" contextid="4" timestamp="2025-04-08T14:16:51.654">
<gc-op id="79" type="compact" timems="2.435" contextid="72" timestamp="2025-04-08T14:16:51.727">
<gc-op id="99" type="compact" timems="11.248" contextid="86" timestamp="2025-04-08T14:16:51.753">
<gc-op id="113" type="compact" timems="1.801" contextid="108" timestamp="2025-04-08T14:16:51.760">
<gc-op id="124" type="compact" timems="9.480" contextid="118" timestamp="2025-04-08T14:16:51.775">
<gc-op id="138" type="compact" timems="1.823" contextid="133" timestamp="2025-04-08T14:16:51.781">
<gc-op id="149" type="compact" timems="8.643" contextid="143" timestamp="2025-04-08T14:16:51.798">
<gc-op id="173" type="compact" timems="7.852" contextid="167" timestamp="2025-04-08T14:16:51.815">
<gc-op id="187" type="compact" timems="2.417" contextid="182" timestamp="2025-04-08T14:16:51.871">
<gc-op id="198" type="compact" timems="6.067" contextid="192" timestamp="2025-04-08T14:16:51.884">
<gc-op id="258" type="compact" timems="1.571" contextid="251" timestamp="2025-04-08T14:16:51.939">
<gc-op id="278" type="compact" timems="7.466" contextid="265" timestamp="2025-04-08T14:16:51.954">
<gc-op id="292" type="compact" timems="2.160" contextid="287" timestamp="2025-04-08T14:16:51.961">
<gc-op id="303" type="compact" timems="9.057" contextid="297" timestamp="2025-04-08T14:16:51.978">
<gc-op id="317" type="compact" timems="1.732" contextid="312" timestamp="2025-04-08T14:16:51.984">
<gc-op id="328" type="compact" timems="8.604" contextid="322" timestamp="2025-04-08T14:16:51.998">
<gc-op id="352" type="compact" timems="8.695" contextid="346" timestamp="2025-04-08T14:16:52.017">
<gc-op id="366" type="compact" timems="1.759" contextid="361" timestamp="2025-04-08T14:16:52.038">
<gc-op id="377" type="compact" timems="7.352" contextid="371" timestamp="2025-04-08T14:16:52.053">
<gc-op id="437" type="compact" timems="1.947" contextid="430" timestamp="2025-04-08T14:16:52.094">
<gc-op id="457" type="compact" timems="7.419" contextid="444" timestamp="2025-04-08T14:16:52.110">
<gc-op id="471" type="compact" timems="2.017" contextid="466" timestamp="2025-04-08T14:16:52.116">
<gc-op id="482" type="compact" timems="10.969" contextid="476" timestamp="2025-04-08T14:16:52.132">
<gc-op id="506" type="compact" timems="7.793" contextid="500" timestamp="2025-04-08T14:16:52.149">
<gc-op id="520" type="compact" timems="1.611" contextid="515" timestamp="2025-04-08T14:16:52.168">
<gc-op id="531" type="compact" timems="6.555" contextid="525" timestamp="2025-04-08T14:16:52.180">
<gc-op id="591" type="compact" timems="1.914" contextid="584" timestamp="2025-04-08T14:16:52.211">
<gc-op id="609" type="compact" timems="8.842" contextid="596" timestamp="2025-04-08T14:16:52.227">
<gc-op id="628" type="compact" timems="11.096" contextid="622" timestamp="2025-04-08T14:16:52.247">
<gc-op id="642" type="compact" timems="2.160" contextid="637" timestamp="2025-04-08T14:16:52.261">
<gc-op id="653" type="compact" timems="6.919" contextid="647" timestamp="2025-04-08T14:16:52.275">
<gc-op id="713" type="compact" timems="1.922" contextid="706" timestamp="2025-04-08T14:16:52.309">
<gc-op id="731" type="compact" timems="8.129" contextid="718" timestamp="2025-04-08T14:16:52.326">
<gc-op id="750" type="compact" timems="7.998" contextid="744" timestamp="2025-04-08T14:16:52.345">
<gc-op id="764" type="compact" timems="2.251" contextid="759" timestamp="2025-04-08T14:16:52.356">
<gc-op id="775" type="compact" timems="11.352" contextid="769" timestamp="2025-04-08T14:16:52.375">
<gc-op id="833" type="compact" timems="1.822" contextid="827" timestamp="2025-04-08T14:16:52.420">
<gc-op id="852" type="compact" timems="9.841" contextid="846" timestamp="2025-04-08T14:16:52.439">
<gc-op id="871" type="compact" timems="8.849" contextid="865" timestamp="2025-04-08T14:16:52.459">
<gc-op id="885" type="compact" timems="2.387" contextid="880" timestamp="2025-04-08T14:16:52.469">
<gc-op id="896" type="compact" timems="7.279" contextid="890" timestamp="2025-04-08T14:16:52.483">
<gc-op id="969" type="compact" timems="8.984" contextid="963" timestamp="2025-04-08T14:16:52.528">
<gc-op id="988" type="compact" timems="9.436" contextid="982" timestamp="2025-04-08T14:16:52.548">
<gc-op id="1002" type="compact" timems="2.659" contextid="997" timestamp="2025-04-08T14:16:52.558">
<gc-op id="1013" type="compact" timems="7.580" contextid="1007" timestamp="2025-04-08T14:16:52.576">
<gc-op id="1086" type="compact" timems="9.141" contextid="1080" timestamp="2025-04-08T14:16:52.620">
<gc-op id="1105" type="compact" timems="12.761" contextid="1099" timestamp="2025-04-08T14:16:52.644">
<gc-op id="1119" type="compact" timems="2.038" contextid="1114" timestamp="2025-04-08T14:16:52.652">
<gc-op id="1130" type="compact" timems="10.855" contextid="1124" timestamp="2025-04-08T14:16:52.672">
<gc-op id="1169" type="compact" timems="3.080" contextid="1164" timestamp="2025-04-08T14:16:52.708">
<gc-op id="1180" type="compact" timems="9.193" contextid="1157" timestamp="2025-04-08T14:16:52.725">
<gc-op id="1204" type="compact" timems="9.120" contextid="1198" timestamp="2025-04-08T14:16:52.747">
<gc-op id="1218" type="compact" timems="2.346" contextid="1213" timestamp="2025-04-08T14:16:52.755">
<gc-op id="1229" type="compact" timems="7.411" contextid="1223" timestamp="2025-04-08T14:16:52.769">
<gc-op id="1278" type="compact" timems="9.411" contextid="1252" timestamp="2025-04-08T14:16:52.805">
<gc-op id="1292" type="compact" timems="2.152" contextid="1287" timestamp="2025-04-08T14:16:52.816">
<gc-op id="1303" type="compact" timems="6.912" contextid="1297" timestamp="2025-04-08T14:16:52.830">

@amicic
Copy link
Contributor Author

amicic commented Apr 8, 2025 •
edited
Loading

Hotspot/G1 needs double the heap size (will fail with -Xmx2050M, pass with -Xmx2100M) probably due to internal fragmentation (1MB object requiring 2 regions of 1MB), and also has a few longish global compactions.

root@off1:/home/openj9-openjdk-jdk21.offheap/build/linux-x86_64-server-release/jdk/bin# time /home/OpenJDK21U-jre_x64_linux_hotspot_21.0.6_7/jdk-21.0.6+7-jre/bin/java -Xint -Xmx2100M -Xms2100M -verbose:gc  LargeArrays &> LargeArrays_hotspot.out

real	0m3.754s
user	0m2.987s
sys	0m1.193s
root@off1:/home/openj9-openjdk-jdk21.offheap/build/linux-x86_64-server-release/jdk/bin# grep Compact LargeArrays_hotspot.out
[0.908s][info][gc] GC(6) Pause Full (G1 Compaction Pause) 1073M->1073M(2100M) 4.566ms
[1.044s][info][gc] GC(7) Pause Full (G1 Compaction Pause) 1073M->1073M(2100M) 135.468ms
[1.303s][info][gc] GC(11) Pause Full (G1 Compaction Pause) 1073M->1073M(2100M) 4.631ms
[1.542s][info][gc] GC(12) Pause Full (G1 Compaction Pause) 1073M->1073M(2100M) 239.070ms
[1.794s][info][gc] GC(16) Pause Full (G1 Compaction Pause) 1197M->1197M(2100M) 4.449ms
[2.074s][info][gc] GC(17) Pause Full (G1 Compaction Pause) 1197M->1197M(2100M) 279.677ms
[2.317s][info][gc] GC(21) Pause Full (G1 Compaction Pause) 1327M->1327M(2100M) 4.200ms
[2.611s][info][gc] GC(22) Pause Full (G1 Compaction Pause) 1327M->1327M(2100M) 293.610ms
[2.741s][info][gc] GC(26) Pause Full (G1 Compaction Pause) 807M->807M(2100M) 3.734ms
[2.939s][info][gc] GC(27) Pause Full (G1 Compaction Pause) 807M->807M(2100M) 198.694ms
[3.122s][info][gc] GC(33) Pause Full (G1 Compaction Pause) 803M->803M(2100M) 4.064ms
[3.315s][info][gc] GC(34) Pause Full (G1 Compaction Pause) 803M->803M(2100M) 192.970ms
[3.450s][info][gc] GC(38) Pause Full (G1 Compaction Pause) 1045M->1M(2100M) 5.813ms

@amicic amicic self-assigned this Apr 8, 2025
@amicic
Copy link
Contributor Author

amicic commented Apr 8, 2025

@dmitripivkine FYI

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@amicic amicic

Labels
comp:gc
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@amicic