Fix VM snapshotting failures relating to OpenJDK MethodHandles

[ThanHenderson]

VM snapshotting is tested and functional (however limited) on JDK 8 and JDK 11 (both of which currently use OpenJ9 Method Handles (MH) by default). From JDK 17 and on, OpenJDK Method Handles are used by default.

I have seen an issue when testing on JDK 17+ where OpenJDK MH specific code is functioning incorrectly when snapshotting is enabled. Namely, addMemberNameToClass walks over J9Class::memberNames that consists of J9MemberNameListNodes. However, the J9MemberNameListNode::memberName is in an invalid state s.t. this line fails with a seg fault.

I think the MemberNames need to be added to the VM snapshot and/or fixed up when writing J9Classes to the snapshot, and handled appropriately on restore.

[babsingh]

@lzhou2025 J9Class->memberNames points to a list of object references. These refs would need to freed, and J9Class->memberNames should be set to NULL before a snapshot is created.

The following code shows how J9Class->memberNames can be freed:

openj9/runtime/vm/jvminit.c

Lines 8358 to 8376 in 3354452

	omrthread_monitor_enter(vm->memberNameListsMutex);
	if (NULL != clazz->memberNames) {
	J9InternalVMFunctions *vmFuncs = vm->internalVMFunctions;
	J9MemberNameListNode *node = clazz->memberNames;
	clazz->memberNames = NULL;
	while (NULL != node) {
	J9MemberNameListNode *next = node->next;
	/* The weak JNI ref must have been cleared by now. Otherwise, the
	* MemberName is still live, and this class should be too.
	*/
	Assert_VM_true(NULL == J9_JNI_UNWRAP_REFERENCE(node->memberName));
	vmFuncs->j9jni_deleteGlobalRef((JNIEnv*)currentThread, node->memberName, JNI_TRUE);
	pool_removeElement(vm->memberNameListNodePool, node);
	node = next;
	}
	}
	omrthread_monitor_exit(vm->memberNameListsMutex);

Could you try freeing J9Class->memberNames and setting the field to NULL in VMSnapshotImpl::fixupClass?

openj9/runtime/vm/VMSnapshotImpl.cpp

Lines 683 to 726 in 3354452

	void
	VMSnapshotImpl::fixupClass(J9Class *clazz)
	{
	clazz->classObject = NULL;
	fixupMethodRunAddresses(clazz);
	fixupConstantPool(clazz);
	clazz->initializeStatus = J9ClassInitNotInitialized;
	clazz->jniIDs = NULL;
	clazz->replacedClass = NULL;
	clazz->gcLink = NULL;
	clazz->jitMetaDataList = NULL;
	UDATA totalStaticSlots = totalStaticSlotsForClass(clazz->romClass);
	memset(clazz->ramStatics, 0, totalStaticSlots * sizeof(UDATA));
	UDATA i;
	if (NULL != clazz->staticSplitMethodTable) {
	for (i = 0; i < clazz->romClass->staticSplitMethodRefCount; i++) {
	clazz->staticSplitMethodTable[i] = (J9Method *)_vm->initialMethods.initialStaticMethod;
	}
	}
	if (NULL != clazz->specialSplitMethodTable) {
	for (i = 0; i < clazz->romClass->specialSplitMethodRefCount; i++) {
	clazz->specialSplitMethodTable[i] = (J9Method *)_vm->initialMethods.initialSpecialMethod;
	}
	}
	if (NULL != clazz->callSites) {
	memset(clazz->callSites, 0, sizeof(UDATA) * clazz->romClass->callSiteCount);
	}
	#if defined(J9VM_OPT_OPENJDK_METHODHANDLE)
	if (NULL != clazz->invokeCache) {
	memset(clazz->invokeCache, 0, sizeof(UDATA) * clazz->romClass->invokeCacheCount);
	}
	#else /* defined(J9VM_OPT_OPENJDK_METHODHANDLE) */
	if (NULL != clazz->methodTypes) {
	memset(clazz->methodTypes, 0, sizeof(UDATA) * clazz->romClass->methodTypeCount);
	}
	if (NULL != clazz->varHandleMethodTypes) {
	memset(clazz->varHandleMethodTypes, 0, sizeof(UDATA) * clazz->romClass->varHandleMethodTypeCount);
	}
	#endif /* defined(J9VM_OPT_OPENJDK_METHODHANDLE) */
	}

[lzhou2025]

OK, I will change it and test it in openjdk11

[babsingh]

According to the issue description, this problem only occurs in JDK17 and above. You can test it in your JDK21 environment.

[lzhou2025]

I see