JVMTI RetransformClasses() is taking a lot of time

Question

I deployed a simple JVMTI agent to test bytecode instrumentation. My strategy is to call RetransformClasses function in CompiledMethodLoad call back to invoke ClassFileLoadHook. I wrote following code to do so:

    err = (*jvmti)->GetMethodDeclaringClass(jvmti, method, &klass);
    check_jvmti_error(jvmti, err, "Get Declaring Class");

    err = (*jvmti)->RetransformClasses(jvmti, 1, &klass);
    check_jvmti_error(jvmti, err, "Retransform class");

This function works correctly by invoking ClassFileLoadHook event, but it takes a lot of time while I'm just passing the same class inside it. My ClassFileLoadHook callback function is empty. I'm counting time of a simple matrix multiplication algorithm. By commenting out RetransformClasses function I get the execution time of the order of 0.8 seconds. Whereas just writing this function elevates the execution time to around 15 seconds.

Is it supposed to take that much overhead or am I doing something wrong?

Regards

Code:

static int x = 1;
void JNICALL
compiled_method_load(jvmtiEnv *jvmti, jmethodID method, jint code_size,
        const void* code_addr, jint map_length, const jvmtiAddrLocationMap* map,
        const void* compile_info) {
    jvmtiError err;
    jclass klass;

    char* name = NULL;
    char* signature = NULL;
    char* generic_ptr = NULL;

    err = (*jvmti)->RawMonitorEnter(jvmti, lock);
    check_jvmti_error(jvmti, err, "raw monitor enter");

    err = (*jvmti)->GetMethodName(jvmti, method, &name, &signature,
            &generic_ptr);
    check_jvmti_error(jvmti, err, "Get Method Name");

    printf("\nCompiled method load event\n");
    printf("Method name %s %s %s\n\n", name, signature,
            generic_ptr == NULL ? "" : generic_ptr);

    if (strstr(name, "main") != NULL && x == 1) {
        x++;
        err = (*jvmti)->GetMethodDeclaringClass(jvmti, method, &klass);
        check_jvmti_error(jvmti, err, "Get Declaring Class");

        err = (*jvmti)->RetransformClasses(jvmti, 1, &klass);
        check_jvmti_error(jvmti, err, "Retransform class");

    }

    if (name != NULL) {
        err = (*jvmti)->Deallocate(jvmti, (unsigned char*) name);
        check_jvmti_error(jvmti, err, "deallocate name");
    }
    if (signature != NULL) {
        err = (*jvmti)->Deallocate(jvmti, (unsigned char*) signature);
        check_jvmti_error(jvmti, err, "deallocate signature");
    }
    if (generic_ptr != NULL) {
        err = (*jvmti)->Deallocate(jvmti, (unsigned char*) generic_ptr);
        check_jvmti_error(jvmti, err, "deallocate generic_ptr");
    }

    err = (*jvmti)->RawMonitorExit(jvmti, lock);
    check_jvmti_error(jvmti, err, "raw monitor exit");
}

Well, retransformation isn't cheap. Are you executing the code above for all compiled methods? `CompiledMethodLoad` is called for hot methods that impact execution time much, and by retransforming their classes you 1) pay the cost of recreating a class 2) drop the compiled methods, effectively forcing VM to execute hot methods in interpreter — Stanislav Lukyanov, Dec 26 '16 at 12:01
I'm just re-transforming one class file and it contains only one method. It shouldn't be that costly. There's no comparison between `0.8` second and `15` second. It doesn't seem to be a cost. Moreover, even if program starts executing in interpreter, because of `TieredCompilation` mode, it should again revert into compiled mode. Correct me if I'm wrong. \\ Actually I want to instrument only "hot" methods. Is there any way I can use the JIT even after re-transformation? — Saqib Ahmed, Dec 27 '16 at 04:38
I'm expecting kind of a partial dead lock situation. [JVMTI Agent Deadlock](http://stackoverflow.com/questions/12291756/jvmti-agent-deadlock) — Saqib Ahmed, Dec 27 '16 at 04:43
1) It is hard to say more seeing only that small portion of code. Can you show the whole `CompiledMethodLoad` callback? 2) Are you sure you're only retransforming one class? Do you specifically ensure that you're retransforming the class you want? You would need to, e.g., check the declaring class name before calling retransform. — Stanislav Lukyanov, Dec 27 '16 at 13:59
3) `TieredCompilation` only means that there are multiple compilation levels. After you've retransformed a class VM has to throw away all the code compiled for it. If you retransform a class each time VM compiles its method, it will always throw compiled code away. You can use JIT after retransformation, you just shouldn't retransform the same class each time JIT have done something for you. 4) Deadlock is when an application cannot continue due to mutual locking, not when it runs slow. However, retransformation has to do some synchronization, which also may be the source of slow. — Stanislav Lukyanov, Dec 27 '16 at 14:04
I'm just retransforming the compiled methods once. During their first run. After that, they are free to execute. Anyway, I figured out a new problem and asked it in a separate question. Have a look. I'd be grateful for any help. [Dynamic Bytecode Instrumentation fails without any error](http://stackoverflow.com/questions/41355383/dynamic-bytecode-instrumentation-fails-without-any-error) — Saqib Ahmed, Dec 28 '16 at 04:38
I've added the whole `CompiledLoadMethod` call back function in the question. The code inside the `if` block executes only once. Tell me if any other information is required. — Saqib Ahmed, Dec 28 '16 at 04:48
I'm sure I'm transforming one class only. I'm sure I'm transforming the class I want. — Saqib Ahmed, Dec 28 '16 at 07:30

score 2 · Accepted Answer · edited Jun 20 '20 at 09:12

To answer my question:

No. I wasn't doing anything wrong. It's supposed to take that much overhead.

And here's the proof:

I used Jitwatch to get some insight in the problem. I profiled both ClassLoad time instrumentation and instrumentation after JIT invocation. I'm using same application code in both cases.

Class Load Time Instrumentation

Execution Time: 18 seconds approx.

Instrumentation During JIT invocation

Execution Time: 80 seconds approx.

Conclusion

We can clearly see here that when I try to instrument my code by calling RetransformClasses -> CLassFileLoadHook sequence in CompiledLoadEvent, JIT simply halts and then never gets invoked for the function I tried to instrument. It doesn't even do OSR compilations afterward. I summed up the reason for this behaviour of JIT in this answer. The follow up question is given here. Anybody who knows a workaround is most welcome to answer.

I tried to retransforming every class that was loaded before I attached to the JVM and my CPU usage takes a huge dip, Like the JVM does less work when retransformations happen. Did you notice anything similar? — Sam Thomas, May 15 '19 at 21:04

JVMTI RetransformClasses() is taking a lot of time

Code:

1 Answers1

Class Load Time Instrumentation

Instrumentation During JIT invocation

Conclusion