diff --git a/doc/compiler/jitserver/Build.md b/doc/compiler/jitserver/Build.md index 5c046c13504..d66c89f8fa6 100644 --- a/doc/compiler/jitserver/Build.md +++ b/doc/compiler/jitserver/Build.md @@ -1,5 +1,5 @@ # Caching in JITServer -Caching is an important aspect of `JITServer` and one of the main reasons for why `JITServer` client consumes less CPU time compared to local compilation. In this document I will explain why caching is important, what kind of entities we cache, and how some specific caches work. +Caching is an important aspect of `JITServer` and one of the main reasons for why `JITServer` client consumes less CPU time compared to local compilation. This document will explain why caching is important, what kind of entities we cache, and how some specific caches work. -# Why caching is important +## Why caching is important When server is compiling a method, it needs to know a lot of information that is located on the client side, e.g. the addresses of RAM classes, GC mode, class hierarchy information, interpreter profiling data and much more. To obtain this information, server will make a remote call over the network to the client, client will find it, and send it back to the server. During the course of a compilation server will make hundreds of such calls (on average), assuming no caching is involved. Why is it bad? First of all, remote calls are expensive, both in terms of latency and CPU, especially in terms of CPU. If the server makes hundreds of requests to the client just for one compilation, client will spend more CPU time just sending and receiving data over the network than it would take for it to compile a method locally. What makes it worse is that in order for the client to fetch the requested data it needs to do some work, which additionally increases CPU consumption. Caching helps alleviate this problem by storing the result of frequent remote calls on the server, so that if the same data is needed again, it can accesss it from the cache, instead of making a remote call. -# Types of caching +## Types of caching There are 2 types of caching that `JITServer` does: - Global (in persistent memory) and per-compilation (on the heap). Global caching (in persistent memory) is done for entities that will not change (or very unlikely to change) over the lifetime of a client JVM, e.g. GC mode, IProfiler data for compiled methods, class hierarchy. @@ -36,21 +36,18 @@ Global caching (in persistent memory) is done for entities that will not change Both types of caching are done on per-client basis, that is, if multiple clients are connected to the same server, they will not share caches, as that would make entities very complicated. Whenever possible, caching should be done globally, because hit rates will be higher, but one should be careful and make sure that the client data will not actually change. -# Important caches -## `ClientSessionData` +## Important caches +### `ClientSessionData` Stores all of the globally cached data; for a detailed description read [this](ClientSession.md). -## `CompilationInfoPerThreadRemote` +### `CompilationInfoPerThreadRemote` Most of the per-compilation caches are stored in `CompilationInfoPerThreadRemote`. Since most caches have very similar structure, i.e. hash map, we added templated methods for working with these caches. The methods are `initializePerCompilationCache`, `cacheToPerCompilationMap`, `getCachedValueFromPerCompilationMap`, `clearPerCompilationCache`. If a new cache is added, it should use these methods. At the end of compilation all local caches need to be reset by adding a call to `TR::CompilationInfoPerThreadRemote::clearPerCompilationCaches`. Some important per-compilation caches: -### `TR_ResolvedMethodInfoCache` -This cache stores pointers to resolved methods created for the current compilaton. It is important, because messages requesting the creation of resolved methods are some of the most frequent messages. Unfortunately, persistent caching does not seem possible, since every compilation creates its own resolved methods. -### `IPTableHeap_t` +#### `TR_ResolvedMethodInfoCache` +This cache stores pointers to resolved methods created for the current compilaton. It is important, because messages requesting the creation of resolved methods are some of the most frequent messages. Unfortunately, persistent caching does not seem possible, since every compilation creates its own resolved methods. For more information on resolved method caching, read [this](ResolvedMethod.md). +#### `IPTableHeap_t` This cache stores IProfiler data for methods whose profiling data might get updated. It is one of the two caches used by IProfiler. - Per-compilation cache: stores data for interpreted methods and a method currently being compiled. It is a nested hash table. The outer cache uses `IPTableHeap_t` hash table type, and takes `J9Method *` as a key. The inner table uses `IPTableHeapEntry`, and takes bytecode index as a key, and stores profiling data as a value. It is possible for this cache to contain outdated profiling data, because IProfiler might collect additional data on the client after we cache it. However, the lifetime of a compilation is pretty short, so using suboptimal profiling information does not affect performance. - Persistent cache: stores data for already compiled methods, because their interpreter profiling data will definitely not change. Uses a slightly different hash table type `IPTable_t`, because it's located inside entries of `J9MethodInfo`, so it only takes bytecode index as a key. -## `TR_ResolvedJ9JITServerMethod` -Some per-compilation caches are stored inside resolved methods, they store results of some resolved method query. These caches do not need to be explicitly cleared, since resolved methods only exist for the duration of one compilation. I am not sure if it is a good thing to have local caches spread out across 2 files, maybe it would be a good idea to move all caches from resolved method to `CompilationInfoPerThreadRemote`. - diff --git a/doc/compiler/jitserver/Problem.md b/doc/compiler/jitserver/Problem.md index 5f6860bdbe2..76f2d40da1b 100644 --- a/doc/compiler/jitserver/Problem.md +++ b/doc/compiler/jitserver/Problem.md @@ -1,5 +1,5 @@ +# Debugging JITServer -Usually typical OpenJ9 debugging procedures can be followed for JITServer. But there are a few JITServer specific tips and tricks. +Typical OpenJ9 debugging procedures can be followed for JITServer, but there are a few JITServer specific tips and tricks. +Unlike regular OpenJ9 where everything runs in one process, JITServer technology moves the JIT compiler activity into a separate process while constantly +working with addresses/objects that are only valid on the client. This makes it very easy to cause a bug on JITServer and may make it hard to catch. +In this document we will explore some common types of bugs and crashes that are encountered in JITServer development and how to deal with them. +This document should be expanded upon whenever possible. +## Server-side crashes +The most common type of a server crash is a segmentation fault caused by the compiler trying to dereference an address that is only valid on the client. +For example, dereferencing `J9Class` pointer will lead to immediate crash in the best case, or some corrupt data being written in the worst case. +If you suddenly discover a crash on the server that you haven't seen before, it's most likely caused by a change to mainline JIT code that did not account +for JITServer. If that's the case, the crash should be in the newly added code. You should find the commit that introduced it, and rewrite it in such a way +that accounts for JITServer, i.e. do not dereference client pointers directly but fetch the result from the client first. + +## Client-side crashes If the client crashes in `handleServerMessage` after receiving some bad data from the server, you will need to find out what the server was doing when it sent the message. The easiest way to do this is by running both the server and client in gdb, then sending the server a Ctrl-C after the client crashes. From the gdb prompt on the server you can type `thread apply all backtrace` and find the appropriate compilation thread to determine where the server was. -Please add to this file!! +## Crashes in compiled code +Another way a client can crash is when it runs the compiled method but something is wrong with it. These can be some of the tougher bugs to fix +because you will need to find the failing method and look at its trace log to find the problem. Sometimes, the failure is not caused by one compilation +but by a series of them, which makes debugging even harder. + +Tracing method compilation is the same as for non-JITServer - use `-Xjit:{}(traceFull,log=)` in the client options and both client +and server will produce trace logs. Passing options for limit files and other tracing options also works in the same way. +One useful technique to find JITServer-specific issues in compiled code is to compare trace log for a remotely compiled method +with a trace log for the same method but compiled locally. +Differences in logs may reveal bugs when JITServer takes incorrect optimizer/codegen paths. +Passing `-Xjit:enableJITServerFollowRemoteCompileWithLocalCompile` to the client makes every compilation take place both on the server +and locally, making it easier to generate logs for comparison. + +Unlike with other types of crashes, there are many possible reasons why JITServer might be producing incorrect compiled method body. +Here are some of them: +- Missing relocation: although most missing relocations are pretty rare in AOT code, it is possible that JITServer needs a relocation +that AOT does not. This is because JITServer supports more optimizations/better code paths than AOT since it does not have to make code +executable on any VM, just on the client one, and it can fetch client addresses during compilation. The fix is to either create a new +relocation or just embed the correct address in the compiled body right away. If it's something that needs to happen infrequently, +we usually go for the latter solution, as it is easier to implement. +- Missing VM information: client needs to inform the server about some VM options that can affect compilation, e.g. GC mode. +If some information about VM setup is missing, server might compile something that will not run correctly. Usually, such information +should go into the `VMInfo` object on the server. + diff --git a/doc/compiler/jitserver/README.md b/doc/compiler/jitserver/README.md index f8642a8bcd1..674d79602a4 100644 --- a/doc/compiler/jitserver/README.md +++ b/doc/compiler/jitserver/README.md @@ -1,5 +1,5 @@ -ResolvedMethod is a wrapper around a Java method which exists for the duration of a single compilation. It collects and manages information about a method and related classes. We need to extend it +# Resolved Methods +## What are resolved methods? -On the server, we extend `TR_ResolvedJ9JITServerMethod` from `TR_ResolvedJ9Method`. Upon instantiation, a mirror instance is created on the client. Instantiation happens either directly via the `createResolvedMethod` family, or indirectly using one of multiple `getResolvedXXXMethod` methods, which perform operations to locate a method of interest and then create the corresponding `ResolvedMethod`. +Resolved methods are wrappers around Java methods which exist for the duration of a single compilation. They collect and manage information about a method and related classes. +Resolved methods are some of the most commonly used entitites by the JIT compiler, thus it is important to handle them correctly and efficiently on JITServer. -Many method calls which require VM access are relayed to the client-side mirror. However, some values are cached at the server to avoid sending remote messages. Since all resolved methods are destroyed at the end of the compilation, this is a good choice as a cache for data which may be invalidated by class unloading/redefinition. +For non-JITServer, we mostly care about 2 classes representing a resolved method - `TR_ResolvedJ9Method` and `TR_ResolvedRelocatableJ9Method`. +The former represents resolved methods in a regular compilation, and the latter in an AOT compilation. + +## JITServer implementation + +### Class hierarchy +On the server, we implement two new classes to represent resolved methods - `TR_ResolvedJ9JITServerMethod` and `TR_ResolvedRelocatableJ9JITServerMethod`. +Same as for non-JITServer, the former is for regular compilations and the latter is for AOT. `TR_ResolvedJ9JITServerMethod` is extended from `TR_ResolvedJ9Method` and +`TR_ResolvedRelocatableJ9JITServerMethod` is extended from `TR_ResolvedJ9JITServerMethod`. + +### Client-side mirrors +Upon instantiation of a server-side resolved method, a resolved method is also created on the client that represents the same Java method. We call that client-side copy "mirror". + +Instantiation happens either directly via the `createResolvedMethod` family, or indirectly using one of multiple `getResolvedXXXMethod` methods, which perform operations to locate a method of interest and then create the corresponding `ResolvedMethod`. + +### Caching inside resolved methods +Some answers to resolved method queries are cached at the server to avoid sending remote messages. +All of this information is sent during method instantiation, inside `TR_ResolvedJ9JITServerMethodInfo`. +The cached values either won't change during the lifetime of a resolved method (i.e. constant pool pointer, J9 class pointer) +or having incorrect value cached for the duration of one compilation does not have impact on correctness or performance (i.e. whether a method is interpreted). + +Resolved methods could also be used as more general per-compilation caches, containing information that does not directly describe the resolved method itself. +Since resolved methods are destroyed at the end of compilation, this would be functionally correct. +However, such usage is discouraged because `TR::CompilationInfoPerThreadRemote` is a better place for such caches. Placing caches there makes them more visible +and it makes more sense to store information not related to resolved methods outside of them. + +### Performance considerations + +Since resolved methods are so frequently created and used, the number of remote messages associated with them is normally a large portion of total messages. Therefore, minimizing the number of requests for resolved method creation is important for improving CPU consumption. + +There are two main ways we optimize resolved method creation: +1. Caching all newly created resolved methods in a per-compilation cache on the server. We use `TR_ResolvedMethodInfoCache` and put it in `TR::CompilationInfoPerThreadRemote`. +Whenever a resolved method needs to be created, server will first check the cache and recreate the resolved method from there, if possible. +2. Prefetching multiple resolved methods before they are first requested. There are parts of the compiler that request many resolved methods in a loop. +For example, ILGen and inliner will walk through method bytecodes and create a resolved method for every method call bytecode. +We use `TR_ResolvedJ9JITServerMethod::cacheResolvedMethodsCallees` to do a preliminary walk and cache all required methods in one remote call. +Techniques like these do not reduce the total number of resolved methods created or data transferred through the network but it reduces the total number of remote calls, +which still has a positive impact on CPU consumption. + +Despite all these optimizations, resolved methods are still responsible for most of the remote messages. This is due to the fact that resolved methods only exist +for the duration of a single compilation so any type of persistent caching is not available to us. diff --git a/doc/compiler/jitserver/Sockets.md b/doc/compiler/jitserver/Sockets.md index fe251f98bd9..81b4c2f5c68 100644 --- a/doc/compiler/jitserver/Sockets.md +++ b/doc/compiler/jitserver/Sockets.md @@ -1,5 +1,5 @@ +# Usage + +This document describes how to use JITServer technology with any Java application and some basic options, such as logging and encryption. + +## Running applications with JITServer + JITServer adds two additional *personas* to the JVM: client mode and server mode. To start the JVM in server mode, run the `jitserver` launcher present under `bin` directory (along side `java` launcher). To start the JVM in client mode, there is a new command line option: `-XX:+UseJITServer` (under bin run `./java -XX:+UseJITServer `) In server mode, the JVM will halt after startup and begin listening for compilation requests from clients. No Java application is given to the `jitserver` launcher on the command line. Any option accepted by JVM can also be passed to the `jitserver` launcher. @@ -47,55 +53,94 @@ JCL - 03cb3a3cb4 based on jdk8u232-b09) ``` Note that `java -version` is itself a small Java application! If you have some application you'd like to test, you can substitute it in for `-version`. Run it as usual, but with the flag `-XX:+UseJITServer` before the application name. +## Logging + You might have noticed that running the client without any server to connect to still appears to work. This is because the client performs required JIT compilations locally if it cannot connect to a server. To ensure that everything is really working as intended, it is a good idea to enable some logging. It's often most convenient on the server side, because log messages will not interfere with application output, but logging can be added to either the server or the client. +### Verbose logs + With verbose logging, if a client connects successfully then server output should look something like this: ``` $ jitserver -Xjit:verbose=\{JITServer\} -JITServer is currently a technology preview. Its use is not yet supported - +JITServer is currently a technology preview. Its use is not yet supported. +#JITServer: JITServer version: 1.17.0 #JITServer: JITServer Server Mode. Port: 38400. Connection Timeout 30000ms -#JITServer: Started JITServer listener thread: 0000000000226C00 +#JITServer: Started JITServer listener thread: 000000000022AD00 + JITServer is ready to accept incoming requests -#JITServer: Server received request for stream 00007FEC658EF720 -#JITServer: Server allocated data for a new clientUID 11129135271614904954 -#JITServer: compThreadID=0 created clientSessionData=00007FEC658EFBE0 for clientUID=11129135271614904954 seqNo=0 -#JITServer: Server will process a list of 0 unloaded classes for clientUID 11129135271614904954 -#JITServer: compThreadID=0 has successfully compiled java/lang/Double.longBitsToDouble(J)D -#JITServer: compThreadID=0 found clientSessionData=00007FEC658EFBE0 for clientUID=11129135271614904954 seqNo=1 -#JITServer: Server will process a list of 0 unloaded classes for clientUID 11129135271614904954 -#JITServer: compThreadID=0 has successfully compiled jdk/internal/reflect/Reflection.getCallerClass()Ljava/lang/Class; -#JITServer: compThreadID=0 found clientSessionData=00007FEC658EFBE0 for clientUID=11129135271614904954 seqNo=2 -#JITServer: Server will process a list of 0 unloaded classes for clientUID 11129135271614904954 -#JITServer: compThreadID=0 has successfully compiled java/lang/System.getEncoding(I)Ljava/lang/String; +#JITServer: Server received request for stream 00007F50BC4E7AB0 +#JITServer: Server allocated data for a new clientUID 7315883206984276314 +#JITServer: compThreadID=0 created clientSessionData=00007F50BC4F0880 for clientUID=7315883206984276314 seqNo=1 (isCritical=1) (criticalSeqNo=0 lastProcessedCriticalReq=0) +#JITServer: compThreadID=0 will ask for address ranges of unloaded classes and CHTable for clientUID 7315883206984276314 +#JITServer: compThreadID=0 will initialize CHTable for clientUID 7315883206984276314 size=1152 +#JITServer: compThreadID=0 has successfully compiled java/lang/Object.()V +#JITServer: compThreadID=0 found clientSessionData=00007F50BC4F0880 for clientUID=7315883206984276314 seqNo=2 (isCritical=1) (criticalSeqNo=1 lastProcessedCriticalReq=1) +#JITServer: compThreadID=0 has successfully compiled com/ibm/jit/JITHelpers.classIsInterfaceFlag()I +#JITServer: compThreadID=0 found clientSessionData=00007F50BC4F0880 for clientUID=7315883206984276314 seqNo=3 (isCritical=1) (criticalSeqNo=2 lastProcessedCriticalReq=2) +#JITServer: compThreadID=0 has successfully compiled jdk/internal/misc/Unsafe.arrayBaseOffset(Ljava/lang/Class;)I +#JITServer: compThreadID=0 found clientSessionData=00007F50BC4F0880 for clientUID=7315883206984276314 seqNo=4 (isCritical=1) (criticalSeqNo=3 lastProcessedCriticalReq=3) ... ``` +JITServer prints out information about each new client connections, new streams, successful compilations, etc. +The same option can be used on the client to log similar information. + + +### JITServer version +Note that JITServer prints out version information that is different from `java -version` output. +To use JITServer, client and server must have matching versions. +To check client's version pass it `verbose=\{JITServer\}` option as well. +``` +$ java -XX:+UseJITServer -Xjit:verbose=\{JITServer\} -version + +JITServer is currently a technology preview. Its use is not yet supported +#JITServer: JITServer version: 1.17.0 +#JITServer: JITServer Client Mode. Server address: localhost port: 38400. Connection Timeout 2000ms +#JITServer: Identifier for current client JVM: 9238414254742342824 +... +``` + +### JITServer heartbeat +Server has a statistics thread running in the background, which can periodically print general information +about the state of the server. Use `-Xjit:statisticsFrequency=` option to enable heartbeat logging. +The output looks like this: +``` +$ jitserver -Xjit:statisticsFrequency=3000` # print heartbeat info every 3 seconds + +JITServer is currently a technology preview. Its use is not yet supported. + +JITServer is ready to accept incoming requests +#JITServer: Number of clients : 0 +#JITServer: Total compilation threads : 63 +#JITServer: Active compilation threads : 1 +#JITServer: Physical memory available: 12570 MB +#JITServer: CpuLoad 6% (AvgUsage 1%) JvmCpu 6% +``` -### Configuration +## Configuration -#### Hostname +### Hostname On the client, you can specify the server hostname or IP. ``` -$ java -XX:+UseJITServerAddress=example.com +$ java -XX:JITServerAddress=example.com ``` -#### Port +### Port By default, communication occurs on port 38400. You can change this by specifying the `-XX:JITServerPort` suboption as follows: ``` $ jitserver -XX:JITServerPort=1234 $ java -XX:+UseJITServer -XX:JITServerPort=1234 MyApplication ``` -#### Timeout +### Timeout If your network connection is flaky, you may want to adjust the timeout. Timeout is given in milliseconds using `-XX:JITServerTimeout` suboption. Client and server timeouts do not need to match. By default there is timeout of 30000 ms at the server and 2000 ms at the client. Typically the timeout at the server can be larger; it can afford to wait because there is nothing else to do anyway. Waiting too much at the client can be detrimental because the client has the option of compiling locally and make progress. ``` $ jitserver -XX:JITServerTimeout=5000 $ java -XX:+UseJITServer -XX:JITServerTimeout=5000 MyApplication ``` -#### Encryption (TLS) +### Encryption (TLS) By default, communication is not encrypted. If messages sent between the client and server need to traverse some untrusted network, you may want to set up encryption. Encryption reduces performance, so consider whether it is required for your use case. Encryption support is currently experimental. **The implementation has not yet undergone a security audit.** Additionally, only certain modes of operation are supported. If you require encryption but run into problems with the current implementation, then please consider opening an issue with your requirements.