Improve handling of HttpOutput.close() for pending writes

[sbordet]

Jetty version
9.4.23
Java version
Any
Description
Applications, especially reactive ones, may issue a final write and then think they can call AsyncContext.complete() without waiting for the write to be complete:

public void onWritePossible() {
  // Only one buffer to write.
  httpOutput.write(byteBuffer);
  asyncContext.complete();
}

This does not really respect the Servlet 3.1 Async I/O contract although the matter is underspecified.

Would be better if applications did this:

private boolean finished;
public void onWritePossible() {
  // Only one buffer to write.
  while (httpOutput.isReady()) {
    if (finished) {
      asyncContext.complete();
    } else {
      httpOutput.write(byteBuffer);
      finished = true;
  }
}

That is, applications should always call isReady() and make sure that the call to AsyncContext.complete() happens when the write is completed.

@gregw can you clarify this with the Servlet Expert Group?

[gregw]

See jakartaee/servlet#273

[gregw]

In addition, review of #4377 suggests that we need to better handle the ERROR state of HttpOuput and decide if it is a terminal or transient state.