Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

spring-boot-devtools java.net.SocketException: Unexpected end of file from server #10317

Closed
evgeniycheban opened this issue Sep 16, 2017 · 27 comments
Assignees
Labels
type: bug A general bug
Milestone

Comments

@evgeniycheban
Copy link
Contributor

I have an issue to reload my app running inside docker container using
spring-boot-devtools:1.5.6.RELEASE
When I connect to the remote app withorg.springframework.boot.devtools.RemoteSpringApplication and do some changes then after my changes were uploaded I'm getting:
java.net.SocketException: Unexpected end of file from server
I tried to get it work on my Mac and Linux laptop but the result is always the same. Here is my log:

2017-09-16 22:20:29.693  INFO 54043 --- [   File Watcher] o.s.b.d.r.c.ClassPathChangeUploader      : Uploaded 1 class resource
Exception in thread "File Watcher" java.lang.IllegalStateException: java.net.SocketException: Unexpected end of file from server
	at org.springframework.boot.devtools.remote.client.ClassPathChangeUploader.onApplicationEvent(ClassPathChangeUploader.java:100)
	at org.springframework.boot.devtools.remote.client.ClassPathChangeUploader.onApplicationEvent(ClassPathChangeUploader.java:58)
	at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:167)
	at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:139)
	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:393)
	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:347)
	at org.springframework.boot.devtools.classpath.ClassPathFileChangeListener.publishEvent(ClassPathFileChangeListener.java:68)
	at org.springframework.boot.devtools.classpath.ClassPathFileChangeListener.onChange(ClassPathFileChangeListener.java:64)
	at org.springframework.boot.devtools.filewatch.FileSystemWatcher$Watcher.fireListeners(FileSystemWatcher.java:306)
	at org.springframework.boot.devtools.filewatch.FileSystemWatcher$Watcher.updateSnapshots(FileSystemWatcher.java:299)
	at org.springframework.boot.devtools.filewatch.FileSystemWatcher$Watcher.scan(FileSystemWatcher.java:259)
	at org.springframework.boot.devtools.filewatch.FileSystemWatcher$Watcher.run(FileSystemWatcher.java:239)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketException: Unexpected end of file from server
	at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:792)
	at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1569)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474)
	at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
	at org.springframework.http.client.SimpleClientHttpResponse.getRawStatusCode(SimpleClientHttpResponse.java:52)
	at org.springframework.http.client.AbstractClientHttpResponse.getStatusCode(AbstractClientHttpResponse.java:33)
	at org.springframework.boot.devtools.remote.client.ClassPathChangeUploader.performUpload(ClassPathChangeUploader.java:116)
	at org.springframework.boot.devtools.remote.client.ClassPathChangeUploader.onApplicationEvent(ClassPathChangeUploader.java:97)
	... 12 more
2017-09-16 22:20:37.719  INFO 54043 --- [pool-1-thread-1] o.s.b.d.r.c.DelayedLiveReloadTrigger     : Remote server has changed, triggering LiveReload
2017-09-16 22:20:38.734  INFO 54043 --- [pool-1-thread-1] o.s.b.d.r.c.DelayedLiveReloadTrigger     : Remote server has changed, triggering LiveReload

Maybe it related to #1106

@spring-projects-issues spring-projects-issues added the status: waiting-for-triage An issue we've not yet triaged label Sep 16, 2017
@eddumelendez
Copy link
Contributor

@evgeniycheban I did the same using version 1.5.7.RELEASE and everything works well. Can you confirm this?

@snicoll
Copy link
Member

snicoll commented Sep 18, 2017

@evgeniycheban I second Eddu here. For a start, it would be nice to see if you can reproduce this outside of Docker.

@snicoll snicoll added the status: waiting-for-feedback We need additional information before we can continue label Sep 18, 2017
@evgeniycheban
Copy link
Contributor Author

evgeniycheban commented Sep 18, 2017

@snicoll outside of Docker everything works fine. @eddumelendez using 1.5.7.RELEASE I'm getting the same result.

@spring-projects-issues spring-projects-issues added status: feedback-provided Feedback has been provided and removed status: waiting-for-feedback We need additional information before we can continue labels Sep 18, 2017
@wilkinsona
Copy link
Member

@evgeniycheban Spring Boot doesn't do anything special in a Docker environment and it expects the network to function as it would elsewhere. Can you please explain why you think this is a problem with Spring Boot rather than Docker?

@wilkinsona wilkinsona added status: waiting-for-feedback We need additional information before we can continue and removed status: feedback-provided Feedback has been provided labels Sep 18, 2017
@evgeniycheban
Copy link
Contributor Author

evgeniycheban commented Sep 18, 2017

@wilkinsona I tried to run org.springframework.boot.devtools.RemoteSpringApplication in debug mode and put breakpoint on ClassPathChangeUploader.performUpload and if I wait 3-5 seconds it will be successfully complete, it seem I need some sort of delay. For now I'm not sure it is a Docker network issue or Spring. @eddumelendez can you please provide your Dockerfile so I can compare it with mine?

@spring-projects-issues spring-projects-issues added status: feedback-provided Feedback has been provided and removed status: waiting-for-feedback We need additional information before we can continue labels Sep 18, 2017
@eddumelendez
Copy link
Contributor

@evgeniycheban I put the Dockerfile at the root of my project

FROM java:8

COPY target/*.jar /opt/demo.jar

ENTRYPOINT ["java", "-jar", "/opt/demo.jar"]

Then, perform the following commands to build the image and run the container. Ensure you are exposing the port 8080 or whatever you are using

docker build -t bootdemo .

docker run -p 8080:8080 bootdemo

@evgeniycheban
Copy link
Contributor Author

It seems that this error occurs only if I running it from Intellij IDEA I tried to run it from Eclipse and everything works well now.

@wilkinsona
Copy link
Member

@evgeniycheban Thanks for the update. Boot doesn't doing anything special when running in IDEA vs Eclipse. I'm going to close this one as there's no evidence at this stage that it's a problem with Boot.

@wilkinsona wilkinsona added status: invalid An issue that we don't feel is valid and removed status: feedback-provided Feedback has been provided status: waiting-for-triage An issue we've not yet triaged labels Sep 18, 2017
@evgeniycheban
Copy link
Contributor Author

@wilkinsona ok thank you.

@steveratay
Copy link

@evgeniycheban Did you ever figure this out? I have the exact same issue using IDEA to run the RemoteSpringApplication and connecting to a Boot app running inside a Docker container. I'm using Docker Compose and running on a Mac.

@evgeniycheban
Copy link
Contributor Author

@steveratay No I didn't find any solution. I'm also having this issue in Eclipse, but it happens very rarely. I think it's Docker's issue because I tried to update app running on the VPS server and it works fine.

@alexIsAtGitHub
Copy link

@evgeniycheban @wilkinsona I am also having the same issue. Did some additional testing and it appears to be caused by dockers bridge networking. When running the container on the host network it works fine. When its behind the bridge it looks like the stream is being cut pre-maturely and the exception "java.net.SocketException: Unexpected end of file from server" is being thrown as it would.
@wilkinsona I can't tell if this is an issue with Dockers network stack or the way DevTools is tunnelling jdwp. I'm not sure if I can get between the two to view sniff the traffic. Could this be a combination of the two technologies and at least worth an investigation as it is easily reproducible?

Thanks,
Alex

@wilkinsona
Copy link
Member

@alexIsAtGitHub Thanks for the analysis. Spring Boot isn’t doing any tunnelling in this scenario. It’s a normal HTTP connection to the application so there’s no custom network code involved on Boot’s side.

@alexIsAtGitHub
Copy link

alexIsAtGitHub commented Oct 31, 2017 via email

@wilkinsona
Copy link
Member

Is Spring devtools just piggy
backing on the tomcat server port specified at launch?

Yes, that's right. It's just a plain HTTP connection made to Tomcat (or Jetty or Undertow).

@alexIsAtGitHub
Copy link

@wilkinsona ok good to know, I have it working now. @evgeniycheban & @steveratay I did manage to get this working properly after receiving the same error as you. Dev tools either isn't loaded inside docker or you're trying to connect to the incorrect port with your remote session. Check your output from docker when starting the spring boot application and verify that the port its bound to is the same one you are trying to reference in your remote session. Spring Remote is piggy backing on the existing tomcat http socket.

@fabietto1975
Copy link

I all, I've the same issue reported above; I'm using Spring Boot 1.5.8.BUILD-SNAPSHOT.
All the port mapping looks correct.

This is the docker file:

FROM java:8
VOLUME /tmp
ADD myapp-0.0.1-SNAPSHOT.jar app.jar
EXPOSE 8080
RUN sh -c 'touch /app.jar'
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"]

and i run the docker container with the -p 8080:8080 argument.
The remote org.springframework.boot.devtools.RemoteSpringApplication java application ha "http://localhost:8080/" as parameter.
I'm using Eclipse and when i perform a local change I see rising the exception:

Exception in thread "File Watcher" java.lang.IllegalStateException: java.net.SocketException: Unexpected end of file from server

The bytecode change is published but the RemoteSpringApplication dies.

TIA and regards.

@wilkinsona
Copy link
Member

@fabietto1975 As far as we know, this is an issue with Docker's networking. If you have evidence that suggests it's Spring Boot that's at fault, please share it and we'll take a look.

@steveratay
Copy link

steveratay commented Nov 27, 2017

The problem for me appears to be that the application I am trying to reload dies with a java.lang.ClassNotFoundException when the Spring Boot app is restarted. The class it's saying it can't load is the one that I modified in IntelliJ.

Logs from IntelliJ:

  .   ____          _                                              __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _          ___               _      \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` |        | _ \___ _ __  ___| |_ ___ \ \ \ \
 \\/  ___)| |_)| | | | | || (_| []::::::[]   / -_) '  \/ _ \  _/ -_) ) ) ) )
  '  |____| .__|_| |_|_| |_\__, |        |_|_\___|_|_|_\___/\__\___|/ / / /
 =========|_|==============|___/===================================/_/_/_/
 :: Spring Boot Remote ::  (v1.5.8.RELEASE)

2017-11-27 09:33:17.963  INFO 63956 --- [           main] o.s.b.devtools.RemoteSpringApplication   : Starting RemoteSpringApplication v1.5.8.RELEASE on Steve-Ratays-MacBook-Pro.local with PID 63956 (/Users/steveratay/.m2/repository/org/springframework/boot/spring-boot-devtools/1.5.8.RELEASE/spring-boot-devtools-1.5.8.RELEASE.jar started by steveratay in /Users/steveratay/te2/code/data-connect)
2017-11-27 09:33:17.968  INFO 63956 --- [           main] o.s.b.devtools.RemoteSpringApplication   : No active profile set, falling back to default profiles: default
2017-11-27 09:33:18.002  INFO 63956 --- [           main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@32910148: startup date [Mon Nov 27 09:33:17 PST 2017]; root of context hierarchy
2017-11-27 09:33:18.321  INFO 63956 --- [           main] f.a.AutowiredAnnotationBeanPostProcessor :  'javax.inject.Inject' annotation found and supported for autowiring
2017-11-27 09:33:18.523  WARN 63956 --- [           main] o.s.b.d.r.c.RemoteClientConfiguration    : The connection to http://localhost:9000 is insecure. You should use a URL starting with 'https://'.
The Class-Path manifest attribute in /Users/steveratay/.m2/repository/org/liquibase/liquibase-core/3.5.3/liquibase-core-3.5.3.jar referenced one or more files that do not exist: file:/Users/steveratay/.m2/repository/org/liquibase/liquibase-core/3.5.3/lib/snakeyaml-1.13.jar
2017-11-27 09:33:18.653  WARN 63956 --- [           main] o.s.b.d.a.OptionalLiveReloadServer       : Unable to start LiveReload server
2017-11-27 09:33:18.681  INFO 63956 --- [           main] o.s.b.devtools.RemoteSpringApplication   : Started RemoteSpringApplication in 1.435 seconds (JVM running for 2.185)
2017-11-27 09:33:39.721  INFO 63956 --- [pool-1-thread-1] o.s.b.d.r.c.DelayedLiveReloadTrigger     : Remote server has changed, triggering LiveReload
2017-11-27 09:33:39.771  INFO 63956 --- [   File Watcher] o.s.b.d.r.c.ClassPathChangeUploader      : Uploaded 1 class resource
Exception in thread "File Watcher" java.lang.IllegalStateException: java.net.SocketException: Unexpected end of file from server
	at org.springframework.boot.devtools.remote.client.ClassPathChangeUploader.onApplicationEvent(ClassPathChangeUploader.java:100)
	at org.springframework.boot.devtools.remote.client.ClassPathChangeUploader.onApplicationEvent(ClassPathChangeUploader.java:58)
	at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:172)
	at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:165)
	at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:139)
	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:393)
	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:347)
	at org.springframework.boot.devtools.classpath.ClassPathFileChangeListener.publishEvent(ClassPathFileChangeListener.java:68)
	at org.springframework.boot.devtools.classpath.ClassPathFileChangeListener.onChange(ClassPathFileChangeListener.java:64)
	at org.springframework.boot.devtools.filewatch.FileSystemWatcher$Watcher.fireListeners(FileSystemWatcher.java:307)
	at org.springframework.boot.devtools.filewatch.FileSystemWatcher$Watcher.updateSnapshots(FileSystemWatcher.java:300)
	at org.springframework.boot.devtools.filewatch.FileSystemWatcher$Watcher.scan(FileSystemWatcher.java:260)
	at org.springframework.boot.devtools.filewatch.FileSystemWatcher$Watcher.run(FileSystemWatcher.java:240)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketException: Unexpected end of file from server
	at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:792)
	at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1536)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)
	at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
	at org.springframework.http.client.SimpleClientHttpResponse.getRawStatusCode(SimpleClientHttpResponse.java:52)
	at org.springframework.http.client.AbstractClientHttpResponse.getStatusCode(AbstractClientHttpResponse.java:33)
	at org.springframework.boot.devtools.remote.client.ClassPathChangeUploader.performUpload(ClassPathChangeUploader.java:116)
	at org.springframework.boot.devtools.remote.client.ClassPathChangeUploader.onApplicationEvent(ClassPathChangeUploader.java:97)
	... 13 more

Logs from remote application:

data-connect_1                | 2017-11-27 17:33:36.238  INFO 1 --- [nio-9000-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring FrameworkServlet 'dispatcherServlet'
data-connect_1                | 2017-11-27 17:33:36.239  INFO 1 --- [nio-9000-exec-1] o.s.web.servlet.DispatcherServlet        : FrameworkServlet 'dispatcherServlet': initialization started
data-connect_1                | 2017-11-27 17:33:38.283  INFO 1 --- [nio-9000-exec-1] o.s.web.servlet.DispatcherServlet        : FrameworkServlet 'dispatcherServlet': initialization completed in 2042 ms
data-connect_1                | 2017-11-27 17:33:38.975  INFO 1 --- [       Thread-0] s.c.a.AnnotationConfigApplicationContext : Closing org.springframework.context.annotation.AnnotationConfigApplicationContext@25d250c6: startup date [Tue Nov 21 07:18:33 GMT 2017]; root of context hierarchy
data-connect_1                | 2017-11-27 17:33:38.986  INFO 1 --- [       Thread-0] ationConfigEmbeddedWebApplicationContext : Closing org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@29d80d2b: startup date [Tue Nov 21 07:18:35 GMT 2017]; parent: org.springframework.context.annotation.AnnotationConfigApplicationContext@25d250c6
data-connect_1                | 2017-11-27 17:33:39.022  INFO 1 --- [       Thread-0] o.s.c.support.DefaultLifecycleProcessor  : Stopping beans in phase 2147483647
data-connect_1                | 2017-11-27 17:33:39.027  INFO 1 --- [       Thread-0] o.s.c.support.DefaultLifecycleProcessor  : Stopping beans in phase 0
data-connect_1                | 2017-11-27 17:33:39.046  INFO 1 --- [       Thread-0] o.s.j.e.a.AnnotationMBeanExporter        : Unregistering JMX-exposed beans on shutdown
data-connect_1                | 2017-11-27 17:33:39.047  INFO 1 --- [       Thread-0] o.s.j.e.a.AnnotationMBeanExporter        : Unregistering JMX-exposed beans
data-connect_1                | 2017-11-27 17:33:39.401  INFO 1 --- [       Thread-0] s.c.a.AnnotationConfigApplicationContext : Closing org.springframework.context.annotation.AnnotationConfigApplicationContext@4339e0de: startup date [Tue Nov 21 07:18:44 GMT 2017]; parent: org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@29d80d2b
data-connect_1                | 2017-11-27 17:33:39.406  INFO 1 --- [       Thread-0] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
data-connect_1                | 2017-11-27 17:33:42.025  INFO 1 --- [  restartedMain] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@3ef5829d: startup date [Mon Nov 27 17:33:42 GMT 2017]; root of context hierarchy
data-connect_1                | 2017-11-27 17:33:42.179  INFO 1 --- [  restartedMain] f.a.AutowiredAnnotationBeanPostProcessor :  'javax.inject.Inject' annotation found and supported for autowiring
data-connect_1                | 2017-11-27 17:33:42.272  INFO 1 --- [  restartedMain] trationDelegate$BeanPostProcessorChecker : Bean 'configurationPropertiesRebinderAutoConfiguration' of type [org.springframework.cloud.autoconfigure.ConfigurationPropertiesRebinderAutoConfiguration$$EnhancerBySpringCGLIB$$aca54ed6] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
data-connect_1                | 
data-connect_1                |   .   ____          _            __ _ _
data-connect_1                |  /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
data-connect_1                | ( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
data-connect_1                |  \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
data-connect_1                |   '  |____| .__|_| |_|_| |_\__, | / / / /
data-connect_1                |  =========|_|==============|___/=/_/_/_/
data-connect_1                |  :: Spring Boot ::        (v1.5.8.RELEASE)
data-connect_1                | 
data-connect_1                | 2017-11-27 17:33:42.625  INFO 1 --- [  restartedMain] c.c.c.ConfigServicePropertySourceLocator : Fetching config from server at: http://spring-cloud-config-server:8888
data-connect_1                | 2017-11-27 17:33:43.918  INFO 1 --- [  restartedMain] c.c.c.ConfigServicePropertySourceLocator : Located environment: name=data-connect, profiles=[local,compose], label=null, version=null, state=null
data-connect_1                | 2017-11-27 17:33:43.918  INFO 1 --- [  restartedMain] b.c.PropertySourceBootstrapConfiguration : Located property source: CompositePropertySource [name='configService', propertySources=[MapPropertySource {name='file:config/application.yml'}]]
data-connect_1                | 2017-11-27 17:33:43.934  INFO 1 --- [  restartedMain] b.t.d.DataConnectServiceApplication      : The following profiles are active: local,compose
data-connect_1                | 2017-11-27 17:33:43.944  INFO 1 --- [  restartedMain] ationConfigEmbeddedWebApplicationContext : Refreshing org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@7d62afc7: startup date [Mon Nov 27 17:33:43 GMT 2017]; parent: org.springframework.context.annotation.AnnotationConfigApplicationContext@3ef5829d
data-connect_1                | 2017-11-27 17:33:43.965  WARN 1 --- [  restartedMain] ationConfigEmbeddedWebApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanDefinitionStoreException: Failed to read candidate component class: URL [jar:file:/opt/data-connect-service.jar!/BOOT-INF/classes!/biz/te2/dataconnect/repo/WranglerExecutionRepository.class]; nested exception is org.springframework.beans.factory.CannotLoadBeanClassException: Cannot find class [biz.te2.dataconnect.controllers.WranglerController] for bean with name 'wranglerController' defined in URL [jar:file:/opt/data-connect-service.jar!/BOOT-INF/classes!/biz/te2/dataconnect/controllers/WranglerController.class]; nested exception is java.lang.ClassNotFoundException: biz.te2.dataconnect.controllers.WranglerController
data-connect_1                | 2017-11-27 17:33:43.967 ERROR 1 --- [  restartedMain] o.s.b.f.s.DefaultListableBeanFactory     : Destroy method on bean with name 'org.springframework.boot.autoconfigure.internalCachingMetadataReaderFactory' threw an exception
data-connect_1                | 
data-connect_1                | java.lang.IllegalStateException: ApplicationEventMulticaster not initialized - call 'refresh' before multicasting events via the context: org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@7d62afc7: startup date [Mon Nov 27 17:33:43 GMT 2017]; parent: org.springframework.context.annotation.AnnotationConfigApplicationContext@3ef5829d
data-connect_1                | 	at org.springframework.context.support.AbstractApplicationContext.getApplicationEventMulticaster(AbstractApplicationContext.java:414) [spring-context-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
data-connect_1                | 	at org.springframework.context.support.ApplicationListenerDetector.postProcessBeforeDestruction(ApplicationListenerDetector.java:97) ~[spring-context-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
data-connect_1                | 	at org.springframework.beans.factory.support.DisposableBeanAdapter.destroy(DisposableBeanAdapter.java:253) ~[spring-beans-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
data-connect_1                | 	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroyBean(DefaultSingletonBeanRegistry.java:578) [spring-beans-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
data-connect_1                | 	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingleton(DefaultSingletonBeanRegistry.java:554) [spring-beans-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
data-connect_1                | 	at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingleton(DefaultListableBeanFactory.java:961) [spring-beans-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
data-connect_1                | 	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingletons(DefaultSingletonBeanRegistry.java:523) [spring-beans-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
data-connect_1                | 	at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.destroySingletons(FactoryBeanRegistrySupport.java:230) [spring-beans-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
data-connect_1                | 	at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingletons(DefaultListableBeanFactory.java:968) [spring-beans-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
data-connect_1                | 	at org.springframework.context.support.AbstractApplicationContext.destroyBeans(AbstractApplicationContext.java:1030) [spring-context-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
data-connect_1                | 	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:556) [spring-context-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
data-connect_1                | 	at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.refresh(EmbeddedWebApplicationContext.java:122) [spring-boot-1.5.8.RELEASE.jar!/:1.5.8.RELEASE]
data-connect_1                | 	at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:693) [spring-boot-1.5.8.RELEASE.jar!/:1.5.8.RELEASE]
data-connect_1                | 	at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:360) [spring-boot-1.5.8.RELEASE.jar!/:1.5.8.RELEASE]
data-connect_1                | 	at org.springframework.boot.SpringApplication.run(SpringApplication.java:303) [spring-boot-1.5.8.RELEASE.jar!/:1.5.8.RELEASE]
data-connect_1                | 	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1118) [spring-boot-1.5.8.RELEASE.jar!/:1.5.8.RELEASE]
data-connect_1                | 	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1107) [spring-boot-1.5.8.RELEASE.jar!/:1.5.8.RELEASE]
data-connect_1                | 	at biz.te2.dataconnect.DataConnectServiceApplication.main(DataConnectServiceApplication.java:37) [classes!/:na]
data-connect_1                | 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_131]
data-connect_1                | 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_131]
data-connect_1                | 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_131]
data-connect_1                | 	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_131]
data-connect_1                | 	at org.springframework.boot.devtools.restart.RestartLauncher.run(RestartLauncher.java:49) [spring-boot-devtools-1.5.8.RELEASE.jar!/:1.5.8.RELEASE]
data-connect_1                | 
data-connect_1                | 2017-11-27 17:33:43.979 ERROR 1 --- [  restartedMain] o.s.boot.SpringApplication               : Application startup failed
data-connect_1                | 
data-connect_1                | org.springframework.beans.factory.BeanDefinitionStoreException: Failed to read candidate component class: URL [jar:file:/opt/data-connect-service.jar!/BOOT-INF/classes!/biz/te2/dataconnect/repo/WranglerExecutionRepository.class]; nested exception is org.springframework.beans.factory.CannotLoadBeanClassException: Cannot find class [biz.te2.dataconnect.controllers.WranglerController] for bean with name 'wranglerController' defined in URL [jar:file:/opt/data-connect-service.jar!/BOOT-INF/classes!/biz/te2/dataconnect/controllers/WranglerController.class]; nested exception is java.lang.ClassNotFoundException: biz.te2.dataconnect.controllers.WranglerController
data-connect_1                | 	at org.springframework.context.annotation.ClassPathScanningCandidateComponentProvider.findCandidateComponents(ClassPathScanningCandidateComponentProvider.java:311) ~[spring-context-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
data-connect_1                | 	at org.springframework.context.annotation.ClassPathBeanDefinitionScanner.doScan(ClassPathBeanDefinitionScanner.java:272) ~[spring-context-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
data-connect_1                | 	at org.springframework.context.annotation.ComponentScanAnnotationParser.parse(ComponentScanAnnotationParser.java:135) ~[spring-context-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
data-connect_1                | 	at org.springframework.context.annotation.ConfigurationClassParser.doProcessConfigurationClass(ConfigurationClassParser.java:287) ~[spring-context-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
data-connect_1                | 	at org.springframework.context.annotation.ConfigurationClassParser.processConfigurationClass(ConfigurationClassParser.java:245) ~[spring-context-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
data-connect_1                | 	at org.springframework.context.annotation.ConfigurationClassParser.parse(ConfigurationClassParser.java:198) ~[spring-context-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
data-connect_1                | 	at org.springframework.context.annotation.ConfigurationClassParser.parse(ConfigurationClassParser.java:167) ~[spring-context-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
data-connect_1                | 	at org.springframework.context.annotation.ConfigurationClassPostProcessor.processConfigBeanDefinitions(ConfigurationClassPostProcessor.java:308) ~[spring-context-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
data-connect_1                | 	at org.springframework.context.annotation.ConfigurationClassPostProcessor.postProcessBeanDefinitionRegistry(ConfigurationClassPostProcessor.java:228) ~[spring-context-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
data-connect_1                | 	at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanDefinitionRegistryPostProcessors(PostProcessorRegistrationDelegate.java:272) ~[spring-context-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
data-connect_1                | 	at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanFactoryPostProcessors(PostProcessorRegistrationDelegate.java:92) ~[spring-context-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
data-connect_1                | 	at org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:687) ~[spring-context-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
data-connect_1                | 	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:525) ~[spring-context-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
data-connect_1                | 	at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.refresh(EmbeddedWebApplicationContext.java:122) ~[spring-boot-1.5.8.RELEASE.jar!/:1.5.8.RELEASE]
data-connect_1                | 	at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:693) [spring-boot-1.5.8.RELEASE.jar!/:1.5.8.RELEASE]
data-connect_1                | 	at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:360) [spring-boot-1.5.8.RELEASE.jar!/:1.5.8.RELEASE]
data-connect_1                | 	at org.springframework.boot.SpringApplication.run(SpringApplication.java:303) [spring-boot-1.5.8.RELEASE.jar!/:1.5.8.RELEASE]
data-connect_1                | 	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1118) [spring-boot-1.5.8.RELEASE.jar!/:1.5.8.RELEASE]
data-connect_1                | 	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1107) [spring-boot-1.5.8.RELEASE.jar!/:1.5.8.RELEASE]
data-connect_1                | 	at biz.te2.dataconnect.DataConnectServiceApplication.main(DataConnectServiceApplication.java:37) [classes!/:na]
data-connect_1                | 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_131]
data-connect_1                | 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_131]
data-connect_1                | 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_131]
data-connect_1                | 	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_131]
data-connect_1                | 	at org.springframework.boot.devtools.restart.RestartLauncher.run(RestartLauncher.java:49) [spring-boot-devtools-1.5.8.RELEASE.jar!/:1.5.8.RELEASE]
data-connect_1                | Caused by: org.springframework.beans.factory.CannotLoadBeanClassException: Cannot find class [biz.te2.dataconnect.controllers.WranglerController] for bean with name 'wranglerController' defined in URL [jar:file:/opt/data-connect-service.jar!/BOOT-INF/classes!/biz/te2/dataconnect/controllers/WranglerController.class]; nested exception is java.lang.ClassNotFoundException: biz.te2.dataconnect.controllers.WranglerController
data-connect_1                | 	at org.springframework.beans.factory.support.AbstractBeanFactory.resolveBeanClass(AbstractBeanFactory.java:1385) ~[spring-beans-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
data-connect_1                | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.determineTargetType(AbstractAutowireCapableBeanFactory.java:641) ~[spring-beans-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
data-connect_1                | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.predictBeanType(AbstractAutowireCapableBeanFactory.java:609) ~[spring-beans-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
data-connect_1                | 	at org.springframework.beans.factory.support.AbstractBeanFactory.isFactoryBean(AbstractBeanFactory.java:1484) ~[spring-beans-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
data-connect_1                | 	at org.springframework.beans.factory.support.DefaultListableBeanFactory.doGetBeanNamesForType(DefaultListableBeanFactory.java:425) ~[spring-beans-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
data-connect_1                | 	at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanNamesForType(DefaultListableBeanFactory.java:395) ~[spring-beans-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
data-connect_1                | 	at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeansOfType(DefaultListableBeanFactory.java:515) ~[spring-beans-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
data-connect_1                | 	at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeansOfType(DefaultListableBeanFactory.java:508) ~[spring-beans-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
data-connect_1                | 	at org.springframework.boot.context.TypeExcludeFilter.match(TypeExcludeFilter.java:65) ~[spring-boot-1.5.8.RELEASE.jar!/:1.5.8.RELEASE]
data-connect_1                | 	at org.springframework.context.annotation.ClassPathScanningCandidateComponentProvider.isCandidateComponent(ClassPathScanningCandidateComponentProvider.java:349) ~[spring-context-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
data-connect_1                | 	at org.springframework.context.annotation.ClassPathScanningCandidateComponentProvider.findCandidateComponents(ClassPathScanningCandidateComponentProvider.java:288) ~[spring-context-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
data-connect_1                | 	... 24 common frames omitted
data-connect_1                | Caused by: java.lang.ClassNotFoundException: biz.te2.dataconnect.controllers.WranglerController
data-connect_1                | 	at org.springframework.boot.devtools.restart.classloader.RestartClassLoader.loadClass(RestartClassLoader.java:143) ~[spring-boot-devtools-1.5.8.RELEASE.jar!/:1.5.8.RELEASE]
data-connect_1                | 	at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[na:1.8.0_131]
data-connect_1                | 	at org.springframework.util.ClassUtils.forName(ClassUtils.java:250) ~[spring-core-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
data-connect_1                | 	at org.springframework.beans.factory.support.AbstractBeanDefinition.resolveBeanClass(AbstractBeanDefinition.java:401) ~[spring-beans-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
data-connect_1                | 	at org.springframework.beans.factory.support.AbstractBeanFactory.doResolveBeanClass(AbstractBeanFactory.java:1432) ~[spring-beans-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
data-connect_1                | 	at org.springframework.beans.factory.support.AbstractBeanFactory.resolveBeanClass(AbstractBeanFactory.java:1377) ~[spring-beans-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
data-connect_1                | 	... 34 common frames omitted
data-connect_1                | 
dataconnectservice_data-connect_1 exited with code 0

@philwebb
Copy link
Member

@steveratay That's usually a sign that the classloader split is incorrect. From the stacktrace above It looks like you have a data-connect-service.jar. Is that one loaded into your IDE? You might want to try setting a restart.include property to pull it up. See this section of the docs.

@steveratay
Copy link

I tried adding a restart.include property. That seems to have allowed me to do one reload, but the second reload fails with the same error. Seems like the line of code throwing the exception is https://github.com/spring-projects/spring-boot/blob/v1.5.8.RELEASE/spring-boot-devtools/src/main/java/org/springframework/boot/devtools/restart/classloader/RestartClassLoader.java#L143. What does Kind.DELETED mean?

@tunesmith
Copy link

A bit of a twist in my case. My app has the main app running on port 8082, and the admin section running on port 8081. When I configure SpringRemoteApplication against 8082 (via an IntelliJ run/debug config using "http://localhost:8082" as the "Program arguments:") and change a class that runs on 8082 (the app runs fine using both these ports), I get this error upon Rebuilding. I have both 8081 and 8082 exposed in docker-compose.yml. (As well as 50505 for debugger purposes.)

If I reconfigure to run the entire app to run only on 8081 and configure SpringRemoteApplication to use 8081, but with the same ports exposed, I still get the same problem.

But if I turn off the port exposure for 8082 in docker-compose-yml, the problem goes away. (Note that 50505 is still turned on.)

I verified this a few times. App only runs on 8081, SpringRemoteApplication is configured against 8081, 8081 is exposed. If 8082 is exposed, it stops working. If it is unexposed, it starts working again. Same docker image, just an altered docker-compose config.

Finally, if I go back to using two ports, but instead of using 8081 and 8082, I use 8081 and 9082, then everything works fine.

So, I've either discovering something really special about port 8082 either with devtools or with docker, or I've got some kind of idiosyncrasy on my own system, perhaps another running container somewhere that is listening for that port. lsof doesn't show anything, but I could be missing something.

I guess the moral of the story is to try changing the ports in your configs and see if the problem goes away.

@wilkinsona
Copy link
Member

Thanks for the analysis, @tunesmith. There's nothing special in DevTools for port 8082. It just uses HTTP over whatever port you tell it to use. I would guess, but it is really just a guess, that this is an oddity of docker or your own system.

@BertrandNOEL
Copy link

Digging this issue up for people who still encounter this.
Issues appear only when using a port exposed to the host (using the -p 8080:8080). I did not manage to find why, but as a workaround, you can use the container ip instead of localhost in the program argument of RemoteSpringApplication (change http://localhost:8080 to http://172.17.0.2:8080 for example. Get container ip using docker inspect $container_name). It's not perfect because ip changes, but it solves the above issues at least.

@asoltesz
Copy link

asoltesz commented Feb 12, 2019

I am having the same issue with my Docker environment, using Spring Boot 2.0.1, Docker 18.06 on Ubuntu 18.04 x64, IntelliJ IDEA 18.2.2 with Gradle.

It would be nice if the FileWatcher capability wouldn't die in RemoteSpringApplication even if there is a networking error. This may happen not only with Docker but for any reason, say a short-lived networking outage between the remote Spring Boot application and the client proxy application.

Currently, if there is a networking error like the above, RemoteSpringApplication stops watching for the filesystem and no updates ill be sent to the remote app.

I can confirm that using the container IP works properly. If I connect via the container IP, there is no error and I can submit code changes repeatedly.

@wilkinsona
Copy link
Member

We added some retry logic for connection failures in b1dd928. That should cover the majority of cases where there's a network outage. When the server returns an incomplete or empty response, I'm not completely sure that carrying on or retrying would be the right thing to do as we don't know what state things are in. Piling changes on top may just make things worse. That said, given that this is development-only, perhaps carrying on as best we can is the better option. The app can always be stopped and restarted if things get in a mess.

@wilkinsona wilkinsona added the for: team-attention An issue we'd like other members of the team to review label Feb 13, 2019
@wilkinsona
Copy link
Member

We're going to broaden the retry.

@wilkinsona wilkinsona reopened this Feb 13, 2019
@wilkinsona wilkinsona added type: bug A general bug and removed for: team-attention An issue we'd like other members of the team to review status: invalid An issue that we don't feel is valid labels Feb 13, 2019
@wilkinsona wilkinsona added this to the 2.1.x milestone Feb 13, 2019
@wilkinsona wilkinsona self-assigned this Feb 14, 2019
@wilkinsona wilkinsona modified the milestones: 2.1.x, 2.1.3 Feb 14, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: bug A general bug
Projects
None yet
Development

No branches or pull requests