Check failure stack trace

[sevenold]

部署paddleOCR脚本

python -m paddle_serving_server_gpu.serve --model server_model/server_rec  --port 9294 --gpu_id 2  --thread 2 --max_body_size 1073741824

python -m paddle_serving_server_gpu.serve --model server_model/server_det  --port 9293 --gpu_id 2  --thread 2 --max_body_size 1073741824

版本

• paddle-serving-server-gpu 0.3.2.post10
• Ubuntu 16.10
• docker images paddlepaddle/serving:latest-cuda10.0-cudnn7

部署成功后，40万图片经过256线程的并发测试，跑几个小时后，会堵塞住，然后停留在这个错误！！！

2020-10-15T03:45:09.161677292Z E1015 11:45:08.967919    14 factory.h:149] RAW: Insert duplicate with tag: WeightedRandomRender
2020-10-15T03:45:09.161723927Z I1015 11:45:08.967919    14 abtest.h:67] RAW: Factory has been registed: WeightedRandomRender->EndpointRouterBase.
2020-10-15T03:45:09.177337994Z E1015 11:45:08.967919    14 factory.h:149] RAW: Insert duplicate with tag: WeightedRandomRender
2020-10-15T03:45:09.177399889Z I1015 11:45:08.967919    14 abtest.h:67] RAW: Factory has been registed: WeightedRandomRender->EndpointRouterBase.
2020-10-15T03:45:10.039389100Z INFO:     192.168.88.35:43330 - "POST /api/models/predict HTTP/1.1" 200 OK
2020-10-15T03:45:10.096505252Z E1015 11:45:08.967919    14 factory.h:149] RAW: Insert duplicate with tag: WeightedRandomRender
2020-10-15T03:45:10.096573100Z I1015 11:45:08.967919    14 abtest.h:67] RAW: Factory has been registed: WeightedRandomRender->EndpointRouterBase.
2020-10-15T03:45:10.096589761Z F1015 11:45:10.096339    14 doubly_buffered_data.h:283] Fail to pthread_key_create: Resource temporarily unavailable
2020-10-15T03:45:10.096766297Z *** Check failure stack trace: ***

server服务运行正常

[bjjwwang]

好的 我马上压力测试复现一下

[sevenold]

@wangjiawei04 有复现这个问题吗？

[AnMoran]

@wangjiawei04 我也遇到了，如果每预测一张图，

都初始化一次这个，预测完一张调用det_client.release()和rec_client.release()
跑几百张图就会崩溃，崩溃如下