Issue streaming and displaying with cv2.imshow()

[selamie]

Hi,

I have a pretty consistent issue with streaming and displaying with cv2.imshow. The tl;dr is that the server and client seem to be communicating fine, but displaying the frames with cv2.imshow keeps running into problems.

Here's an example of my code.

Server:

image_hub = imagezmq.ImageHub()

count = 0
while True:  # show streamed images until Ctrl-C
    try:
        rpi_name, buffer = image_hub.recv_jpg()
        frame = cv2.imdecode(np.frombuffer(buffer, dtype='uint8'), -1)
        cv2.imshow('image', frame)  # 1 window for each RPi
        cv2.waitKey(int(1/15*100))
        image_hub.send_reply(b'OK')
        count += 1
        print(count)

    except KeyboardInterrupt:
        cv2.destroyAllWindows()
        break

Client:

sender = imagezmq.ImageSender(connect_to='tcp://my.ip.addr:5555')

rpi_name = socket.gethostname()  

cap = cv2.VideoCapture(0)

time.sleep(2.0)  

count = 0
while True:  
    frame = threadcap.frame
    (flags, buffer) = cv2.imencode(".jpg", frame, [int(cv2.IMWRITE_JPEG_QUALITY), 95])
    sender.send_jpg(rpi_name, buffer)
    count += 1
    print(count)

As you can see, I'm printing the "count" on both the client and server.

By printing the "count", I can confirm that there were no issues with sending or receiving images between server and client, and they appear to be in sync. But without fail, cv2.imshow seems to stall or freeze after a while, and I'm not sure how to fix this issue. It appears to be more of an issue with cv2.imshow() sycning with the stream than an issue imagezmq, but since this frequently called by imagezmq users to display video streams, I'd like to know if:

1. Others have used different approaches to display the video stream
2. If there are any known workarounds for this imshow() issue.

I did research this issue before posting here, including: https://stackoverflow.com/questions/37038606/opencv-imshow-freezes-when-updating

Thanks in advance!

[jeffbass]

Hi @selamie,
I, too, have had cv2.imshow() issues with stalling or freezing. I don't have anything to add to the suggestions in the stackoverflow link you shared. Some of those suggestions have worked for me, but not reliably. I don't use cv2.imshow() for anything other than brief tests because of this. I have found that my issues with cv2.imshow occur differently on different machines (I have used it on Macs, Ubuntu laptops, and Raspberry Pi's). My suggestion is to try the various suggestions in the stackoverflow post. You may also want ask a question on the OpenCV Question / Answer forums. Here is one about cv2.imshow freezing on Macs:. I don't believe that the cv2.imshow issues are related to imageZMQ. My own issues with cv2.imshow have occurred just as often in programs that are not using imageZMQ at all.

There are a number of alternatives to using cv2.imshow. I have used both the matplotlib and Pillow image display methods as alternatives to cv2.imshow. Adrian Rosebrock has written a blog post about displaying images in matplotlib. And here are the docs for displaying images using Pillow. For color images, both matplotlib and Pillow require you to convert the image from OpenCV's BGR to RGB. Adrian discusses that in his blog post and there are a number of posts on stackoverflow about it.

I am going to leave this issue open in case someone has a helpful suggestion. Good luck!
Jeff

[selamie]

Thanks so much! I was confused by how exactly to display an image stream in matplotlib, I will check out those tutorials and docs!