Darknet Python IMAGE object #289
Comments
|
Not sure if this will work but try it out @FranciscoGomez90 |
|
Thank you for your answer! |
|
I forgot some lines after line |
|
I'm sorry. I can't make it work. |
|
Can you post the python script you are using to use the webcam, I will see if I can make it work and report back :) |
|
There is a extract of the code def runLive():
handDetector = HandDetectorYolo() #Yolo-darknet encapsulation
video_capture = cv2.VideoCapture(1)
while True:
ret, frame = video_capture.read()
im = cv2.resize(frame,(640,480))
# somehow transform im from opencv python to an IMAGE object
# network_detect is self.network_detect.argtypes = [c_void_p, IMAGE, c_float, c_float, c_float, POINTER(BOX), POINTER(POINTER(c_float))]
handDetector.network_detect(self.net, im, self.thresh, self.hier_thresh, self.nms, boxes, probs)So i need to convert from an opencv image to an IMAGE object |
|
sorry i closed it by mistake |
|
@FranciscoGomez90 I've managed to compile with webcam, this is what needs to be changed. Then in in your And lastly also in Then recompile the library using I've managed to detect using tiny-yolo.cfg and weights from website by using my webcam to take an image and send it to darknet and get output back This is the script i've used named |
|
Yay! That worked like a charm!! |
|
Hi @TheMikeyR , That works it seems. Thanks very much. Another worry on restreaming this tagged output video. My current understanding is that the output will be written in an MJPEG container and then that file is taken for further processing. FFMPEG can be involved for this and we can restream. |
|
@sleebapaul I haven't tested maximum fps from video stream, but when loading images (2208x1242) from disk I can achieve 15 fps with tiny yolo + sort (online tracker) on an m1200 gpu. I'm primarily using multiprocessing package, where I have an imageLoader() function in its own process: Where it only focus on loading images (according to my results it went 3x faster) and then I'm saving the results to a file also with a separate process. (tears on the memory though, but if I reach a limit I will make the loop sleep until I cleared the buffer). About restreaming, i've tried to restream the images I save to disk with great success (using flask), where I modified the code to stream images as they arrive in a folder, I had to run the python script separately though. I think it would be much better to go with the FFMPEG method, but I haven't tried that yet, since I couldn't find a good example. Feel free to share if you have a great example or solution :) |
|
Hi @TheMikeyR , I'm trying to follow your instructions. Could you post the entire python code? I need to find a way to have both the webcam image displayed and get text output. Thanks! |
|
|
Hi @TheMikeyR, Would you mind sharing your implementation of sort with YOLO, or point me to good resources? Thank you |
|
@iraadit there are not many steps to add into the pipeline with the code above, just import sort.py into your code and follow the steps from https://github.com/abewley/sort/#using-sort-in-your-own-project Furthermore you need to do some data-juggling, since you have to go from YOLO output to SORT. I hope this provide enough info to help you add it to your own implementation. |
|
@TheMikeyR, thank you for your answer. |
|
Hi @TheMikeyR There are issues with YOLOV3 I guess, followed all the instructions but still getting errors on
|
|
@sleebapaul the python wrapper isn't updated for yolov3, so the "hack" above doesn't work. It still works for yolov2 though. |
|
@TheMikeyR I'm trying to detect from numpy array without adding another function, so before then in function it get the right detection, but after |
added ndarray_to_image
|
After following TheMikeyR's darknet compilation steps, I too was getting the make_boxes error with Yolov3. This worked for me: |
|
@TheMikeyR, I followed every step you recommended to get rid of the error.. I'm still getting the same I'm trying to run it on a TX2 board. Can you help me? |
Thnk u for ur help. I want to test it on a single channel image. How do I do that. |
|
@PallawiSinghal You will need to modify the network. So far, the input volume of the architecture is (batchSize x height x width x 3), being the last 3 the RGB channels of an image. You will need to change that in order to take (batchSize x height x width x 1), being the 1 a greyscale image. |
Thank you, But where do I make changes in the code. Can you please help. |
I am getting the same error, can you help me if u have solved the problem. |
|
@PallawiSinghal You should change the architecture itself so I think you must hack the .cfg files. For instance, the yolov3-voc.cfg has this input layer: You must change the parameter channels to 1 |
|
@TheMikeyR Does this also work for the current darknet? Because when i try to adapt things in the current darknet the lines are not valid. And when i put it in spots where i think it belong i get the error:
|
|
@MarviB16 I don't believe so, but you can try this out #289 (comment) |
|
@TheMikeyR @julianweisbord I tried the code you shared for Yolov3 but I'm observing that bounding boxes aren't aligned correctly as given below. I looked into the coordinates of boxes and I confirm that they are wrong. Do you have any idea what would be the cause of this ? |
Ok, I solved the issue. For anyone who's been struggling with this problem, the following lines of code may help you: custom_image = cv2.cvtColor(custom_image_bgr, cv2.COLOR_BGR2RGB)
custom_image = cv2.resize(custom_image, (lib.network_width(
self.net), lib.network_height(self.net)), interpolation=cv2.INTER_LINEAR)
im, _ = self.array_to_image(custom_image)
num = c_int(0)
pnum = pointer(num)
predict_image(self.net, im)
dets = get_network_boxes(self.net, custom_image_bgr.shape[1], custom_image_bgr.shape[
0], self.thresh, hier_thresh, None, 0, pnum, 0)
num = pnum[0]
if nms:
do_nms_obj(dets, num, self.meta.classes, nms)
Edit: custom_image = cv2.resize(custom_image, (lib.network_width(
self.net), lib.network_height(self.net)), interpolation=cv2.INTER_NEAREST) |
|
@TheMikeyR I tried the modifications in darknet and got this error: File "/usr/lib/python3.6/ctypes/init.py", line 366, in getitem since I'm using python3.6, I change the makefile as: I don't know what's the problem? Can you help me with that? |
|
To commit above changes for YOLO v3 go to my link---------------- https://github.com/ganeshkharad2/darknet-yolo_v3_numpy I have uploaded necessary files there just replace all 3 files with original ones |
I have the same promblem,how to ``` |
|
@julianweisbord I tried your code and my kernel simply crashes when it attempts to predict on the webcam image. There is no error message, the code simply crashes. I'm assuming it's because of some problem to occur in the C-code rather than the python code. It could also be crashing because some variable isn't being set up correctly. Could you (or anyone else) point me in some direction fix this problem? |
|
@saddiesh Did you solve the problem? |
Yes. I updated some libraries and then it worked. I couldn't say which library updating helped to fix this problem and I can't remember clearly now. If you still have a problem with that I can check the libraries next week. |
That's fine I solved it myself. I'm not sure why, but my problem is that the flag NUMPY=1 is not recognized in image.c, and thus it's not compiled. Hence, I had to manually set it in image.c. |
|
where is libdarknet.so file |
|
@TheMikeyR Could you please explain why you need to swap the channels with rgbgr_image()? As far as I understand darknet takes rgb images for training and test without the swapping. Thank you! |
|
This code can process the video and record the video and FPS using opencv. "darknet.py" def c_array(ctype, values): class BOX(Structure): class DETECTION(Structure): class IMAGE(Structure): class METADATA(Structure): lib = CDLL("/home/**/git-space/darknet/libdarknet.so", RTLD_GLOBAL) predict = lib.network_predict set_gpu = lib.cuda_set_device make_image = lib.make_image get_network_boxes = lib.get_network_boxes make_network_boxes = lib.make_network_boxes free_detections = lib.free_detections free_ptrs = lib.free_ptrs network_predict = lib.network_predict reset_rnn = lib.reset_rnn load_net = lib.load_network do_nms_obj = lib.do_nms_obj do_nms_sort = lib.do_nms_sort free_image = lib.free_image letterbox_image = lib.letterbox_image load_meta = lib.get_metadata load_image = lib.load_image_color rgbgr_image = lib.rgbgr_image predict_image = lib.network_predict_image def classify(net, meta, im): def array_to_image(arr): def convertBack(x,y,w,h): def cvDrawBoxes(detections, img): def detect(net, meta, image, thresh=.3, hier_thresh=.5, nms=.45): if name == "main": |
|
@TheMikeyR! Thank u so much! I just make a change in your Makefile commands that can help some people: If you're using another version of Python or a specific interpreter of Python, just change the |
|
Hİ I have a question for you this my error and I did what was said above but nothing change how can ı fix this. pygame 2.0.1 (SDL 2.0.14, Python 3.6.9) |
|
Hello everyone, I recently ran into this same problem and saw TheMikeyR's solution. It helped me find a solution that is 100% implemented in Python. There's no need to recompile darknet, unless the c-code implementation is perhaps a little bit faster and you need the speed, however my solution uses numpy so it's pretty fast as well, and surprisingly simple! TL;DR: All you have to do is the following (it's explained inline): import cv2
import ctypes
import darknet as dn
import numpy as np
def np_image_to_c_IMAGE(input_frame):
"""
converts a numpy image (w x h x c dim ndarray) to a C type IMAGE as defined in darknet.
"""
# The image data is stored in IMAGE.data as a c float pointer therefore...
# create a flattened image and normalize by devision by 255., needs to be float32 (to be converted to float pointer)
flattened_image = input_frame.transpose(2, 0, 1).flatten().astype(np.float32)/255.
# NOTE ON THE ABOVE CODE:
# transpose(2, 0, 1) permutes the axes from 0,1,2 to 2,0,1 (clockwise cycle) which is
# required because on the C-side the data is structured slightly differently
# define LP_c_float type (*float)
c_float_p = ctypes.POINTER(ctypes.c_float)
# cast flattened_image to LP_c_float
c_float_p_frame = flattened_image.ctypes.data_as(c_float_p)
# create empty C_IMAGE type object
w, h, c = input_frame.shape
C_IMAGE_frame = dn.make_image(w, h, c)
# set data to the pointer containing the image data
C_IMAGE_frame.data = c_float_p_frame
return C_IMAGE_frame
# initialize net
net, class_names, class_colors = dn.load_network("path/to/config", "path/to/meta", "path/to/weights")
cap = cv2.VideoCapture(0)
ret, img = cap.read()
C_IMAGE = np_image_to_c_IMAGE(img)
r = dn.detect(net, class_names, C_IMAGE)
# do something with r. ...Note that the darknet.py module I use is AlexeyAB's version The function without all the comments: def np_image_to_c_IMAGE(input_frame):
"""converts a numpy image (w x h x c dim ndarray) to a C type IMAGE as defined in darknet."""
flattened_image = input_frame.transpose(2, 0, 1).flatten().astype(np.float32)/255.
c_float_p = ctypes.POINTER(ctypes.c_float)
c_float_p_frame = flattened_image.ctypes.data_as(c_float_p)
w, h, c = input_frame.shape
C_IMAGE_frame = dn.make_image(w, h, c)
C_IMAGE_frame.data = c_float_p_frame
return C_IMAGE_frameSome more info for the interested reader: for(i = 0; i < h; ++i){
for(k= 0; k < c; ++k){
for(j = 0; j < w; ++j){
index1 = k*w*h + i*w + j;
index2 = step_h*i + step_w*j + step_c*k;
//fprintf(stderr, "w=%d h=%d c=%d step_w=%d step_h=%d step_c=%d \n", w, h, c, step_w, step_h, step_c);
//fprintf(stderr, "im.data[%d]=%u data[%d]=%f \n", index1, src[index2], index2, src[index2]/255.);
im.data[index1] = src[index2]/255.;
}
}
}I initially implemented the code above exactly in Python. It worked, but it was rather slow. I tested my code a little and found that the above for loop it is simply converting the (w x h x c) ndarray to a one dimensional array with the data being strung together column-wise (the columns as they are visually represented by |
and others
Hi,
I'm trying to forward the feed of a webcam to YOLO in python.
My problem is that I capture the images from OpenCV but i don't know how to transform them to an IMAGE object in order to perform the .network_detect()
Could you please throw some insight on this matter?
Thank you!
The text was updated successfully, but these errors were encountered: