Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Running Youtube8m model_inference with local videos takes long time to render the video with labels and my computer CRASH #204

Closed
tl32rodan opened this issue Oct 29, 2019 · 9 comments
Closed

Running Youtube8m model_inference with local videos takes long time to render the video with labels and my computer CRASH #204

tl32rodan opened this issue Oct 29, 2019 · 9 comments
Assignees
@jiuqiant
Labels
legacy:youtube8M Youtube8M related issues

Comments

@tl32rodan
Copy link

I'm trying to run https://github.com/google/mediapipe/tree/master/mediapipe/examples/desktop/youtube8m

  • Steps to run the YouTube-8M model inference graph with a local video
    • 3.Build and run the inference binary.
      GLOG_logtostderr=1 bazel-bin/mediapipe/examples/desktop/youtube8m/model_inference --calculator_graph_config_file=mediapipe/graphs/youtube8m/local_video_model_inference.pbtxt --input_side_packets=input_sequence_example_path=/tmp/mediapipe/output.tfrecord,input_video_path=/home/tl32rodan/Downloads/v1.mp4,output_video_path=/tmp/mediapipe/annotated_video.mp4,segment_size=5,overlap=4 &> model_inference_error_log

After a while, my computer will crash and automatically delete /tmp/mediapipe
I tried one more time and recorded the log before it crashes as below.
Needs help.

input_side_packet: "input_video_path"
input_side_packet: "output_video_path"
input_side_packet: "segment_size"
input_side_packet: "overlap"

node {
  calculator: "LocalFileContentsCalculator"
  input_side_packet: "FILE_PATH:input_sequence_example_path"
  output_side_packet: "CONTENTS:input_sequence_example"
}

node {
  calculator: "StringToSequenceExampleCalculator"
  input_side_packet: "STRING:input_sequence_example"
  output_side_packet: "SEQUENCE_EXAMPLE:parsed_sequence_example"
}

node {
  calculator: "UnpackMediaSequenceCalculator"
  input_side_packet: "SEQUENCE_EXAMPLE:parsed_sequence_example"
  output_stream: "FLOAT_FEATURE_RGB:rgb_feature_vector"
  output_stream: "FLOAT_FEATURE_AUDIO:audio_feature_vector"
}

node {
  calculator: "ConcatenateFloatVectorCalculator"
  input_stream: "rgb_feature_vector"
  input_stream: "audio_feature_vector"
  output_stream: "feature_vector"
}

node {
  calculator: "VectorFloatToTensorCalculator"
  input_stream: "feature_vector"
  output_stream: "feature_tensor"
}

node {
  calculator: "StringToInt32Calculator"
  input_side_packet: "segment_size"
  output_side_packet: "segment_size_int"
}

node {
  calculator: "StringToInt32Calculator"
  input_side_packet: "overlap"
  output_side_packet: "overlap_int"
}

node {
  calculator: "LappedTensorBufferCalculator"
  input_stream: "feature_tensor"
  output_stream: "lapped_feature_tensor"
  input_side_packet: "BUFFER_SIZE:segment_size_int"
  input_side_packet: "OVERLAP:overlap_int"
  node_options: {
    [type.googleapis.com/mediapipe.LappedTensorBufferCalculatorOptions] {
      add_batch_dim_to_tensors: true
    }
  }
}

node {
  calculator: "SidePacketToStreamCalculator"
  input_side_packet: "segment_size_int"
  output_stream: "AT_ZERO:segment_size_int_stream"
}

node {
  calculator: "VectorIntToTensorCalculator"
  input_stream: "SINGLE_INT:segment_size_int_stream"
  output_stream: "TENSOR_OUT:segment_size_tensor"
}

node {
  calculator: "PacketClonerCalculator"
  input_stream: "segment_size_tensor"
  input_stream: "lapped_feature_tensor"
  output_stream: "synced_segment_size_tensor"
}

node {
  calculator: "TensorFlowSessionFromSavedModelCalculator"
  output_side_packet: "SESSION:session"
  node_options: {
    [type.googleapis.com/mediapipe.TensorFlowSessionFromSavedModelCalculatorOptions]: {
      saved_model_path: "/tmp/mediapipe/saved_model"
    }
  }
}

node: {
  calculator: "TensorFlowInferenceCalculator"
  input_side_packet: "SESSION:session"
  input_stream: "NUM_FRAMES:synced_segment_size_tensor"
  input_stream: "RGB_AND_AUDIO:lapped_feature_tensor"
  output_stream: "PREDICTIONS:prediction_tensor"
  node_options: {
    [type.googleapis.com/mediapipe.TensorFlowInferenceCalculatorOptions]: {
      batch_size: 32
    }
  }
}

node {
  calculator: "TensorToVectorFloatCalculator"
  input_stream: "prediction_tensor"
  output_stream: "prediction_vector"
}

node {
  calculator: "TopKScoresCalculator"
  input_stream: "SCORES:prediction_vector"
  output_stream: "TOP_K_INDEXES:top_k_indexes"
  output_stream: "TOP_K_SCORES:top_k_scores"
  output_stream: "TOP_K_LABELS:top_k_labels"
  node_options: {
    [type.googleapis.com/mediapipe.TopKScoresCalculatorOptions]: {
      top_k: 3
      label_map_path: "mediapipe/graphs/youtube8m/label_map.txt"
    }
  }
}

node {
  calculator: "OpenCvVideoDecoderCalculator"
  input_side_packet: "INPUT_FILE_PATH:input_video_path"
  output_stream: "VIDEO:input_video"
  output_stream: "VIDEO_PRESTREAM:input_video_header"
}

node {
  calculator: "LabelsToRenderDataCalculator"
  input_stream: "LABELS:top_k_labels"
  input_stream: "SCORES:top_k_scores"
  input_stream: "VIDEO_PRESTREAM:input_video_header"
  output_stream: "RENDER_DATA:render_data"
  node_options: {
    [type.googleapis.com/mediapipe.LabelsToRenderDataCalculatorOptions]: {
      color { r: 255 g: 0 b: 0 }
      color { r: 0 g: 255 b: 0 }
      color { r: 0 g: 0 b: 255 }
      thickness: 2.0
      font_height_px: 20
      max_num_labels: 3
      location: TOP_LEFT
    }
  }
}

node {
  calculator: "PacketClonerCalculator"
  input_stream: "render_data"
  input_stream: "input_video"
  output_stream: "synchronized_render_data"
}

node {
  calculator: "AnnotationOverlayCalculator"
  input_stream: "INPUT_FRAME:input_video"
  input_stream: "synchronized_render_data"
  output_stream: "OUTPUT_FRAME:output_video"
}

node {
  calculator: "OpenCvVideoEncoderCalculator"
  input_stream: "VIDEO:output_video"
  input_stream: "VIDEO_PRESTREAM:input_video_header"
  input_side_packet: "OUTPUT_FILE_PATH:output_video_path"
  node_options: {
    [type.googleapis.com/mediapipe.OpenCvVideoEncoderCalculatorOptions]: {
      codec: "avc1"
      video_format: "mp4"
    }
  }
}

I1029 21:10:19.224232  4893 simple_run_graph_main.cc:120] Initialize the calculator graph.
I1029 21:10:19.224892  4893 simple_run_graph_main.cc:133] Start running the calculator graph.
2019-10-29 21:10:19.225491: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:31] Reading SavedModel from: /tmp/mediapipe/saved_model
I1029 21:10:19.227170  4894 unpack_media_sequence_calculator.cc:203] Found feature timestamps: RGB/feature/timestamp with size: 120
I1029 21:10:19.227187  4894 unpack_media_sequence_calculator.cc:203] Found feature timestamps: AUDIO/feature/timestamp with size: 120
2019-10-29 21:10:19.229736: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:54] Reading meta graph with tags { serve }
2019-10-29 21:10:19.234491: I external/org_tensorflow/tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2019-10-29 21:10:19.307794: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:202] Restoring SavedModel bundle.
2019-10-29 21:10:19.534830: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:311] SavedModel load for tags { serve }; Status: success. Took 309549 microseconds.
W1029 21:10:21.283898  4899 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: render_data to: 112. Consider increasing max_queue_size for better performance.
W1029 21:11:03.794173  4895 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 150. Consider increasing max_queue_size for better performance.
W1029 21:11:03.942543  4902 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 200. Consider increasing max_queue_size for better performance.
W1029 21:11:04.114259  4898 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 250. Consider increasing max_queue_size for better performance. ```
@tl32rodan
Copy link
Author

Besides, if the model_inference can't run, I'd like to run starter code's inference.py , so output.tfrecord's format should be same as test data of YT8M dataset.
But I found that output_tfrecord's rgb/audio feature was floatlist instead of bytelist.
Can I just convert it into bytelist so that I can use it as input of inference.py , or it is needed to do some specific convert to do inference.py ?

@jiuqiant
Copy link
Contributor

Please give the model_inference more time to finish the video rendering. The warning messages is ok in this case because video rendering is expected to be super slow and it's not a real deadlock issue. Also, may I know what you mean by crashing your computer?

For your second question, if you want to use the output_tfrecord generated by the feature extraction pipeline as the input of inference.py, a converter script is needed. It needs to convert the audio and rgb features from floatlist to bytelist and also rewrite the feature_list keys by applying following mappings:

  • AUDIO/feature/floats -> audio
  • RGB/feature/floats -> rgb

@jiuqiant jiuqiant changed the title Running Youtube8m model_inference will cause DEADLOCK and my computer CRASH Running Youtube8m model_inference with local videos takes long time to render the video with labels and my computer CRASH Oct 29, 2019
@jiuqiant jiuqiant self-assigned this Oct 29, 2019
@tl32rodan
Copy link
Author

My computer had stunned, I couldn't even move my mouse cursor. But maybe it's just GUI problem, I'll run again later.Thanks a lot!

@jiuqiant
Copy link
Contributor

My computer had stunned, I couldn't even move my mouse cursor. But maybe it's just GUI problem, I'll run again later.Thanks a lot!

In theory, MediaPipe should pick an appropriate number of threads to use depending on the number of available processors. However, if you prefer to limit the cpu resources used by MediaPipe, you can add num_threads: 4 or num_threads: 8 into local_video_model_inference.pbtxt.

@tl32rodan
Copy link
Author

@jiuqiant I re-run the code on another computer and it seems work. But in the last step, it had run for a while and got Killed message. The annotated video was created but I can't open it, and it's header was deprecated as the link below. Is there any suggestion?

Here's the annotated video's info:
annotated_video_info

And process log:

$ GLOG_logtostderr=1 bazel-bin/mediapipe/examples/desktop/youtube8m/model_inference --calculator_graph_config_file=mediapipe/graphs/youtube8m/local_video_model_inference.pbtxt --input_side_packets=input_sequence_example_path=/tmp/mediapipe/features.pb,input_video_path=/home/project/v1.mp4,output_video_path=/tmp/mediapipe/annotated_video.mp4,segment_size=5,overlap=4

I1031 21:25:25.569058 19085 simple_run_graph_main.cc:105] Get calculator graph config contents: input_side_packet: "input_sequence_example_path"
input_side_packet: "input_video_path"
input_side_packet: "output_video_path"
input_side_packet: "segment_size"
input_side_packet: "overlap"

node {
  calculator: "LocalFileContentsCalculator"
  input_side_packet: "FILE_PATH:input_sequence_example_path"
  output_side_packet: "CONTENTS:input_sequence_example"
}

node {
  calculator: "StringToSequenceExampleCalculator"
  input_side_packet: "STRING:input_sequence_example"
  output_side_packet: "SEQUENCE_EXAMPLE:parsed_sequence_example"
}

node {
  calculator: "UnpackMediaSequenceCalculator"
  input_side_packet: "SEQUENCE_EXAMPLE:parsed_sequence_example"
  output_stream: "FLOAT_FEATURE_RGB:rgb_feature_vector"
  output_stream: "FLOAT_FEATURE_AUDIO:audio_feature_vector"
}

node {
  calculator: "ConcatenateFloatVectorCalculator"
  input_stream: "rgb_feature_vector"
  input_stream: "audio_feature_vector"
  output_stream: "feature_vector"
}

node {
  calculator: "VectorFloatToTensorCalculator"
  input_stream: "feature_vector"
  output_stream: "feature_tensor"
}

node {
  calculator: "StringToInt32Calculator"
  input_side_packet: "segment_size"
  output_side_packet: "segment_size_int"
}

node {
  calculator: "StringToInt32Calculator"
  input_side_packet: "overlap"
  output_side_packet: "overlap_int"
}

node {
  calculator: "LappedTensorBufferCalculator"
  input_stream: "feature_tensor"
  output_stream: "lapped_feature_tensor"
  input_side_packet: "BUFFER_SIZE:segment_size_int"
  input_side_packet: "OVERLAP:overlap_int"
  node_options: {
    [type.googleapis.com/mediapipe.LappedTensorBufferCalculatorOptions] {
      add_batch_dim_to_tensors: true
    }
  }
}

node {
  calculator: "SidePacketToStreamCalculator"
  input_side_packet: "segment_size_int"
  output_stream: "AT_ZERO:segment_size_int_stream"
}

node {
  calculator: "VectorIntToTensorCalculator"
  input_stream: "SINGLE_INT:segment_size_int_stream"
  output_stream: "TENSOR_OUT:segment_size_tensor"
}

node {
  calculator: "PacketClonerCalculator"
  input_stream: "segment_size_tensor"
  input_stream: "lapped_feature_tensor"
  output_stream: "synced_segment_size_tensor"
}

node {
  calculator: "TensorFlowSessionFromSavedModelCalculator"
  output_side_packet: "SESSION:session"
  node_options: {
    [type.googleapis.com/mediapipe.TensorFlowSessionFromSavedModelCalculatorOptions]: {
      saved_model_path: "/tmp/mediapipe/saved_model"
    }
  }
}

node: {
  calculator: "TensorFlowInferenceCalculator"
  input_side_packet: "SESSION:session"
  input_stream: "NUM_FRAMES:synced_segment_size_tensor"
  input_stream: "RGB_AND_AUDIO:lapped_feature_tensor"
  output_stream: "PREDICTIONS:prediction_tensor"
  node_options: {
    [type.googleapis.com/mediapipe.TensorFlowInferenceCalculatorOptions]: {
      batch_size: 32
    }
  }
}

node {
  calculator: "TensorToVectorFloatCalculator"
  input_stream: "prediction_tensor"
  output_stream: "prediction_vector"
}

node {
  calculator: "TopKScoresCalculator"
  input_stream: "SCORES:prediction_vector"
  output_stream: "TOP_K_INDEXES:top_k_indexes"
  output_stream: "TOP_K_SCORES:top_k_scores"
  output_stream: "TOP_K_LABELS:top_k_labels"
  node_options: {
    [type.googleapis.com/mediapipe.TopKScoresCalculatorOptions]: {
      top_k: 3
      label_map_path: "mediapipe/graphs/youtube8m/label_map.txt"
    }
  }
}

node {
  calculator: "OpenCvVideoDecoderCalculator"
  input_side_packet: "INPUT_FILE_PATH:input_video_path"
  output_stream: "VIDEO:input_video"
  output_stream: "VIDEO_PRESTREAM:input_video_header"
}

node {
  calculator: "LabelsToRenderDataCalculator"
  input_stream: "LABELS:top_k_labels"
  input_stream: "SCORES:top_k_scores"
  input_stream: "VIDEO_PRESTREAM:input_video_header"
  output_stream: "RENDER_DATA:render_data"
  node_options: {
    [type.googleapis.com/mediapipe.LabelsToRenderDataCalculatorOptions]: {
      color { r: 255 g: 0 b: 0 }
      color { r: 0 g: 255 b: 0 }
      color { r: 0 g: 0 b: 255 }
      thickness: 2.0
      font_height_px: 20
      max_num_labels: 3
      location: TOP_LEFT
    }
  }
}

node {
  calculator: "PacketClonerCalculator"
  input_stream: "render_data"
  input_stream: "input_video"
  output_stream: "synchronized_render_data"
}

node {
  calculator: "AnnotationOverlayCalculator"
  input_stream: "INPUT_FRAME:input_video"
  input_stream: "synchronized_render_data"
  output_stream: "OUTPUT_FRAME:output_video"
}

node {
  calculator: "OpenCvVideoEncoderCalculator"
  input_stream: "VIDEO:output_video"
  input_stream: "VIDEO_PRESTREAM:input_video_header"
  input_side_packet: "OUTPUT_FILE_PATH:output_video_path"
  node_options: {
    [type.googleapis.com/mediapipe.OpenCvVideoEncoderCalculatorOptions]: {
      codec: "avc1"
      video_format: "mp4"
    }
  }
}

I1031 21:25:25.573375 19085 simple_run_graph_main.cc:120] Initialize the calculator graph.
I1031 21:25:25.576467 19085 simple_run_graph_main.cc:133] Start running the calculator graph.
2019-10-31 21:25:25.577248: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:31] Reading SavedModel from: /tmp/mediapipe/saved_model
I1031 21:25:25.587728 19094 unpack_media_sequence_calculator.cc:203] Found feature timestamps: RGB/feature/timestamp with size: 120
I1031 21:25:25.587746 19094 unpack_media_sequence_calculator.cc:203] Found feature timestamps: AUDIO/feature/timestamp with size: 120
2019-10-31 21:25:25.590532: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:54] Reading meta graph with tags { serve }
2019-10-31 21:25:25.596413: I external/org_tensorflow/tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2019-10-31 21:25:25.616382: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:202] Restoring SavedModel bundle.
2019-10-31 21:25:25.835479: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:311] SavedModel load for tags { serve }; Status: success. Took 258262 microseconds.
W1031 21:25:26.616621 19088 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: render_data to: 113. Consider increasing max_queue_size for better performance.
W1031 21:25:53.793540 19092 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 150. Consider increasing max_queue_size for better performance.
W1031 21:25:53.886016 19087 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 200. Consider increasing max_queue_size for better performance.
W1031 21:25:54.000036 19097 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 250. Consider increasing max_queue_size for better performance.
W1031 21:25:54.120272 19097 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 300. Consider increasing max_queue_size for better performance.
W1031 21:25:54.245237 19090 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 350. Consider increasing max_queue_size for better performance.
W1031 21:25:54.368718 19092 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 400. Consider increasing max_queue_size for better performance.
W1031 21:25:54.492698 19087 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 450. Consider increasing max_queue_size for better performance.
W1031 21:25:54.622370 19087 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 500. Consider increasing max_queue_size for better performance.
W1031 21:25:54.751965 19096 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 550. Consider increasing max_queue_size for better performance.
W1031 21:25:54.881501 19096 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 600. Consider increasing max_queue_size for better performance.
W1031 21:25:55.004333 19090 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 650. Consider increasing max_queue_size for better performance.
W1031 21:25:55.137701 19092 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 700. Consider increasing max_queue_size for better performance.
W1031 21:25:55.271704 19096 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 750. Consider increasing max_queue_size for better performance.
W1031 21:25:55.397341 19097 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 800. Consider increasing max_queue_size for better performance.
W1031 21:25:55.520668 19091 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 850. Consider increasing max_queue_size for better performance.
W1031 21:25:55.647541 19091 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 900. Consider increasing max_queue_size for better performance.
W1031 21:25:55.779991 19091 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 950. Consider increasing max_queue_size for better performance.
W1031 21:25:55.912314 19092 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 1000. Consider increasing max_queue_size for better performance.
W1031 21:25:56.037524 19096 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 1050. Consider increasing max_queue_size for better performance.
W1031 21:25:56.168604 19097 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 1100. Consider increasing max_queue_size for better performance.
W1031 21:25:56.303138 19092 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 1150. Consider increasing max_queue_size for better performance.
W1031 21:25:56.426353 19088 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 1200. Consider increasing max_queue_size for better performance.
W1031 21:25:56.553753 19091 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 1250. Consider increasing max_queue_size for better performance.
W1031 21:25:56.692029 19086 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 1300. Consider increasing max_queue_size for better performance.
W1031 21:25:56.813877 19090 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 1350. Consider increasing max_queue_size for better performance.
W1031 21:25:56.943153 19091 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 1400. Consider increasing max_queue_size for better performance.
W1031 21:25:57.080940 19086 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 1450. Consider increasing max_queue_size for better performance.
W1031 21:25:57.207360 19096 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 1500. Consider increasing max_queue_size for better performance.
W1031 21:25:57.332010 19090 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 1550. Consider increasing max_queue_size for better performance.
W1031 21:25:57.461563 19089 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 1600. Consider increasing max_queue_size for better performance.
W1031 21:25:57.585266 19091 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 1650. Consider increasing max_queue_size for better performance.
W1031 21:25:57.713424 19094 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 1700. Consider increasing max_queue_size for better performance.
W1031 21:25:57.843329 19093 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 1750. Consider increasing max_queue_size for better performance.
W1031 21:25:57.966703 19088 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 1800. Consider increasing max_queue_size for better performance.
W1031 21:25:58.090462 19089 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 1850. Consider increasing max_queue_size for better performance.
W1031 21:25:58.219684 19089 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 1900. Consider increasing max_queue_size for better performance.
W1031 21:25:58.348937 19095 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 1950. Consider increasing max_queue_size for better performance.
W1031 21:25:58.481753 19097 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 2000. Consider increasing max_queue_size for better performance.
W1031 21:25:58.606221 19091 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 2050. Consider increasing max_queue_size for better performance.
W1031 21:25:58.737759 19093 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 2100. Consider increasing max_queue_size for better performance.
W1031 21:25:58.868942 19088 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 2150. Consider increasing max_queue_size for better performance.
W1031 21:25:59.000164 19087 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 2200. Consider increasing max_queue_size for better performance.
W1031 21:25:59.126252 19092 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 2250. Consider increasing max_queue_size for better performance.
W1031 21:25:59.244422 19093 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 2300. Consider increasing max_queue_size for better performance.
W1031 21:25:59.369397 19096 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 2350. Consider increasing max_queue_size for better performance.
W1031 21:25:59.493105 19087 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 2400. Consider increasing max_queue_size for better performance.
W1031 21:25:59.617084 19086 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 2450. Consider increasing max_queue_size for better performance.
W1031 21:25:59.748052 19094 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 2500. Consider increasing max_queue_size for better performance.
W1031 21:25:59.871553 19093 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 2550. Consider increasing max_queue_size for better performance.
W1031 21:26:00.000365 19086 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 2600. Consider increasing max_queue_size for better performance.
W1031 21:26:00.128849 19095 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 2650. Consider increasing max_queue_size for better performance.
W1031 21:26:00.251487 19092 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 2700. Consider increasing max_queue_size for better performance.
W1031 21:26:00.374985 19096 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 2750. Consider increasing max_queue_size for better performance.
W1031 21:26:00.500679 19093 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 2800. Consider increasing max_queue_size for better performance.
W1031 21:26:00.619832 19090 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 2850. Consider increasing max_queue_size for better performance.
W1031 21:26:00.745216 19087 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 2900. Consider increasing max_queue_size for better performance.
W1031 21:26:00.876139 19093 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 2950. Consider increasing max_queue_size for better performance.
W1031 21:26:01.004670 19086 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 3000. Consider increasing max_queue_size for better performance.
W1031 21:26:01.131801 19094 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 3050. Consider increasing max_queue_size for better performance.
W1031 21:26:01.254413 19096 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 3100. Consider increasing max_queue_size for better performance.
W1031 21:26:01.375080 19090 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 3150. Consider increasing max_queue_size for better performance.
W1031 21:26:01.503315 19086 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 3200. Consider increasing max_queue_size for better performance.
W1031 21:26:01.621018 19097 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 3250. Consider increasing max_queue_size for better performance.
W1031 21:26:01.749037 19094 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 3300. Consider increasing max_queue_size for better performance.
W1031 21:26:01.874925 19093 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 3350. Consider increasing max_queue_size for better performance.
W1031 21:26:01.999327 19095 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 3400. Consider increasing max_queue_size for better performance.
W1031 21:26:02.129403 19090 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 3450. Consider increasing max_queue_size for better performance.
W1031 21:26:02.254400 19088 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 3500. Consider increasing max_queue_size for better performance.
W1031 21:26:02.376797 19087 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 3550. Consider increasing max_queue_size for better performance.
W1031 21:26:02.499802 19090 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 3600. Consider increasing max_queue_size for better performance.
W1031 21:26:02.626557 19092 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 3650. Consider increasing max_queue_size for better performance.
W1031 21:26:02.746040 19090 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 3700. Consider increasing max_queue_size for better performance.
W1031 21:26:02.870661 19094 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 3750. Consider increasing max_queue_size for better performance.
W1031 21:26:02.996584 19093 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 3800. Consider increasing max_queue_size for better performance.
W1031 21:26:03.122262 19091 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 3850. Consider increasing max_queue_size for better performance.
W1031 21:26:03.245393 19090 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 3900. Consider increasing max_queue_size for better performance.
W1031 21:26:03.368260 19088 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 3950. Consider increasing max_queue_size for better performance.
W1031 21:26:03.492913 19092 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 4000. Consider increasing max_queue_size for better performance.
W1031 21:26:03.618468 19092 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 4050. Consider increasing max_queue_size for better performance.
W1031 21:26:03.741801 19090 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 4100. Consider increasing max_queue_size for better performance.
W1031 21:26:03.866073 19087 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 4150. Consider increasing max_queue_size for better performance.
W1031 21:26:03.992866 19088 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 4200. Consider increasing max_queue_size for better performance.
W1031 21:26:04.124851 19092 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 4250. Consider increasing max_queue_size for better performance.
W1031 21:26:04.246923 19094 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 4300. Consider increasing max_queue_size for better performance.
W1031 21:26:04.377209 19088 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 4350. Consider increasing max_queue_size for better performance.
W1031 21:26:04.503204 19086 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 4400. Consider increasing max_queue_size for better performance.
W1031 21:26:04.627650 19097 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 4450. Consider increasing max_queue_size for better performance.
W1031 21:26:04.752900 19097 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 4500. Consider increasing max_queue_size for better performance.
W1031 21:26:04.883294 19096 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 4550. Consider increasing max_queue_size for better performance.
W1031 21:26:05.005065 19092 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 4600. Consider increasing max_queue_size for better performance.
W1031 21:26:05.132372 19096 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 4650. Consider increasing max_queue_size for better performance.
W1031 21:26:05.263226 19089 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 4700. Consider increasing max_queue_size for better performance.
W1031 21:26:05.385084 19096 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 4750. Consider increasing max_queue_size for better performance.
W1031 21:26:05.518621 19091 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 4800. Consider increasing max_queue_size for better performance.
W1031 21:26:05.644104 19096 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 4850. Consider increasing max_queue_size for better performance.
W1031 21:26:05.767717 19092 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 4900. Consider increasing max_queue_size for better performance.
W1031 21:26:05.893537 19093 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 4950. Consider increasing max_queue_size for better performance.
W1031 21:26:06.016335 19096 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 5000. Consider increasing max_queue_size for better performance.
W1031 21:26:06.146034 19091 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 5050. Consider increasing max_queue_size for better performance.
W1031 21:26:06.280050 19096 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 5100. Consider increasing max_queue_size for better performance.
W1031 21:26:06.408025 19093 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 5150. Consider increasing max_queue_size for better performance.
W1031 21:26:06.535462 19087 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 5200. Consider increasing max_queue_size for better performance.
W1031 21:26:06.663843 19089 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 5250. Consider increasing max_queue_size for better performance.
W1031 21:26:06.788637 19089 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 5300. Consider increasing max_queue_size for better performance.
W1031 21:26:06.915647 19088 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 5350. Consider increasing max_queue_size for better performance.
W1031 21:26:07.049458 19089 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 5400. Consider increasing max_queue_size for better performance.
W1031 21:26:07.179040 19093 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 5450. Consider increasing max_queue_size for better performance.
W1031 21:26:07.315174 19091 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 5500. Consider increasing max_queue_size for better performance.
W1031 21:26:07.457113 19095 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 5550. Consider increasing max_queue_size for better performance.
W1031 21:26:07.631956 19088 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 5600. Consider increasing max_queue_size for better performance.
W1031 21:26:07.795706 19089 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 5650. Consider increasing max_queue_size for better performance.
W1031 21:26:07.978233 19090 calculator_graph.cc:1136] Resolved a deadlock by increasing max_queue_size of input stream: input_video to: 5700. Consider increasing max_queue_size for better performance.
Killed

@jiuqiant jiuqiant added the legacy:youtube8M Youtube8M related issues label Oct 31, 2019
@jiuqiant
Copy link
Contributor

Thanks for reporting this issue. After investigation, I realize that there is a bug in the code if the duration of the video is longer than the number of the feature points in /tmp/mediapipe/features.pb. In your particular case, I believe your video is longer than 120 seconds and the feature extractor pipeline is only asked to extract the features for the first 120 seconds of the video. That triggers the bug.

Please do the following change in the opencv_video_decoder_calculator.cc. Then, rebuild and rerun the model_inference binary. Rebuild should be super fast because of the bazel cache you already have.

diff --git a/mediapipe/calculators/video/opencv_video_decoder_calculator.cc b/mediapipe/calculators/video/opencv_video_decoder_calculator.cc
index b333e3c..49c7f05 100644
--- a/mediapipe/calculators/video/opencv_video_decoder_calculator.cc
+++ b/mediapipe/calculators/video/opencv_video_decoder_calculator.cc
@@ -123,6 +123,7 @@ class OpenCvVideoDecoderCalculator : public CalculatorBase {
       cc->Outputs()
           .Tag("VIDEO_PRESTREAM")
           .Add(header.release(), Timestamp::PreStream());
+      cc->Outputs().Tag("VIDEO_PRESTREAM").Close();
     }
     // Rewind to the very first frame.
     cap_->set(cv::CAP_PROP_POS_AVI_RATIO, 0);

We will have this fix in the next release if that works for you. Thanks!

@tl32rodan
Copy link
Author

tl32rodan commented Nov 3, 2019 •
edited
Loading

Thanks for helping ! The annotated video ran successfully !
But when I trying to convert output.tfrecord's rgb and audio feature, I found that in the old extractor , the quantize function convert float to byte via quantize value of range [-2,2]. Is me using this quantize range okay ? Or there's some other suggestions to the range ?

@joeyhng
Copy link

joeyhng commented Nov 4, 2019

Yes, the [-2,2] quantize range is ok.

@jiuqiant
Copy link
Contributor

jiuqiant commented Nov 4, 2019

@sedez, joeyhng is the author of the old yt8m feature extractor. His answer can be trusted.
I believe the original issue is solved. We close this issue for now.
Feel free to create another github issue if you face any new problems. Thanks!

@jiuqiant jiuqiant closed this as completed Nov 4, 2019
arttupii pushed a commit to arttupii/mediapipe that referenced this issue Nov 18, 2023
@khanhlvg
Merge pull request google-ai-edge#204 from priankakariatyml/ios-objec…
e4dd265 
…t-detector-rearch

Refactored iOS Object Detector
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jiuqiant jiuqiant

Labels
legacy:youtube8M Youtube8M related issues
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@joeyhng @jiuqiant @tl32rodan