dataset_pb2 not cannot be imported

[SagnikD95]

After succesfully downloading all the dependencies, I tried running the test program of your colab on my local machine setup, but it cannot import dataset_pb2 because it does not exist in waymo-open-dataset.

Also running pip3 install waymo-open-dataset returns a "no matching distribution found"

[peisun1115]

1. Try to run:
   pip3 install --upgrade pip

manylinux pip package needs a recent pip version

(we only support python2.7, python3.5, python3.6)

2. If you have a different python version or an os not supported, try to build the pip package on your own. something like:

export PYTHON_VERSION=3 && ./pip_pkg_scripts/build.sh

[SagnikD95]

Hello, Thank you for your reply. I have run the build.sh and it did compile everything successfully but it still fails to import the dataset_pb2 folder. Regards, Sagnik

…

On Fri, Sep 13, 2019 at 6:42 PM Pei Sun ***@***.***> wrote: 1. Try to run: pip3 install --upgrade pip manylinux pip package needs a recent pip version (we only support python2.7, python3.5, python3.6) 1. If you have a different python version or an os not supported, try to build the pip package on your own. something like: export PYTHON_VERSION=3 && ./pip_pkg_scripts/build.sh — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#35?email_source=notifications&email_token=AKTTYVA2QV4TEQLLEK3OR3DQJQQM7A5CNFSM4IWUUZPKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6WN5OQ#issuecomment-531422906>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKTTYVGE4O7C2FMO2A7ADJDQJQQM7ANCNFSM4IWUUZPA> .

[peisun1115]

did you install pip package output by build.sh? something like:
pip (or pip3) install ${WHEEL_FILE} --user

note that if you do not install the pip package, it can also work. but you need to copy the proto generated files to waymo_open_dataset dir.

See the tutorial
https://github.com/waymo-research/waymo-open-dataset/blob/r1.0/tutorial/tutorial.ipynb

[SagnikD95]

Yes i ran it by pip3. Every time i run it returns and error “Cannot import dataset_pb2 from frame_utils”

…

On Mon, Sep 16, 2019 at 11:48 AM Pei Sun ***@***.***> wrote: did you install pip package output by build.sh? something like: pip (or pip3) install ${WHEEL_FILE} --user note that if you do not install the pip package, it can also work. but you need to copy the proto generated files to waymo_open_dataset dir. See the tutorial https://github.com/waymo-research/waymo-open-dataset/blob/r1.0/tutorial/tutorial.ipynb — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#35?email_source=notifications&email_token=AKTTYVASZQORBKDBB3L6RCLQJ62GPA5CNFSM4IWUUZPKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6ZY6PA#issuecomment-531861308>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKTTYVDKD67KRRFUAWYJ76TQJ62GPANCNFSM4IWUUZPA> .

[peisun1115]

Can you copy/paste the error msg and all the commands you used?

• make sure you have installed the package.

[SagnikD95]

Thank you for helping me out so patiently, it is highly appreciated. Please find attached a text file with details of the execution and the error. Regards, Sagnik

On Mon, Sep 16, 2019 at 12:19 PM Pei Sun ***@***.***> wrote: Can you copy/paste the error msg and all the commands you used? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#35?email_source=notifications&email_token=AKTTYVAR4XEONMURXFJNTL3QJ65ZHA5CNFSM4IWUUZPKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6Z33LI#issuecomment-531873197>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKTTYVBNPX32XKLIQNI6WWDQJ65ZHANCNFSM4IWUUZPA> .

git clone https://github.com/waymo-research/waymo-open-dataset.git waymo-od cd waymo-od git checkout remotes/origin/r1.0 ./configure.sh bazel clean bazel test waymo_open_dataset/metrics:all pip3 install numpy tensorflow ./configure.sh bazel build waymo_open_dataset/metrics/ops/... bazel test waymo_open_dataset/metrics/ops/... pip3 install upgrade --pip -- from pip import main ImportError: cannot import name 'main' pip3 install waymo-open-dataset==1.0.1 --user -- Without upgrading pip, no compatible distribution found. So I decided to build from scratch using pip_pkg_scripts as follows: ./pip_pkg_scripts/build.sh Everything compiled without any erros. When trying the sample collab file code on my machine, the following error appears: CODE:import os import tensorflow as tf import math import numpy as np import itertools tf.enable_eager_execution() from waymo_open_dataset.utils import range_image_utils from waymo_open_dataset.utils import transform_utils from waymo_open_dataset.utils import frame_utils from waymo_open_dataset import dataset_pb2 as open_dataset ERROR: from waymo_open_dataset import dataset_pb2 ImportError: cannot import name dataset_pb2

[peisun1115]

1. You cannot install waymo-open-dataset because you cannot upgrade your pip.

2. You did not install the pip package after './pip_pkg_scripts/build.sh'. You need to do something like
   pip3 install $YOUR_PIP_PACKAGE_BUILT_LOCALLY

[peisun1115]

Is this resolved?

[turboxin]

@SagnikD95 you need to install protobuf and run protoc ./waymo_open-dataset/dataset.proto --python_out=.

[peisun1115]

You can also copy the files generated by bazel directly though i think it is better to use the pip package.

[khsrhhh]

I had same issue, and I referenced @turboxin 's comment.

Actually 2 lines had a problem to compile.
=>
from waymo_open_dataset.utils import frame_utils
from waymo_open_dataset import dataset_pb2 as open_dataset

and I ran commands on terminal
=>
$ protoc ./waymo_open_dataset/dataset.proto -- python_out=.
$ protoc ./waymo_open_dataset/label.proto -- python_out=.

After ran them, change directory to waymo_open_dataset(master)/waymo_open_dataset
Now, you can see dataset_pb2.py and label_pb2.py files exist.

[peisun1115]

1. Did you install from the pre-compiled pip package?

2. If answer to 1 is no, then yes, you will need to copy the proto generated files. Bazel generates that automatically for you. It locates at bazel-genfiles/waymo_open_dataset/

[SagnikD95]

@SagnikD95 you need to install protobuf and run protoc ./waymo_open-dataset/dataset.proto --python_out-.

Thanks a ton. Worked like a charm!

[SagnikD95]

I had same issue, and I referenced @turboxin 's comment.

Actually 2 lines had a problem to compile.
=>
from waymo_open_dataset.utils import frame_utils from waymo_open_dataset import dataset_pb2 as open_dataset

and I ran commands on terminal
=>
$ protoc ./waymo_open_dataset/dataset.proto -- python_out=. $ protoc ./waymo_open_dataset/label.proto -- python_out=.

After ran them, change directory to waymo_open_dataset(master)/waymo_open_dataset
Now, you can see dataset_pb2.py and label_pb2.py files exist.

Yes it worked. Thanks