Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BigQueryInsertJobOperator may start the same job twice #17493

Closed
hsej opened this issue Aug 8, 2021 · 3 comments
Closed

BigQueryInsertJobOperator may start the same job twice #17493

hsej opened this issue Aug 8, 2021 · 3 comments
Labels
kind:bug This is a clearly a bug won't fix

Comments

@hsej
Copy link

hsej commented Aug 8, 2021

Apache Airflow version: 1.10.15

Apache Airflow Provider versions (please include all providers that are relevant to your bug):
apache-airflow-backport-providers-google==2021.3.3

What happened:
Looking at the _submit_job method the job is started by using job.result() - in bold below. However, the job IS already started by just callnig the hook.insert_job() method. Thus I suggest removing the extra job.result() call in the _submit_job() method below.

    def _submit_job(
        self,
        hook: BigQueryHook,
        job_id: str,
    ) -> BigQueryJob:
        # Submit a new job
        job = hook.insert_job(
            configuration=self.configuration,
            project_id=self.project_id,
            location=self.location,
            job_id=job_id,
        )
        # Start the job and wait for it to complete and get the result.
        **job.result()**
        return job
@hsej hsej added the kind:bug This is a clearly a bug label Aug 8, 2021
@boring-cyborg
Copy link

boring-cyborg bot commented Aug 8, 2021

Thanks for opening your first issue here! Be sure to follow the issue template!

@hsej
Copy link
Author

hsej commented Aug 8, 2021

Sorry!

Thank you very much for that.

@uranusjr
Copy link
Member

uranusjr commented Aug 9, 2021

We have ceased maintenance of Airflow 1.10. Please consider upgrading to Airflow 2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind:bug This is a clearly a bug won't fix
Projects
None yet
Development

No branches or pull requests

2 participants