Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(PubSub): publisher compression type #6417

Merged

Conversation

vishwarajanand
Copy link
Contributor

@vishwarajanand vishwarajanand commented Jul 5, 2023

@product-auto-label product-auto-label bot added the api: pubsub Issues related to the Pub/Sub API. label Jul 5, 2023
@vishwarajanand vishwarajanand added the next release PRs to be included in the next release label Jul 5, 2023
@vishwarajanand vishwarajanand marked this pull request as ready for review July 5, 2023 20:45
@vishwarajanand vishwarajanand requested review from a team as code owners July 5, 2023 20:45
Copy link
Contributor

@bshaffer bshaffer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bit worrisome because, while removing the typehint might get rid of the error, it's not the best way to fix this issue. Instead we should be looking at why these properties weren't initialized by the time they were accessed. They're being set properly in the constructor, so they should be initialized in every instance they're used.

My guess is that since we're using Sysv, there may be some strange behavior here. Those values aren't showing up on the other side when passed to the batch daemon.

When I switched the order so that the call to setCommonBatchProperties happened after setting those properties, the error went away. I believe this is because when this method is called, it looks at what the initialized properties are, and only serializes those. So we need to initialize them beforehand.

I think this is a better solution, so please test and see if it works!

$this->setCommonBatchProperties($options + [
'identifier' => sprintf(self::ID_TEMPLATE, $topicName),
'batchMethod' => 'publishDeferred'
]);
$this->enableCompression = $options['enableCompression'] ?? false;
$this->compressionBytesThreshold = $options['compressionBytesThreshold'] ??
Topic::DEFAULT_COMPRESSION_BYTES_THRESHOLD;

(move lines 90-93 to be after line 96)

@yash30201
Copy link
Contributor

yash30201 commented Jul 6, 2023

This is a bit worrisome because, while removing the typehint might get rid of the error, it's not the best way to fix this issue. Instead we should be looking at why these properties weren't initialized by the time they were accessed. They're being set properly in the constructor, so they should be initialized in every instance they're used.

My guess is that since we're using Sysv, there may be some strange behavior here. Those values aren't showing up on the other side when passed to the batch daemon.

When I switched the order so that the call to setCommonBatchProperties happened after setting those properties, the error went away. I believe this is because when this method is called, it looks at what the initialized properties are, and only serializes those. So we need to initialize them beforehand.

I think this is a better solution, so please test and see if it works!

$this->setCommonBatchProperties($options + [
'identifier' => sprintf(self::ID_TEMPLATE, $topicName),
'batchMethod' => 'publishDeferred'
]);
$this->enableCompression = $options['enableCompression'] ?? false;
$this->compressionBytesThreshold = $options['compressionBytesThreshold'] ??
Topic::DEFAULT_COMPRESSION_BYTES_THRESHOLD;

(move lines 90-93 to be after line 96)

Thank you so much @bshaffer for zeroing this in🔥.
I was investigating why this was happening and cornered down to Sysv but wasn't able to understand the exact issue. This was a valuable insight.🙌

I've raised a PR fixing this. #6423

@yash30201 yash30201 closed this Jul 6, 2023
@yash30201 yash30201 reopened this Jul 6, 2023
@yash30201
Copy link
Contributor

Discussed internally offline that since the case when we do not use type checks eliminates the issue, hence we should go ahead with priortising the fixes until we properly figure out what exactly is causing this issue.

@vishwarajanand
Copy link
Contributor Author

Thanks for looking into this @bshaffer. We fixed your recommendation.
But think we should merge this as well, for two reasons:

  1. Seems some linters still complain (maybe they are checking on older PHP versions?) for private int $var_name; and private bool $var_name; while our lint has no issue with it.
  2. These are the only two source files to follow the explicit type declaration pattern.

I am also not sure why not setting the properties in setCommonBatchProperties could lead to a variable not initialized error, when it's always initialized to false by default in constructor.

@bshaffer
Copy link
Contributor

bshaffer commented Jul 6, 2023

My main concern with the approach here is that it will fix the fatal error (which is obviously super important) but won't fix the underlying issue of the expected variable values being passed to the batch runner. I think if you test, you'll see that even without the typehints, those values are not initialized as expected.

I don't mind getting rid of the typehint - they aren't that important. But we still need to switch the order if we want the local variables to be set.

EDIT: I see now we already merged the other PR, so this comment is not necessary!

As for the WHY this works - take this example:

class ExampleFoo
{
    protected string $typedVariable1;
    protected string $typedVariable2;
    protected $untypedVariable;

    public function __construct() 
    {
        // initialize one of the typed variables, but not the other
        $this->typedVariable1 = 'initialized!';
    }

    public function getObjectVars()
    {
        return get_object_vars($this);
    }
}

In this example, we initialize one typed variable, but not the other. When we call getObjectVars on this object, only the typed variable we initialized, along with the untyped variable, will show up:

$f = new ExampleFoo();
var_dump($f->getObjectVars());
// array(2) {
//  ["typedVariable1"]=>
//  string(12) "initialized!"
//  ["untypedVariable"]=>
//  NULL
//}

This is a guess, I'd need to do more digging to be sure.

@bshaffer bshaffer self-requested a review July 6, 2023 14:54
@bshaffer bshaffer dismissed their stale review July 6, 2023 14:54

I no longer require changes

@bshaffer
Copy link
Contributor

bshaffer commented Jul 6, 2023

Seems some linters still complain

Can you give me an example of this?

These are the only two source files to follow the explicit type declaration pattern

I found that strange as well - I wonder when they were added?

I removed my review so feel free to merge it if you feel it's important!

@yash30201 yash30201 merged commit 6811945 into googleapis:main Jul 7, 2023
@vishwarajanand
Copy link
Contributor Author

It seems it was complaining because of an older php version. When I test the following code in php7.3 or below it complains.

Checked the below code in http://phptester.net/

<?php

class MyClass {
    private int $myInt;
    private bool $myBool;
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: pubsub Issues related to the Pub/Sub API. next release PRs to be included in the next release
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants