Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix errors assigning bool to DateTime properties in Keys #326

Merged
merged 9 commits into from
Jun 21, 2022

Conversation

Nextra
Copy link
Contributor

@Nextra Nextra commented May 23, 2022

What does this PR do?

Fixes #321

Comments on the format choices:

MeiliSearch outputs various slight variations of timestamp formats depending on the precise moment represented by it (see #321). This SDK can neither assume that expiresAt never contains milli- or microseconds, nor that createdAt or updatedAt always contains microseconds. I don't think PHP provides a way to handle this in a single date_create_from_format call, so this change makes it always try both.

In my opinion .vu as a format term for microseconds is unnecessarily restrictive, forcing the milli-/microsecond part to always contain four or more digits, while using just u covers all possibilities. - Not quite true, see #326 (comment)

Similar goes for forcing the timezone to appear as Z. While MeiliSearch seems to adhere to this, I don't think it is necessary to disallow generic time zone notations.

If an invalid timestamp is provided, so either a non-string is passed or the string has an invalid format, it defaults to returning null. This reliably prevents the assignment error, but might not be desired behavior as it eats formatting errors.

Please do not hesitate to make style or implementation changes. I would consider this a high priority issue, so please do not wait for me to get back to you. The SDK is virtually unusable in practice if you need to interact with API keys.

PR checklist

Please check if your PR fulfills the following requirements:

  • Does this PR fix an existing issue?
  • Have you read the contributing guidelines?
  • Have you made sure that the title is accurate and descriptive of the changes?

@brunoocasali brunoocasali self-requested a review June 9, 2022 12:42
@Nextra
Copy link
Contributor Author

Nextra commented Jun 9, 2022

@brunoocasali

I think I have fixed the issues found by the various checks and tests.

More information on what I said up top: .vu as a format for the fractional seconds is not actually restrictive, but just plain wrong:

>>> date_create_from_format('Y-m-d\TH:i:s.uP', '2001-02-03T04:05:06.123456Z')
=> DateTime @978310861 {#11847
     date: 2001-02-03 04:05:06.123456 +00:00,
   }
>>> date_create_from_format('Y-m-d\TH:i:s.vuP', '2001-02-03T04:05:06.123456Z')
=> DateTime @978310861 {#11849
     date: 2001-02-03 04:05:06.456 +00:00,
   }

Note how .vu results in .456 because u only starts reading after the first three digits, but then replaces whatever v did before.

Also: I found another variant of timestamp that can again not be parsed by even this updated version:

>>> app(MeiliSearch\Client::class)->getRawKeys()['results'][1]['createdAt']
=> "2022-05-20T08:51:30.4825815Z"
>>> date_create_from_format('Y-m-d\TH:i:s.uP', '2022-05-20T08:51:30.4825815Z')
=> false

If it is not necessary to only accept just these two precise timestamp formats, it would probably be advisable to simply use the DateTime constructor which seems to handle all variants properly, discarding additional microsecond digits if necessary.

>>> new DateTime('2001-02-03T04:05:06Z')
=> DateTime @981173106 {#11848
     date: 2001-02-03 04:05:06.0 UTC (+00:00),
   }
>>> new DateTime('2001-02-03T04:05:06.123Z')
=> DateTime @981173106 {#11852
     date: 2001-02-03 04:05:06.123 UTC (+00:00),
   }
>>> new DateTime('2001-02-03T04:05:06.123456Z')
=> DateTime @981173106 {#11855
     date: 2001-02-03 04:05:06.123456 UTC (+00:00),
   }
>>> new DateTime('2001-02-03T04:05:06.123456789Z')
=> DateTime @981173106 {#11850
     date: 2001-02-03 04:05:06.123456 UTC (+00:00),
   }

@Nextra
Copy link
Contributor Author

Nextra commented Jun 10, 2022

@brunoocasali
Two possible variants that pass the tests:

protected function createDate($attribute): ?DateTime
{
    if (!\is_string($attribute)) {
        return null;
    }

    if (false === strpos($attribute, '.')) {
        $date = date_create_from_format(DateTime::ATOM, $attribute);
    } else {
        $attribute = preg_replace('/(\.\d{6})\d+/', '$1', $attribute, 1);
        $date = date_create_from_format('Y-m-d\TH:i:s.uP', $attribute);
    }

    return false === $date ? null : $date;
}

protected function createDate($attribute): ?DateTime
{
    if (!\is_string($attribute)) {
        return null;
    }

    try {
        return new DateTime($attribute);
    } catch (\Exception $e) {
        return null;
    }
}

Edit for posterity:

The DateTime constructor fails to discard extra microsecond digits in PHP7.4 and early versions of PHP8.0, which is why the implementation was reverted to the date_create_from_format variant (ref).

@brunoocasali
Copy link
Member

I'll give a look in your PR today, thanks for being so involved :)

src/Endpoints/Keys.php Outdated Show resolved Hide resolved
@brunoocasali
Copy link
Member

@Nextra I have a hard time understanding the issue, I was not able to reproduce it, can you create a test case to cover this bug?

About your variants, I think the second one is better to read, but I'm not sure if it will solve the problem of multiple formats simultaneously, will it?

@Nextra
Copy link
Contributor Author

Nextra commented Jun 14, 2022

@Nextra I have a hard time understanding the issue, I was not able to reproduce it, can you create a test case to cover this bug?

Any information I provided in #321 and in this thread comes from a real-world test. When I present output from a Tinker REPL session12, that actually talked to a proper QA Meilisearch installation, and shows it outputting timestamps that this SDK fails to parse. I know it is hard to test because Meilisearch doesn't output consistent timestamp formats, and instead three "variants":

  1. Timestamp with second precision: 2001-02-03T04:05:06Z
  2. Timestamp with milisecond precision: 2001-02-03T04:05:06.123Z
  3. Timestamp with microsecond precision: 2001-02-03T04:05:06.12356Z or 2001-02-03T04:05:06.123567Z (and possibly more digits)

The reason your tests don't fail outright is that your expiresAt timestamps are not created as they might be in a real-world scenario3. These artificially truncated timestamps will always be output back to the SDK/tests by Meilisearch as variant 1, which the SDK relies on by not parsing any fractional seconds4. This means in a real world scenario where expiry timestamps are not always truncated1, almost every single key will be unparsable2.

After working around this issue in my personal setup, I noticed that createdAt and updatedAt also at least output variant 22, which then also fails to parse correctly. They might also output variant 1 if a timestamp was randomly created at exactly 0 seconds (incredibly unlikely), which makes this hard to test. This SDK however relies on them outputting variant 35. The format used is also subtly incorrect6 which should be mostly harmless, but still get fixed.

About your variants, I think the second one is better to read, but I'm not sure if it will solve the problem of multiple formats simultaneously, will it?

I showed it accepting the three variants I mentioned, and correctly discarding microsecond digits that PHP cannot handle6. I wanted to provide both of them to provide you will all the information you might need for a decision. Otherwise I have no preference.

Footnotes

  1. https://github.com/meilisearch/meilisearch-php/issues/321#issuecomment-1114667800 2

  2. https://github.com/meilisearch/meilisearch-php/issues/321#issuecomment-1123358789 2 3

  3. https://github.com/meilisearch/meilisearch-php/blob/e47796269169f89f1d4062c7525e54f7386b5762/tests/Endpoints/KeysAndPermissionsTest.php#L121

  4. https://github.com/meilisearch/meilisearch-php/blob/e47796269169f89f1d4062c7525e54f7386b5762/src/Endpoints/Keys.php#L46

  5. https://github.com/meilisearch/meilisearch-php/blob/e47796269169f89f1d4062c7525e54f7386b5762/src/Endpoints/Keys.php#L48-L53

  6. https://github.com/meilisearch/meilisearch-php/pull/326#issuecomment-1151213711 2

@brunoocasali
Copy link
Member

Hi @Nextra I was able to create a failing test for your issue:

<?php

declare(strict_types=1);

namespace Tests\Endpoints;

use MeiliSearch\Http\Client;
use MeiliSearch\MeiliSearch;
use PHPUnit\Framework\MockObject\MockObject;
use PHPUnit\Framework\TestCase;
use Psr\Http\Client\ClientInterface;
use Psr\Http\Message\RequestInterface;
use Psr\Http\Message\ResponseInterface;
use Psr\Http\Message\StreamInterface;

use Tests\Http\ClientTest;

final class DateHandlingTest extends ClientTest
{
    /**
     * @group failing
     * Tests the api edit form
     */
    public function testHandleDateFormat(): void
    {
        $httpClient = $this->createHttpClientMock(200, '
        {
            "results": [
              {
                "description": "my key",
                "key": "z6ySBsnp002e8bc6a31b794a95d623333be1fe4fd2d7eacdeaf7baf2c439866723e659ee",
                "actions": ["*"],
                "indexes": ["*"],
                "expiresAt": "2023-06-14T10:34:03.629690014Z",
                "createdAt": "2022-06-14T10:34:03.627606639Z",
                "updatedAt": "2022-06-14T10:34:03.627606639Z"
              }
            ]
          }
        ');
        $newClient = new \MeiliSearch\Client('https://localhost:7700', null, $httpClient);

        $response = $newClient->getKeys();

        $this->assertCount(2, $response);
        $this->assertNull($response[0]->getExpiresAt());
    }

     /**
     * @return ClientInterface|MockObject
     */
    private function createHttpClientMock(int $status = 200, string $content = '{', string $contentType = 'application/json')
    {
        $stream = $this->createMock(StreamInterface::class);
        $stream->expects(self::once())
            ->method('getContents')
            ->willReturn($content);

        $response = $this->createMock(ResponseInterface::class);
        $response->expects(self::any())
            ->method('getStatusCode')
            ->willReturn($status);
        $response->expects(self::any())
            ->method('getHeader')
            ->with('content-type')
            ->willReturn([$contentType]);
        $response->expects(self::once())
            ->method('getBody')
            ->willReturn($stream);

        $httpClient = $this->createMock(ClientInterface::class);
        $httpClient->expects(self::once())
            ->method('sendRequest')
            ->with(self::isInstanceOf(RequestInterface::class))
            ->willReturn($response);

        return $httpClient;
    }
}

It was a pure CTRL+C/V with the createHttpClientMock defined in other class (maybe it's a good time to share them). What do you think? Can you apply your code to make it pass?

image

@Nextra
Copy link
Contributor Author

Nextra commented Jun 15, 2022

I don't quite have the time to setup a test environment today, but both of my createDate variants parse the timestamps you have chosen in your example, so the tests should also pass.

@Nextra
Copy link
Contributor Author

Nextra commented Jun 15, 2022

I thought I should mention this: The two variants should not cause different behavior if ISO8601 timestamps are used as input (Y-m-d\TH:i:s.uP in PHP parlance), which should always be the case here. The difference is that the native DateTime constructor allows more, completely different formats aswell. This should not be a problem for this usage, but here are some examples:

>>> new DateTime('2001-02-03 04:05:06')
=> DateTime @981173106 {
     date: 2001-02-03 04:05:06.0 UTC (+00:00),
   }
>>> new DateTime('Sat Feb 03, 2001 04:05:06')
=> DateTime @981173106 {
     date: 2001-02-03 04:05:06.0 UTC (+00:00),
   }
>>> new DateTime('@981173106')
=> DateTime @981173106 {
     date: 2001-02-03 04:05:06.0 +00:00,
   }

@brunoocasali brunoocasali added the bug Something isn't working label Jun 21, 2022
Copy link
Member

@brunoocasali brunoocasali left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for your hard work in fixing this issue @Nextra

Meilisearch is better with contributors like you! ❤️

@brunoocasali
Copy link
Member

bors merge

@bors
Copy link
Contributor

bors bot commented Jun 21, 2022

@bors bors bot merged commit b3daf75 into meilisearch:main Jun 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Internal error assigning bool to property expiresAt
3 participants