Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH Zero limit no results #10652

Merged
merged 3 commits into from
Jan 26, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 19 additions & 13 deletions src/ORM/ArrayList.php
Original file line number Diff line number Diff line change
Expand Up @@ -171,23 +171,29 @@ public function toNestedArray()
return $result;
}

/**
* Get a sub-range of this dataobjectset as an array
* Pass null to "remove the limit" - this is for consistency with DataList::limit(null) which itself will
* call SQLSelect::setLimit(null)
*
* @param int|null $length
* @param int $offset
* @return static
*/
public function limit($length, $offset = 0)
sabina-talipova marked this conversation as resolved.
Show resolved Hide resolved
public function limit(?int $length, int $offset = 0): static
{
if (!$length) {
$length = count($this->items ?? []);
if ($length === null) {
// If we unset the limit, we set the length to the size of the list. We still want the offset to be picked up
$length = count($this->items);
}

if ($length < 0) {
throw new InvalidArgumentException("\$length can not be negative. $length was provided.");
}

if ($offset < 0) {
throw new InvalidArgumentException("\$offset can not be negative. $offset was provided.");
}
Comment on lines +181 to 187
Copy link
Contributor Author

@maxime-rainville maxime-rainville Jan 18, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We weren't validating for this before. If you give a negative length, methods like array slice will want to start from the end.

While this might make sense for an array, most DB don't think like that. Seems more logical to throw an exception.


$list = clone $this;
$list->items = array_slice($this->items ?? [], $offset ?? 0, $length);

if ($length === 0) {
// If we set the limit to 0, we return an empty list.
$list->items = [];
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you let array slice below do its thing, you'll end up with the same results. Seems clearer to do it explicitly.

} else {
$list->items = array_slice($this->items ?? [], $offset ?? 0, $length);
}

return $list;
}
Expand Down
18 changes: 11 additions & 7 deletions src/ORM/DataList.php
Original file line number Diff line number Diff line change
Expand Up @@ -287,15 +287,19 @@ public function canFilterBy($fieldName)
/**
* Return a new DataList instance with the records returned in this query
* restricted by a limit clause.
*
* @param int $limit
* @param int $offset
* @return static
sabina-talipova marked this conversation as resolved.
Show resolved Hide resolved
*/
public function limit($limit, $offset = 0)
public function limit(?int $length, int $offset = 0): static
{
return $this->alterDataQuery(function (DataQuery $query) use ($limit, $offset) {
$query->limit($limit, $offset);
if ($length !== null && $length < 0) {
throw new InvalidArgumentException("\$length can not be negative. $length was provided.");
}

if ($offset < 0) {
throw new InvalidArgumentException("\$offset can not be negative. $offset was provided.");
}

return $this->alterDataQuery(function (DataQuery $query) use ($length, $offset) {
$query->limit($length, $offset);
});
}

Expand Down
6 changes: 1 addition & 5 deletions src/ORM/DataQuery.php
Original file line number Diff line number Diff line change
Expand Up @@ -752,12 +752,8 @@ public function reverseSort()

/**
* Set the limit of this query.
*
* @param int $limit
* @param int $offset
* @return $this
*/
public function limit($limit, $offset = 0)
public function limit(?int $limit, int $offset = 0): static
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not necessary to meet the ACs, but seems logical to change this while we are at it.

{
$this->query->setLimit($limit, $offset);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No exceptions on this one?

Copy link
Contributor Author

@maxime-rainville maxime-rainville Jan 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method is really just an alias for calling $query->setLimit(). So it seems rational to just let setLimit do all the work.

return $this;
Expand Down
8 changes: 4 additions & 4 deletions src/ORM/Limitable.php
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,9 @@ interface Limitable extends SS_List
* If $offset is specified, then that many records at the beginning of the list will be skipped.
* This matches the behaviour of the SQL LIMIT clause.
*
* @param int $limit
* @param int $offset
* @return static
* If `$length` is null, then no limit is applied. If `$length` is 0, then an empty list is returned.
*
* @throws InvalidArgumentException if $length or offset are negative
*/
public function limit($limit, $offset = 0);
public function limit(?int $length, int $offset = 0): Limitable;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to return a static here ... but list decorator doesn't actually return a version of itself, so I couldn't do that.

}
17 changes: 6 additions & 11 deletions src/ORM/ListDecorator.php
Original file line number Diff line number Diff line change
Expand Up @@ -17,21 +17,16 @@ abstract class ListDecorator extends ViewableData implements SS_List, Sortable,
/**
* @var SS_List
*/
protected $list;
protected SS_List&Sortable&Filterable&Limitable $list;
Copy link
Contributor Author

@maxime-rainville maxime-rainville Jan 25, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basically, every method on the class assumes that we will return a List that implements all these interfaces.


public function __construct(SS_List $list)
public function __construct(SS_List&Sortable&Filterable&Limitable $list)
{
$this->setList($list);

parent::__construct();
}

/**
* Returns the list this decorator wraps around.
*
* @return SS_List
*/
public function getList()
public function getList(): SS_List&Sortable&Filterable&Limitable
{
return $this->list;
}
Expand All @@ -44,7 +39,7 @@ public function getList()
*
* @return SS_List
*/
public function setList($list)
public function setList(SS_List&Sortable&Filterable&Limitable $list): self
{
$this->list = $list;
$this->failover = $this->list;
Expand Down Expand Up @@ -246,9 +241,9 @@ public function filterByCallback($callback)
return $output;
}

public function limit($limit, $offset = 0)
public function limit(?int $length, int $offset = 0): SS_List&Sortable&Filterable&Limitable
{
return $this->list->limit($limit, $offset);
return $this->list->limit($length, $offset);
}

/**
Expand Down
2 changes: 1 addition & 1 deletion src/ORM/Queries/SQLSelect.php
Original file line number Diff line number Diff line change
Expand Up @@ -249,7 +249,7 @@ public function setLimit($limit, $offset = 0)
throw new InvalidArgumentException("SQLSelect::setLimit() only takes positive values");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make this the same as the other exceptions

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SQLSelect::setLimit() does substantially different things than the other limit methods. It accepts all sort of weird values like arrays and strings that the other don't.

We could change this error message to something more specific ... but I think it needs to be different than the other methods.

}

if (is_numeric($limit) && ($limit || $offset)) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The second part of this IF statement would fail if 0,0 was provided. This would effectively prevent you from setting a zero limit.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

setLimit() actually accept strings and array. This seems a bit weird but refactoring this is probably beyond the scope of this card.

if (is_numeric($limit)) {
$this->limit = [
'start' => (int)$offset,
'limit' => (int)$limit,
Expand Down
8 changes: 4 additions & 4 deletions src/ORM/Search/SearchContext.php
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@ protected function applyBaseTableFields()
* for example "Comments__Name" instead of the filter name "Comments.Name".
* @param array|bool|string $sort Database column to sort on.
* Falls back to {@link DataObject::$default_sort} if not provided.
* @param array|bool|string $limit
* @param array|null|string $limit
* @param DataList $existingQuery
* @return DataList
* @throws Exception
Expand Down Expand Up @@ -168,7 +168,7 @@ private function search(DataList $query): DataList
* Prepare the query to begin searching
*
* @param array|bool|string $sort Database column to sort on.
* @param array|bool|string $limit
* @param array|null|string $limit
*/
private function prepareQuery($sort, $limit, ?DataList $existingQuery): DataList
{
Expand Down Expand Up @@ -320,11 +320,11 @@ private function applyFilter(SearchFilter $filter, DataQuery $dataQuery, array $
*
* @param array $searchParams
* @param array|bool|string $sort
* @param array|bool|string $limit
* @param array|null|string $limit
* @return DataList
* @throws Exception
*/
public function getResults($searchParams, $sort = false, $limit = false)
public function getResults($searchParams, $sort = false, $limit = null)
{
$searchParams = array_filter((array)$searchParams, [$this, 'clearEmptySearchFields']);

Expand Down
57 changes: 46 additions & 11 deletions tests/php/ORM/ArrayListTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -117,34 +117,69 @@ function ($item) use (&$count, $test) {
$this->assertEquals($list->Count(), $count);
}

public function testLimit()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a bit weird how limited the test coverage was for such a critical method.

public function limitDataProvider(): array
{
$list = new ArrayList(
[
$all = [ ['Key' => 1], ['Key' => 2], ['Key' => 3] ];
list($one, $two, $three) = $all;

return [
'smaller limit' => [2, 0, [$one, $two]],
'limit equal to array' => [3, 0, $all],
'limit bigger than array' => [4, 0, $all],
'zero limit' => [0, 0, []],
'false limit' => [0, 0, []],
'null limit' => [null, 0, $all],

'smaller limit with offset' => [1, 1, [$two]],
'limit to end with offset' => [2, 1, [$two, $three]],
'bigger limit with offset' => [3, 1, [$two, $three]],
'offset beyond end of list' => [4, 3, []],
'zero limit with offset' => [0, 1, []],
'null limit with offset' => [null, 2, [$three]],

];
}

/**
* @dataProvider limitDataProvider
*/
public function testLimit($length, $offset, array $expected)
{
$data = [
['Key' => 1], ['Key' => 2], ['Key' => 3]
]
];
$list = new ArrayList($data);
$this->assertEquals(
$list->limit($length, $offset)->toArray(),
$expected
);
$this->assertEquals(
$list->limit(2, 1)->toArray(),
[
['Key' => 2], ['Key' => 3]
]
$list->toArray(),
$data,
'limit is immutable and does not affect the original list'
);
}

public function testLimitNull()
public function testLimitNegative()
{
$this->expectException(\InvalidArgumentException::class, 'Calling limit with a negative length throws exception');
$list = new ArrayList(
[
['Key' => 1], ['Key' => 2], ['Key' => 3]
]
);
$this->assertEquals(
$list->limit(null, 0)->toArray(),
$list->limit(-1);
}

public function testLimitNegativeOffset()
{
$this->expectException(\InvalidArgumentException::class, 'Calling limit with a negative offset throws exception');
$list = new ArrayList(
[
['Key' => 1], ['Key' => 2], ['Key' => 3]
]
);
$list->limit(1, -1);
}

public function testAddRemove()
Expand Down
50 changes: 31 additions & 19 deletions tests/php/ORM/DataListTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -121,31 +121,43 @@ public function testListCreationSortAndLimit()
$this->assertEquals(['Joe', 'Phil'], $list->limit(2, 1)->column('Name'));
}

public function testLimitAndOffset()
public function limitDataProvider(): array
{
$list = TeamComment::get();
$check = $list->limit(3);

$this->assertEquals(3, $check->count());

$check = $list->limit(1);
$this->assertEquals(1, $check->count());

$check = $list->limit(1, 1);
$this->assertEquals(1, $check->count());
return [
'no limit' => [null, 0, 3],
'smaller limit' => [2, 0, 2],
'greater limit' => [4, 0, 3],
'one limit' => [1, 0, 1],
'zero limit' => [0, 0, 0],
'limit and offset' => [1, 1, 1],
'false limit equivalent to 0' => [false, 0, 0],
'offset only' => [null, 2, 1],
'offset greater than list length' => [null, 3, 0],
'negative length' => [-1, 0, 0, true],
'negative offset' => [0, -1, 0, true],
];
}

$check = $list->limit(false);
$this->assertEquals(3, $check->count());
/**
* @dataProvider limitDataProvider
*/
public function testLimitAndOffset($length, $offset, $expectedCount, $expectException = false)
{
$list = TeamComment::get();

$check = $list->limit(null);
$this->assertEquals(3, $check->count());
if ($expectException) {
$this->expectException(\InvalidArgumentException::class);
}

$check = $list->limit(null, 2);
$this->assertEquals(1, $check->count());
$this->assertCount($expectedCount, $list->limit($length, $offset));
$this->assertCount(
$expectedCount,
$list->limit(0, 9999)->limit($length, $offset),
'Follow up limit calls unset previous ones'
);

// count()/first()/last() methods may alter limit/offset, so run the query and manually check the count
$check = $list->limit(null, 1)->toArray();
$this->assertEquals(2, count($check ?? []));
$this->assertCount($expectedCount, $list->limit($length, $offset)->toArray());
}

public function testDistinct()
Expand Down
5 changes: 3 additions & 2 deletions tests/php/ORM/ListDecoratorTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -243,8 +243,9 @@ public function testOffsetUnset()

public function testLimit()
{
$this->list->expects($this->once())->method('limit')->with(5, 3)->willReturn('mock');
$this->assertSame('mock', $this->decorator->limit(5, 3));
$mockLimitedList = $this->list;
$this->list->expects($this->once())->method('limit')->with(5, 3)->willReturn($mockLimitedList);
$this->assertSame($mockLimitedList, $this->decorator->limit(5, 3));
}

public function testTotalItems()
Expand Down
13 changes: 13 additions & 0 deletions tests/php/ORM/SQLSelectTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,7 @@ public function testSelectWithLimitClause()
$this->assertSQLEquals("SELECT * FROM MyTable LIMIT 99 OFFSET 97", $query->sql($parameters));
}


public function testSelectWithOrderbyClause()
{
$query = new SQLSelect();
Expand Down Expand Up @@ -255,6 +256,18 @@ public function testNullLimit()
);
}

public function testZeroLimit()
{
$query = new SQLSelect();
$query->setFrom("MyTable");
$query->setLimit(0);

$this->assertSQLEquals(
'SELECT * FROM MyTable LIMIT 0',
$query->sql($parameters)
);
}

public function testNegativeLimit()
{
$this->expectException(\InvalidArgumentException::class);
Expand Down