Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

3D Tiles Traversal cleanup #6390

Merged
merged 64 commits into from
Aug 16, 2018
Merged
Show file tree
Hide file tree
Changes from 34 commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
d683e4d
Change request queue from a heap to a sorted array
lilleyse Feb 20, 2018
366a1d2
Set highest priority for globe tiles that you are inside
lilleyse Feb 21, 2018
3f09e0c
Move tileset cache to its own class
lilleyse Feb 22, 2018
15700e4
Temp
lilleyse Feb 27, 2018
7500d7d
Stable
lilleyse Mar 20, 2018
b285439
Optimizations
lilleyse Mar 21, 2018
b44933d
Fix bug with skipping. Solid
lilleyse Mar 21, 2018
b975f1e
Combine base and skip traversal together
lilleyse Mar 22, 2018
5a36e0b
Remove touched frame checks
lilleyse Mar 22, 2018
66f977d
Remove desiredTiles
lilleyse Mar 22, 2018
e0a2a86
Better handling of processing tiles
lilleyse Mar 23, 2018
dbdd7ef
Sort children by distance in main traversal
lilleyse Mar 26, 2018
2894551
Select empty tiles so their debug bounding volumes get drawn
lilleyse Mar 26, 2018
b9219db
Limit selection traversal by SSE and visibility
lilleyse Mar 26, 2018
5e7e764
Update tile outline properly
lilleyse Mar 26, 2018
63ad7c3
Cleanup
lilleyse Mar 26, 2018
c804dd3
Fix for outlines
lilleyse Mar 26, 2018
a6aad57
Fixing tests
lilleyse Mar 27, 2018
4bbacf8
Cleanup
lilleyse Mar 28, 2018
248c443
Better handling of request volume
lilleyse Mar 28, 2018
db216c1
Include additive tiles in final selection
lilleyse Mar 28, 2018
1f51e9b
Add empty tiles to a different selection list
lilleyse Mar 28, 2018
978f9f6
Reorganization of checking refinement
lilleyse Mar 28, 2018
582a51f
Fix geometric errors on TilesetOfTilesets
lilleyse Mar 28, 2018
49c005b
Add back loadSiblings and fix more tests
lilleyse Mar 28, 2018
4509017
Fix 2 tests
lilleyse Mar 29, 2018
7d21281
Fix expiration
lilleyse Mar 29, 2018
ca07240
Remove unneeeded scene.renderForSpecs
lilleyse Mar 29, 2018
fae93f7
Fix remaining tests and starting to reorg for pick pass
lilleyse Mar 29, 2018
72ec719
Fix picking statistics
lilleyse Mar 29, 2018
e558f2f
Fix refine for skip-lods off
lilleyse Mar 29, 2018
0cb8dcd
Always include root tile in base traversal
lilleyse Mar 29, 2018
6d29c37
Move tileset cache to its own class
lilleyse Feb 22, 2018
5c97c9b
Rename file
lilleyse Mar 27, 2018
15921c2
Rename file back to lowercase
lilleyse Mar 27, 2018
bc9c55a
Fix eslint error
lilleyse Mar 29, 2018
ee36a46
Temp CHANGES.md
lilleyse Mar 29, 2018
059d2b0
Merge branch 'master' into tileset-cache
lilleyse Apr 2, 2018
7254492
Removed hasTilesetContent check when loading because it will always b…
lilleyse Apr 3, 2018
9046570
Merge branch 'master' into request-performance
lilleyse Apr 4, 2018
593826f
Merge branch 'request-performance' into tileset-cache
lilleyse Apr 4, 2018
8e58835
Add back comment about expired tiles
lilleyse Apr 4, 2018
57627f4
Merge pull request #6391 from AnalyticalGraphicsInc/tileset-cache
pjcozzi Apr 11, 2018
ac0cf76
Merge branch 'master' into traversal-cleanup
lilleyse Jun 13, 2018
3f6714b
Add and update comments
lilleyse Jun 13, 2018
fc65396
Merge branch 'master' into request-performance
lilleyse Jun 29, 2018
5e57ca2
Merge branch 'request-performance' into traversal-cleanup
lilleyse Jun 29, 2018
5856714
Simplify update visibility checking
lilleyse Jul 4, 2018
6bda10e
Simplify visibility code and tweak comments
lilleyse Jul 4, 2018
79e77c7
Tweak and fix tests to get better code coverage
lilleyse Jul 4, 2018
716c6a4
Draw empty bounding volumes as a different color (green)
lilleyse Jul 4, 2018
a0d5c5c
Comments and .peek
lilleyse Jul 5, 2018
6a8fcfa
Updated CHANGES.md
lilleyse Jul 5, 2018
2705996
Remove RequestQueue code
lilleyse Jul 5, 2018
c8c981e
Added visited frame to tile
lilleyse Jul 6, 2018
1dd5e68
Fix update visibility code again
lilleyse Jul 6, 2018
8df3ddb
Merge branch 'master' into traversal-cleanup
lilleyse Jul 6, 2018
6e84390
Remove TODO
lilleyse Jul 6, 2018
e95a9a2
Merge branch 'master' into traversal-cleanup
lilleyse Jul 25, 2018
8d85e60
Updates
lilleyse Jul 28, 2018
4fc6f7e
More concise handling of finalResolution
lilleyse Jul 28, 2018
6991103
Remove tile.renderableContent
lilleyse Jul 28, 2018
30c0459
Merge branch 'master' into traversal-cleanup
lilleyse Aug 9, 2018
b527503
Merge branch 'master' into traversal-cleanup
lilleyse Aug 15, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
186 changes: 186 additions & 0 deletions Source/Core/RequestQueue.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,186 @@
define([
'./Check',
'./defineProperties'
], function(
Check,
defineProperties) {
'use strict';

/**
* Priority queue for the {@link RequestScheduler} implemented as a sorted array.
* The request with the highest priority is placed at index 0 and the request
* with lowest priority is placed at index <code>length - 1</code>.
* <p>
* A lower <code>request.priority</code> value indicates that the request has higher priority. See {@link Request#priority}.
* </p>
*
* @alias RequestQueue
* @constructor
* @private
*
* @param {Number} maximumLength The maximum length of the queue.
*/
function RequestQueue(maximumLength) {
//>>includeStart('debug', pragmas.debug);
Check.typeOf.number('maximumLength', maximumLength);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpicky, but since we have it should this check number.greaterThanOrEquals?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update in #6240: 9b9d8fd

//>>includeEnd('debug');

this._array = new Array(maximumLength);
this._length = 0;
this._maximumLength = maximumLength;
}

defineProperties(RequestQueue.prototype, {
/**
* Gets the length of the queue.
*
* @memberof RequestQueue.prototype
*
* @type {Number}
* @readonly
*/
length : {
get : function() {
return this._length;
}
}
});

/**
* Get the request at the given index.
*
* @param {Number} index The index of the request.
*
* @return {Request} The request at the given index.
*/
RequestQueue.prototype.get = function(index) {
//>>includeStart('debug', pragmas.debug);
Check.typeOf.number.greaterThanOrEquals('index', index, 0);
Check.typeOf.number.lessThan('index', index, this._length);
//>>includeEnd('debug');
return this._array[index];
};

/**
* Insert a request into the queue. If the length would grow greater than the maximum length
* of the queue, the lowest priority request is removed and returned.
*
* @param {Request} request The request to insert.
*
* @return {Request|undefined} The request that was removed from the queue if the queue is at full capacity.
*/
RequestQueue.prototype.insert = function(request) {
//>>includeStart('debug', pragmas.debug);
Check.defined('request', request);
//>>includeEnd('debug');

var array = this._array;
var previousLength = this._length;
var length = this._length;
var maximumLength = this._maximumLength;

if (length < maximumLength)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like this can also just be previousLength.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update in #6240: 9b9d8fd

{
++this._length;
}

if (previousLength === 0)
{
array[0] = request;
return;
}

var removedRequest;
var lastIndex = previousLength - 1;

if (previousLength === maximumLength) {
var lastRequest = array[lastIndex];
if (request.priority >= lastRequest.priority) {
// The array is full and the priority value of this request is too high to be inserted.
return request;
}
// The array is full and the inserted request pushes off the last request
removedRequest = lastRequest;
--lastIndex;
}

while (lastIndex >= 0 && request.priority < array[lastIndex].priority) {
array[lastIndex + 1] = array[lastIndex]; // Shift element to the right
--lastIndex;
}
array[lastIndex + 1] = request;

return removedRequest;
};

/**
* Call the given function for each request in the queue.
*
* @type {RequestQueue~ForEachCallback} The function to call.
*/
RequestQueue.prototype.forEach = function(callback) {
//>>includeStart('debug', pragmas.debug);
Check.typeOf.func('callback', callback);
//>>includeEnd('debug');

var array = this._array;
var length = this._length;
for (var i = 0; i < length; ++i) {
callback(array[i]);
}
};

/**
* Sorts the queue.
*/
RequestQueue.prototype.sort = function() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I asked this in another PR, but do you want this to delegate to a new generic function just like how mergeSort is separate? https://github.com/AnalyticalGraphicsInc/cesium/blob/master/Source/Core/mergeSort.js

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I replied really late to your comment in the other PR, but it is here: #6240 (comment).

var array = this._array;
var length = this._length;

// Use insertion sort since our array is small and likely to be mostly sorted already.
// Additionally length may be smaller than the array's actual length, so calling array.sort will lead to incorrect results for uninitialized values.
for (var i = 1; i < length; ++i) {
var j = i;
while ((j > 0) && (array[j - 1].priority > array[j].priority)) {
var temp = array[j - 1];
array[j - 1] = array[j];
array[j] = temp;
--j;
}
}
};

/**
* Remove <code>length</code> number of requests from the top of the queue.
*
* @param {Number} length The number of requests to remove.
*/
RequestQueue.prototype.remove = function(length) {
//>>includeStart('debug', pragmas.debug);
Check.typeOf.number.greaterThanOrEquals('length', length, 0);
Check.typeOf.number.lessThanOrEquals('length', length, this._length);
//>>includeEnd('debug');
if (length === 0) {
return;
}
if (length === this._length) {
this._length = 0;
return;
}

// Shift remaining requests back to the left
var array = this._array;
for (var i = length; i < this._length; ++i) {
array[i - length] = array[i];
}
this._length -= length;
};

/**
* The callback to use in forEach.
* @callback RequestQueue~ForEachCallback
* @param {Request} request The request.
*/

return RequestQueue;
});
82 changes: 39 additions & 43 deletions Source/Core/RequestScheduler.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@ define([
'./defined',
'./defineProperties',
'./Event',
'./Heap',
'./isBlobUri',
'./isDataUri',
'./RequestQueue',
'./RequestState'
], function(
Uri,
Expand All @@ -16,16 +16,12 @@ define([
defined,
defineProperties,
Event,
Heap,
isBlobUri,
isDataUri,
RequestQueue,
RequestState) {
'use strict';

function sortRequests(a, b) {
return a.priority - b.priority;
}

var statistics = {
numberOfAttemptedRequests : 0,
numberOfActiveRequests : 0,
Expand All @@ -35,12 +31,8 @@ define([
numberOfActiveRequestsEver : 0
};

var priorityHeapLength = 20;
var requestHeap = new Heap({
comparator : sortRequests
});
requestHeap.maximumLength = priorityHeapLength;
requestHeap.reserve(priorityHeapLength);
var requestQueueLength = 20;
var requestQueue = new RequestQueue(requestQueueLength);

var activeRequests = [];
var numberOfActiveRequestsByServer = {};
Expand Down Expand Up @@ -112,29 +104,28 @@ define([
},

/**
* The maximum size of the priority heap. This limits the number of requests that are sorted by priority. Only applies to requests that are not yet active.
* The maximum length of the request queue. This limits the number of requests that are sorted by priority. Only applies to requests that are not yet active.
*
* @memberof RequestScheduler
*
* @type {Number}
* @default 20
*
* @private
*/
priorityHeapLength : {
requestQueueLength : {
get : function() {
return priorityHeapLength;
return requestQueueLength;
},
set : function(value) {
// If the new length shrinks the heap, need to cancel some of the requests.
// Since this value is not intended to be tweaked regularly it is fine to just cancel the high priority requests.
if (value < priorityHeapLength) {
while (requestHeap.length > value) {
var request = requestHeap.pop();
cancelRequest(request);
}
// Cancel all requests and resize the queue
var length = requestQueue.length;
for (var i = 0; i < length; ++i) {
var request = requestQueue.get(i);
cancelRequest(request);
}
priorityHeapLength = value;
requestHeap.maximumLength = value;
requestHeap.reserve(value);
requestQueue = new RequestQueue(value);
RequestScheduler.requestQueue = requestQueue;
}
}
});
Expand Down Expand Up @@ -242,21 +233,19 @@ define([
}
activeRequests.length -= removeCount;

// Update priority of issued requests and resort the heap
var issuedRequests = requestHeap.internalArray;
var issuedLength = requestHeap.length;
for (i = 0; i < issuedLength; ++i) {
updatePriority(issuedRequests[i]);
}
requestHeap.resort();
// Update priority of issued requests and resort the queue
requestQueue.forEach(updatePriority);
requestQueue.sort();

// Get the number of open slots and fill with the highest priority requests.
// Un-throttled requests are automatically added to activeRequests, so activeRequests.length may exceed maximumRequests
var openSlots = Math.max(RequestScheduler.maximumRequests - activeRequests.length, 0);
var filledSlots = 0;
while (filledSlots < openSlots && requestHeap.length > 0) {
// Loop until all open slots are filled or the heap becomes empty
request = requestHeap.pop();
var processedRequests = 0;
var totalRequests = requestQueue.length;
while (filledSlots < openSlots && processedRequests < totalRequests) {
// Loop until all open slots are filled or the queue becomes empty
request = requestQueue.get(processedRequests++);
if (request.cancelled) {
// Request was explicitly cancelled
cancelRequest(request);
Expand All @@ -272,6 +261,7 @@ define([
startRequest(request);
++filledSlots;
}
requestQueue.remove(processedRequests);

updateStatistics();
};
Expand Down Expand Up @@ -344,10 +334,9 @@ define([
return undefined;
}

// Insert into the priority heap and see if a request was bumped off. If this request is the lowest
// priority it will be returned.
// Insert into the priority queue and see if a request was bumped off. If this request is the lowest priority it will be returned.
updatePriority(request);
var removedRequest = requestHeap.insert(request);
var removedRequest = requestQueue.insert(request);

if (defined(removedRequest)) {
if (removedRequest === request) {
Expand Down Expand Up @@ -397,12 +386,19 @@ define([
* @private
*/
RequestScheduler.clearForSpecs = function() {
while (requestHeap.length > 0) {
var request = requestHeap.pop();
var request;
var length;
var i;

length = requestQueue.length;
for (i = 0; i < length; ++i) {
request = requestQueue.get(i);
cancelRequest(request);
}
var length = activeRequests.length;
for (var i = 0; i < length; ++i) {
requestQueue.remove(length);

length = activeRequests.length;
for (i = 0; i < length; ++i) {
cancelRequest(activeRequests[i]);
}
activeRequests.length = 0;
Expand Down Expand Up @@ -431,7 +427,7 @@ define([
*
* @private
*/
RequestScheduler.requestHeap = requestHeap;
RequestScheduler.requestQueue = requestQueue;

return RequestScheduler;
});
Loading