Skip to content

Commit

Permalink
[Management] Allows for imports to select existing index (elastic#14137)
Browse files Browse the repository at this point in the history 
* Adds ability to change index pattern on import

Signed-off-by: Tyler Smalley <[email protected]>

* UI changes. Use a table in the modal grouped by index pattern id instead of multiple modals.

* PR feedback

* PR feedback

* PR updates

* Handle skip properly

* Fix error when there were no existing index patterns

* Tests for the new import conflict logic

* Fix invisible filters caused by missing index pattern (elastic#14131)

"invisible filters" occur when the mapping chain throws an error. If a single filter throws an error, the entire chain rejects. As a result, not even the valid filters appear in the filter bar because they never get added to the scope. However the filters still exist in app state and still get sent with each search request.

The most common error occurs when the filter's meta.index property points to a non-existing index pattern. Since this property is only used for looking up field formatters and it is not essential for a working filter, we now fall back on raw values instead of failing if the index pattern is not found. See the PR this one replaces for discussion about other solutions we tried and why we chose to go this route.

* Show query and filter bars even when there's a linked search (elastic#14212)

The query bar used to be hidden in the presence of a linked search because unlike filters, queries didn't get merged when flattening a SearchSource hierarchy. That means a query in the query bar would override the query in the linked search. This is no longer the case. As of 6.0 we include all queries in the SearchSource hierarchy in the final request, so there's no longer any reason to hide the query bar.

Since filters created via a Vis show up in the query bar when Kuery is selected, these filters now appear correctly even when there's a linked search in the vis editor.

Previously when unlinking a saved search visualize would insert the query and filters from the saved search into app state before removing the SearchSource from the hierarcy. This posed a problem because combining two lucene query strings isn't as easy as combing two sets of filters. We decided this behavior was a bit counterintuitive anyway. If the user wants to unlink the saved search, they probably want to discard it, not combine it with their local changes. So I've also updated the unlinking functionality to discard the saved search.

* limit wait time for baselayer (elastic#14047)

* adding scope appy back (elastic#14269)

* remove junk tests (elastic#14191)

* We are using the index pattern id now

* Use the index pattern id here too

* Use an isolated es env for these tests

* Revert "Fix invisible filters caused by missing index pattern (elastic#14131)"

This reverts commit e09d7ad.

* Revert "Show query and filter bars even when there's a linked search (elastic#14212)"

This reverts commit 3aee7c2.

* Revert "limit wait time for baselayer (elastic#14047)"

This reverts commit 44a7107.

* Revert "adding scope appy back (elastic#14269)"

This reverts commit 51b6b51.

* Revert "remove junk tests (elastic#14191)"

This reverts commit f06c183.

* Revert these
  • Loading branch information
@chrisronline
chrisronline committed Oct 5, 2017
1 parent e6b6d22 commit 896cb26
Show file tree
Hide file tree
Showing 14 changed files with 762 additions and 57 deletions.
5 changes: 4 additions & 1 deletion src/core_plugins/kibana/public/management/sections/objects/_objects.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ <h1 class="kuiTitle">
<div class="kuiBarSection">
<button
class="kuiButton kuiButton--basic kuiButton--iconText"
data-test-subj="exportAllObjects"
ng-click="exportAll()"
>
<span class="kuiButton__inner">
Expand Down Expand Up @@ -49,6 +50,7 @@ <h1 class="kuiTitle">
ng-class="{ 'kuiTab-isSelected': state.tab === service.title }"
ng-repeat="service in services"
ng-click="changeTab(service)"
data-test-subj="objectsTab-{{ service.title }}"
>
{{ service.title }}
<small aria-label="{{:: service.data.length + ' of ' + service.total + ' ' + service.title }}">
Expand Down Expand Up @@ -126,7 +128,7 @@ <h1 class="kuiTitle">
</div>

<!-- Table -->
<table class="kuiTable" ng-if="service.data.length">
<table class="kuiTable" ng-if="service.data.length" data-test-subj="objectsTable-{{service.title}}">
<thead>
<tr>
<th class="kuiTableHeaderCell kuiTableHeaderCell--checkBox">
Expand Down Expand Up @@ -154,6 +156,7 @@ <h1 class="kuiTitle">
<tr
ng-repeat="item in service.data | orderBy:'title'"
class="kuiTableRow"
data-test-subj="objectsTableRow"
>
<td class="kuiTableRowCell kuiTableRowCell--checkBox">
<div class="kuiTableRowCell__liner">
Expand Down
146 changes: 96 additions & 50 deletions src/core_plugins/kibana/public/management/sections/objects/_objects.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,25 @@ import 'ui/directives/file_upload';
import uiRoutes from 'ui/routes';
import { SavedObjectsClientProvider } from 'ui/saved_objects';
import { uiModules } from 'ui/modules';
import { showChangeIndexModal } from './show_change_index_modal';
import { SavedObjectNotFound } from 'ui/errors';

const indexPatternsResolutions = {
indexPatterns: function (Private) {
const savedObjectsClient = Private(SavedObjectsClientProvider);

return savedObjectsClient.find({
type: 'index-pattern',
fields: ['title'],
perPage: 10000
}).then(response => response.savedObjects);
}
};

uiRoutes
.when('/management/kibana/objects', {
template: objectIndexHTML
template: objectIndexHTML,
resolve: indexPatternsResolutions
});

uiRoutes
Expand All @@ -19,7 +34,7 @@ uiRoutes
});

uiModules.get('apps/management')
.directive('kbnManagementObjects', function (kbnIndex, Notifier, Private, kbnUrl, Promise, confirmModal) {
.directive('kbnManagementObjects', function ($route, kbnIndex, Notifier, Private, kbnUrl, Promise, confirmModal) {
const savedObjectsClient = Private(SavedObjectsClientProvider);

return {
Expand Down Expand Up @@ -188,61 +203,92 @@ uiModules.get('apps/management')
}
);
})
.then((overwriteAll) => {
function importDocument(doc) {
const { service } = find($scope.services, { type: doc._type }) || {};
.then((overwriteAll) => {
const conflictedIndexPatterns = [];

if (!service) {
const msg = `Skipped import of "${doc._source.title}" (${doc._id})`;
const reason = `Invalid type: "${doc._type}"`;
function importDocument(doc) {
const { service } = find($scope.services, { type: doc._type }) || {};

notify.warning(`${msg}, ${reason}`, {
lifetime: 0,
});

return;
}
if (!service) {
const msg = `Skipped import of "${doc._source.title}" (${doc._id})`;
const reason = `Invalid type: "${doc._type}"`;

return service.get()
.then(function (obj) {
obj.id = doc._id;
return obj.applyESResp(doc)
.then(() => {
return obj.save({ confirmOverwrite : !overwriteAll });
})
.catch((err) => {
// swallow errors here so that the remaining promise chain executes
err.message = `Importing ${obj.title} (${obj.id}) failed: ${err.message}`;
notify.error(err);
});
});
}

function groupByType(docs) {
const defaultDocTypes = {
searches: [],
other: [],
};
notify.warning(`${msg}, ${reason}`, {
lifetime: 0,
});

return docs.reduce((types, doc) => {
switch (doc._type) {
case 'search':
types.searches.push(doc);
break;
default:
types.other.push(doc);
}
return types;
}, defaultDocTypes);
return;
}

const docTypes = groupByType(docs);
return service.get()
.then(function (obj) {
obj.id = doc._id;
return obj.applyESResp(doc)
.then(() => {
return obj.save({ confirmOverwrite : !overwriteAll });
})
.catch((err) => {
if (err instanceof SavedObjectNotFound && err.savedObjectType === 'index-pattern') {
conflictedIndexPatterns.push({ obj, doc });
return;
}

// swallow errors here so that the remaining promise chain executes
err.message = `Importing ${obj.title} (${obj.id}) failed: ${err.message}`;
notify.error(err);
});
});
}

function groupByType(docs) {
const defaultDocTypes = {
searches: [],
other: [],
};

return Promise.map(docTypes.searches, importDocument)
.then(() => Promise.map(docTypes.other, importDocument))
.then(refreshData)
.catch(notify.error);
});
return docs.reduce((types, doc) => {
switch (doc._type) {
case 'search':
types.searches.push(doc);
break;
default:
types.other.push(doc);
}
return types;
}, defaultDocTypes);
}

const docTypes = groupByType(docs);

return Promise.map(docTypes.searches, importDocument)
.then(() => Promise.map(docTypes.other, importDocument))
.then(() => {
if (conflictedIndexPatterns.length) {
showChangeIndexModal(
(objs) => {
return Promise.map(
conflictedIndexPatterns,
({ obj }) => {
const oldIndexId = obj.searchSource.getOwn('index');
const newIndexId = objs.find(({ oldId }) => oldId === oldIndexId).newId;
if (newIndexId === oldIndexId) {
// Skip
return;
}
return obj.hydrateIndexPattern(newIndexId)
.then(() => obj.save({ confirmOverwrite : !overwriteAll }));
}
).then(refreshData);
},
conflictedIndexPatterns,
$route.current.locals.indexPatterns,
);
} else {
return refreshData();
}
})
.catch(notify.error);
});
};

// TODO: Migrate all scope methods to the controller.
Expand Down
Loading

0 comments on commit 896cb26

Please sign in to comment.