Skip to content

Commit

Permalink
Nuclio as a plugin in CVAT, improved system to check installed plugins (
Browse files Browse the repository at this point in the history
#2192)

* allow to run cvat without nuclio

* fix new line

* fix comments

* Updated core version

* refactoring

* minor refactoring, fixed eslint issues, added documentation to cvat-core, updated ui version, updated changelog

* move plugins to serverViewSet

Co-authored-by: Boris Sekachev <[email protected]>
  • Loading branch information
dmitryagapov and bsekachev authored Sep 25, 2020
1 parent 6370f98 commit f2c84a2
Show file tree
Hide file tree
Showing 22 changed files with 225 additions and 161 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- UI models (like DEXTR) were redesigned to be more interactive (<https://github.com/opencv/cvat/pull/2054>)
- Used Ubuntu:20.04 as a base image for CVAT Dockerfile (<https://github.com/opencv/cvat/pull/2101>)
- Right colors of label tags in label mapping when a user runs automatic detection (<https://github.com/openvinotoolkit/cvat/pull/2162>)
- Nuclio became an optional component of CVAT (<https://github.com/openvinotoolkit/cvat/pull/2192>)
- A key to remove a point from a polyshape [Ctrl => Alt] (<https://github.com/openvinotoolkit/cvat/pull/2204>)

### Deprecated
Expand Down
1 change: 1 addition & 0 deletions components/analytics/docker-compose.analytics.yml
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ services:
DJANGO_LOG_SERVER_PORT: 5000
DJANGO_LOG_VIEWER_HOST: kibana
DJANGO_LOG_VIEWER_PORT: 5601
CVAT_ANALYTICS: 1
no_proxy: kibana,logstash,nuclio,${no_proxy}

volumes:
Expand Down
7 changes: 7 additions & 0 deletions components/serverless/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
## Serverless for Computer Vision Annotation Tool (CVAT)

### Run docker container
```bash
# From project root directory
docker-compose -f docker-compose.yml -f components/serverless/docker-compose.serverless.yml up -d
```
28 changes: 28 additions & 0 deletions components/serverless/docker-compose.serverless.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
version: '2.3'
services:
serverless:
container_name: nuclio
image: quay.io/nuclio/dashboard:1.4.8-amd64
restart: always
networks:
default:
aliases:
- nuclio
volumes:
- /tmp:/tmp
- /var/run/docker.sock:/var/run/docker.sock
environment:
http_proxy:
https_proxy:
no_proxy: 172.28.0.1,${no_proxy}
NUCLIO_CHECK_FUNCTION_CONTAINERS_HEALTHINESS: "true"
ports:
- "8070:8070"

cvat:
environment:
CVAT_SERVERLESS: 1
no_proxy: kibana,logstash,nuclio,${no_proxy}

volumes:
cvat_events:
2 changes: 1 addition & 1 deletion cvat-core/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion cvat-core/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "cvat-core",
"version": "3.7.1",
"version": "3.8.0",
"description": "Part of Computer Vision Tool which presents an interface for client-side integration",
"main": "babel.config.js",
"scripts": {
Expand Down
5 changes: 5 additions & 0 deletions cvat-core/src/api-implementation.js
Original file line number Diff line number Diff line change
Expand Up @@ -223,6 +223,11 @@
return tasks;
};

cvat.server.installedApps.implementation = async () => {
const result = await serverProxy.server.installedApps();
return result;
};

return cvat;
}

Expand Down
54 changes: 34 additions & 20 deletions cvat-core/src/api.js
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,6 @@ function build() {
return result;
},
/**
* Method allows to register on a server
* @method register
* @async
Expand Down Expand Up @@ -272,6 +271,20 @@ function build() {
.apiWrapper(cvat.server.request, url, data);
return result;
},

/**
* Method returns apps that are installed on the server
* @method installedApps
* @async
* @memberof module:API.cvat.server
* @returns {Object} map {installedApp: boolean}
* @throws {module:API.cvat.exceptions.PluginError}
* @throws {module:API.cvat.exceptions.ServerError}
*/
async installedApps() {
const result = await PluginRegistry.apiWrapper(cvat.server.installedApps);
return result;
},
},
/**
* Namespace is used for getting tasks
Expand Down Expand Up @@ -470,34 +483,35 @@ function build() {
return result;
},
/**
* Install plugin to CVAT
* @method register
* @async
* @memberof module:API.cvat.plugins
* @param {Plugin} [plugin] plugin for registration
* @throws {module:API.cvat.exceptions.PluginError}
*/
* Install plugin to CVAT
* @method register
* @async
* @memberof module:API.cvat.plugins
* @param {Plugin} [plugin] plugin for registration
* @throws {module:API.cvat.exceptions.PluginError}
*/
async register(plugin) {
const result = await PluginRegistry
.apiWrapper(cvat.plugins.register, plugin);
return result;
},
},

/**
* Namespace is used for serverless functions management (mainly related with DL models)
* @namespace lambda
* @memberof module:API.cvat
*/
* Namespace is used for serverless functions management (mainly related with DL models)
* @namespace lambda
* @memberof module:API.cvat
*/
lambda: {
/**
* Method returns list of available serverless models
* @method list
* @async
* @memberof module:API.cvat.lambda
* @returns {module:API.cvat.classes.MLModel[]}
* @throws {module:API.cvat.exceptions.ServerError}
* @throws {module:API.cvat.exceptions.PluginError}
*/
* Method returns list of available serverless models
* @method list
* @async
* @memberof module:API.cvat.lambda
* @returns {module:API.cvat.classes.MLModel[]}
* @throws {module:API.cvat.exceptions.ServerError}
* @throws {module:API.cvat.exceptions.PluginError}
*/
async list() {
const result = await PluginRegistry
.apiWrapper(cvat.lambda.list);
Expand Down
13 changes: 13 additions & 0 deletions cvat-core/src/server-proxy.js
Original file line number Diff line number Diff line change
Expand Up @@ -812,6 +812,18 @@
}
}

async function installedApps() {
const { backendAPI } = config;
try {
const response = await Axios.get(`${backendAPI}/server/plugins`, {
proxy: config.proxy,
});
return response.data;
} catch (errorData) {
throw generateError(errorData);
}
}

Object.defineProperties(this, Object.freeze({
server: {
value: Object.freeze({
Expand All @@ -828,6 +840,7 @@
register,
request: serverRequest,
userAgreements,
installedApps,
}),
writable: false,
},
Expand Down
52 changes: 22 additions & 30 deletions cvat-ui/src/actions/plugins-actions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,43 +3,35 @@
// SPDX-License-Identifier: MIT

import { ActionUnion, createAction, ThunkAction } from 'utils/redux';
import { SupportedPlugins } from 'reducers/interfaces';
import PluginChecker from 'utils/plugin-checker';
import { PluginsList } from 'reducers/interfaces';
import getCore from '../cvat-core-wrapper';

const core = getCore();

export enum PluginsActionTypes {
CHECK_PLUGINS = 'CHECK_PLUGINS',
CHECKED_ALL_PLUGINS = 'CHECKED_ALL_PLUGINS',
GET_PLUGINS = 'GET_PLUGINS',
GET_PLUGINS_SUCCESS = 'GET_PLUGINS_SUCCESS',
GET_PLUGINS_FAILED = 'GET_PLUGINS_FAILED',
}

type PluginObjects = Record<SupportedPlugins, boolean>;

const pluginActions = {
checkPlugins: () => createAction(PluginsActionTypes.CHECK_PLUGINS),
checkedAllPlugins: (list: PluginObjects) => (
createAction(PluginsActionTypes.CHECKED_ALL_PLUGINS, {
list,
})
checkPlugins: () => createAction(PluginsActionTypes.GET_PLUGINS),
checkPluginsSuccess: (list: PluginsList) => createAction(
PluginsActionTypes.GET_PLUGINS_SUCCESS, { list },
),
checkPluginsFailed: (error: any) => createAction(
PluginsActionTypes.GET_PLUGINS_FAILED, { error },
),
};

export type PluginActions = ActionUnion<typeof pluginActions>;

export function checkPluginsAsync(): ThunkAction {
return async (dispatch): Promise<void> => {
dispatch(pluginActions.checkPlugins());
const plugins: PluginObjects = {
ANALYTICS: false,
GIT_INTEGRATION: false,
};

const promises: Promise<boolean>[] = [
// check must return true/false with no exceptions
PluginChecker.check(SupportedPlugins.ANALYTICS),
PluginChecker.check(SupportedPlugins.GIT_INTEGRATION),
];

const values = await Promise.all(promises);
[plugins.ANALYTICS, plugins.GIT_INTEGRATION] = values;
dispatch(pluginActions.checkedAllPlugins(plugins));
};
}
export const getPluginsAsync = (): ThunkAction => async (dispatch): Promise<void> => {
dispatch(pluginActions.checkPlugins());
try {
const list: PluginsList = await core.server.installedApps();
dispatch(pluginActions.checkPluginsSuccess(list));
} catch (error) {
dispatch(pluginActions.checkPluginsFailed(error));
}
};
12 changes: 8 additions & 4 deletions cvat-ui/src/components/cvat-app.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ interface CVATAppProps {
authActionsInitialized: boolean;
notifications: NotificationsState;
user: any;
isModelPluginActive: boolean;
}

class CVATApplication extends React.PureComponent<CVATAppProps & RouteComponentProps> {
Expand Down Expand Up @@ -115,6 +116,7 @@ class CVATApplication extends React.PureComponent<CVATAppProps & RouteComponentP
userAgreementsInitialized,
authActionsFetching,
authActionsInitialized,
isModelPluginActive,
} = this.props;

this.showErrors();
Expand Down Expand Up @@ -150,7 +152,7 @@ class CVATApplication extends React.PureComponent<CVATAppProps & RouteComponentP
loadAbout();
}

if (!modelsInitialized && !modelsFetching) {
if (isModelPluginActive && !modelsInitialized && !modelsFetching) {
initModels();
}

Expand Down Expand Up @@ -248,11 +250,12 @@ class CVATApplication extends React.PureComponent<CVATAppProps & RouteComponentP
switchSettingsDialog,
user,
keyMap,
isModelPluginActive,
} = this.props;

const readyForRender = (userInitialized && (user == null || !user.isVerified))
|| (userInitialized && formatsInitialized
&& pluginsInitialized && usersInitialized && aboutInitialized);
|| (userInitialized && formatsInitialized && pluginsInitialized
&& usersInitialized && aboutInitialized);

const subKeyMap = {
SWITCH_SHORTCUTS: keyMap.SWITCH_SHORTCUTS,
Expand Down Expand Up @@ -316,7 +319,8 @@ class CVATApplication extends React.PureComponent<CVATAppProps & RouteComponentP
<Route exact path='/tasks/create' component={CreateTaskPageContainer} />
<Route exact path='/tasks/:id' component={TaskPageContainer} />
<Route exact path='/tasks/:tid/jobs/:jid' component={AnnotationPageContainer} />
<Route exact path='/models' component={ModelsPageContainer} />
{ isModelPluginActive
&& <Route exact path='/models' component={ModelsPageContainer} /> }
<Redirect push to='/tasks' />
</Switch>
</GlobalHotKeys>
Expand Down
Loading

0 comments on commit f2c84a2

Please sign in to comment.