Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

doc: update RAEDME of dspot-web mojo #941

Merged
merged 106 commits into from
Feb 18, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
106 commits
Select commit Hold shift + click to select a range
afd13ca
Refactor: move dspot-web to add mojo.
borisbaldassari Sep 21, 2019
0b0bef1
Feat: Add mojo dspot-web.
borisbaldassari Sep 21, 2019
da5d831
Feat [dspot-mojo] update readme.
borisbaldassari Sep 29, 2019
5a0863e
Feat [dspot-mojo] rename application - conf.
borisbaldassari Sep 29, 2019
74e0897
Feat [dspot-mojo] rename application - lib.
borisbaldassari Sep 29, 2019
32d20a9
Feat [dspot-mojo] rename application - controller.
borisbaldassari Sep 29, 2019
87f0d04
Feat [dspot-mojo] Add log dir.
borisbaldassari Sep 29, 2019
acc1cce
Feat [dspot-mojo] rename application - script.
borisbaldassari Sep 29, 2019
6d7a6bd
Feat [dspot-mojo] rename application - tests.
borisbaldassari Sep 29, 2019
53b0052
Feat [dspot-mojo] Add pages for repos + single repo.
borisbaldassari Sep 29, 2019
bda4c2c
Feat [dspot-mojo] Update templates for all new pages.
borisbaldassari Sep 29, 2019
26a6c25
Feat [dspot-mojo] Add template for single repo page.
borisbaldassari Sep 29, 2019
fe59167
Feat [dspot-mojo] Add readme-developers.
borisbaldassari Sep 29, 2019
d294370
Update text in pages.
borisbaldassari Sep 29, 2019
8e9b7ac
Feat [dspot-mojo] Update templates.
borisbaldassari Sep 30, 2019
85f59e0
Feat [dspot-mojo] Comment conf file.
borisbaldassari Sep 30, 2019
a700fc1
Feat [dspot-mojo] add about + create.
borisbaldassari Sep 30, 2019
bd4dd82
Feat [dspot-mojo] Update layout.
borisbaldassari Sep 30, 2019
56a0838
Feat [dspot-mojo] Add forms and create/new endpoit.
borisbaldassari Sep 30, 2019
e29d22c
Feat [dspot-mojo] Add font awesome, update routers and tasks.
borisbaldassari Oct 8, 2019
c76b698
Feat [dspot-mojo] Update Controller for jobs.
borisbaldassari Oct 8, 2019
844db84
Feat [dspot-mojo] Update html layout to include font awesome.
borisbaldassari Oct 8, 2019
23512eb
Feat [dspot-mojo] Add page to list jobs.
borisbaldassari Oct 8, 2019
9e9d6e5
Feat [dspot-mojo] Fix jobs execution.
borisbaldassari Oct 8, 2019
90de4cd
Feat [dspot-mojo] Update jobs execution: git, mvn, dspot.
borisbaldassari Oct 8, 2019
416b306
Feat [dspot-mojo] Update jobs, conf file, dirs.
borisbaldassari Oct 9, 2019
9030613
Feat [dspot-mojo] Improve UIs.
borisbaldassari Oct 9, 2019
c89ccb9
Feat [dspot-mojo] Add zip create + download.
borisbaldassari Oct 10, 2019
557adf1
Feat [dspot-mojo] Fix install doc in readme.
borisbaldassari Oct 16, 2019
d868368
Feat [dspot-mojo] Fix maven path issue.
borisbaldassari Oct 16, 2019
a0f7b1f
Feat [dspot-mojo] Fix maven path issue.
borisbaldassari Oct 16, 2019
ac5e6b8
Feat [dspot-mojo] Fix missing projects.conf issue.
borisbaldassari Oct 16, 2019
6f1afc2
Feat [dspot-mojo] Fix install doc in readme.
borisbaldassari Oct 16, 2019
839f190
Feat [dspot-mojo] Fix zip management and repo page.
borisbaldassari Oct 16, 2019
572b434
Feat [dspot-mojo] Fix mvn call issue.
borisbaldassari Oct 17, 2019
a590c61
Feat [dspot-mojo] Fix mvn call issue.
borisbaldassari Oct 17, 2019
bae6270
Feat [dspot-mojo] D2 improve/fix ui.
borisbaldassari Oct 19, 2019
b7bc1f5
Feat [dspot-mojo] D2 improve/fix jobs + add info to result.
borisbaldassari Oct 19, 2019
7238cd9
Feat [dspot-mojo] D2 improve/fix ui + controller.
borisbaldassari Oct 19, 2019
ed2b20d
Feat [dspot-mojo] D2 fix link in home page.
borisbaldassari Oct 19, 2019
ec643f9
Feat [dspot-mojo] D2 update jobs, repo, home page.
borisbaldassari Oct 19, 2019
6c3865b
Feat [dspot-mojo] D2 update text in new repo page.
borisbaldassari Oct 19, 2019
6d7a3ad
Feat [dspot-mojo] Add extended exec command for dspot.
borisbaldassari Oct 19, 2019
c31dc4f
Feat [dspot-mojo] D2 Improve new repo page.
borisbaldassari Oct 19, 2019
3065147
Feat [dspot-mojo] Fix undef parameters in ui.
borisbaldassari Oct 19, 2019
a12a877
Feat [dspot-mojo] D2 Remove controller sub for repos page.
borisbaldassari Oct 20, 2019
9dd8fac
Feat [dspot-mojo] D2 Improve jobs + repo pages.
borisbaldassari Oct 20, 2019
cb0a7fa
Feat [dspot-mojo] D2 Improve tasks + add info in projects file.
borisbaldassari Oct 20, 2019
707f418
Feat [dspot-mojo] D2 Improve jobs + repo pages.
borisbaldassari Oct 20, 2019
8c677ff
Feat [dspot-mojo] D2 Add email management.
borisbaldassari Oct 21, 2019
5e2d46b
Feat [dspot-mojo] D2 Improve email management.
borisbaldassari Oct 21, 2019
3d6204d
Feat [dspot-mojo] D2 Rename dspot-web dirs.
borisbaldassari Oct 21, 2019
717c1bc
Feat [dspot-mojo] D2 Update 'from' address.
borisbaldassari Oct 21, 2019
ccbc49f
Feat [dspot-mojo] D2 Fix minor UI items.
borisbaldassari Oct 21, 2019
412017b
Merge branch 'master' into dspot-mojo
borisbaldassari Oct 21, 2019
44847cb
Feat [dspot-mojo] D2 Various fixes to jobs management.
borisbaldassari Oct 21, 2019
dc4f270
Feat [dspot-mojo] D2 Various fixes to repo page.
borisbaldassari Oct 21, 2019
7bd8178
Feat [dspot-mojo] WIP add jobs artefacts management.
borisbaldassari Oct 21, 2019
b6f2b97
Feat [dspot-mojo] Add single job information page.
borisbaldassari Oct 23, 2019
2299a10
Feat [dspot-mojo] Remove useless copyright.
borisbaldassari Oct 23, 2019
82599f0
Feat [dspot-mojo] Improve jobs page, add link to single job.
borisbaldassari Oct 23, 2019
d2b7d43
Feat [dspot-mojo] Fix minor warnings.
borisbaldassari Oct 25, 2019
30d18d1
Feat [dspot-mojo] Add info to new repo page.
borisbaldassari Oct 25, 2019
21483dd
Feat [dspot-mojo] Add amplifier selection, improve form validation.
borisbaldassari Oct 25, 2019
903e606
Feat [dspot-mojo] Finalise amplifier selection.
borisbaldassari Oct 25, 2019
bc3da29
Feat [dspot-mojo] Fix url in emails.
borisbaldassari Oct 25, 2019
d614062
Merge branch 'master' into dspot-mojo
borisbaldassari Oct 25, 2019
c7fc965
Feat [dspot-mojo] Fix wrong tests in job page.
borisbaldassari Oct 26, 2019
693a320
Feat [dspot-mojo] Fix remove old files before execution.
borisbaldassari Oct 26, 2019
8a0d422
Feat [dspot-mojo] Fix default amplification method in create.
borisbaldassari Oct 26, 2019
6b3790c
Feat [dspot-mojo] Prevent double exec of jobs on same repo.
borisbaldassari Oct 26, 2019
91783a7
Feat [dspot-mojo] Fix various minor issues in UI.
borisbaldassari Oct 26, 2019
9baeefa
Feat [dspot-mojo] Set HEAD as default for git hash.
borisbaldassari Nov 1, 2019
fcad372
Feat [dspot-mojo] Minor improvement to logging.
borisbaldassari Nov 1, 2019
02b15ad
Feat [dspot-mojo] Implement link to test files from repo.
borisbaldassari Nov 1, 2019
bd0dcba
Feat [dspot-mojo] Add list of jobs for specific repo.
borisbaldassari Nov 1, 2019
43f436a
Feat [dspot-mojo] Add link to test files, improve ui.
borisbaldassari Nov 1, 2019
55ec5bf
Feat [dspot-mojo] Remove top text.
borisbaldassari Nov 1, 2019
1e07ff2
Feat [dspot-mojo] Add link to result in jobs.
borisbaldassari Nov 1, 2019
07b3d1a
Feat [dspot-mojo] Add syntax highlighting.
borisbaldassari Nov 1, 2019
5dc0139
Feat [dspot-mojo] Increase file display height.
borisbaldassari Nov 16, 2019
c1507d3
Feat [dspot-mojo] Send email when jobs are started.
borisbaldassari Nov 16, 2019
87c3398
Merge branch 'master' into dspot-mojo
borisbaldassari Nov 16, 2019
126820a
Feat [dspot-mojo] Fix uninitialised var in init.
borisbaldassari Nov 16, 2019
c5ab33d
Feat [dspot-mojo] Fix various minor warnings.
borisbaldassari Nov 16, 2019
0983f11
Merge branch 'master' into dspot-mojo
borisbaldassari Dec 1, 2019
e02f182
[touch:166] Amplifiers update + fixes for dspot 3.0.0.
borisbaldassari Dec 2, 2019
46bd861
Feat [dspot-mojo] Add descartes output download, improve consistency.
borisbaldassari Dec 3, 2019
94a45fc
Feat [dspot-mojo] Add descartes output zipping
borisbaldassari Dec 3, 2019
9690016
Feat [dspot-mojo] Improve consistency job/repo pages.
borisbaldassari Dec 3, 2019
d8cd7a3
Feat [dspot-mojo] Improve readme doc.
borisbaldassari Dec 13, 2019
6d33244
Feat [dspot-mojo] Fix uninit warning.
borisbaldassari Dec 13, 2019
de34c78
Feat [dspot-mojo] Sort repos table according to time created.
borisbaldassari Dec 13, 2019
a14db47
Feat [dspot-mojo] Remove useless repos page.
borisbaldassari Dec 18, 2019
382e4e5
Feat [dspot-mojo] Keep jobs for one year.
borisbaldassari Dec 18, 2019
958e1e5
Feat [dspot-mojo] Add list of 3 top jobs.
borisbaldassari Dec 23, 2019
746ae0c
Feat [dspot-mojo] Remove specifics on dspot-mvn version.
borisbaldassari Dec 23, 2019
0a000a0
Feat [dspot-mojo] Fix sort in welcome page.
borisbaldassari Dec 23, 2019
d2cf047
Feat [dspot-mojo] Fix wrong job number in welcome page.
borisbaldassari Dec 24, 2019
eec5cb7
Feat [dspot-mojo] Fix order in welcome page.
borisbaldassari Jan 27, 2020
075073c
Feat [dspot-mojo] Update readme.
borisbaldassari Feb 10, 2020
6aaffb3
Feat [dspot-mojo] Move dspot-web dev readme to main dev readme.
borisbaldassari Feb 10, 2020
351b431
Merge pull request #1 from STAMP-project/master
borisbaldassari Feb 15, 2020
2552b82
Feat [dspot-mojo] Re-fetch, re-merge, re-push.
borisbaldassari Feb 15, 2020
129852d
Update README.md
monperrus Feb 17, 2020
b8fbe74
Update README.md
monperrus Feb 18, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 22 additions & 1 deletion README-developers.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ We accept contribution in form of pull requests. Pull requests must include at l

For each pull request opened, travis is triggered. Our CI contains different jobs that must all pass.

There are jobs that execute the test for the different module of DSpot: `DSpot Core`, `DSpot Maven plugin`, `DSpot diff test selection`, and `DSpot prettifier`.
There are jobs that execute the test for the different module of DSpot: `DSpot Core`, `DSpot Maven plugin`, `DSpot diff test selection`, and `DSpot prettifier`.

There are also jobs for different kind of execution: from command line, using the maven plugin from command line and from a configuration in the pom, on large and complex code base.

Expand Down Expand Up @@ -103,6 +103,27 @@ dspot.amplifyTest(String fulQualifiedName, List<String> testCasesName); // will
dspot.amplifyAllTests(); // will amplify all test in the test suite.
```

## Using DSpot-web

## Directories

Projects being analysed by the DSpot Web UI are cloned and analysed in a dedicated directory, which can be configured in section `work_dir` of the main configuration file of the application: `d_spot_web.conf`.

In this directory each directory is a project, with all required files and information for the project stored inside. A typical architecture looks as follows:

* `work_dir`
- `project_a`
- `src` is the git extract (clone) of the repository.
- `output` is the hierarchy generated during the dspot execution.
- `results.zip` is the compressed file of the results.
- `logs` contains the logs of all major actions:
- `dspot.log` is the log of the dspot execution.
- `git_clone.log` is the log of the `git clone` command.
- `git_pull.log` is the log of the last `git pull` command, if relevant.

Please refer to [the dedicated README](dspot-web/v2/README.md) for information about how to install, configure, start and stop the application.


## Contributing

DSpot is licensed under LGPLv3. Contributors and pull requests are welcome.
92 changes: 89 additions & 3 deletions dspot-web/v2-mojo/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,9 @@ $ cpanm Mojo::Pg
$ cpanm Archive::Zip File::Copy::Recursive DateTime
```

### Initialising the application
### Initialising the database

Setup a PostgreSQL DB for Minion:
Setup a PostgreSQL DB for Minion, the job management framework:

```
postgres=# CREATE USER dspot_minion WITH PASSWORD 'pass4minion';
Expand All @@ -30,11 +30,97 @@ postgres=# CREATE DATABASE dspot_minion OWNER dspot_minion;
CREATE DATABASE
```

## Configuration

Configuration is done in file `d_spot_web.conf`:

```
{
'secrets' => ['abc7qzverq3rcqevq0d178qzabvftq96evfab563'],

# See README.md for Posgres configuration. Only used for job management.
'conf_pg_minion' => 'postgresql://dspot_pg_user:dspot_pg_password@/dspot_minion',

# URL of the host (used for emails).
'hostname' => 'https://dspot.myhost.com',

# MAVEN_HOME.
'mvn_home' => '~/apache-maven-3.6.0/',

# Command to run when invoking mvn.
'mvn_cmd' => 'mvn clean install -DskipTests',

# Command to run when invoking dspot-maven.
'dspot_cmd' => 'mvn eu.stamp-project:dspot-maven:3.1.0:amplify-unit-tests -Doutput-path=../output/dspot/',

# Places where all repos will be cloned (and analysed).
'workspace' => '/data/dspot_workspace/',
}
```

Parameters that can be adjusted are:

* **secrets** is a customised string used to create safe cookies.
* **conf_pg_minion** is the URL + credentials of the PostGreSQL cluster used by the job management component (Minion).
* **hostname** is used to build the instance's URL in the emails sent.
* **mvn_home** is the path to the maven installation that will be used for analysis.
* **mvn_cmd** is the pre-analysis command to use in order to build the dependency tree.
* **dspot_command** is the command to execute to run dspot (parameters for the mutation method are added dynamically).
* **workspace** is the directory where the projects will be cloned and analysed. Please remember that cloned projects can be substancially big and used disk space might increase substancially.


## Run the application

There are two processes to start: the UI and the job management framework.

### UI

Regarding the UI Mojolicious runs [hypnotoad](https://mojolicious.org/perldoc/Mojo/Server/Hypnotoad) to serve the application. It can be safely used for production-grade setups.

You can run the same command again for automatic hot deployment.

```
$ hypnotoad script/dspot_web
* Using workspace from conf [/data/dspot_workspace/].
* Using hostname from conf [http://ci4.castalia.camp:3000].
* Work dir [/data/dspot_workspace/projects] exists.
* Jobs dir [/data/dspot_workspace/jobs] exists.
* JSON projects file [/data/dspot_workspace/projects/projects.json] exists.
* Checking config..
- Using mvn home [~/Applis/apache-maven-3.6.3/].
- Using mvn command [MAVEN_HOME=~/Applis/apache-maven-3.6.3/ ~/Applis/apache-maven-3.6.3/bin/mvn clean install -DskipTests].
- Using dspot cmd [MAVEN_HOME=~/Applis/apache-maven-3.6.3/ ~/Applis/apache-maven-3.6.3/bin/mvn eu.stamp-project:dspot-maven:3.1.0:amplify-unit-tests -Doutput-path=../output/dspot].

Starting hot deployment for Hypnotoad server 16493.
```

The server can be stopped using the `-s` flag:

```
$ hypnotoad -s script/dspot_web
* Using workspace from conf [/data/dspot_workspace/].
* Using hostname from conf [http://ci4.castalia.camp:3000].
* Work dir [/data/dspot_workspace/projects] exists.
* Jobs dir [/data/dspot_workspace/jobs] exists.
* JSON projects file [/data/dspot_workspace/projects/projects.json] exists.
* Checking config..
- Using mvn home [~/Applis/apache-maven-3.6.3/].
- Using mvn command [MAVEN_HOME=~/Applis/apache-maven-3.6.3/ ~/Applis/apache-maven-3.6.3/bin/mvn clean install -DskipTests].
- Using dspot cmd [MAVEN_HOME=~/Applis/apache-maven-3.6.3/ ~/Applis/apache-maven-3.6.3/bin/mvn eu.stamp-project:dspot-maven:3.1.0:amplify-unit-tests -Doutput-path=../output/dspot].

Stopping Hypnotoad server 16571 gracefully.
```

The application is served on http://localhost:3000.
### Job management framework

```
$ cd ~/dspot/dspot-web/v2-mojo/
$ script/dspot-web minion worker
```

To check if the worker is running is to go to the admin UI <https://dspot-demo.stamp-project.eu/admin/minion>

The worker is running inside a screen on the dspot server. To see it just run screen -r to re-attach the screen (it's a foreground process so there should be lines printed on stdout).