Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate Dynamic Settings #15

Merged
merged 70 commits into from
May 1, 2023
Merged
Show file tree
Hide file tree
Changes from 62 commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
546a4ff
Standardize formatting of mappers
davidshq Apr 12, 2023
ba49754
Fixing `user_identified_artifacts` type def
davidshq Apr 12, 2023
f4dea93
Adding Setting model and adjusting code accordingly
davidshq Apr 14, 2023
8fc466b
Minor formatting changes to match linting rules
davidshq Apr 14, 2023
f02fa26
Standardizing naming of mutations
davidshq Apr 14, 2023
bc1f822
Updates primarily around standardized mutation names
davidshq Apr 14, 2023
f7b6448
Updating comments
davidshq Apr 14, 2023
2bacb08
Fixing setLock in ControlPanel, ...LoadingFrame in index
davidshq Apr 14, 2023
d224de4
Using isort on imports
davidshq Apr 14, 2023
5d4019b
Fix linting issues
davidshq Apr 14, 2023
92ffb72
Refactoring function names...
davidshq Apr 17, 2023
9926aea
Refactoring getImageData's id to frameId, update comments
davidshq Apr 17, 2023
8ee6feb
Refactoring getNextFrame's params:
davidshq Apr 17, 2023
664e3c4
Refactoring params of mutations...
davidshq Apr 17, 2023
a4c4acf
Updating mutation params...
davidshq Apr 17, 2023
5a57680
Refactoring mutation params, removing unused mutations
davidshq Apr 17, 2023
f98d927
Removing unused constants
davidshq Apr 17, 2023
10bf28b
Refactoring variable names...
davidshq Apr 17, 2023
1fa05d3
Refactoring k in loadProject to frameIndex
davidshq Apr 17, 2023
8ce2863
Refactoring action getScan to loadScan (for consistent naming)
davidshq Apr 17, 2023
623b10c
Refactoring fileP to cachedFile, making loadFileAndGetData async
davidshq Apr 17, 2023
bb52721
Fix decisionToRating sorting, refactoring state to scanTaskState in s…
davidshq Apr 17, 2023
09fe974
Refactoring naming in Projects.vue and add await in refreshAllTaskOve…
davidshq Apr 17, 2023
12a6a29
Refactoring reset to continueSession in TimeoutDialog, add await; rem…
davidshq Apr 17, 2023
7d5ff69
Adding await in restoreLogin and removing unused ColorMaps
davidshq Apr 17, 2023
e92d1c2
Fix VRangeSlider TS error, update playwright
davidshq Apr 17, 2023
5ecb4ca
Fix minor errors
davidshq Apr 17, 2023
d9a044d
Updating loadAllUsers form to use mapGetters, minor code formatting a…
davidshq Apr 18, 2023
51b1e8f
Extract editRights from ControlPanel into Vuex store, refactoring var…
davidshq Apr 18, 2023
95d2163
Adding AUTO_ADVANCE switch, updating Decision Buttons to support cust…
davidshq Apr 18, 2023
ef31448
Adding editRight and experimentIsEditable to Vuex store (instead of i…
davidshq Apr 18, 2023
e960fa0
Minor refactoring of Vuex store...
davidshq Apr 18, 2023
55fce72
Updating packages in package.json, package-lock.json was already usin…
davidshq Apr 18, 2023
364cb6d
Adding await to proceedToNext in Projects.vue
davidshq Apr 18, 2023
41a5f23
Reorging types to match older code for easier merging, can reorder th…
davidshq Apr 18, 2023
3f148d1
Refactoring Scan...
davidshq Apr 18, 2023
044cbc4
Refactoring value to newSlice
davidshq Apr 19, 2023
a5ba832
Fixing show and store crosshairs switches, making snackbar injectable…
davidshq Apr 20, 2023
156eeff
Move constants for timeout into constants
davidshq Apr 25, 2023
7515409
Updating CI config
davidshq Apr 25, 2023
6c44b91
Pull in comment changes
davidshq Apr 25, 2023
84a767d
Remove pyppeteer from tox
davidshq Apr 26, 2023
ebc9eff
Removing unneeded files, updating dependencies
davidshq Apr 27, 2023
2b4f334
Bringing in changes from minor refactorings
davidshq Apr 28, 2023
815ff4e
Updating GitHub Actions yaml files
davidshq Apr 28, 2023
79d70e0
Explicitly set nii.gz as binary
davidshq Apr 28, 2023
f4593f2
Fixing gitattributes
davidshq Apr 28, 2023
b96536b
Fixing gitattributes
davidshq Apr 28, 2023
675fedd
Merge in dynamic settings changes
davidshq Apr 28, 2023
f3fee67
Minor bump to playwright dependency
davidshq Apr 28, 2023
453ba89
Adding settings for eslint rule to suppress unnecessary errors relate…
davidshq Apr 28, 2023
53bb6be
Merge remote-tracking branch 'knowledgevis/master' into dynamic-setti…
davidshq Apr 28, 2023
bfcc7fa
Fixing new issues with ESLint caused by fixing issues with ESLint. :-D
davidshq Apr 28, 2023
f50f917
Updating test to expect artifacts to be returned when querying projec…
davidshq Apr 28, 2023
6c1cb89
CI failing, rollback actions/checkout to v2 for pytests, tox
davidshq Apr 28, 2023
2a22a36
CI failing, rollback action versions across board
davidshq Apr 28, 2023
c7f0f73
Remove ci.yml.old
annehaley May 1, 2023
5c5ebcf
Remove commented questions from types.ts
annehaley May 1, 2023
006d0ac
Remove commented code in playwright.config.js
annehaley May 1, 2023
1d59fc6
Remove comment in .eslintrc.js
annehaley May 1, 2023
cbb81ba
Undo actions version regressions
annehaley May 1, 2023
0402bd2
Remove comments and debug logs from models/project.py
annehaley May 1, 2023
f01ab8f
Remove empty list `artifact_options` from Miqa Config object
annehaley May 1, 2023
3d5c023
Fix incorrect ID refs in setting.json
annehaley May 1, 2023
5a00963
Call loaddata command as part of migration
annehaley May 1, 2023
2a2ecab
Use default groups when project does not specify
annehaley May 1, 2023
4d7a9a3
Lint fix
annehaley May 1, 2023
422834b
Fix itk dependency for check-migrations test
annehaley May 1, 2023
7e7ebfd
Remove debug logging in tasks.py
annehaley May 1, 2023
0dfa4c5
Fix failing pytest
annehaley May 1, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .gitattributes
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
*.png -text
* text eol=lf
*.png binary
*.pth* binary
*.ico binary
*.nii.gz binary

9 changes: 8 additions & 1 deletion miqa/core/admin.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
from django.contrib import admin
from guardian.admin import GuardedModelAdmin

from .models import Evaluation, Experiment, Frame, Project, Scan, ScanDecision
from .models import Evaluation, Experiment, Frame, Project, Scan, ScanDecision, Setting


@admin.register(Experiment)
Expand Down Expand Up @@ -52,3 +52,10 @@ class ProjectAdmin(GuardedModelAdmin):
)
list_filter = ('created', 'modified', 'creator')
search_fields = ('name',)


@admin.register(Setting)
class SettingAdmin(admin.ModelAdmin):
list_display = ('key', 'value', 'type', 'group', 'is_type')
list_filter = ('type', 'group', 'is_type')
list_editable = ('type', 'group', 'is_type')
6 changes: 1 addition & 5 deletions miqa/core/migrations/0023_user_identified_artifacts.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,6 @@

from django.db import migrations, models

import miqa.core.models.scan_decision


class Migration(migrations.Migration):
dependencies = [
Expand All @@ -14,8 +12,6 @@ class Migration(migrations.Migration):
migrations.AddField(
model_name='scandecision',
name='user_identified_artifacts',
field=models.JSONField(
default=miqa.core.models.scan_decision.default_identified_artifacts
),
field=models.JSONField(default=dict),
),
]
141 changes: 141 additions & 0 deletions miqa/core/migrations/0036_add_setting_alter_project.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
# Generated by Django 3.2.16 on 2023-01-19 19:41

import uuid

from django.db import migrations, models
import django.db.models.deletion


class Migration(migrations.Migration):
replaces = [
('core', '0036_setting_settingsgroup'),
('core', '0037_project_artifact_group'),
('core', '0038_project_model_mapping_group'),
('core', '0039_auto_20221207_2028'),
('core', '0040_auto_20221207_2037'),
('core', '0041_auto_20221208_1429'),
('core', '0042_auto_20230110_1852'),
('core', '0043_alter_setting_group'),
('core', '0044_alter_project_artifact_group'),
('core', '0045_auto_20230110_2004'),
('core', '0046_alter_setting_group'),
('core', '0047_alter_setting_type'),
('core', '0048_auto_20230113_1751'),
('core', '0049_alter_project_models_group'),
('core', '0050_delete_settingsgroup'),
('core', '0051_alter_setting_type'),
('core', '0052_alter_setting_type'),
('core', '0053_auto_20230119_1815'),
('core', '0054_auto_20230119_1938'),
]

dependencies = [
('core', '0035_allow_null_decision_creation_times'),
]

operations = [
migrations.CreateModel(
name='Setting',
fields=[
(
'id',
models.UUIDField(
default=uuid.uuid4, editable=False, primary_key=True, serialize=False
),
),
('key', models.CharField(max_length=255)),
('value', models.TextField(blank=True)),
('is_type', models.BooleanField(blank=True, default=False)),
(
'group',
models.ForeignKey(
blank=True,
limit_choices_to=models.Q(
('type__in', ['GST', 'GAOT', 'GAT', 'GDCT', 'GEFMMT', 'GEMPT', 'GEMT'])
),
null=True,
on_delete=django.db.models.deletion.SET_NULL,
related_name='setting_group',
to='core.setting',
),
),
(
'type',
models.CharField(
choices=[
('AOT', 'Anatomy Orientation'),
('AT', 'Artifact'),
('DCT', 'Decision Choice'),
('EFMMT', 'Evaluation File to Model Mapping'),
('EMPT', 'Evaluation Model Prediction'),
('EMT', 'Evaluation Model'),
('ST', 'Scan'),
('GIP', 'Global Import Path'),
('GEP', 'Global Export Path'),
('NS', 'Not Set'),
('GAOT', 'Group of Anatomy Orientations'),
('GAT', 'Group of Artifacts'),
('GDCT', 'Group of Decision Choices'),
('GEFMMT', 'Group of Evaluation File to Model Mappings'),
('GEMPT', 'Group of Evaluation Model Predictions'),
('GEMT', 'Group of Evaluation Models'),
('GST', 'Group of Scans'),
],
default='NS',
max_length=20,
),
),
],
options={
'ordering': ('key',),
},
),
migrations.AddField(
model_name='project',
name='artifacts_group',
field=models.ForeignKey(
blank=True,
limit_choices_to={'type': 'GAT'},
null=True,
on_delete=django.db.models.deletion.SET_NULL,
related_name='artifacts_group',
to='core.setting',
),
),
migrations.AddField(
model_name='project',
name='files_to_models_group',
field=models.ForeignKey(
blank=True,
limit_choices_to={'type': 'GEFMMT'},
null=True,
on_delete=django.db.models.deletion.SET_NULL,
related_name='files_to_models_group',
to='core.setting',
),
),
migrations.AddField(
model_name='project',
name='models_group',
field=models.ForeignKey(
blank=True,
limit_choices_to={'type': 'GEMT'},
null=True,
on_delete=django.db.models.deletion.SET_NULL,
related_name='models_group',
to='core.setting',
),
),
migrations.AddField(
model_name='project',
name='predictions_group',
field=models.ForeignKey(
blank=True,
limit_choices_to={'type': 'GEMPT'},
null=True,
on_delete=django.db.models.deletion.SET_NULL,
related_name='predictions_group',
to='core.setting',
),
),
]
2 changes: 2 additions & 0 deletions miqa/core/models/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
from .project import Project
from .scan import Scan
from .scan_decision import ScanDecision
from .setting import Setting

__all__ = [
'Evaluation',
Expand All @@ -14,4 +15,5 @@
'Project',
'Scan',
'ScanDecision',
'Setting',
]
84 changes: 83 additions & 1 deletion miqa/core/models/project.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@
from guardian.shortcuts import assign_perm, get_perms, get_users_with_perms, remove_perm

from miqa.core.models.scan import SCAN_TYPES, Scan
from miqa.core.models.scan_decision import ScanDecision
from miqa.core.models.scan_decision import ArtifactState, ScanDecision
from miqa.core.models.setting import Setting


def default_evaluation_model_mapping():
Expand Down Expand Up @@ -54,6 +55,87 @@ class Project(TimeStampedModel, models.Model):
)
evaluation_models = models.JSONField(default=default_evaluation_model_mapping)
default_email_recipients = models.TextField(blank=True)
artifacts_group = models.ForeignKey(
'Setting',
null=True,
blank=True,
on_delete=models.SET_NULL,
related_name='artifacts_group',
limit_choices_to={'type': 'GAT'},
)
files_to_models_group = models.ForeignKey(
'Setting',
null=True,
blank=True,
on_delete=models.SET_NULL,
related_name='files_to_models_group',
limit_choices_to={'type': 'GEFMMT'},
)
models_group = models.ForeignKey(
'Setting',
null=True,
blank=True,
on_delete=models.SET_NULL,
related_name='models_group',
limit_choices_to={'type': 'GEMT'},
)
predictions_group = models.ForeignKey(
'Setting',
null=True,
blank=True,
on_delete=models.SET_NULL,
related_name='predictions_group',
limit_choices_to={'type': 'GEMPT'},
)

@property
def artifacts(self) -> dict:
"""Gets the list of artifacts associated with the project."""
if self.artifacts_group:
artifacts = Setting.objects.filter(group__id=self.artifacts_group_id)

return {artifact_name.key: ArtifactState.UNDEFINED.value for artifact_name in artifacts}
else:
return {}

@property
def model_source_type_mappings(self) -> dict:
"""Gets the list of file type to model mappings associated with the project."""
if self.files_to_models_group:
model_source_type_mappings = Setting.objects.filter(
group__id=self.files_to_models_group_id
)
this_model_source_type_mapping = {
model_source_type_mapping.key: model_source_type_mapping.value
for model_source_type_mapping in model_source_type_mappings
}
return this_model_source_type_mapping
else:
return {}

@property
def model_mappings(self) -> dict:
"""Gets the list of models associated with the project."""
if self.models_group:
model_mappings = Setting.objects.filter(group__id=self.models_group_id)
this_model_mapping = {
model_mapping.key: model_mapping.value for model_mapping in model_mappings
}
return this_model_mapping
else:
return {}

@property
def model_predictions(self) -> dict:
"""Gets the list of predictions associated with the project."""
if self.predictions_group:
prediction_mappings = Setting.objects.filter(group__id=self.predictions_group_id)
this_prediction_mapping = {}
for prediction in prediction_mappings:
this_prediction_mapping.setdefault(prediction.key, []).append(prediction.value)
return this_prediction_mapping
else:
return {}

def __str__(self):
return self.name
Expand Down
25 changes: 1 addition & 24 deletions miqa/core/models/scan_decision.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,19 +11,6 @@
if TYPE_CHECKING:
from miqa.core.models import Experiment

artifacts = [
'normal_variants',
'lesions',
'full_brain_coverage',
'misalignment',
'swap_wraparound',
'ghosting_motion',
'inhomogeneity',
'susceptibility_metal',
'flow_artifact',
'truncation_artifact',
]

DECISION_CHOICES = [
('U', 'Usable'),
('UE', 'Usable-Extra'),
Expand All @@ -38,16 +25,6 @@ class ArtifactState(Enum):
UNDEFINED = -1


def default_identified_artifacts():
return {
(
artifact_name if artifact_name != 'full_brain_coverage' else 'partial_brain_coverage'
): ArtifactState.UNDEFINED.value
for artifact_name in artifacts
if artifact_name != 'normal_variants'
}


class ScanDecision(models.Model):
class Meta:
ordering = ['-created']
Expand All @@ -61,7 +38,7 @@ class Meta:
creator = models.ForeignKey(User, on_delete=models.PROTECT, null=True, blank=True)
decision = models.CharField(max_length=2, choices=DECISION_CHOICES, blank=False)
note = models.TextField(max_length=3000, blank=True)
user_identified_artifacts = models.JSONField(default=default_identified_artifacts)
user_identified_artifacts = models.JSONField(default=dict)
location = models.JSONField(default=dict)

@property
Expand Down
Loading