Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GMM clustering and fixed compute pca #179

Closed
wants to merge 96 commits into from
Closed

GMM clustering and fixed compute pca #179

wants to merge 96 commits into from

Conversation

alishbaimran
Copy link
Collaborator

@alishbaimran alishbaimran commented Oct 8, 2024 •
edited
Loading

  • added GMMClustering class w/ various methods for GMM clustering using BIC and AIC
  • fixed compute_pca so values aren't hardcoded
  • example script to run gmm clustering w/ ground truth annotations
  • visualizing Feb + June datasets using UMAPs colored by GMM cluster weights and by predicted cluster labels

mattersoflight and others added 30 commits June 30, 2024 22:06
@mattersoflight
first draft of contrastive learning model
602e428
@mattersoflight
fixed stem and projection head, drafted lightning module
438895c
@mattersoflight
Contrastive_dataloader (#99)
94cac32 
* initial dataloader.py

* Update dataloader_test.py

* Update dataloader_test.py

* Update dataloader_test.py

* Update dataloader_test.py

* rename training script
@mattersoflight
move contrastive network to viscy.representation module
0e2fbd2
@alishbaimran
Update hcs.py
41ba83b
@mattersoflight
refactored class names
2ec3e67
@mattersoflight
correct imports
4da8572
@mattersoflight
cleaner names for model arch and module
45f73a7
new imports
a933bdd
@alishbaimran
Merge branch 'contrastive_phenotyping' of https://github.com/mehta-la…
ae6f7ce 
…b/VisCy into contrastive_phenotyping
@alishbaimran
Fixed epoch loss logging and WandB integration in ContrastiveModule
247cef3
@alishbaimran
updated training_script.py
e7b5121
@alishbaimran
Update hcs.py
de7188b
@alishbaimran
contrastive.py
94cd28d
@alishbaimran
engine.py
ebd4a78
@mattersoflight
script to test data i/o speed from different filesystems
8162f20
@mattersoflight
moved applications folder to viscy.applications so that pip install -…
2459d82 
…e . works.
@duopeng
add resnet50 to ContrastiveEncoder
72862e9
@duopeng
rename training_script.py to training_script_resnet.py
859de82
@mattersoflight
test dataloader on lustre and vast
23bd087
@mattersoflight
Merge remote-tracking branch 'origin/contrastive_resnet' into contras…
9751174 
…tive_phenotyping
@mattersoflight
move training_script_resnet to viscy.applications so that `pip instal…
955fea8 
…l -e .` works
@mattersoflight
refined the tests for contrastive dataloader
f521ceb
@mattersoflight
sbatch script for dataloader
3f6d3cd
@mattersoflight
delete redundant module
346e7c5
@mattersoflight
nits: updated the model construction of contrastive resnet encoder.
3a2a865
@alishbaimran
Merge branch 'contrastive_phenotyping' of https://github.com/mehta-la…
9c11b0f 
…b/VisCy into contrastive_phenotyping
@alishbaimran
Updated training script, HCS data handling, engine, and contrastive r…
675c87c 
…epresentation
@alishbaimran
Fix normalization, visualization issues, logging and multi-channel pr…
6422060 
…ediction
@alishbaimran
updated training and prediction
d968334
Soorya19Pradeep and others added 7 commits September 27, 2024 13:23
@Soorya19Pradeep
formatted with black
a58ab83
@Soorya19Pradeep
reduce to single arrow on plot
df9a5cd
@Soorya19Pradeep
remove reduntant script
17a2e48
@Soorya19Pradeep @ziw-liu
Fixes on correlation of PCA and UMAP components to computed_feature s…
ad74176 
…cript (#159)

* reduce initial patch size

* add radial profiling

* add function descriptions

* add umap correlation

* add def comments

* change umap for all data

* add script for 1 chan

* add p-value analysis

* add PCA analysis

* remove duplicate script

* Refactor and format code

* Format code

* Removed umap correlation

* note for future refactor

---------

Co-authored-by: Ziwen Liu <[email protected]>
@alishbaimran @mattersoflight
@Soorya19Pradeep @ziw-liu
updated eval module & cosine sim figures (#168)
582952d 
* updated files

* format fixed for tests

* updated scripts

* umap dist code

* bug fixes and linting

* logistic regression script

* add infection figure script

* Add script for generating infection figure and perform prediction on the June dataset

* Format code

* Black format evaluation module and fix import in figure_cell_infection script

* Refactor scatterplot colors and markers

* Calculate model accuracy

* Add script for appendix video

* formatted code

* updated displacement funcs for full embeddings

* script for displacement computation

* fix style

* fix docstring format

---------

Co-authored-by: Shalin Mehta <[email protected]>
Co-authored-by: Soorya Pradeep <[email protected]>
Co-authored-by: Ziwen Liu <[email protected]>
@alishbaimran
GMM clustering and fixed compute pca
0d6a473
@alishbaimran
format issues fixed
8e4a028
@mattersoflight mattersoflight requested review from Soorya19Pradeep and removed request for mattersoflight October 12, 2024 17:24
@mattersoflight
Copy link
Member

@alishbaimran @Soorya19Pradeep I am booked with multiple planning and writing projects through the end of the year. I think Soorya is situated to provide a solid review and testing of this code. She implemented GMM clustering over time for infection state discrimination for Fig. 5 of https://academic.oup.com/pnasnexus/article/3/9/pgae323/7731083

@ziw-liu ziw-liu added enhancement New feature or request representation Representation learning (SSL) labels Oct 16, 2024
@ziw-liu ziw-liu added this to the v0.4.0 milestone Oct 16, 2024
@Soorya19Pradeep
Copy link
Contributor

@alishbaimran , can you pull the changes from the representation branch into this branch? There seem to be some conflicts with the representation branch.

@alishbaimran
Copy link
Collaborator Author

@alishbaimran , can you pull the changes from the representation branch into this branch? There seem to be some conflicts with the representation branch.

Yes I can do this and then push changes again.

Base automatically changed from representation to main October 17, 2024 23:06
@mattersoflight
Copy link
Member

mattersoflight commented Oct 18, 2024 •
edited
Loading

@alishbaimran it looks like you started a branch or merged changes in a way that affected files that are not related to this feature.

You may need to do this:

  1. start a new branch from main after release 0.3.0.
  2. transfer the relevant files by hand from this branch.
  3. open a fresh PR and close this PR.

@Soorya19Pradeep
Copy link
Contributor

@alishbaimran , as you open the new branch to make the changes Shalin mentioned please keep in mind to clean up the scripts like this inside applications/contrastive_phenotyping/figures folder as these doesn't make sense anymore.

@alishbaimran
Copy link
Collaborator Author

@alishbaimran , as you open the new branch to make the changes Shalin mentioned please keep in mind to clean up the scripts like this inside applications/contrastive_phenotyping/figures folder as these doesn't make sense anymore.

@Soorya19Pradeep I'm not sure if these scripts are even suppose to be included in new branch. I pulled the files that were pushed to this branch and it seems some really old commits were moved here?
Screenshot 2024-10-21 at 11 37 39 AM

@mattersoflight could you clarify if/what to do with these olds commits?

@Soorya19Pradeep
Copy link
Contributor

@alishbaimran , I think it is best to close this and start a new branch. Let us do that and then you can add the new scripts for GMM clustering and remove those figure scripts there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Soorya19Pradeep Soorya19Pradeep Awaiting requested review from Soorya19Pradeep

Assignees
No one assigned
Labels
enhancement New feature or request representation Representation learning (SSL)
Projects
None yet
Milestone
v0.4.0
Development

Successfully merging this pull request may close these issues.
5 participants
@alishbaimran @mattersoflight @Soorya19Pradeep @ziw-liu @duopeng