[upd] README

keyu-tian · Mar 20, 2023 · 11d4f51 · 11d4f51
1 parent 4826bc8
commit 11d4f51
Show file tree

Hide file tree

Showing 7 changed files with 149 additions and 4 deletions.
diff --git a/README.md b/README.md
@@ -49,6 +49,8 @@ Check [pretrain/viz_reconstruction.ipynb](pretrain/viz_reconstruction.ipynb) whi
 <img src="https://user-images.githubusercontent.com/39692511/226376648-3f28a1a6-275d-4f88-8f3e-cd1219882488.png" width=60%>
 <p>
 
+We also provide [pretrain/viz_reconstruction.ipynb](pretrain/viz_spconv.ipynb) that shows the "mask pattern vanishing" issue of dense conv layers.
+
 
 ## What's new here?
 
@@ -85,14 +87,15 @@ Check [pretrain/viz_reconstruction.ipynb](pretrain/viz_reconstruction.ipynb) whi
 
 - [x] Pre-training code
 - [x] Fine-tuning code
-- [x] Colab visualization playground
+- [x] Colab visualization playground ([reconstruction](/pretrain/viz_reconstruction.ipynb), [sparse conv](pretrain/viz_spconv.ipynb))
 - [ ] Weights & visualization playground on `Huggingface`
 - [ ] Weights in `timm`
 
 
 ## Pre-trained weights
 
 **Note: for network definitions, we directly use `timm.models.ResNet` and [official ConvNeXt](https://github.com/facebookresearch/ConvNeXt/blob/048efcea897d999aed302f2639b6270aedf8d4c8/models/convnext.py).**
+
 `reso.`: the image resolution; `acc@1`: IN1k fine-tuned acc (top-1)
 
 

diff --git a/pretrain/viz1.png → pretrain/viz_imgs/recon.png b/pretrain/viz1.png → pretrain/viz_imgs/recon.png
diff --git a/pretrain/viz_imgs/spconv1.png b/pretrain/viz_imgs/spconv1.png
diff --git a/pretrain/viz_imgs/spconv2.png b/pretrain/viz_imgs/spconv2.png
diff --git a/pretrain/viz_imgs/spconv3.png b/pretrain/viz_imgs/spconv3.png
diff --git a/pretrain/viz_reconstruction.ipynb b/pretrain/viz_reconstruction.ipynb
@@ -138,7 +138,7 @@
    "outputs": [],
    "source": [
     "import matplotlib.pyplot as plt\n",
-    "def show(spark: SparK, img_file='viz1.png', active_b1ff: torch.BoolTensor = None):\n",
+    "def show(spark: SparK, img_file='viz_imgs/recon.png', active_b1ff: torch.BoolTensor = None):\n",
     "    inp_bchw = load_image(img_file)\n",
     "    spark.forward\n",
     "    inp_bchw, masked_bchw, rec_or_inp = spark(inp_bchw, active_b1ff=active_b1ff, vis=True)\n",
@@ -202,7 +202,7 @@
    ],
    "source": [
     "# specify the mask\n",
-    "show(spark, 'viz1.png', active_b1ff=torch.tensor([\n",
+    "show(spark, 'viz_imgs/recon.png', active_b1ff=torch.tensor([\n",
     "    [0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0],\n",
     "    [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n",
     "    [0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0],\n",
@@ -244,7 +244,7 @@
     }
    ],
    "source": [
-    "show(spark, 'viz1.png', active_b1ff=None)"
+    "show(spark, 'viz_imgs/recon.png', active_b1ff=None)"
    ]
   },
   {

diff --git a/pretrain/viz_spconv.ipynb b/pretrain/viz_spconv.ipynb