CLIP FROM SCRATCH

Integrating Vision and Language

Developed with :

---

Quick Links

• Repository Structure
• Modules
• Getting Started
  • Installation
• Contributing
• TODO

---

TODO

• Text Encoder
• Vision Encoder
• Projection Matrix
• Basic Training Script
• Make Model GPU Compatible (Just little changes in Text Encoder Part)
• Write Distributed training scripts
• Add options to switch to custom text and vision encoder

---

Repository Structure

└── CLIP_From_Scratch/
    ├── CLIP.py
    ├── encoder.py
    ├── projection_head.py
    └── train.py

---

Modules

File	Summary
encoder.py	The encoder.py provides text and image embeddings functionality.
train.py	The train.py script trains a CLIP model, managing data loading, training steps, updating metrics, and learning rate retrieval.
CLIP.py	CLIP.py defines a multimodal CLIP model, integrating image and text encoders with projection heads for joint embedding.
projection_head.py	The projection_head.py defines a neural network module for transforming embeddings

---

Getting Started

Installation

1. Clone the CLIP_From_Scratch repository:

git clone https://github.com/anonymous-atom/CLIP_From_Scratch

2. Change to the project directory:

cd CLIP_From_Scratch

3. Install the dependencies:

pip install -r requirements.txt

---

---

Contributing

Contributions are welcome! Here are several ways you can contribute:

• Submit Pull Requests: Review open PRs, and submit your own PRs.
• Join the Discussions: Share your insights, provide feedback, or ask questions.
• Report Issues: Submit bugs found or log feature requests for Clip_from_scratch.

Contributing Guidelines

1. Fork the Repository: Start by forking the project repository to your GitHub account.
2. Clone Locally: Clone the forked repository to your local machine using a Git client.

   git clone https://github.com/anonymous-atom/CLIP_From_Scratch

3. Create a New Branch: Always work on a new branch, giving it a descriptive name.

   git checkout -b new-feature-x

4. Make Your Changes: Develop and test your changes locally.
5. Commit Your Changes: Commit with a clear message describing your updates.

   git commit -m 'Implemented new feature x.'

6. Push to GitHub: Push the changes to your forked repository.

   git push origin new-feature-x

7. Submit a Pull Request: Create a PR against the original project repository. Clearly describe the changes and their motivations.

Once your PR is reviewed and approved, it will be merged into the main branch.

---

Return

---