GitHub - Faizan711/multimodal-search · GitHub
/" data-turbo-transient="true" />
Skip to content
Search or jump to...
Search code, repositories, users, issues, pull requests...
-->
Search
Clear
Search syntax tips
Provide feedback
--><br>We read every piece of feedback, and take your input very seriously.
Include my email address so I can be contacted
Cancel
Submit feedback
Saved searches
Use saved searches to filter your results more quickly
-->
Name
Query
To see all available qualifiers, see our documentation.
Cancel
Create saved search
Sign in
/;ref_cta:Sign up;ref_loc:header logged out"}"<br>Sign up
Appearance settings
Resetting focus
You signed in with another tab or window. Reload to refresh your session.<br>You signed out in another tab or window. Reload to refresh your session.<br>You switched accounts on another tab or window. Reload to refresh your session.
Dismiss alert
{{ message }}
Faizan711
multimodal-search
Public
Notifications<br>You must be signed in to change notification settings
Fork
Star
main
BranchesTags
Go to file
CodeOpen more actions menu
Folders and files<br>NameNameLast commit message<br>Last commit date<br>Latest commit
History<br>16 Commits<br>16 Commits
app
app
data
data
scripts
scripts
tests
tests
ui
ui
.env.example
.env.example
.gitignore
.gitignore
Dockerfile
Dockerfile
README.md
README.md
gradio_app.py
gradio_app.py
pyproject.toml
pyproject.toml
render.yaml
render.yaml
requirements-ui.txt
requirements-ui.txt
requirements.txt
requirements.txt
start.sh
start.sh
View all files
Repository files navigation
Multimodal Search
Search images using natural language or image upload — powered by OpenAI CLIP and Qdrant vector search.
This project was built from scratch to explore how modern image search works under the hood. Instead of relying on manual tags or filenames, this app "understands" visual content by embedding both text and images into a shared 512-dimensional vector space.
Features
Text-to-Image Search: Describe what you are looking for (e.g., "sunset over mountains").
Image-to-Image Search: Upload an image to find visually similar ones.
Pipeline Visualization: A real-time UI animation that shows the AI processing your query step-by-step: Tokenization → CLIP Encoder → Vector Embedding → Qdrant Search → Ranked Results.
Tech Stack
Component<br>Technology
Embedding Model<br>OpenAI CLIP (ViT-B/32)
Vector Database<br>Qdrant (HNSW indexing)
Frontend/UI<br>HTML / CSS / Vanilla JS
Backend<br>FastAPI, Python (PyTorch, Transformers)
Local Setup Instructions
Follow these steps to run the project locally on your machine.
1. Prerequisites
Python 3.10+
Docker (to run the local Qdrant database)
2. Start Qdrant (Vector Database)
Open a terminal and start a local Qdrant instance using Docker:
docker run -p 6333:6333 -p 6334:6334 \<br>-v $(pwd)/qdrant_storage:/qdrant/storage:z \<br>qdrant/qdrant
3. Install Python Dependencies
Open a new terminal window in the project directory, create a virtual environment, and install the required packages:
# Create and activate virtual environment<br>python3 -m venv .venv<br>source .venv/bin/activate
# Install dependencies<br>pip install -r requirements.txt
4. Run the Application
The repository includes a pre-exported file of vectors (data/vectors.json) containing 541 sample images.
Start the FastAPI server:
uvicorn app.api:app --host 0.0.0.0 --port 8000
Note: On the very first run, the app will automatically read data/vectors.json and import the vectors into your local Qdrant database. It will also download the CLIP model weights.
5. Use the App
Open your browser and navigate to:<br>http://localhost:8000
Type a query or upload an image, and watch the pipeline animate as it searches!
Project Structure
app/api.py: FastAPI server routes and static file serving.
ui/app.html: Custom HTML/JS frontend with the animated pipeline.
app/embeddings.py: CLIP model initialization and inference code.
app/vector_store.py: Qdrant client connection and search logic.
app/config.py: Configuration settings and environment variables.
scripts/: Utilities for downloading datasets and importing/exporting vectors.
data/: Contains the image dataset and the exported vectors JSON.
About
No description, website, or topics provided.
Resources
Readme
Uh oh!
There was an error while loading. Please reload this page.
Activity
Stars
star
Watchers
watching
Forks
forks
Report repository
Releases
No releases published
Packages
Uh oh!
There was an error while loading. Please reload this page.
Contributors
Uh oh!
There was an error while loading. Please reload this page.
Languages
Python<br>73.3%
HTML<br>24.7%
Dockerfile<br>1.3%
Shell<br>0.7%
You can’t perform that action at this time.