-
-
Notifications
You must be signed in to change notification settings - Fork 42
Description
What feature do you want to see added?
I was setting up the project locally to start contributing, and I noticed that the default make api command triggers the full "Production-like" setup. This forces a download of PyTorch (800MB+) and the Mistral LLM (~4GB), and it requires NVIDIA CUDA libraries,even if I just want to work on API endpoints or data pipelines.
- Setup Time: Can take 30+ minutes depending on internet speed.
- Resources: Requires >5GB disk space and heavy RAM usage.
- Barrier: It might discourage new contributors who just want to fix a small logic bug or work on the frontend but don't have a GPU or fast internet.
While exploring docs/setup.md, I realized that setting PYTEST_VERSION=1 forces the backend into "API-only mode".
This skips the heavy LLM load, installs only ~300 packages, and starts the server instantly.
Currently, this is only documented under "Testing," but it’s actually a perfect environment for lightweight development.
Proposed Solution
We should formalize this as a first-class feature for contributors.
1. Add a dev-lite command to Makefile:
We can add a shortcut so contributors can spin up the backend immediately without the heavy ML deps.
# Proposed addition to Makefile
dev-lite: ## Run API in Lite Mode (No LLM loaded) for fast dev
@echo "Starting API in Lite Mode..."
PYTEST_VERSION=1 uvicorn api.main:app --reload --host 0.0.0.0 --port 8000**2. Update Documentation: **
I can update README.md and docs/setup.md to recommend make dev-lite as the default "Quick Start" for new contributors.
Why this helps :
Speed: Reduces setup time from ~30 mins to ~3 mins.
Access: Allows contributors on lower-end laptops (or no GPU) to participate easily.
Focus: Lets us iterate on backend logic/APIs without waiting for model inference.
Upstream changes
No response
Are you interested in contributing this feature?
I’ve verified this workflow locally and it works great. I’m happy to submit a PR for the Makefile and Docs update if the issue seems relevant...!