LucyDetect helps developers and AI researchers analyze LLM response drift over time. It tracks and compares responses from GPT models, measuring consistency and detecting unexpected changes.
Right now, AI teams, researchers, and businesses have no standard way to:
- Detect when an LLM response changes over time.
- Measure how different two LLM-generated answers are.
- Track drift after fine-tuning, retraining, or updates.
- Compare two LLM models (e.g., GPT-4o vs. GPT-3.5) for reliability.
- Prevent AI hallucinations and ensure factual consistency.
LucyDetect quantifies response drift by:
✔ Logging all GPT responses in a database.
✔ Using FAISS for similarity search to compare past answers.
✔ Computing drift scores based on semantic similarity.
✔ Visualizing trends over time with an interactive UI.
✅ Detect Response Drift → Compare past and current LLM outputs.
✅ Log & Analyze Trends → Track drift scores over time.
✅ Store & Retrieve Data → Securely save responses for future comparison.
✅ Custom API Key Support → Users enter their own OpenAI API key.
✅ Web UI (Streamlit) → Easy-to-use interface for drift detection.
- Monitor model consistency across deployments.
- Detect drift when fine-tuning or retraining LLMs.
- Benchmark different OpenAI models before production use.
- Ensure AI-generated responses stay consistent for users.
- Prevent sudden answer changes in customer-facing applications.
- Study long-term model behavior under different conditions.
- Investigate semantic drift in language models over time.
Anyone can use LucyDetect online without setup!
🔗 **LucyDetect Web UI
Clone the repo and install dependencies:
git clone https://github.com/your-repo/lucydetect.git
cd lucydetect
pip install -r requirements.txt