Vectors, Python, AI, and Your Face
Building a Facial Recognition System with Cursor AI
At Don’t Panic Labs, I recently embarked on an prototype project to explore the intersection of computer vision, Python, and AI-assisted development. Our goal was to build a light-weight facial recognition system that could identify faces, create a searchable database, and analyze images. All while leveraging Cursor AI to write as much code as possible.
The Challenge
I set out with six objectives:
- Identify faces in an image – Detect and extract facial features with confidence scores
- Create a searchable database of faces – Store face embeddings for fast similarity search
- Search the database by providing an image – Find similar faces using vector similarity
- Display matching search results – Generate professional HTML reports with visual results
- Identify objects, brands, etc. in an image – Image analysis beyond just faces
- Create an API as well as a console app – Use FastAPI to expose functionality of the system
- Use Cursor AI to write as much of the code as possible – Test the limits of AI-assisted development
The Architecture
The development process was iterative without a predetermined architecture, with Cursor AI helping refactor and improve the code structure as I went (not a recommended process for production code). This led to a very functionally decomposed system. This was acceptable considering the project was only a prototype and would never be deployed to a production system. What started as a simple app.py file quickly evolved into a service-oriented architecture (SOA).
Service-Oriented Design
Cursor AI helped refactor the monolithic app.py into focused services, each with clear responsibilities:
The Cursor AI did well with the refactoring as long as the prompts were clear about what code should move to what service.
Cursor AI is good at writing its own rules. During development as part of refactoring to a SOA, when asked to create a rule to enforce SOA, the AI created a decent service-oriented architecture rules file (service-architecture.mdc) that defined strict patterns for the project. This included requirements like “all public service methods MUST use a single request object and return a single response object” and “services should be completely decoupled from each other.” Cursor doesn’t always follow these rules perfectly when adding new code, but more often than not it will adhere to the rules.
User Query: “/Generate Cursor Rules create a project level rule in the correct folder. The rule should state that a service oriented architecture should always be used”
Result:
The AI Assisted Coding Examples
The development process was highly dependent on Cursor AI writing the Python code, involving real problem-solving and iterative refinement. Here are some actual examples from our conversations:
1. Giving Cursor a List of Tasks to Work Through
At times, micromanaging Cursor’s work provided the best results. Cursor doesn’t seem to have a planning mode and I had to be explicit in telling it to pause and ask for feedback before implementing a change
One of better the experiences was working through this prompt with Cursor:
User Query: “Let’s start by breaking this down into steps to implement. As you complete each step pause to ask for my input before continuing. We may need to pivot and adujst the steps as we go. Be concise with your responses. At each step pause to tell me the changes you will make before implementing them and ask if you should proceed or adjust the plan.”
- “Update the @face_detection_service.py and to get the embeddings from the image. Theses embeddings will be needed for face matching. Update the response object from the face_detection_service to include the embedding data that will need to be saved for searching. You will need to use multiple models to get sufficient data for creating a complete face record”
- “Create an archive_service that will be used for storing, retrieving, and searching of the embeddings data. We don’t know how we will implement these functions yet so just stub in these functions. The Store function, as part of its request object, will need to take in the embedding information returned from the face_detection_service.”
- “Research vector databases and choose one to use for the archive_service.”
- “Implement storing and searching in the archive_service.”
- “Improve the filtering so that low quality results are not included in the final result.”
What happened in reality was that by the time Cursor got to Step 3, I had already opened another chat tab and done this research. I was able to tell Cursor to skip Step 3 when it got there because I had already made a decision.
2. Choosing the Right Vector Database
User Query: “With regards to the FaceDetectionService and the idea of savings embeddings from the images for later searching. Which vector database would be best for saving embeddings so that they can be searched to find similar images? The criteria for the vector database is that it should have an official docker image available that can easily be run locally as a container in docker.”
AI Response: “Based on your requirements and the current face detection service structure, I recommend Qdrant as the best vector database for your face recognition application. Here’s why: Qdrant has excellent official Docker images with multiple deployment options, is specifically designed for high-dimensional vectors like face embeddings, and is open source with no usage restrictions.”
This led to setting up Qdrant with Docker and configuring the vector database for our multi-model approach.
Vector Database Integration
I chose Qdrant as our vector database for its excellent Docker support and face recognition optimization:
The multi-model approach was developed as we discovered that different face recognition models produce embeddings of different dimensions, requiring separate collections in Qdrant.
3. Solving Bugs Along the Way
User Query: “The Qdrant database appears to be broken. I get this error when trying to search it: ‘Service internal error: task 317 panicked with message “called Result::unwrap() on an Err value: OutputTooSmall’”
AI Analysis: “The ‘OutputTooSmall’ error in Qdrant typically occurs when there’s a mismatch between the vector dimensions stored in the database and what the search is expecting. The problem appears to be a vector dimension mismatch between what’s stored in Qdrant collections and what the search is expecting.”
Solution Applied: We recreated the Qdrant collections with correct dimensions for each model (VGG-Face: 4096, Facenet: 128, ArcFace: 512) and fixed configuration conflicts.
4. Architecture Evolution
User Query: “Create a service-oriented architecture for face recognition with dependency injection. Each service should use request/response objects and be completely decoupled.”
This request led to the evolution from a monolithic app.py file to a sophisticated service-oriented architecture with:
FaceDetectionServicefor face detection and embedding extractionArchiveServicefor vector database operationsDisplayServicefor HTML report generationAnalysisServicefor Azure Computer Vision integration
5. Debugging and Development Setup
User Query: “instead of using uvicorn.run could we instead setup a launch settings file to use the FastAPI CLI when I debug in Cursor?”
AI Response: “Absolutely! Setting up a launch configuration for the FastAPI CLI is a much cleaner approach. Let me help you create a VS Code/Cursor launch configuration file.”
This resulted in a debugging setup with multiple configurations for different parts of the application.
6. Azure SDK vs REST API
One of the interesting development experiences was implementing Azure Computer Vision for image analysis. This highlighted both the strengths and limitations of AI-assisted development:
Initial Approach – Azure SDK:
Final Solution – REST API:
The Learning: Cursor AI struggled with the complex Azure SDK setup, authentication, and version compatibility issues. However, when we switched to the REST API approach, it worked almost immediately. This was a combination of the doc for the SDK not being accurate and Cursor AI not being able to determine if it had the right version of the SDK to match the version of the service that we were trying to use.
Face Storage and Search Implementation
We were able to demonstrate finding faces in an image using both the DeepFace library running locally and using the Azure Face Recognition Service. Once the bounding boxes of the faces were found, we were able to use DeepFace to create the embeddings for each face and store those embeddings in the Qdrant database.
Here’s how we store face embeddings in Qdrant:
And here’s how we search for similar faces:
Key Learnings
Cursor AI proved helpful for:
- Technical decision-making – Recommending Qdrant over other vector databases based on specific requirements
- Problem diagnosis – Identifying root causes of complex issues like the Qdrant “OutputTooSmall” error
- Refactoring – Doing the heavy lifting of moving code from from a monolithic to service-oriented design
- Development quality of life – Setting up debugging configurations, launch settings, writing the ReadMe files and Cursor Rules
The AI-assisted approach was genuinely collaborative at times and strictly managed at other times. Rather than following predetermined prompts, I engaged in natural conversations about real problems:
- “Which vector database should I use?” led to a comprehensive analysis and Qdrant recommendation
- “My database is broken” resulted in systematic diagnosis and a complete fix
- “How do I set up debugging?” created a professional development environment
- “Create a service architecture” evolved the entire codebase structure
However, AI assistance needs you, the human to know when to pivot on a solution. The Azure Computer Vision integration revealed Cursor’s limited ability to sort through bad documentation and realize that there was a simpiler approach.
