About the Smart Pillbox

According to Taiwan's Ministry of Health and Welfare (MOHW), multiple chronic conditions are prevalent among older adults, often requiring several medications daily. Missed doses, repeated ingestion, and poor medication compliance present serious health risks.

"Smart Medicine Box" aims to assist caregivers and patients in tracking medication routines accurately, eliminating medication mistakes, and establishing structured adherence logs.

flowchart TD
  subgraph Client
    A["Raspberry Pi Pillbox (Python/Vue)"]
    D["Mobile App (React Native)"]
  end
  subgraph Cloud
    B[".NET API Server"]
    C[("SQL Server")]
    E["Prescription OCR Service (Python/FastAPI)"]
    F["Google Cloud Vision & GPT-4o"]
  end

  A <-->|"REST API"| B
  D <-->|"REST API"| B
  B <--> C
  D <-->|"Upload Prescription Image"| E
  E <-->|"Extract Text & Parse JSON"| F
  E -->|"Save Medication Schedule"| B

The system comprises three major components:

Raspberry Pi Embedded Pillbox: Features internal micro-switches, lighting control, and camera modules to monitor compartment status and photograph placed pills.
Backend API Server: Handles business logic, user auth, and historical logs powered by .NET and SQL Server.
Cross-Platform Mobile App: Allows caregivers and patients to scan paper prescriptions, configure notification reminders, and monitor pillbox conditions remotely.

System Features & Demonstration

1. Hardware Pillbox

Pillbox Exterior

Equipped with an integrated camera and physical drawer detection. When a compartment is pulled out, the system automatically triggers internal LED fill lighting and captures high-resolution imagery of the medications placed inside.

2. Real-Time Pill Detection (YOLO)

Utilizing a custom-trained YOLO object detection model, the Raspberry Pi analyzes drawer contents to recognize medication shapes, colors, and blister pack markings, verifying whether the correct pills were inserted for scheduled times.

3. Mobile App Companion

App Overview

Upon sign-in, the home screen displays current pill storage status and ambient environment data.

Pillbox Status

Prescription OCR Scanning

Combines Google Cloud Vision for high-accuracy text recognition with GPT-4o Structured Outputs to convert unstructured prescription slips into validated JSON payloads.

Scanning UI

Users capture a photo of their clinic prescription slip. The system extracts medication names, dosage frequencies, and duration, parsing them into structured server data.

Parsed Result

Medication Reminders

Reminder Configuration

After medications are organized into the smart pillbox, users customize scheduled reminders via the in-app calendar. When medication time arrives, push notifications alert patients to take their dose on schedule.

Prescription History

Users can inspect medication regimens, course durations, and instructions under "My Prescriptions" to ensure accurate consumption.

Responsibilities & Project Division

As team lead for this capstone project, I coordinated technical execution across the team and personally oversaw Raspberry Pi embedded programming & hardware integration, backend server API engineering, system analysis & specifications, as well as assisting with mobile app development:

Raspberry Pi & Hardware Engineering
- Embedded software control (camera shutter orchestration, fill-light switching, limit switch sensing, custom circuitry)
- Circuit assembly and physical device prototyping
- Touchscreen UI development (leveraging Python Eel and Vue.js)
Backend & Mobile Architecture
- Core .NET REST API development with Dapper ORM and SQL Server
- Mobile client features (prescription scanning pipeline, authentication UI, and modularizing team codebases with AI-assisted tooling)
- Database schema architecture and query optimization
System Analysis & Specifications
- Drafting system requirements, hardware architecture diagrams, and comprehensive academic thesis documentation.