RememberMe CareGrid

AI memory, wandering safety, community care coordination, and Gemma 4-powered dementia training for Indian families.

RememberMe does not just track dementia patients. It gives them back context: who they met, where they are, what was said, and who can help.

Dementia care in Indian families is usually handled by one exhausted caregiver. Patients may forget trusted people, recent conversations, medicine routines, and safe routes. Existing tools often stop at GPS tracking or family-only reminders.

RememberMe CareGrid creates a consent-aware memory layer around a dementia patient. It recognizes enrolled trusted people, remembers meaningful conversations, maps safe places, alerts caregivers during wandering risk, coordinates community helpers, generates CareLearn training resources, and prepares doctor-ready summaries.

The product is built around three rings of care:

• Patient: Lumo Companion, SmritiLens, memory cues, SafePath.
• Family: caregiver dashboard, alerts, timeline, doctor brief.
• Community: neighbour, ASHA worker, pharmacy partner, RWA volunteer, student volunteer, Gemma 4 CareLearn training.

• Gemma 4 via local Ollama: Lumo cues, conversation summaries, memory journal summaries, privacy decisions, SafePath alerts, CareCircle task text, CareLearn training cards, RWA awareness session, doctor report.
• Google ADK style workflow: Memory Ingestion Agent, Face Match Agent, Conversation Agent, Privacy Agent, Wandering Agent, Community Agent, CareLearn Agent, Doctor Report Agent.
• Fallbacks: If local inference is unavailable, AI endpoints return high-quality deterministic demo JSON with the same shape.

CareLearn powered by Gemma 4 generates practical learning topics, in-app lessons, role-specific training cards, quizzes, completion records, and certificate-ready training history for caregivers, neighbours, ASHA workers, pharmacy partners, RWA volunteers, and student volunteers.

• Next.js 16 App Router + React 19 + TypeScript.
• Tailwind CSS 4 with a warm editorial care-system UI.
• Optional Firebase Firestore realtime sync with local Demo Mode fallback.
• Browser SmritiLens with real trusted-person enrollment and matching through @vladmandic/human face embeddings.
• Android phone camera relay route at /mobile-camera with QR onboarding, frame capture, optional GPS, and Firebase/local fallback sync.
• SafePath map uses Leaflet/OpenStreetMap with home geofence, safe/risky markers, browser GPS, phone GPS, and simulation fallback.
• Wear OS companion scaffold under /wearos plus a web mirror at /watch.
• API routes for structured Gemma 4/mock generation.
• Local seed data for Rajamma, Ananya, Lakshmi, ASHA, pharmacy, RWA, places, events, and CareLearn resources.

• Patient Home Hub with Lumo Companion and memory journal.
• SmritiLens trusted-person recognition with consented averaged embeddings, webcam/photo/phone-frame matching, and unknown-face privacy behavior.
• Conversation Memory summarization and care timeline updates.
• SafePath Leaflet geofence map with GPS capture, wandering simulation, and patient calming cue.
• SOS workflow for watch GPS geofence exits and cognitive confusion, with Twilio SMS/call delivery when configured and demo-safe simulation otherwise.
• CareCircle neighbour/ASHA/pharmacy/RWA task flow.
• CareLearn powered by Gemma 4 with in-app lessons, role cards, quizzes, training completion, and RWA awareness event generation.
• Doctor Brief with print and PDF export.
• Enrollment screen for trusted profiles: upload 5-10 consented photos, compute descriptors, and store an averaged embedding.
• Galaxy Watch 4 companion app: continuous Lumo talk, I'm okay, notify caregiver, SOS SMS/call, periodic GPS pings, health sync, vibration alert, acknowledgement flow, AAC communication cards for word-loss moments, and bystander QR help card.

RememberMe builds a consent-aware care timeline from important events, not continuous surveillance. It recognizes only enrolled trusted people using consented face embeddings. It does not diagnose dementia, predict disease progression, label unknown strangers, or store raw video forever.

1. Open / and introduce the three rings of care.
2. Open /patient; Lumo gives a calm morning orientation cue.
3. Save the memory journal: “I remember going to Mysore with my husband.”
4. Open /lens; recognize Lakshmi from the sample or webcam.
5. Optional hardware branch: open /enroll, upload 5-10 photos for a trusted person, then match a live/photo/phone frame in /lens.
6. Optional phone branch: open /mobile-camera on Android over HTTPS, send a frame, then analyze it in /lens.
7. Summarize the Lakshmi transcript and see the memory timeline update.
8. Open /safe-path; use browser/phone GPS or simulate a safe-zone exit near MSRIT gate.
9. Trigger a risky road watch ping; SafePath evaluates the radius/risky place and starts the SOS workflow.
10. Open /watch; show the SOS button and bystander rescue QR card.
11. Open /rescue/patient_rajamma; show how a bystander can notify the caregiver with their location.
12. Open /care-circle; accept, mark reached, and mark safe for the neighbour task.
13. Open /carelearn; start an in-app CareLearn lesson, generate the neighbour training card, and mark trained.
14. Generate the RWA awareness session.
15. Open /doctor-report; generate the weekly doctor brief.

• Next.js 16.2.6
• React 19.2.6
• Tailwind CSS 4.2.4
• Firebase 12.13.0
• @vladmandic/human 3.3.6
• motion, lucide-react, recharts, sonner, jspdf
• leaflet, react-leaflet, qrcode.react

npm install
npm run dev

Open http://localhost:3000.

Optional:

cp .env.example .env.local

Add Firebase web config values for Firestore sync. Without them, the app runs in Demo Mode.

For strict local Gemma 4 reasoning, run Ollama with OLLAMA_MODEL=gemma4:e2b. For raw watch audio, start the local Faster-Whisper ASR service before using Talk to Lumo:

.\scripts\start-local-asr.ps1

The watch sends a short AAC clip to /api/watch/talk-audio; the laptop transcribes it locally at ASR_URL, then sends the transcript through the local Gemma/Ollama /api/watch/talk logic.

For real SOS SMS/calls with Twilio, add:

TWILIO_ACCOUNT_SID=
TWILIO_AUTH_TOKEN=
TWILIO_FROM_NUMBER=
TWILIO_EMERGENCY_TO=
TWILIO_ENABLE_CALLS=true

If these are missing, SOS alerts are still recorded in CareGrid and displayed as simulated delivery, so judging does not break.

For Android phone camera:

npm run dev

Expose the app through HTTPS with Vercel preview, Cloudflare Tunnel, ngrok, or local HTTPS. Open /mobile-camera on the phone. Browser camera access works on secure origins (https://) or localhost.

For Galaxy Watch 4:

1. Open wearos in Android Studio.
2. Set caregridApiBase in wearos/gradle.properties to your HTTPS app URL.
3. Enable developer mode and wireless debugging on the watch.
4. Install the debug app from Android Studio.

For the camera-to-watch flow, the phone or future helmet/body camera is the input device, the laptop backend is the memory and Gemma 4 reasoning layer, and the Galaxy Watch is the patient-facing output layer.

1. /mobile-camera or /lens captures a frame.
2. SmritiLens extracts a face embedding and matches it only against consented trusted people.
3. Gemma 4 generates a short Lumo cue, for example: “This is Lakshmi aunty. She is your neighbour from downstairs.”
4. /watch receives the latest cue, vibrates, shows the text, and can speak it using browser speech synthesis.
5. The native Wear OS app calls /api/watch/cue over HTTPS for the same flow.

1. The Galaxy Watch sends GPS to /api/watch/location manually and every few minutes while the app is active.
2. The backend evaluates distance from the configured home safe-zone radius and nearby risky places.
3. High/critical risk creates a SafePath alert, memory event, CareCircle task, and watch cue.
4. Twilio sends SMS and a spoken call to the configured emergency number when credentials are present.
5. Lumo calmly tells the patient to stop near a familiar person and show the bystander help card if needed.
6. /rescue/patient_rajamma lets a bystander notify the caregiver with location without exposing private medical records.

• Production Wear OS app with paired-phone Data Layer fallback.
• Raspberry Pi edge home hub.
• On-device face recognition and encrypted embeddings.
• Multilingual local Gemma 4 Lumo Companion.
• QR bystander rescue band.
• Offline mode.
• Encrypted medical records.
• Real ASHA WhatsApp integration.
• Pharmacy refill integration.
• RWA emergency protocol templates.
• Caregiver stress support journeys.
• Consent and lawful guardian workflow.
• Integration with dementia care NGOs.
• Role-based certificates for CareLearn training.