AvatarMediKi - Technical process for taking medication

System logic in detail

• Trigger by cup removal:
  The integrated sensor in the medication dispenser detects the removal of a cup and initiates the monitoring process.
• Activation of the avatar:
  AvatarMediKi switches to attention mode (State: active monitoring).
• Motion detection via YOLO:
  A locally running instance of YOLO analyzes the environment for relevant movements. Goal: Detection of a person with targeted hand movement towards the mouth.
• Image filtering & forwarding:
  Only frames with potentially relevant hand-mouth interaction are sent to the Deep-AI.
• Deep-AI analysis:
  The Deep-AI classifies the action as “medication taken” or “not taken” and justifies this evaluation.
• Auditory prompt in case of uncertainty:
  If the Deep-AI cannot recognize intake, an audio dialogue is automatically triggered: “Have you taken your medication?”
• Data transfer to nursing staff:
  Results of the visual analysis and verbal feedback are transmitted to the care platform including timestamp.

Goal

Robust, AI-supported monitoring of medication intake with minimal intervention, optimized for use in nursing facilities, assisted living, or at home with remote monitoring.

Outlook: Further applications of the AvatarMediKi technology

The combination of sensor-based event detection, visual AI analysis (YOLO, Deep-AI) and adaptive user interaction (e.g., audio prompts) offers potential far beyond medication monitoring. Future fields of application could include:

• Fall detection and emergency response: Visual analysis detects atypical movement patterns or inactivity, responds automatically, and informs help.
• Food and fluid monitoring: Automated recording of whether residents drink or eat – relevant for care and nutritional control.
• Dementia support: Reminder systems that recognize whether important daily activities have been performed, and intervene kindly if necessary.
• Rehabilitation and therapy monitoring: Recording of exercise executions or progress in movement to support physiotherapy.
• Access or object security: Monitoring whether certain objects (e.g., cleaning agents, tools) are used correctly or stored safely.