Problem
A physical-therapy product used Swift, Core ML, and TensorFlow to read patient movement from a camera and score it against a clinical protocol. They needed the model to run on the web as well as on iOS, needed better ML data hygiene, and needed a deterministic way to inspect frame-by-frame model output.
What I did
- Translated the Core ML model to TensorFlow.js so the same scoring could run in the browser.
- Compiled the core scoring logic in Swift to WebAssembly and shared it between the web client and the iOS app.
- Built a live synchronization layer between the web canvas and the iOS app so therapist and patient stayed in sync.
- Implemented complex physical-therapy tests in Swift / SwiftUI.
- Integrated the Bullet physics engine into the WASM module to lift prediction quality against noisy input.
- Improved ML data management with Python tooling.
- Built data-analysis tooling for deterministic frame-by-frame insight into the machine-vision pipeline.
- Set up Terraform deployments for the back-end services.
Outcome
The same scoring model ran on iOS and on the web from a single Swift source, with live therapist/patient sync, better training data pipelines, and reproducible per-frame analysis.
Stack
- Languages. Swift, TypeScript, Python
- Platforms. GCP
- Frameworks. TensorFlow (Python / iOS / JS), SwiftUI, React, Bullet Physics, Terraform