-
Notifications
You must be signed in to change notification settings - Fork 19
Stream camera frame to ObjectDetector? #173
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Yeah same issue |
This is a good question -- I think there is likely a way to implement this, but it'll take some work. MLKit definitely supports streaming, but we'd need to implement a module interface for streams. We should probably remove the option in the meantime, to prevent confusion. |
Took a look and this could be implemented by writing a Frame Processor for something like I'm going to take a swing at writing one that we can export from the module. But they are not all that difficult to implement. |
Cool! If there’s anything I can do to help let me know. Would be a learning experience for me too. Yes a frame processor makes sense! |
I have been using React-Native-Vision-Camera + react-native-fast-tflite. Unfortunately, I need to investigate an alternate solution to passing images to the model. This happens seamlessly with RNFastTFLite. I noticed with this MLKIt library, that a
uri
path "string" is needed to pass to the model. That would require converting theframe
from RNVisionCamera'sframeProcessor
to an image on the device and then create a temp file path. This is going to be very memory & storage intensive, so not a good idea.Does this MLKIt work well with frame streams, or is it designed to be more for single image processing? I see you can set
detectorMode: 'stream'
option for when initializing the original model options, but it still requires a string path?The text was updated successfully, but these errors were encountered: