I have a program I'm working on that uses JavaScript (or typescript and react native, more specifically) to classify images taken by the camera or selected from the device's gallery. I have a TFLite model ready to use, but I'm not sure how to get those images to work with the model.
I think my biggest issue is I don't fully understand how the code I'm using interacts with the different modules I've been trying.
Here's what I have so far.
export default class CameraPage extends React.Component { camera = null; state = { captures: [], capturing: null, hasCameraPermission: null, cameraType: Camera.Constants.Type.back, flashMode: Camera.Constants.FlashMode.off, }; setFlashMode = (flashMode) => this.setState({ flashMode }); setCameraType = (cameraType) => this.setState({ cameraType }); handleCaptureIn = () => this.setState({ capturing: true }); handleCaptureOut = () => { if (this.state.capturing) this.camera.stopRecording(); }; handleShortCapture = async () => { const photoData = await this.camera.takePictureAsync(); this.setState({ capturing: false, captures: [photoData, ...this.state.captures] }) }; handleLongCapture = async () => { const videoData = await this.camera.recordAsync(); this.setState({ capturing: false, captures: [videoData, ...this.state.captures] }); }; async componentDidMount() { const camera = await Permissions.askAsync(Permissions.CAMERA); const audio = await Permissions.askAsync(Permissions.AUDIO_RECORDING); const hasCameraPermission = (camera.status === 'granted'&& audio.status === 'granted'); this.setState({ hasCameraPermission }); }; render() { const { hasCameraPermission, flashMode, cameraType, capturing, captures } = this.state; if (hasCameraPermission === null) { return <View />; } else if (hasCameraPermission === false) { return <Text>Access to camera has been denied.</Text>; } return (<React.Fragment><View><Camera type={cameraType} flashMode={flashMode} style={styles.preview} ref={camera => this.camera = camera} /></View> {captures.length > 0 && <Gallery captures={captures}/>}<Toolbar capturing={capturing} flashMode={flashMode} cameraType={cameraType} setFlashMode={this.setFlashMode} setCameraType={this.setCameraType} onCaptureIn={this.handleCaptureIn} onCaptureOut={this.handleCaptureOut} onLongCapture={this.handleLongCapture} onShortCapture={this.handleShortCapture} /></React.Fragment> ); };}
My first issue is I need to know where the images are going and how to reference them. Second, how to use those images in a TFLite model. And third how to do the same thing pulling the images instead from the gallery.
After all that, I need to see about taking optimal images from a short video to use in the model, but I'll tackle that in another ask if I can't figure it out.
Thank you.