Quantcast
Channel: Active questions tagged react-native+typescript - Stack Overflow
Viewing all articles
Browse latest Browse all 6287

How to use a TFLite model on a javascript program?

$
0
0

I have a program I'm working on that uses JavaScript (or typescript and react native, more specifically) to classify images taken by the camera or selected from the device's gallery. I have a TFLite model ready to use, but I'm not sure how to get those images to work with the model.

I think my biggest issue is I don't fully understand how the code I'm using interacts with the different modules I've been trying.

Here's what I have so far.

export default class CameraPage extends React.Component {  camera = null;    state = {        captures: [],        capturing: null,        hasCameraPermission: null,        cameraType: Camera.Constants.Type.back,        flashMode: Camera.Constants.FlashMode.off,    };    setFlashMode = (flashMode) => this.setState({ flashMode });    setCameraType = (cameraType) => this.setState({ cameraType });    handleCaptureIn = () => this.setState({ capturing: true });    handleCaptureOut = () => {        if (this.state.capturing)            this.camera.stopRecording();    };    handleShortCapture = async () => {        const photoData = await this.camera.takePictureAsync();        this.setState({ capturing: false, captures: [photoData, ...this.state.captures] })    };    handleLongCapture = async () => {        const videoData = await this.camera.recordAsync();        this.setState({ capturing: false, captures: [videoData, ...this.state.captures] });    };    async componentDidMount() {        const camera = await Permissions.askAsync(Permissions.CAMERA);        const audio = await Permissions.askAsync(Permissions.AUDIO_RECORDING);        const hasCameraPermission = (camera.status === 'granted'&& audio.status === 'granted');        this.setState({ hasCameraPermission });    };    render() {        const { hasCameraPermission, flashMode, cameraType, capturing, captures } = this.state;        if (hasCameraPermission === null) {            return <View />;        } else if (hasCameraPermission === false) {            return <Text>Access to camera has been denied.</Text>;        }        return (<React.Fragment><View><Camera                        type={cameraType}                        flashMode={flashMode}                        style={styles.preview}                        ref={camera => this.camera = camera}                    /></View>                {captures.length > 0 && <Gallery captures={captures}/>}<Toolbar                     capturing={capturing}                    flashMode={flashMode}                    cameraType={cameraType}                    setFlashMode={this.setFlashMode}                    setCameraType={this.setCameraType}                    onCaptureIn={this.handleCaptureIn}                    onCaptureOut={this.handleCaptureOut}                    onLongCapture={this.handleLongCapture}                    onShortCapture={this.handleShortCapture}                /></React.Fragment>        );    };}

My first issue is I need to know where the images are going and how to reference them. Second, how to use those images in a TFLite model. And third how to do the same thing pulling the images instead from the gallery.

After all that, I need to see about taking optimal images from a short video to use in the model, but I'll tackle that in another ask if I can't figure it out.

Thank you.


Viewing all articles
Browse latest Browse all 6287

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>