- Frontend runs on ReactJS.
- To initalize node packages run npm i.
- to run the frontend run the command npm start
- 'Add Assets' button is used to add asset image and asset names and send it to the server.
- 'Upload File' button is used to upload target image.
- 'Record Video' button is used to record video and get geolocation in real-time and send it to backend.
- User must upload asset images and name and give either target image or record video to send the data to backend.
- handleInputType() function is called to store the input type whether the input type is adding assets, target image or recording images.
- type of input is stored in useState 'inputType'. Vale of 'inputType' is changed by 'toggleInputType'.
- according to the value of 'inputType' respective div is made visible for the user to give input. Rest other divs are set to style {display:'none'}.
Adding assets
- assets are added by using the component 'AssetForm'.
- useState 'assets' is passed to 'AssetForm' as props.
- 'assets' is an array of objects containing the details of assets in the following format: [ {id: string, name: string, image: file}, {id: string, name: string, image: file}, {id: string, name: string, image: file}, ]
- the component 'CiCirclePlus' calls addAsset() function on click.
- addAsset() function adds another div containing required fields to the useState 'assets'.
- On entering asset name, handleNameChange() function is called which adds the name of the asset to the 'assets' useState array.
- on uploading asset image, file is added to 'assets'.
- 'id' for each asset is handled automatically by the function window.crypto.randomUUID().
- on clicking 'IoIosCloseCircleOutline' symbol, removeAsste() function is called which removes that particular asset from 'assets' array.
Uploading Target Image
- a simple input tag to upload the image.
- uploaded target image is stored in 'file' useState.
Recording Video
- For recording video, 'RecordView' component is used.
- When video recording is started, video recording, live geolocation and counter, these 3 functions run simultaneously.
- recorded video is stored in 'recordedBlob' useState.
- 'RecordView' further use 'ReactMediaRecorder' library to record video and store it in 'recordedBlob'.
- 'RecordView' also uses 'useStopwatch' and 'useGeolocated' libraries in react to start timer and live geolocation when video recording is started.
Sending data to server
- below the input fields, 2 buttons are there, 'Detect with detectron' and 'Detect with OWLViT'.
- For now, both buttons call the function sendData() and send api to OWLViT.
- sendData() accepts 1 parameter: 'model' which is the name of the model, 'Detectron' or 'OWLViT'.
- Inside sendData(), url of api is saved to variable 'url'.
- if the target image is provided, then target image is added to FormData object and then assets are appended to FormData object.
- Else if recorded video is provided then recordedBlob is converted to mp4 format and stored in 'targetFile' variable. Then 'targetFile' and assets are appended to FormData object.
- FormData object is sent to server using POST method.
- Resulted image is received and stored in 'resultImage' useState.