Data validation and processing services
Summary
Data validation and processing services
S00115
Version: 1.10
- arable farming
- horticulture
- viticulture
- data augmentation; data analysis
- data
- Location: at user’s premises; in Italy
- Offered by: POLIMI; UNIMI
Description
Through this service, we offer customers to validate and augment the data collected during the different testing phases.
This validation process is important to check, for instance, that all relevant data streams have been collected as expected, that no drop-outs have occurred and that the collected data adhere to high technical quality standards (e.g., by checking the focus and exposure of camera images).
In this validation phase, we will also opportunely prepare the data so that they are ready for the performance evaluation. This includes checking the data formatting, identifying images that contain reference visual markers, associating collected images with ground truth annotations, and so forth.
One especially important type of data augmentation that can be performed by Service S00115 is the labeling of the data, i.e. the association of portions of data (e.g., image regions) to descriptions that provide information about the significance of those portions. In particular, labeling is required whenever the data are meant to be used for model training.
Example service: In the context of autonomously monitoring the hydric stress of grapevines, the customer has already collected hyperspectral and RGB images of Red Globe bunches at their premises. Analysing this dataset, we notice that 10% of the provided frames are corrupted and need to be filtered out before being fed to any inference model. To enhance the quality of the hyperspectral data, we apply opportune noise reduction methods. Moreover, let us assume the customer data were collected in a single run at 9:00 AM. In this case, we would diversify the brightness of RGB images in post-processing, to obtain a more heterogeneous and rich set that resembles different lighting conditions.
Finally, for each collected image we annotate image regions and provide text labels categorising each region (macro class: vine, support infrastructure, ground, sky). Since the customer is also interested in using the images to train a disease detection neural network, we involve agronomists to annotate the same images to highlight damaged leaf regions and assign a disease-related labels to each region. Multiple experts are involved to cross-check the correctness of the labeling and to characterise the reliability of individual labels.