From July to October 2016, we have conducted an evaluation of Garden Monitor with eight users in Milton Keynes. This evaluation was designed to assess the performance of the machine learning of Garden Monitor, as well as its value to users and its usability. At the end of the evaluation, we asked our users to answer a questionnaire focusing on the last two aspects.
The results of this evaluation are promising.
About the performance of the machine learning engine, the computation of Root Mean Square Error (RMSE) values show models becoming more accurate with time. Moreover, users reported positive feedback regarding the correctness of the predicted soil moisture values.
About the value of Garden Monitor to users, their gardens were in healthy shapes at the end of the evaluation even though users were watering less frequently. This observation suggests Garden Monitor effectively helped users in reducing water wastage related to gardening activities.
About the usability of Garden Monitor, answers to a System Usability Scale questionnaire yielded a mean score of 72.5. This score is above the standard threshold of 68, which means that the application can be considered easy-to-use, even though this is still an initial prototype.