Applying Artificial Intelligence to Classify the Maturity Level of Coffee Beans During Roasting

,


Introduction
Based on the Republic of Indonesia Law Number 86 of 2019, the definition of food encompasses all products derived from biological sources such as agriculture, plantations, forestry, fisheries, livestock, and aquatic sources.This includes both processed and unprocessed food products intended for human consumption, whether in the form of food or beverages.The term also covers food ingredients, raw materials, and other substances used in the preparation, management, and production of food and beverages [1].Coffee is an immensely popular beverage widely consumed across the globe.Superior coffee quality can be identified through its distinct aroma and unique flavor profile.The parameters for assessing high-quality coffee are measured during the coffee bean roasting process [2].The coffee bean roasting process is a stage where green coffee beans are heated to specific high temperatures, leading to chemical reactions such as hydrolysis, polymerization, and pyrolysis.The colors formed during roasting are due to melanoidins, which are a result of the Maillard and caramelization reactions.These color changes significantly impact the flavor quality produced by the coffee beans [3].Each type of coffee bean has distinct differences in shape, color, and flavor, depending on the roasting process used, which also influences its price and quality.However, not all coffee shop owners or coffee farmers can visually recognize the types of coffee beans accurately.This lack of knowledge can lead to errors in identifying the coffee bean types if the coffee shop owners do not possess adequate knowledge.In this research, image segmentation using ROI (Region Of Interest) will be employed to identify specific areas in the coffee bean images.Subsequently, RGB color features will be extracted from each image to obtain relevant characteristics.The classification process will be carried out using CNN (Convolutional Neural Network), which is effective in recognizing patterns and features in images, thereby enabling the classification of roasted coffee bean types with high accuracy [4][5][6] [7].The entire algorithm and model developed will be implemented in an Android smartphone application to enable easy and practical detection of roasted coffee bean types in Coffee shops.With this application, coffee shop owners and coffee farmers are expected to have a more straightforward and convenient way to recognize and present high-quality coffee beans to their customers.By utilizing this application, it is hoped that coffee shop owners and farmers can readily identify and serve high-quality coffee beans to their customers, thereby enhancing the overall coffee drinking experience.

Image Classification
Image classification is the process of categorizing an image into a specific class or category.It finds extensive application in various fields, such as classifying plant leaf diseases, facial expression recognition, and fruit ripeness classification.Image classification is the task of assigning labels or classes to entire images, where each image is expected to belong to only one class.Image classification models take images as input and return predictions about the class to which the image belongs [8].

Artificial Intelligence
Artificial Intelligence (AI) is the implementation of human intelligence replicated in computational systems and aimed at processing information in a human-like manner.The use of data is crucial in this technology to strengthen knowledge, facilitate advancements, and promote learning based on past experiences [9].Artificial Intelligence has the ability to self-improve through its learning capabilities from past mistakes.Artificial Intelligence falls under one of the following four factors: acting like a human, thinking like a human, thinking rationally, and acting rationally [10].

Machine Learning
Machine learning is the use of computers and mathematical algorithms to adopt knowledge from data and generate predictions for the future.This learning process consists of two main stages, namely training and testing, with the aim of acquiring intelligence through experience.The field of machine learning focuses on developing computer programs that can automatically self-improve based on the experiences gained [11].

MobileNet
The use of Convolutional Neural Network (CNN) models in image formation and visual identification greatly influences the accuracy of the results.In this process, a model is created from the existing dataset to recognize images using the MobileNet architecture.MobileNet is one type of CNN architecture that is effective in handling large-scale data computations.The uniqueness of MobileNet lies in the utilization of convolutional layers with appropriate filter thickness according to the size of the input images [12][13][14].

Python
Python is a high-level programming language that allows the direct execution of various instructions interpretively with an Object-Oriented Programming approach.By using dynamic semantics, Python's syntax has a high level of readability.The advantage of being a high-level language makes Python easy to learn due to its efficient automatic memory management [15][16] [17].

Tensorflow
TensorFlow is an open-source and free software library used in the field of machine learning.Its primary focus is on training and inferring deep neural networks.The library is based on the concepts of dataflow and programming.TensorFlow serves as a computational framework that facilitates the construction of machine learning models.It provides a variety of toolkits, allowing users to create models with appropriate levels of abstraction according to their needs and execute these graphs on various hardware devices, such as CPUs, GPUs, and TPUs [18][19].

System Overview
This program is a smartphone application that has the capability to detect and classify coffee bean images based on their roasting levels.After the user launches the application, they can use their smartphone camera for real-time coffee bean detection or choose images from the gallery to process.The application utilizes artificial intelligence algorithms, including Convolutional Neural Network (CNN) models, to perform image classification and accurately determine the roasting level of the coffee beans.The user-friendly interface allows users to easily interact with the app and obtain quick and reliable results.With this application, coffee shop owners, farmers, or coffee enthusiasts can effortlessly identify and categorize different coffee bean roasting levels, thereby improving the overall quality of coffee preparation and serving [20].

Fig 1. Implementation System
Figure 1 shows the implementation of the system.The program will use data from a pre-trained image dataset.The detection results will display images of successfully detected roasted coffee beans, while the classification results will show labels containing the coffee bean class names and the confidence level of the classification results.The application uses a Convolutional Neural Network (CNN) model to carry out both the detection and classification tasks.The pre-trained model has been trained on a diverse set of coffee bean images with different roasting levels to ensure accurate and reliable performance.When the user runs the application, they can either use the smartphone camera for real-time coffee bean detection or select images from the gallery for processing.The system will then analyze the input images and provide the corresponding detection and classification results.With this system, users, such as coffee shop owners, farmers, or coffee enthusiasts, can easily and efficiently identify the roasting levels of coffee beans and gain insights into the quality of their coffee products.

Research methods
This research phase involves several steps, including data preprocessing, which consists of image segmentation, image resizing, and data splitting.The next steps are CNN architecture modeling, model testing, model conversion, and the design of a smartphone-based application.Figure 2 illustrates the research methodology.In the data preprocessing stage, the images are segmented to identify specific regions of interest, resized to a standardized format, and then divided into training and testing sets to ensure proper model evaluation.Next, the Convolutional Neural Network (CNN) architecture is designed to create a suitable model for coffee bean classification based on the input images.After the model is trained, it is thoroughly tested to assess its performance and accuracy in detecting and classifying roasted coffee beans.Upon achieving satisfactory results, the model is converted into a format compatible with the smartphone application, allowing seamless integration and real-time inference on the smartphone.Finally, the smartphone-based application is designed and developed to provide users with a user-friendly interface for capturing coffee bean images and obtaining instant detection and classification results.This research methodology ensures a comprehensive and systematic approach to building an effective coffee bean roasting detection and classification system for smartphone use.The initial stage begins with data collection, which involves acquiring data in the form of roasted coffee bean images.This is followed by the data preprocessing stage.The next step involves designing the CNN model to be used and conducting model testing.Finally, the application is designed.The flowchart demonstrates the systematic approach taken in the research, starting from data collection, preprocessing, model design, testing, and ultimately the application development for coffee bean roasting detection and classification.

Dataset Requirement Analysis
The dataset used in this research consists of four types of roasted coffee beans, namely Green Roasting, Light Roasting, Medium Roasting, and Dark Roasting.The coffee bean images were obtained through direct observations at "Underline Coffee and Noodles," a coffee shop in Cibinong Bogor, and through observations at a coffee plantation.The process of collecting the dataset is illustrated in Figure 3.The dataset collection process involved capturing images of coffee beans of different roasting levels at the mentioned locations.These images were then labeled and organized according to their respective roasting types.The dataset serves as the foundation for training and testing the Convolutional Neural Network (CNN) model in the subsequent stages of the research.Figure 3

Image Segmentation
This research performs image segmentation because the image data used contains shadows from the image acquisition process.The segmentation process involves normalizing RGB values and converting the RGB image data into the HSV color space to facilitate defining upper and lower boundaries for color segmentation.By normalizing RGB values and using the HSV color space, the research aims to enhance the accuracy and effectiveness of color-based image segmentation.This segmentation technique enables the identification and extraction of specific color ranges, which are crucial in distinguishing different components or objects in the images, such as coffee beans and shadows, for further analysis and classification.

Image Resizing and Data Splitting
In the next step of data preprocessing, the images are resized to ensure uniform pixel size, with each image being resized to 224x224 pixels.
Resizing the image dataset signifies that each image is now ready for analysis using the Convolutional Neural Network (CNN) method.Furthermore, data splitting is performed as part of the preprocessing process.The dataset is divided into three sets: 80% for training data, 10% for testing data, and 10% for validation data.This division allows for a comprehensive evaluation of the model's performance on different subsets of the dataset, ensuring that the model generalizes well to unseen data.With these preprocessing steps completed, the dataset is now prepared for training, testing, and validation of the CNN model to accurately classify the roasted coffee bean images based on their respective roasting levels.

CNN modeling
After preprocessing the data, the next step is to train the CNN model.MobileNet is a lightweight architecture and easy to understand.In

Model Conversion
Converting a CNN model to .tflite is for optimizing and executing models on devices with limited computing power, such as mobile devices, microcontrollers, or edge environments.TFLite (TensorFlow Lite) is a platform for running machine learning models efficiently on devices with limited resources.

Mobile Application Design
In this phase, the design of the application interface that will implement the built model will be created.The research utilizes the Android framework provided on the TensorFlow Example website and makes design modifications to the interface, adding functions for inputting images via the gallery and removing the background.The source code can be found at the following link: https://github.com/tensorflow/examples/tree/master/lite/examples/image_classification/android_java. Figure 5 displays the design of the main menu interface of the application.During this stage, the researchers will work on customizing the user interface of the Android application to ensure a seamless and user-friendly experience.The modifications will include integrating the image classification model (converted to .tfliteformat) into the application's codebase, allowing users to input images from their gallery for classification.Additionally, a feature for background removal will be implemented to enhance the accuracy and usability of the application.The design of the main menu will provide users with clear and intuitive options for accessing the image classification functionality and other features of the application.The goal is to create an efficient and visually appealing interface that allows users to easily interact with the AI-powered coffee bean roasting detection and classification system.

Result and Discussion
Based on the training that has been done using 100 epochs, there are 2 test results, the first is based on training accuracy and the second is based on confusion matrix, precision, recall, and f1-score.Testing with 50 epochs obtained an accuracy of 83.33%.Figure 6 shows the results of plotting the accuracy of the training data for 50 epochs.In this menu, there are two buttons: one for removing the background if needed and another for performing the classification.The images or pictures input through the gallery will be displayed on the designed image view.When the "Classify Coffee" button is clicked, the application will display the prediction results, including the predicted label and its accuracy.The "Remove Background" button provides users with the option to remove the background from the input image if necessary.This feature can be useful for enhancing the accuracy of the coffee bean roasting classification.The "Classify Coffee" button triggers the model to perform the classification based on the input image.Once the classification is completed, the application will display the predicted label, indicating the type of roasted coffee bean, along with its accuracy score.

Conclusion
From the results of creating and analyzing the coffee bean roasting classification application, it can be concluded that machine learning technology with image classification has been successfully implemented to classify different types of coffee bean roasting.The use of Google Colab and Jupyter Notebook as software for model training, along with the MobileNet V2 architecture, produced a well-performing classification model.The training process with 100 epochs resulted in a model with a loss value of 0.12 and a training accuracy of 94.79%.This classification model achieved an average accuracy of 85.83% on the test data with a loss value of 0.35.Furthermore, the trained model was converted into a Tensorflow Lite embedded system, suitable for deployment on Android devices.The implementation of the model into the Android embedded system resulted in an application that can be executed as an .apkfile on devices with a minimum Android OS version 8 (Oreo).The application's testing results showed successful real-time classification under the condition of 1275 lumens light intensity and a 90-degree angle based on the horizontal line.Additionally, testing the application using images stored in the gallery also demonstrated its excellent capability in classifying different types of coffee bean roasting.Overall, the application's performance showcases the successful integration of machine learning and image classification techniques, allowing users to conveniently and accurately identify the roasting type of coffee beans in real-time using their Android devices.

Fig 2 .
Fig 2. Flowchart of research methods Figure 2 illustrates the flowchart of the research method.The initial stage begins with data collection, which involves acquiring data in the form of roasted coffee bean images.This is followed by the data preprocessing stage.The next step involves designing the CNN model to be used and conducting model testing.Finally, the application is designed.The flowchart demonstrates the systematic approach taken in the research, starting from data collection, preprocessing, model design, testing, and ultimately the application development for coffee bean roasting detection and classification.

Fig 3 .
Fig 3. Dataset Requirements Based on Figure 3, there are two images showing the process of collecting the dataset from two locations.Figure (a) displays ripe coffee bean fruits ready for harvesting, and Figure (b) shows the coffee bean roasting process at "Underline Coffee and Noodles" coffee shop.The entire dataset used consists of 1200 images, with each class containing 300 images.

Fig 4 .
Fig 4. MobileNet V2 architecture flowchart Figure 4 displays the modeling of the MobileNet V2 architecture model.Then carry out the training process using 100 epochs.

Fig 5 .
Fig 5.The main menu design of the application

Fig 7 .
Fig 7. Results of training 100 epochsAfter data training with two trials at 50 epochs and 100 epochs, the best results were obtained at 100 epochs with an accuracy of 85.83% and a loss accuracy of 0.35.This study also compares the results using the Confusion Matrix, Precision, Recall, and F1-score as shown in Figure8.

Fig 8 .
Fig 8. Confussion Matrix,Precission, recall dan f1-score Based on Figure 8 the results of the Confusion Matrix model use 120 validation data, can predict the type of coffee beans Light Roasting 23 data with 6 prediction errors, Green Roasting 31 data with 3 prediction errors, Dark Roasting 31 data with 3 prediction errors, and Medium Roasting 18 data with 5 prediction errors.Precision results for Light Roasting are 92.00%,Green Roasting 86.11%, Dark Roasting 79.49%, and Medium Roasting 90.00%.Recall results for Light Roasting are 79.31%, Green Roasting 91.18%, Dark Roasting 91.18%, and Medium Roasting 78.26%.The F1-score results for Light Roasting are 85.19%, Green Roasting 88.57%, Dark Roasting 84.93%, and Medium Roasting 83.72%.The camera menu page is a page for carrying out the process of identifying coffee beans in real-time to identify which image object belongs to which type of roast coffee bean.Figure 9 displays the camera menu.

Fig 10 .
Fig 10.Gallery menu page addition, MobileNet can run on CPU, GPU, and TPU, making it easier to use deep learning and convolutional neural networks for the first time.The following is MobileNet's implementation of the prepared dataset, which consists of a folder named "SEEDS" which contains several class folders that have been separated by class.Here are the details: