ICTworks

Speech-based conversational interfaces have several potential benefits for farmers and the ICTforAg professionals who support them:

• There is little requirement for literacy and it offers a natural and familiar modality that does not require a user to learn new technical concepts or interaction methods.
• The knowledge base of the conversational system can be easily edited or customized by agriculture experts.

However, chatbots for farmers still calls for empirical understanding for the acceptability and usability of this new type of technology among the farmers population. The research paper, FarmChat: A Conversational Agent to Answer Farmer Queries, set out to fill this gap.

FarmChat Agriculture Chatbot Research

The authors acknowledge two sources of knowledge that informed the development of FarmChat:

• Farmers’ information inquiries with the Kisan Call Center (KCC)
• Findings from a formative study with local farmers and agri-experts.

The Government of India has made all logs of calls to the KCC from January 2015 to September 2017 publicly available. In total, this corpus contains data for 8,012,856 calls. Each call log has 11 fields, including the date and time of the call, location, crop (one of the 306 crop types), query, and the answer provided by the KCC agri-expert.

The paper authors conducted semi-structured interviews with 14 farmers (9 male, 5 female) and 2 male agri-experts, in September 2017. They worked closely with a local agriculture NGO, where the two agri-experts were employed. They helped recruit the farmers and obtain their consent for participation, following their own internal ethics policies.

The farmers and agri-experts provided the researchers with similar questions as the ones they found in the KCC dataset. Based on both sources, they identified four major areas requiring information support:

• Plant Protection: In the KCC dataset, 60.6% of the potato farming calls were related to remedies for protecting
• Pests and diseases: Agri-experts stated that a majority of farmers seek suggestions on which medicine to spray for a particular crop disease. None of the farmers the researchers interviewed were aware of any disease name. Usually, farmers describe crop diseases by their visible symptoms to the agri-expert; with a few back-and-forth questions, the agri-expert hypothesizes the issue and recommends medicine with dosage information.
• Weather: In the KCC dataset, 39.4% of the overall calls were about weather-related questions; 13.5% of potato farming questions were about weather. Farmers eagerly sought weather information, as rains can wash away expensive sprayed pesticides and weather conditions determine the best time to harvest crops.
• Best Practices: Information related to best practices can help increase yield in terms of the quantity or quality of potatoes. Common questions were: “Till what height should I put water?” “After how many days, should I harvest?” These best practices questions comprise of 6.6% of the potato farming calls in the KCC dataset. Agri-experts also stated that farmers consistently asked them tips to increase yield and, consequently, income.
• Unbiased Recommendations on Products: Farmers wanted recommendations from agri-experts on products they should purchase. Questions such as “Which fertilizer to put and how many times?” and “Which seeds are the best for red potatoes?” were commonly asked. They prefer to ask these questions to agri-experts instead of local shopkeepers, believing that agri-experts would provide unbiased and trustworthy response; they feared that shopkeepers may be motivated by the profit margin of products.

FarmChat Agriculture Chatbot Development

The researchers developed a knowledge base for potato farming using the KCC dataset and information collected from formative interviews with smallholder farmers and agri-experts. For each of the identified topics, they asked the two agri-experts (who participated in the Formative Study) to provide examples of typical farmer questions, the follow-up questions that they would ask in order to understand the problem, and the final advice they would provide.

All such conversations was added to the IBM Watson Conversation dialogue flow, and the informational advice was included in the FarmChat knowledge base. In the current version, the knowledge base is a SQL database consisting of four tables, one for each of the topics they identified above.

The Audio-only FarmChat interface consists of only two buttons:

• A red ‘microphone’ button that the user needs to click to provide speech input,
• A blue ‘play’ button, which enables the user to listen to the chatbot’s last response again.

Users can click on the blue button any number of times to repeat the most recent response. While the bot is playing the last response, the blue button can be clicked again to pause the response. After the user’s speech input is received, the interface shows a waiting icon and does not allow the user to click any of the buttons.

This was done based on the results from a pilot study with 4 farmers that uncovered major design issues. Participants had a tendency to talk further while waiting for the bot’s response, perhaps presuming that the bot did not understand the previous input. Once the Audio-only FarmChat app receives the bot’s response, it removes the waiting icon, and speaks out the response.

The Audio+Text FarmChat user interface closely resembles a typical text-messaging interface, wherein the user input and bot response are presented in message bubbles. There are two major differences from a typical text-messaging interface:

• Audio+Text FarmChat can only receive speech and button click input, not text;
• The text/image output can be processed as audio, i.e., clicking on a message bubble in Audio+Text FarmChat results in it being read aloud through Text-to-Speech.

Audio+Text FarmChat use images in two ways – for asking multiple choice question, and for explaining a farming concept. For example, the user asks FarmChat, “What kind of seeds should I use”, to which the bot responds, “Which potatoes do you want to grow?” with three images as options ‘white potato’, ‘red potato’, and ‘potato for chips’.

The user can either press the microphone button and say the response to the question aloud or click one of the three tick-mark buttons to select a particular option. The user can also click the image to hear a description of the selected image. The response from the bot may also contain images to explain certain concepts

Overall, FarmChat was acceptable by the farmers as an information source to satisfy their farming information needs. All participants expressed willingness to continue using FarmChat in the future. The major reasons that farmers enjoyed using FarmChat were immediate responses to their queries and constant access to farming-related knowledge.

For example: “Information is the key… If I know more, I will earn more!”  and “It gave me new knowledge… like treating the seed initially will help… It even told me medicines.” In particular, responses that included a medicine name and quantity were re-read and replayed most often (2.4±1.5 times for Audio-only). This may be because they wanted to memorize the names or the hard medicine names were not clear to them in the first attempt.

This suggests that compared to existing human information sources like the KCC and agri-experts, a chatbot system has the potential to better serve farmers’ needs for continuous learning.

Seven Lessons Learned from FarmChat Research

There are seven themes that emerged from the user study on information assistance provided by FarmChat.

Precise and Localized Answers

Specificity and localization were identified as keys to the information needs of farmers. With the help of agri-experts, the researchers carefully tailored the system responses to local conditions. Participants appreciated such information contents:

Trust

Trust is another key design requirement. In general, participants trusted the responses provided by FarmChat. Six participants even asked the facilitator to write down the recommended medicines, seeds variety, and/or fertilizer with their quantities for them to refer later. Participants often formed trust in FarmChat by validating its responses with their existing knowledge.

Over-expectation

The researchers found participants’ tendency to overestimate the capabilities of FarmChat, which led to some level of dissatisfaction in usage. Lack of clear affordability is a known challenge for conversational interface, but the novelty effect for the farmer population seemed to exacerbate the problem.

Although FarmChat had high success rate (overall only 21 out of 238 questions went unanswered), some participants (4/34) were still disappointed when one of their questions was not answered and said: “Why are you making excuses?” or “it should have the answer to all questions”

Anthropomorphism

FarmChat by design was not anthropomorphic, as the researchers did not introduce any human-like features such as name or character. In spite of that, they found participants to have a high tendency to anthropomorphize the bot.

A few participants (6/34) referred to the bot as “didi”, which means elder sister in the local language, since the bot had a female voice. Some participants (4/34) said: “ok”, “good”, “yes”, after every sentence said by the bot, as if they were talking to a human. Also, participants were very polite in their interactions. Questions usually began with “please listen …”, “can you please tell me …”, and ended the conversation with “thank you for the help”, as if talking to an agri-expert.

Speech as Input

Participants were pleasantly surprised that FarmChat was able to understand their complex questions in Hindi (rating 4.1±0.6). “The question I am asking, it is able to understand well. Most times, only after a single attempt. ”.

Note that this was partly due to the wizard’s role in fixing errors in speech processing; 36.3% speech inputs were fixed by the wizard, as computed from the wizard log files. Speech as input failed for a few of the illiterate participants (4/11) who were not able to speak Hindi fluently.

Responses by Speech

A majority of the participants (18/34) appreciated the fast responses given by the bot in speech: “very quick response, no wait”.

The average response time was 9.2±2.8 sec, which includes Google transcription and translation time (0.9±0.2), wizard time (5.7±4.5 during edits, 2.1±1.2 without any edit), Watson Conversation service with Python Flask response time (3.1±1.1), and network delays.

Using button clicks for input significantly lowered the response time to 1.7±1.0 sec since neither the wizard nor transcription/translation services were used. Participants were generally satisfied with FarmChat’s response time

Interaction Order as a Usability Challenge

For a majority of illiterate and digitally-illiterate participants, the ordering of pressing the red microphone button, waiting for the beep sound, and then speaking was challenging to follow. Note that this order is required by the current speech input technologies.

Many times, participants did not press the microphone button at all or started speaking before the beep sound. At times, the mic button was pressed but no speech input was received (Audio+Text: 18.5±13.0%, Audio-only: 15.4±14.1%). This often happened when the participants were thinking of what to say next, but the app stopped listening after it detected a long silence.

Farmer Chatbots Are an Option

This study of 34 potato farmers in rural India indicated that it is possible to provide satisfying information support to the farmers through chatbot. The researchers also compared the effectiveness of two interface modalities: Audio-only and Audio+Text.

The study indicated that although text-based output allows for repeated consumption of the same information, participants expressed different preferences due to literacy, digital-literacy, and other environmental and physical factors.

The positive feedback of the farmers indicates that conversational intelligence as a technology delivered through the ubiquitous smartphone can be an effective tool to improve information access in a rural context for people with limited literacy and technology experience.