How do GenZ speakers use and process emoji in chatbot conversations: An eye-tracking study

. This study investigates use and processing of emoji in chatbot conversations. 32 GenZ participants engaged in semi-naturalistic chats in an eye-tracker for ten minutes with a ChatGPT bot that used specific emoji from two personas (GenZ and Millennial). Eye-tracking and sentiment analysis revealed that the GenZ bot was typically positively perceived, while the Millennial bot showed one of two patterns: adaptation or hyperfocus on the ‘wrong’ emoji. The study sheds light on how emoji are used compared to words and their impact on AI assistant communication styles. The findings offer insights for studying open-ended conversations and linguistic patterns using ChatGPT.


Introduction.
Emojis have transformed how Internet users express themselves in digital conversations (Bai et al. 2019;Cohn et al. 2018;Feedman 2018, Giannoulis & Wilde 2019).These small, colorful icons help convey emotions to make text communication more engaging.Their interpretation varies greatly: the way users perceive, interpret, and use emojis is influenced by their age, gender, and culture (An et al. 2018;Freedman 2018;Guntuku et al. 2019;Jones et al. 2020;Miller et al. 2016;Prada et al. 2018;Weiß et al. 2020).Despite clear evidence for userspecific interpretation, previous psycholinguistic studies showed that emojis are processed similarly to words during reading (Barach et al. 2021;Boutet et al. 2021;Howman & Filik 2020;Paggio & Tse 2022;Robus et al. 2020;Weissman & Tanner 2018).This leaves an open puzzle as to how both persona-based and lexical aspects of emojis are integrated in processing.We approach this in the current work using interaction with an AI-based chatbot.As AI-powered agents such as ChatGPT(OpenAI 2023a, 2023b) learn from online data, they can acquire knowledge about conversational patterns, including emoji usage.Nowadays, conversational agents have become more integrated in today's world, and it is important to understand and evaluate the role of emojis in shaping AI-human interactions.By collecting eye-tracking data during live human-AI interactions, we can gain behavioral insights into cognitive processing of emojis and conversational content in context, with clear applications to user experience.
Here, we focus on AI-human interactions with GenZ users (born between 1996-2001).This is the first generation to grow up entirely in the post-digital world, with technology embedded in their lives from a young age.GenZ users are known to have a distinct digital communication style characterized by the avoidance of capitalization and extensive use of abbreviations, acronyms, and emojis to communicate their messages in a more efficient way (Satviki 2020;The Economic Times 2023;Véliz 2023).When it comes to the use of emojis, GenZ users exhibit distinct patterns compared to other generations, creating new, non-literal social meanings of emojis (Abril 2022;Dictionary.com 2022;Gomez 2023;Jines 2023).Some common interpretations of emojis used by GenZ users are "Sparkles" emoji '✨', used to convey excitement, "Skull" emoji '', used to indicate humor or surprise, "Eyes" emoji '', used to express curiosity, and "Nail Polish emoji" '', used to express nonchalance (Emojipedia 2023).In contrast, emojis with a positive literal meaning, such as "Smiling Face with Smiling Eyes" emoji '' and "Thumbs Up" emoji '' may carry passive-aggressive connotations for GenZ users but are often used more literally by older users such as Millennials, the demographic cohort preceding GenZ users (Brandon 2022; The Economic Times 2023).These generational differences are particularly important to understand how emojis shape human-AI interactions for GenZ-a demographic at the leading edge of leveraging emojis in digital communication and creating new social meanings of emojis.
To understand how GenZ users' emoji interpretation patterns shape their interactions with AI systems and whether chatbot emoji choice affects their experience, we conducted an eyetracking experiment where participants interacted with a ChatGPT-powered chatbot with a GenZ or Millennial persona conveyed using stereotypical generation-specific emojis.This, to our knowledge, is the first study to examine emoji use and processing with chatbots.To explore the effect of emoji use on human-AI interactions, we asked three research questions (RQs).
• RQ1: What is the processing pattern for emojis in general: Do GenZ users process emojis similarly to closed-class words (e.g., verbs, nouns, adjectives) or open-class words (e.g., nouns, verbs) during unscripted interactions with an AI-powered chatbot?• RQ2: Is there a pragmatic processing cost to using the 'wrong' emoji: Does the use of typical "GenZ" emojis elicit different eye fixation patterns vs. the use of typical "Millennial" emojis in the processing of emojis produced by the chatbot?• RQ3: How do GenZ users feel about these interactions with chatbots, and does the alignment between the chatbot's persona (GenZ vs Millennial) and the user's generation (GenZ) affect their overall experience?To answer our research questions, we designed a chatbot that was powered by ChatGPT (OpenAI 2023a, 2023b) and used it in an eye-tracking study where 32 participants interacted with the chatbot under one randomly selected persona condition (GenZ or Millennial) for ten minutes and then answered several follow-up questions about their experience.We focused our analysis on two eye tracking metrics (first fixation duration, representing early processing time, and total fixation duration, representing all processing time).We supplemented these data with a content analysis of open-ended survey responses about the user experience to obtain additional insights into perceptions of the chatbot personalities and emoji use.This work makes four unique contributions to research on emoji perception & processing and human-AI interaction.
• The study provides novel insights into the perception of emojis used by conversational agents in human-chatbot interactions.Our findings extend prior research on conversational agents by better understanding the role of emojis in shaping user experience, information processing, and conversational dynamics during human-AI interaction.• By collecting and analyzing conversational data with eye-tracking, the study investigates how emojis are processed in a dialogue context, contributing to the existing body of work on emoji processing and interpretation.• The study highlights generation-specific patterns in emoji processing, usage, and human-AI interaction.
• The study provides recommendations on the integration of emojis into chatbot design.

Related work.
Prior studies have examined GenZ use of emojis, linguistic processing of emojis, and the implications of emoji use by conversational agents.In this section, we review literature in these areas to situate our research questions within the current understanding of generational differences in emoji interpretation and of the emojis' role in human-AI interactions.
2.1.GENZ USE OF EMOJIS.Studies and popular news articles (Abril 2022;Adobe Fonts 2022;Brandon 2022;Duijst 2017;Herring & Dainas 2020;Kirakowski et al. 2009;Slack 2023) have discussed how GenZ users use emojis differently compared to older generations.They emphasize how emoji meanings can shift across generations, leading to misunderstandings between users.Research by Abril (2022), Adobe Fonts (2022), andSlack (2023) shows that compared to other generations, GenZ users are more likely to creatively use emojis and find messages feel incomplete without emojis.However, their interpretations and use patterns can confuse older colleagues.Herring & Dainas (2020) show that users over 30 often interpret emojis like Fire '' literally whereas younger users interpret them metaphorically (e.g. as meaning "lit" or "sexy").Dictionary.com (2022) provides some specific examples of how Gen Z gives emojis different meanings than Millennials: Folded Hands '' emoji can mean "thank you" instead of a high five; Skull '' emoji and Loudly Crying Face '' emoji are used to convey intense laughter rather than literal crying or death.GenZ users also use the Slightly Smiling Face '' emoji ironically or passive-aggressively, whereas older generations interpret it literally (Brandon 2023).The Thumbs up '' emoji is perceived especially negatively by GenZ users, as confrontational or passive aggressive, whereas older generations view it positively (Katz 2022).
2.2.EMOJI PROCESSING: EYE TRACKING STUDIES.Eye tracking studies have proven to be a valuable technique for investigating emoji processing during reading (Barach et al. 2021;Boutet et al. 2021;Howman & Filik 2020;Paggio & Tse 2022;Robus et al. 2020).Eye tracking measures reflect the time course cognitive processing, providing an online measure of how emojis are handled during reading.First fixation duration indicates the initial time spent fixating on a target and relates to early stages of lexical access, whereas total fixation duration captures all the fixation time, and therefore all of the processing for a target (Rayner 1998).Paggio & Tse (2022) conducted a reading task in which an emoji either follows or replaces a word and found that emojis were sometimes more difficult to process when used to replace words, with shorter total visit times for unambiguous (such as Airplane '✈' or Pizza '') vs. ambiguous (such as Bowl with Spoon '' or Tent '⛺') emojis.This indicates that lexical access may be more effortful when the meaning of an emoji is not immediately clear for the reader.Barach et al. (2021) showed that semantically congruent (synonymous) emojis were fixated less, skipped more often, and elicited shorter total reading times relative to incongruent uses.Specifically, the authors show that for the message "My tall coffee is just the right temperature", followed by the Hot Beverage '☕' emoji compared to the Beer Mug '' emoji, the Hot Beverage emoji, synonymous to the target word in the sentence, was easier to process This suggests that emojis are integrated similarly in sentences to words during comprehension.Expanding on the sentence-position effect, Robus et al. (2020) observed longer fixations and reading times for sentence-final compared to sentence-initial emojis (e.g., "Steven waited at home for his order to be delivered in the morning " vs " When the guest returned to the hotel later there was nobody to be seen").The authors likened this to sentence "wrap-up" processes in reading, showing that emojis, like words, undergo a final integration processing stage.Howman and Filik (2020) and Boutet et al. (2021) showed that the emotional incongruence between the emoji and the emotional valence of the sentence can impact information processing, with faster early processing followed by longer later processing for targets surrounding the emoji.Howman and Filik (2020) showed that the presence of the Winking Face '' emoji that typically signifies sarcasm paired with a literal, non-sarcastic meaning increased processing time.This indicates readers spent more effort processing the conflict between emoji and the overall message tone.Existing studies suggest that emojis may improve chatbot perceptions with younger users but caution about risks of emotional manipulation, highlighting the importance of tailoring emoji usage to the interaction context.Beattie et al. (2020) show that participants rated emoji-using chatbots as more socially attractive, competent, and credible compared to word-only chatbots.Dogan & Collins (2019) focused specifically on emoji use for communication with GenZ users and found emojis help GenZ express their feelings better.This suggests emojis may be particularly beneficial for chatbots targeting younger demographics.It aligns with results from Choi (2023) who showed that such emojis as sparkles and smiling faces increased positive emotions and purchase intentions among GenZ women towards products promoted via TikTok video ads.Wilhelm et al. (2022) also found a positive trend between chatbot emoji use and user satisfaction, though results were not statistically significant.In the tourism industry context, Bezirgan & Kaya (2019) argue emojis are important for digital marketing and communication with younger travelers.They recommend tourism companies strategically incorporate emojis in social media, advertising, reservations, and customer communications.In contrast, Li & Shin (2023) found luxury brand chatbots using emojis negatively influence brand status perceptions and interaction appropriateness.Similarly, Duijst (2017) found user frustration with financial chatbots when emojis were used with a formal chatbot communication style.This suggests that context matters greatly for emoji use.There are also concerns around emotional manipulation.Véliz (2023) argues chatbots that use emojis could manipulate users by eliciting instinctive emotional reactions, undermining autonomy.There are also considerations around user expectations in interaction context.Chaves & Gerosa (2021) found users disliked chatbots using unusually formal language or vocabulary for casual conversations, and, similarly, participants in Duijst ( 2017) study perceived combining formal and informal language styles as inconsistent language use by the chatbot.

Methods.
In this section, we review the methodology of our experiment in detail, including the eye tracking setup, chatbot design, survey materials, participant recruitment, data collection, annotation, and analysis procedures.
3.1.EYE TRACKING SOFTWARE.Participants' eye movements were recorded using SR Research EyeLink and WebLink software (SR Research Ltd 2020a, 2020b).Participants were seated 27 inches in front of a 17.5 x 8.5-inch laptop display screen with their head stabilized using either a chin rest or tracked using a sticker target.A 13-point calibration routine was conducted, followed by video recording of the display screen and the recordings of the location of mouse clicks and eye fixations at a sampling rate of 1000Hz.This enabled continuous tracking of where participants were looking on screen as they interacted with the chatbot.
3.2.CHATBOT DESIGN.We used the ChatGPT API (gpt-3.5-turbo;OpenAI 2023a, 2023b) and OneReach.aiRich Web Chat software (OneReach.ai2023) to design the bot.The chatbot was presented in a Chrome browser.ChatGPT was instructed to act as a friendly GenZ or Millennial chatbot having a conversation with a friend.Both prompts directed the bot to use certain emojis periodically, keep responses relatively short, and discuss neutral topics such as lifestyle, hobbies, technology, travel, music, sports, food, or personal development.The difference in prompts for ChatGPT included the list of recommended emoji used in the conversation (✨ for GenZ and ❤ for Millennial) and the conversation style (GenZ persona or Millennial persona).ChatGPT was also instructed to ask questions and share opinions on these topics.Each conversation had the first turn scripted as "Hi!What have you been up to lately?".After 50 conversational turns, a scripted response of "Great!Let's talk about something else"' was programmed to avoid hitting ChatGPT's token limit1 .Leveraging ChatGPT's capacity for diverse, randomized response generation introduced an element of natural unpredictability into conversations.While this means that persona-based language and emoji use were not the only factors influencing bot responses, it mirrors the organic variation present in real human-AI interactions and enhances ecological validity.
3.3.SURVEY DESIGN.After the chatbot conversation, participants were asked to fill out a survey in the same interface.The survey consisted of two sections of open-ended responses about participant demographics and their interaction with the chatbot.We collected participant demographics including age, gender, sexuality, and race/ethnicity and information about any diagnosed conditions like ADHD, ADD, OCD, and speech/language disorders.These were collected to account for individual differences that may influence emoji perception and eye tracking patterns.Participants were also asked about the operating systems and platforms they engage with and their familiarity with generative AI to account for their familiarity with emoji use and previous experience with generative AI-powered chatbots and smart assistants.Following the demographic section, we asked several open-ended questions about the conversation experience to collect qualitative feedback about how the emojis used by the chatbot impacted the conversational experience.
3.4.PARTICIPANTS.We recruited 34 GenZ users (18-26 years old, median = 22) from a Southern California college town during May and June 2023.As this is exploratory work, we felt that 16 participants per group each chatting for 10 minutes was a reasonable amount of data to afford both quantitative and qualitative comparison.Participants were recruited from social media and mailing lists.After collecting data from 32 participants, data from two participants were excluded due to poor-quality eye tracking data, and an additional two participants were recruited.All participants signed an informed consent form and received a $10 Amazon gift card for their participation in the experiment.The final sample included 16 self-identified females, 12 males, 1 transgender non-binary individual, 1 nonbinary demi femme individual, 1 non-binary gender non-confirming individual, and 1 gender queer individual.All participants indicated that they were fluent in English, had normal or corrected-to-normal vision, and were free of language disorders (e.g., dyslexia).
3.5.PROCEDURE.The study began with participants reviewing and signing a consent form2 , followed by calibration of the eye tracker.Participants then had a ten-minute conversation with a ChatGPT-powered chatbot.After the conversation participants completed a survey about their experience.At the end of the study, participants were debriefed on the study's purpose.The total study duration was approximately 30 minutes.
3.6.DATA EXTRACTION.Conversations were extracted from OneReach.ai(OneReach.ai2023) and transcripts were saved in .csvformat.Demographic information and responses to openended survey questions were compiled separately.The raw eye tracking data (eye fixation data, timestamps, and interest area coordinates) were extracted using SR Research EyeLink Data Viewer (SR Research Ltd 2020).3.7.DATA ANNOTATION.For each conversation, we annotated key information including the chatbot condition and anonymized participant IDs.For each conversation turn, we annotated the conversation agent and turn ID (e.g., chatbot_turn1 or participant_turn2).This resulted in a dataset of 1476 total turns, with approximately 20 turn pairs per participant.We annotated each conversation turn for repeated responses (cases were flagged where the response repeated more than 80% of the previous response and manually checked) and the presence of emojis.From the video recording of each conversation, we also annotated the timestamps for each conversation turn and marked the interest area coordinates for each conversation turn to identify top/bottom/left/right coordinates of the response block on the screen (see Figure 1).3.8.DATA PREPROCESSING.We used lookup tables to align the timestamps of coordinates of the text areas on each screen (which changed dynamically as the chat progressed) with the indices of each turn to identify all tracked eye fixations when the participant's gaze is focused on a specific block of text.Because the chat was presented in a non-fixed width font, we also created a lookup table of pixel measurements for each character (including upper-and lower-case letters, punctuation, and unique emojis) and combined these with the text data to identify the word or emoji in the text area that the participant was looking at for each fixation; the word or emoji fixated became our Interest Area measure3 .To ensure accuracy, we manually checked the script for 10% of turns per recording; all were accurate.This resulted in a dataset of 17,790 rows with aligned fixations, timestamps, participant turns, chatbot turns, the interest area (word or emoji that the participant was looking at), and additional information about each turn.
3.9.DATA ANALYSIS.Quantitative analysis was based on two eye-tracking measures: first fixation duration (FFD: the time period between the first fixation on the interest area until the moment of leaving the interest area by the reader) and total visit duration (TVD: total fixation time on the target during the trial).The amount of time that participants spend fixating elements indexes processing costs in general-splitting this based upon early (FFD) and late (TVD) measures gives insight into when during processing the largest costs occur.Qualitative analysis was based on the open-ended responses.
Mixed-effect Poisson models with by-participant random intercepts were built in the lme4 package in R (R Core Team 2013) to answer the first two RQs.The rationale for choosing Poisson regression over a linear model lies in its appropriateness for count-based data and its interpretability in the context of the research questions being addressed.To answer RQ1, we examined differences in FFD and TVD in based upon: word class (POS; Emoji vs each other word class used; treatment coding), the length of the target (LENGTH: binned at 1-3 vs 4-6 and 7+; treatment coding), sentiment of the message (SENTIMENT; neutral vs negative vs positive; treatment coding), the presence of a question at the end of the sentence (QUESTION; question vs non-question; treatment coding), and the cases of repeated content of the bot (HALLUCINATION; hallucination vs non-hallucination; treatment coding).POS and LENGTH were the key variables of interest: we wanted to know whether emoji were more similar to various open-class or closed-class words, controlling for the length of the item because longer items often take longer to process.As such, POS and LENGTH were allowed to interact, but no other predictors were.
To answer RQ2, we tested the effect of chatbot type (CHATBOT; GenZ vs Millennial; treatment coding) and specific emojis (EMOJI; ✨ vs each other emoji used; treatment coding) on TVD (no interactions).We chose TVD as the proxy for total emoji processing time.
To answer RQ3, we performed content analysis (Fuad Selvi 2019) of the open-ended responses provided by the participants about the interaction with the chatbots.A codebook was developed through an iterative process of open coding a subset of the responses.It contained codes for emoji_influence, emoji_quality, and user_satisfaction.Emoji_influence captured comments about how the chatbot's use of emojis influenced perceptions of the conversation.Emoji_quality coded statements about whether the emoji usage felt natural or unnatural.User_satisfaction coded feedback on enjoyment of the conversation.

Results.
We found that GenZ users process emojis similarly or faster than open-class and slower than closed-class words (RQ1).While the Millennial chatbot overall had shorter total visit durations on emojis suggesting more efficient emoji processing, there were differences between individual emojis indicating that emojis are not processed uniformly and emoji processing was largely not chatbot style dependent (RQ2).While both chatbots received negative feedback from individual users due to frequent memory lapses, the GenZ chatbot was perceived as more friendly with its emoji use contributing to user empathy (RQ3).4.1.RQ1.FFD and TVD showed similar results.As shown in Figure 2, interjections, prepositions, determiners (closed class words) elicited the longest FFD and TVD.This reflects the more complex semantics of these word classes in driving sentence-level meaning.The extended processing time for these parts of speech suggests deeper analysis is required to fully access their contribution to the message.Nouns, pronouns, adjectives, verbs, and adverbs (open class words) showed moderate FFD and TVD.These parts of speech represent more interchangeable or supplementary information in sentences.Emojis showed shorter FFD and TVD, which was close to the average FFD and TVD for numbers and named entities, suggesting more efficient initial processing compared to open-class words.The constrained semantics and syntax of named entities and numbers allows for rapid identification of their function, without extensive meaning integration.However, emojis still required more extensive cognition processing than closed-class function words like modal verbs and existential there, which received longer FFD and TVD 4 .
Several other factors also impacted FFD and TVD 5 .Specifically, the presence of repeated content by the chatbot was associated with shorter processing time, suggesting a repetition prime effect.Questions were associated with longer processing times than non-question responses, consistent with their association with increased effort. .Sentiment also impacted processing, with negative statements eliciting the longest processing times, indicating greater cognitive effort to understand the meaning of the response.Responses that contained positive sentiment were processed faster than neutral and negative responses, which reflects facilitated processing of positive information.Word length effects depended on part-of-speech, where medium length (4-6 letters) words were processed longer than the same parts of speech with a length of 1-3 words.This increased processing as longer words intrinsically require more perceptual effort.However, the longest words (7+ letters) showed mixed effects depending on part-of-speech: "existential there" constructions, as well as named entities, were processed faster (see Figure 2).This divergent effect of long words reveals complexity in how word length interacts with syntactic categories during lexical processing.properties show that emoji processing is closely connected with the semantic and syntactic context of the message.Consequently, for chatbots to select and incorporate emojis naturally, their algorithms must model emoji as intrinsic elements of utterances that interact with other words and phrases Chatbots that can generate emojis situationally based on syntax, tone, and context would achieve more efficient processing and comprehension by users.4.2.RQ2.Overall, we found that Millennial chatbot was associated with shorter total visit durations on all emojis overall (220 ms) compared to the GenZ bot (250 ms), suggesting more efficient emoji processing6 .This was likely because the majority of current chatbots do not use GenZ-specific emojis and it therefore took longer for the participants of the experiment to adapt to the chatbot style.There were also many emoji-specific differences suggesting that emoji are not a uniform class of items 7 .
Results from all fixations on emojis used by the chatbots were combined into a total visit duration measure per interest area.This resulted in a dataset with 199 data points.Among the most visited emojis in the data set were Millennial bot emojis, such as   , and GenZ emojis, such as  ✨ .To test the effect of chatbot type (CHATBOT) and specific emojis (EMOJI) on total visit duration (TVD), we used the "Sparkles emoji" '✨' as the reference level as it represented the mean total visit duration (250 ms) and was one of the commonly fixated emojis for the GenZ chatbot (17 fixations total).Unpacking differences between specific emojis, we find that for emojis associated with the Millennial bot persona like the "Heart Eyes" '' and "OK Hand" '' emojis, long mean total visit durations were observed8 .In particular, the "Heart Eyes" '' emoji had the longest mean total visit duration at 359 ms, suggesting that the use of this emoji was likely incongruent with the dialogue context and did not align with the user's expectations.The other Millennial emojis like "Hundred Points" '' and "Smiling Face with Smiling Eyes" '' were processed similarly to the "Sparkles" '✨' emoji.Prototypical GenZ emojis like Eyes '' and "Smiling Face with Hearts" '' also elicited long mean total visit durations, indicating possible incongruity in emoji usage by the chatbot.Interestingly, other GenZspecific emojis as Fire '' and Pleading Face '' showed shorter mean fixation duration, indicating that the sentiment conveyed by these emojis was easier to process.This meant that while there was a general slowing for GenZ specific emojis, some of these emojis were in fact easier to process than the Millennial set.Icon emojis depicting subjects or objects like "Person Climbing" '' emoji and "Person Lifting Weights" '' emoji had long total visit durations of 400+ ms."Artist Palette" '' emoji and "Video Game" '' emoji, as well as "Cat" '' and "Dog" '' emojis also had long total visit durations.These emojis were mainly used by the bot in conversations about hobbies of the user.In contrast, such emojis such as Water Wave '', Backpack '', "Books" '' had short total visit durations of 100-200 ms, implying easier processing and that these emojis may be used in a more straightforward context.We suggest that these differences in eye fixation patterns may be attributed to varying levels of emoji familiarity, emotional expression, or contextual relevance associated with different emojis, but are likely not associated with a specific chatbot style.

RQ3.
Responses to the open-ended questions about the conversation experience showed a range of opinions about the experience chatting with both GenZ and Millennial-acting chatbots.
While most participants (N = 14) indicated they found the experience of chatting with the GenZ chatbot interesting, fun, helpful, and insightful, suggesting a positive interaction, others (N = 2) mentioned that the conversation felt unnatural: "… personally I'd rather talk to a human, it was a bit odd having such … unnatural conversation… it made me feel like i was definently talking to a robot" (female, 19, white) and repetitive: "bot kept asking me the same question" (male, 20, white).This mixed feedback shows that while most participants enjoyed chatting with the bot and found it useful, others evaluated the experience as artificial and limited.One participant mentioned the emojis gave the chatbot a happy, reassuring tone: "a lil it was reassuring and made me feel happy" (female, 24, white).Several participants reported the emojis did significantly shape the perception of responses and that it made them perceive the bot's personality differently: "it influenced how i perceived the personality of the chatbot" (transgender non-binary, 21, white) or reacted in a more empathic way: "I think I probably used more exclamation points and was more emphatic in how I responded than I might otherwise be, because I was responding to the use of emoji and also things like how many exclamation points the bot used" (Non-binary/gender non-confirming, I do identify as trans, 22, white).
The feedback from the participants was that GenZ chatbot used emojis too frequently and was less ironic than expected.One participant mentioned that the bot seemed to use emojis in a very formal, not fun, way, which made it feel unnatural: "...gen Z typically uses emojis ironically versus using emojis seriously…" (female, 19, white), while the other participant mentioned that they "felt that the bot was using emojis more than I or my friends might" (male, 18, White).Responses about the experience with the Millennial chatbot were more mixed.Conversation issues, such as problems with the bot getting stuck or being repetitive, the bot insisting it didn't have interests/preferences, and using emojis repeatedly despite requests to stop, contributed to feelings that the conversation was odd and frustrating for a larger number of participants (N = 6): "It got stuck at one point, and when I tried to get it to ask more questions it just repeated its earlier questions.I tried to give different answers, but then it got stuck again, so I asked it about its interests and preferences.A few times it stopped answering to insist it didn't have any, but then mostly it would claim to have some, making it a slightly odd experience."(male, 26, White).Several participants found the use of emojis by the Millennial bot positive: "it was cute" (female, 19, Mexican) "it felt cuter and more positive" (non-binary demi femme, 24, white), while another participant consistently told the bot to stop using emojis and therefore experienced a feeling of frustration during the conversation "I kept telling it to stop using emojis" (male, 26, white).At the same time, the participants did not find the Millennial bot's use of emojis to be very similar to conversing with a human: one participant mentioned that the chatbot was using emojis as "a foreigner with diff emoji habits" (non-binary demi femme, 24, white).In terms of influence on the conversation, most participants reported that they "didn't really pay attention to the emojis enough for them to influence my response" (female, 19, white) and "mostly just ignored them" (male, 26, white).
Overall, while most participants responded positively to the conversational experience with both chatbots, the chatbot with the GenZ persona was overall perceived as more friendly to chat with compared to the chatbot with the Millennial persona.Participants mentioned that the GenZ chatbot's use of emojis contributed to more participants feeling empathy and reacting happily, though some still found the emoji use too formal and not natural.This is particularly striking given that the GenZ chatbot was associated with longer overall processing time, and therefore more processing difficulty.Conversational limitations as repetitive questions were the main factors contributing to the negative feedback for both chatbots.5. Discussion.This section covers the implications for chatbot developers on integrating emojis in chatbots and limitations of the current study that suggest future research directions.Key takeaways include strategically using emojis to improve user perceptions, matching bot persona to target demographics, and studying a wider range of emojis and user populations.5.1.STUDY IMPLICATIONS.Quantitative findings from the study have several key takeaways for using emojis in chatbots.The findings from RQ1 show that while emojis can require some additional cognitive processing compared to words, they are still fairly easy for users to interpret in conversation.This makes them a useful tool for adding emotional nuance such as friendliness or empathy that text alone cannot convey.Chatbot developers should consider integrating emoji usage to make conversations more natural and human-like.However, we identified that not all emojis are equally easy to process.RQ2 results showed that popular emojis like the smiling face were processed most quickly, while emojis with more ambiguity or those used incongruently took the user longer to interpret.When selecting a set of emojis for the chatbot to use, developers should lean towards emojis that will convey clear meaning to the end user without ambiguity.
The qualitative results suggest several key implications for chatbot developers.While answering RQ3, we showed that emojis appear able to shape user perceptions of a chatbot's personality, making conversations seem more friendly, happy, and empathetic when emoji use is congruent and aligned with user personality and expectations.This indicates that strategic emoji use will tend to improve user experience.However, overusing emojis or using them in an overly formal way while they are typically expected to be used in a funny or sarcastic manner, may, in contrast, be perceived as fake and unnatural.Second, matching a chatbot's persona and communication style, including emoji selection, to the communication style of the target demographic group can likely improve perceptions of friendliness of the chatbot.Emoji interpretations vary, so chatbots may need customization for different target populations.This means that testing emoji usage with end users is critical to prevent mismatches between the chatbot's intentions and user interpretations.In summary, strategic emoji integration informed by user research and testing can significantly improve chatbot conversational experiences.However, emojis require thoughtful implementation informed by bot/huma persona alignment, demographic trends, and user feedback to realize their full potential for enhancing human-chatbot interaction.5.2.STUDY LIMITATIONS.This study only examined a small set of emojis that are commonly associated with two generations of users, GenZ and Millennials from over 3,000 available emojis (Adobe Fonts 2022) to use.Additional research is needed to understand how people interpret and use the full diverse range of emojis during conversations.Before chatbots can realistically mimic human emoji usage, more empirical work is needed to better understand the nuances of emoji interpretation across various demographic, generational, and cultural contexts.While the study findings provide initial evidence about the role of emoji in human-AI interactions, including across a more gender-diverse selection of users than earlier studies, a wider range of emojis should be investigated in a variety of populations to inform more comprehensive emoji usage guidelines for chatbots.Using ChatGPT as the baseline model for the AI-powered chatbot in the experimental setup introduced some inconsistencies, as the chatbot did not always follow instructions and had no memory for past utterances Running a study with a more controlled chatbot system could provide clearer insights on how specific emoji usage strategies influence user perceptions.In particular, the noise introduced by ChatGPT's repeated responses likely influenced patterns in the data.Another study limitation is that chatbots in this study were based on perceived generational stereotypes about emoji usage and perceptions of ChatGPT about the communication style of GenZ and Millennial users.However, participants likely brought their own biases about emoji usage, motivated by their own patterns of emoji use, influencing their perception and evaluation of the use of emojis by chatbots.Another limitation of this study is that it is difficult to isolate the specific drivers of the qualitative results as the differences in emoji use are confounded with differences in the linguistic content and tone of the chatbot responses across the two personas.It is unclear whether the observed patterns in user satisfaction stem from the emoji themselves, the wording/content of the responses, or the interaction between the two -such as particular generational associations between linguistic expressions and emoji choices.DIRECTIONS.This study demonstrates the value of an experimental linguistics perspective for understanding patterns of human-AI interaction.Analyzing emoji processing through eye-tracking metrics in interactive dialogue settings can provide useful insights for improving user experience that may be overlooked by pure engineering-based approaches.More interdisciplinary work on the intersection of human-computer interaction and psycholinguistics is likely to provide additional insights on how chatbots can effectively integrate emojis into conversational flow and adapt to target user demographics.One other promising direction for future work would be to compare the processing of emojis in human-AI and human-human conversations in similar conversational contexts.For example, one could observe customer service interactions with either a human agent or a chatbot and analyze if emoji processing patterns and user satisfaction ratings would show differences.Future studies could also explore emoji processing in human-AI interactions across different demographic groups, e.g., a comparative study with millennial and GenZ participants.Given the differences between the two chatbot types, studying a wider range of user populations may reveal more generational nuances in emoji perception that chatbots developers should consider.Future research should also aim to better disentangle linguistic factors through more controlled experimental manipulations that systematically vary emoji use on the same linguistic content.This will provide clearer evidence on the isolated influence of emoji selection in shaping user perceptions beyond the linguistic components of the chatbot responses.

Conclusion.
We conducted an eye tracking experiment analyzing how GenZ users process emojis during conversation with an AI-powered chatbot, using eye tracking methodology and open-ended feedback to gather new insights into emoji perception, processing, and effects on user experience with chatbots.First, we demonstrated that GenZ users process emojis during conversations with AI agents faster than other open-class words, indicating easier lexical access for emojis.However, factors like incongruent usage of emoji, the individual emoji used by the bot, and the chatbot's personality can increase processing time leading to a higher processing load.These results therefore expand our understanding of emoji processing in human-AI conversations.Second, by focusing specifically on GenZ users, we identified generational patterns in emoji interpretation that shape processing and perceptions of chatbots.We show that aligning the chatbot's emoji style with the user's communication style can improve perceptions of friendliness and empathy, though it does not improve processing speed.This highlights the need to profile emojis used by the conversational agents, aligning it with the communication patterns of target audiences during the design stage.Third, we found that conversational limitations of AIpowered chatbots like repetitive questions appeared to be a critical factor influencing user satisfaction.Overall, this work makes significant theoretical and practical contributions regarding incorporation of emojis in human-AI conversations.

Figure 1 .
Figure 1.A dialogue excerpt with annotated interest areas: participant current turn, participant previous turn, and chatbot current turn.The squares represent the location of mouse clicks.The circle represents eye gaze at this point in time.

Figure 2 .
Figure 2. Density plots of first fixation duration by POS and word length These results provide evidence that emojis are processed as integrated with accompanying text by GenZ users during human-AI interaction.The significant effects of all linguistic