Facilitating social interactions among sighted and visually impaired students via chatbots
Research Problem
-
Teaching and Learning Services (TLS), a group that implements services to help students succeed, funded my team to investigate how AI can enhance undergraduate students' classroom experience.
-
Given TLS's focus on providing an EDI-centered classroom experience, we investigated how chatbots can promote social interactions between visually impaired and sighted students.
Research Impact
Shared in Communication:
-
Produced 4 actionable recommendations to TLS, aiding their decision-making on ways to incorporate AI technologies into classrooms.
Prompted Future Research
-
Submitted a research report to the flagship HCI conference to initiate cross-functional collaboration and develop a chatbot embodying our recommendations.
Solution Explored
-
Conducted 22 interviews to identify barriers impeding social interactions between two student groups.
-
Ideated 4 chatbot features that can address the barriers.
-
Conducted 3 group brainstorming sessions to validate the features.
-
Team: Research associate (me), 1 Research assistant (UX design), Project supervisor (IT) • Duration: 5 months
CONTEXT
Need for AI-driven solutions to make EDI-centered undergraduate class
Teaching and Learning Services (TLS) at Carleton University seeks ways to develop new services that create an EDI-centered classroom experience. With the rise of AI in education, the group funded my team to investigate how AI technologies can deliver an EDI-centered classroom experience.
Diversity
Inclusion
Equity
PROBLEM
Visually impaired undergraduates feel like an outsider in class
EDI-centred classroom experience involves creating an environment where students of all abilities feel that they belong. However, visually impaired students do not feel they belong because they are socially excluded.
visually impaired students feel like outsiders.
visually impaired students are excluded in classe.
We tackled this core issue and investigated how a chatbot can promote social interactions between sighted and visually impaired students. We focused on the potential of chatbots because they are widely used in universities, and students show favourable attitudes towards chatbots.
students report their universities use chatbots.
students are open to receive support from chatbots.
increase in student participation via chatbots
Our research questions were:
-
Research Question 1: What barriers impede social interactions between sighted and visually impaired students?
-
Research Question 2: What chatbot features can reduce the barriers?
OVERALL PROCESS
Here is an overview of our research process.
Stakeholder Communication
I prepared a research plan that defined research questions that aligned with TLS' overall mission and project scope, including budgets and timelines.
This plan was approved by the TLS research offer and associate vice president.
1:1 Interview
Group Brainstorming
Thematic Analysis
INTERVIEW: SET UP
I independently led this process.
1. Participant Recruitment
I recruited 7 sighted and 15 visually impaired students from diverse academic backgrounds and standings (M = 22.86, SD = 2.98).
2. Interview Guide
I prepared an interview guide and questions for sighted and visually impaired students.
What are the barriers to social interaction for visually impaired students?
What chatbot features could reduce those barriers?
CONSTRAINT: PARTICIPANT RECRUITMENT
Recruiting visually impaired participants, in general, is already hard as it is, and I had an even more difficult time recruiting those who were also undergraduate students. In the first week of recruitment, I found zero participants using convenience and snowball sampling techniques.
I had to think differently and came up with an idea: I searched the keywords "visually impaired students" and "school achievements" in Google, knowing that the universities publicly promote any students' accomplishments and post the featured students' email addresses! Using this strategy, I located 34 potential visually impaired students and successfully recruited 15 in two weeks, achieving my goal.
INTERVIEW: KEY INSIGHTS
What barriers discourage social interactions between two student groups?
Inaccessible class materials prevent visually impaired students from participating in group discussions because they do not have the information available to sighted students.
Disrespectful language or actions also impede social interaction by making visually impaired students feel that they don't belong.
"A teacher gave me a physical hand-out and I just sat there while group members were talking about the topic. I couldn't engage in conversations because I didn't know what they were discussing."
- E
Barrier 2. Sighted students are uncertain how to approach visually impaired students.
Sighted students worry about offending visually impaired students by using "wrong" language; thus, they do not initiate interaction with the students.
"I have a few close friends who couldn't approach me for 2 years being in the same major. They thought I needed a "special" treatment and thought their interactions with me had to be different."
What chatbot features could reduce the barriers?
Everyone in the class asks a chatbot about the language they can use towards a visually impaired student.
Idea 3
A chatbot informs instructors and classmates about ways to make class materials accessible to BLV students.
Idea 2
A chatbot analyzes when class materials and posts are not accessible to BLV students.
Idea 4
A chatbot tracks class conversations and flags language that makes BLV students not belong in the class.
IDEATION
The team and I took the ideation to the next level and evaluated the above 4 features by asking these 2 questions:
-
How technically feasible is the idea?
-
How impactful is the idea in reducing users' pain points?
Thanks to OpenAI's GPT technology, we concluded that each idea passed the two criteria. I then elaborated on each feature's use case to be more realistic in a classroom setting and less intrusive to students and created an initial storyboard.
Feature 1
Everyone can ask a chatbot about inclusive words and phrases they can use toward visually impaired students in class.
Feature 2
A chatbot can analyze when class materials and posts are not accessible to visually impaired students. It makes them accessible.
Feature 3
Everyone can ask a chatbot whether class materials and teaching styles are accessible to visually impaired students.
Feature 4
A chatbot learns when students use non-inclusive language toward visually impaired students based on their posts. It sends a weekly summary of those languages and shows alternatives.
SUBJECT MATTER EXPERT INTERVIEW
I independently led this process.
Goal
To revise the initial features based on the expert's lived experiences as someone who is blind and an undergraduate student. I made the storyboard screen-reader-friendly so that the SME could read the features, and we had an hour-long discussion.
Revision 1
Emphasize that a chatbot is for everyone, not just for visually impaired students. A chatbot does not want to make visually impaired students stand out.
Revision 2
Let a chatbot explain how it is making a certain material accessibility, instead of fixing the mistakes. This way, sighted students can grow.
This is the revised storyboard.
Previously: "...they can use towards visually impaired students in class."
Previously: "...not accessible to visually impaired students."
Initial wording: "I make them accessible."
Previously: "...visually impaired students."
Previously: "...visually impaired students."
GROUP BRAINSTORMING
.png){kind=small}
RA and I participated in this process
We held 3 group brainstorming sessions to refine the features with a new group of visually impaired and sighted undergraduates.
1. Participant Recruitment
We recruited 7 sighted and 7 visually impaired students from diverse academic backgrounds and standings (M = 24.22, SD = 1.77).
2. Brainstorming Guide
I prepared a brainstorming guide. Each session hosted 5 to 6 participants and had a members of mixed-visual abilities.
3. Process
A.
I shared the revised storyboard with participants (screen-reader-friendly) one-week in advance so they could come prepared.
B.
On the day, participants were split into the challenge team or the solution team.
Identify 3 problems in the chatbot.
Challenge Team
Identify 3 benefits in the chatbot.
Solution Team
C.
Three rounds of competitions! During each round, the challenge team described one problem. The solution team then developed a solution to resolve the problem, drawing upon the benefits they brainstormed.
There is no one definition of inclusivity. SAMI's recommendations are to generic.
Example problem:
Problem 1
Problem 2
Round 1
Round 2
Example solution:
SAMI is good at collecting large data and survey class' preference on inclusivity.
Solution 1
Solution 2
Problem 3
Round 3
Solution 3
BRAINSTORMING: KEY INSIGHTS
Given the diverse nature of vision impairments, there is no one-size-fits-all solution for inclusive accommodations. An inclusive practice for a BLV student might not be inclusive for another BLV student.
Across feature #1 to #3, participants voiced concern over its generic suggestions and proposed two solutions:
-
The chatbot can let sighted students know its suggestions are generic and encourage them to ask a given BLV student's unique preference after initial interaction.
-
The chatbot can survey class preferences on inclusivity and make more personalized recommendations.
The chatbot can introduce errors in its fixes due to technical limitations. What if it produces inaccurate alt-texts?
For feature #4, participants voiced concern that a chatbot can produce errors as it fixes inaccessible materials to accessible and proposed one solution:
-
It can ask sighted students to check for its fixes, and this can further their knowledge of accessibility.
I revised each feature to reflect these solutions, as highlighted in pink highlights.
Chatbot feature 1
Chatbot feature 2
Everyone can ask a chatbot about inclusive words and phrases they can use in class.
A chatbot should add that the student should directly ask the student in question of their unique preference.
Based on the data collected at the beginning of the semester, a chatbot learns when students use non-inclusive language based on what they posts and sends a weekly summary of the non-inclusive language used in the class and shows alternatives.
Chatbot feature 3
Chatbot feature 4
Everyone can ask a chatbot whether class materials and teaching styles are accessible to students of diverse abilities.
A chatbot should add that the student should directly ask the student in question of their unique preference.
A chatbot can analyze when class materials and posts are not accessible to some students. It lets students know why their materials are not accessible, invites them to correct, and makes them accessible.
REFLECTION
FUTURE WORK
LESSONS LEARNED
-
Our team submitted our research report to TLS so that we can start a cross-functional collaboration with their software engineers to develop a chatbot that embodies our recommendations.
-
Share study materials with participants ahead of time. I was surprised that my participants proposed realistic and technically feasible chatbot features in the interviews. This was possible because I shared the interview questions one week in advance, allowing them to reflect on the ideas.
-
Empower visually impaired participants to take notes on their computers. Many online collaboration platforms (e.g., Google Docs) are not screen-reader-friendly, so having your visually impaired participants write their ideas locally and copy and paste the notes into Google Docs allows for interactive collaborative sessions with sighted participants.