Mobile Interfaces for Low Literacy and Homeless Users

Mobile Phones are one of the first pieces of technology adapted by remote villages in the middle of the jungle or desert and one of the last pieces of technology given up by people who have lost everything including somewhere to live. This research looked at how the fringes of population used mobile technology.
For research conducted in an Agile fashion using sprints, please see my portfolio.

Deliverables

Patent Filed

  • Mark Bailey, Jim Christensen, Catalina Danis, Jason Ellis, Robert Farrell, “Asynchronous Message Tags,” Patent filed April 2, 2011.
  • Mark Bailey, Jim Christensen, Catalina Danis, Jason Ellis, Robert Farrell, “Data Session Synchronization with Phone Numbers,” Patent filed April 2, 2011.
  • My Contributions

  • Development and testing of mobile SMS interface.
  • Development of backend mobile to server interaction.
  • App Store for Low-End Cell Phones through SMS

    Native African with cell phone

    The Social Computing Research Group created a concept which was built around the idea that there are billions of phones in use in developing economies out there that can not hold specialized apps like the iPhone or even Java games. Instead, the goal was to create a system easy enough that people without programming experience could create applications that would could be used by the local community. We have found that the best people to decide what they need is the community itself. Applications were hosted on our servers that could be interacted with through SMS. Created application ranged from a simple business card where people could find out the next time the salesman would be in the town to a party-line message service where everyone in the group got the shared messages.

    Community Resource Messenger (Joint Study with Georgia Tech)

    Info flowing between mobile, large screen and computers

    To see more about this interface please see the corresponding entry in my portfolio. The design and use of Information and Communication Technologies (ICTs) has evolved beyond its workplace origins to include people living at the margins of contemporary society. Increasingly in the U.S., as in the developing world, people who do not own personal computers to access the Internet are instead using inexpensive mobile phones for access. State and federal agencies in the U.S. have recognized the benefits of these devices, creating programs to provide free phones and airtime to citizens who meet income guidelines (see, e.g., the SafeLink program). IBM, along with others, is engaging in opportunities created by the explosive growth of mobile phones. While much focus in developed nations is on “smart phones” (and small form-factor computers such as netbooks and tablets), the highest growth and adoption rates around the world are for basic cell phones. Because these devices have such limited interfaces (speaker, microphone, small LCD, 12-key keypad), innovative approaches must be used to enable users access to information and services enabled by the Internet and wireless networks. The Social Computing Group is engaged in a joint research study with Christopher Le Dantec and Keith Edwards of the Georgia Institute of Technology. The focus of the collaboration is to explore how ICTs can enhance communication for homeless clients and their case workers. The Community Resource Messenger project approaches the homeless and their case workers as two interacting ‘publics’ – each with shared constraints and unique desires, whose members must be fully aware, and in charge of their communication choices. The solution leverages a web-based communications center for case workers to send messages to their clients’ phones, and basic cell phones for clients to produce user-generated content. The system also includes a large digital message board installed in the shelter for both case workers and clients to broadcast and share information with each other in a public way.

    Deliverables

    Papers Published

  • “A Tale of Two Publics: Democratizing Design at the Margins.” Proceedings of ACM DIS 2010 Conference on Designing Interactive Systems. August 16-20 2010.
  • “Publics in Practice: Ubiquitous Computing at a Shelter for Homeless Mothers.” Proceedings of ACM CHI 2011 Conference on Human Factors in Computing Systems. May 7-12, 2011.
  • My Contributions

  • Development and testing of mobile SMS interface.
  • Development and testing of front-end TV interface.
  • Development of backend mobile to server interaction.
  • Deliverables

    Papers Published

  • “Social Computing Applications for the Next Billion Users.” Position paper for Designing Future Mobile Software for Underserved Users Workshop at ACM CSCW 2008 Conference on Computer Supported Cooperative Work, November 2008.
  • “Mobile Applications for the Next Billions: A Social Computing Perspective.” Position paper for Human-Centered Computing in International Development (HCI4D) at ACM CHI 2009 Conference on Human Factors in Computing Systems, April 2009.
  • “Mobile Applications for the Next Billions: A Social Computing Application and a Perspective on Sustainability.” Position paper for 2nd Workshop on Innovative Mobile Technology & Services for Developing Countries (IMTS-DC 09) in conjunction with International Conference on Computing and ICT Research (ICCIR), Kampala, Uganda, July 30-31 2009.
  • A Picture and a Thousand Worlds: Visual Scaffolding for Mobile Communication in the Developing World” in Handheld Computing for Mobile Commerce: Applications, Concepts and Technologies. IGI Publications.
  • “A Picture and a Thousand Words: Visual Scaffolding for Mobile Communication in the Developing World.” In Handheld Computing for Mobile Commerce: Applications, Concepts and Technologies. Hu, Wen-Chen and Zuo, Yanjun (eds.) IGI Publications 2010.
  • Patents filed

  • Mark Bailey, Jim Christensen, Catalina Danis, Thomas Erickson, Robert Farrell, Wendy Kellogg, “Social Bookmarking for Interactive Services,” Patent filed March, 2010.
  • Mark Bailey, Jim Christensen, Catalina Danis, Thomas Erickson, Robert Farrell, Wendy Kellogg, “Digital Media Voice Tags in Social Networks,” Patent filed March, 2010.
  • My Contributions

  • Development and testing of mobile SMS interface.
  • Development and testing of front-end Android interface.
  • Development of backend mobile to server interaction.
  • Mobile Interaction for Users with Low Literacy

    Three apps created which could be used low literacy

    As part of the Next Billions Research Group research into multi-sensory modes of human interaction using mobile phones can support these sharing information and activities. For example, photos, drawings, and graphics can provide context for and serve to organize aural information and conversely, spoken comments can guide users to appropriate parts of complex visual information. Picture Discussions is a mobile computing application framework that supports asynchronous conversation anchored by pictures. We are exploring several issues in the design of mobile social computing interfaces, including how to create visual representations of digital speech that afford effective interactions between mobile phone users, how to coordinate peer production of content, and how to organize question and answer exchanges. The use of pictures and sound affords the possibility of communicating without text, and thus reaching users who for a variety of reasons — lack of access to computers, insufficient literacy, or disabilities — have been unable to benefit from information technology. Picture Discussions augments speech with three types of visual component: comment proxies, pictorial contexts and visual controls. Comment proxies are visual representations of digital speech that depict various types of meta-information, such as the identity of the speaker and the length of the comment. Pictorial contexts are diagrams or photographs that provide a background for a particular conversation; pictorial contexts serve both to represent the conversation as a whole, and allow comment proxies to take on additional meaning by virtue of their location with respect to the pictorial background. Visual controls are visual user interface components for controlling the system, for example, a message play button. We have implemented Picture Discussions prototypes on several devices: Web browsers, Android phones, and camera phones connected to an interactive voice response system. Each client connects to a centralized Web service that stores photos, audio clips, and links between them. The Android phone implementation of Picture Discussions leverages the phone’s touch screen, high resolution display, and built-in microphone and speaker. The camera phone implementation of Picture Discussions accepts picture messages sent over the telecommunications network using the Multimedia Messaging Service (MMS) and supports audio posting through the voice channel. Users can be notified of new discussion postings by Short Message Service (SMS). The Web implementation uses Flash Media Server to create a rich internet application able to display multiple discussions at once. Picture Discussions Web service transcodes photos and audio recordings to formats appropriate for each device, enabling cross-device discussion. For example, one user can take a picture on a camera phone and call the Picture Discussions phone number to provide a spoken introduction to this picture, then another user might discover this photo on their Android phone’s high resolution display and tap on the picture to respond to the first user’s posting. The limitations of using mobile devices to access rich structured multimedia content by users with limited literacy skills exposes human-computer interaction challenges that are key to enabling broad access to information by the coming billions of information technology users.