Theia: Opportunistic and Real-time Smartphone Photo Search

What is Theia?

Theia is a first-of-its-kind system for real-time search of photos in smartphones. Photos taken by smartphone users may opportunistically contain content of real-time value to others. To realize such value, one can search the smartphone crowds as a distributed data center with each smartphone holding a unique set of data. However, unlike servers in a real data center, smartphones are resource-constrained, individually owned, and have unindexed photo content, despite simple tags. These properties render conventional "crawl-and-fetch"-based search methods inefficient. Theia addresses these properties with three innovations. First, it allows search users to compose a query and execute it inside selected smartphones to analyze photo content. Second, Theia allows a search user to expand the search scope incrementally in order to control search cost, or incremental search. It further exploits the locality of relevant results to reduce search cost. Finally, Theia reduces the energy consumption of search by dynamically partitioning the query execution between the smartphone and cloud, or partitioned search.

What does Opportunistic Mean?

The two photos above show examples of opportunistic and real-time value of photos. The photo on the left is from a real story: A recent CNN news article reported the arrest of a thief whose act of stealing appeared in the background of a family photo. Notice the role of opportunism. The person in the background of the photo would normally have been ignored by the photographer. It is only the context of a theft that makes this person interesting. The second photo is from an imaginary story: Consider a child who gets lost in a large crowd such as the Macy's Thanksgiving Day parade in Manhattan. Searching for the child is a daunting task in these circumstances. Imagine the police geocasting a request to all smartphone owners in the vicinity, asking to search photographs that were taken recently. The background of one of those photographs, such as this photo, may reveal the lost child, soon leading to her rescue. Note the role of opportunism here. The users whose photos are searched were completely unaware of this potential future use. But because of the richness of the sensed data, there are potentially "uninteresting" aspects of the photo (e.g., small child in the corner of the photo) that prove to be very important in hindsight. These two examples also show the importance of searching in real-time. The longer a child is lost, the greater the chances that she will be harmed. The longer a thief is undetected, the more likely that he will escape far from the scene of the crime.