Interactive tables or large displays are great for exploring and interacting with urban data, for example in urban observatories or exhibitions. They turn working with maps, visualizations, or other urban information into a fun and social experience. However, such interactive tables or large displays are also expensive – much too expensive for schools, public libraries, community centres, hobbyists, or bottom up initiatives whose budgets are typically small.

We therefore asked ourselves how we could use the countless tablets and smart phones that are typically idling away in our pockets and bags to compose a low-cost but powerful multi-user and multi-device system from them. How can we enable users to temporarily share their personal devices for creating a joint cross-device system for a social and fun data exploration?



Video of HuddleLamp demo applications

Our result is HuddleLamp, a desk lamp with an integrated low-cost depth camera (e.g. the Creative Senz3d by Intel). It enables users to compose interactive tables (or other multi-device user interfaces) from their tablets and smart phones just by putting them under this desk lamp.

hud1
Technical setup of HuddleLamp with an integrated RGB-D camera (Tracking region: 1.0×0.6m).

HuddleLamp uses our free and open source computer vision software to continuously track the presence and positions of devices on a table with sub-centimetre precision. At any time, users can add or remove devices and reconfigure them without the need of installing any apps or attaching markers. Additionally, the users’ hands are tracked to detect interactions above and between devices.

All this information is provided to our free and open source Web API that enables developers to write cross-device Web applications. These applications can use all this information to become “spatially-aware”. This means that the applications can react to how the devices are arranged or moved in space. For example, physically moving a tablet on a desk can also pan and rotate the content of the screen, so that each device appears to be a kind of peephole through which users can view a spatially-situated virtual workspace. When putting multiple tablets or phones side-by-side, these peepholes turn into one huddle or federation of devices and users can interact with them as if they were just one large display.

hud2

Multiple tablets side-by-side form a huddle of tablets

Peephole navigation and using multiple tablets as one tiled display.

HuddleLamp was created by Hans-Christian Jetter of the Intel ICRI Cities at UCL in London and Roman Rädle of the Human-Computer Interaction Group of the University of Konstanz together with colleagues from the UCL Interaction Centre.

Thanks to the great work of our student research interns Oscar Robinson (UCL), Jonny Manfield (UCL), and Francesco De Gioia (University of Pisa) who visited the ICRI during Summer 2014, we are happy to not only present HuddleLamp in a talk at the ACM ITS Conference 2014 but also to give a live demonstration there.

HuddleLamp is a first step towards a “sharing economy” for excess display and interaction resources in a city. We envision that in future cities users will be able to seamlessly add or remove their devices to or from shared multi-device systems in an ad-hoc fashion without explicit setup or pairing. Instead this will happen implicitly as a by-product of natural use in space, for example, by bringing multiple devices to the same room, placing them side-by-side on a table or desk, and moving them around as needed. Ideally, users will experience these co-located cooperating devices and reconfigurable displays as one seamless and natural user interface for ad-hoc co-located collaboration.

After having created our free and open source base technology, we are now looking at creating and studying examples for the visual exploration of urban data. Our goal is to enable citizens to create their own bottom up urban observatories for community engagement and activism in spaces such as schools, public libraries, community centres, or museums.

Further reading:

To learn how to build your own HuddleLamp and HuddleLamp applications, please visit: http://www.huddlelamp.org or join the HuddleLamp Facebook group.

The ITS paper on HuddleLamp is also available here.

St Andrews Summer School

From 8 July until 12 July, Lisa Koeman attended the SICSA Big Data Information Visualisation summer school at the University of St Andrews. The week was aimed specifically at PhD students, and just over 30 applications were selected. The attendees came from a number of countries and had a variety of backgrounds, including human-computer interaction and statistics.

The summer school had an initial focus on theory, with an increasing amount of practical work. The theoretical part consisted of talks by Professor Peter Triantafillou (University of Glasgow), Professor John Stasko (Georgia Tech) and Professor Sheelagh Carpendale (University of Calgary), among many others.

Not only were practical topics covered, including introductions to Hadoop and various visualisation tools, but also a more general background on small, medium and big data, programming models, network visualisation and interaction.

All attendees were assigned to groups, based on their interest in the different available datasets – which included Skyscraper flight data, UK migration data, the Enron mail corpus and various others. After a short introduction to the data, a sketching session was organised by Professor Sheelagh Carpendale, to allow each group to explore different methods of visualising.

The remaining part of the week was spent on the implementation of the visualisation ideas and final presentations were given to a jury on the last day of the summer school. The consensus among the attendees seemed to be that the week had been very intensive, but more importantly: very productive. A valuable experience, of great relevance to the work done on data analysis and visualisation by us at the ICRI Cities.

(Photo by Professor Aaron Quigley)

In their pursuit of a “natural” or “intuitive” interaction, researchers and designers in Human-Computer Interaction have created a multitude of post-WIMP (post-“Windows Icons Menu Pointer”) user interfaces and interaction techniques during the recent years. Examples range from “perceptual computing” with depth cameras for gesture/body tracking and simultaneous pen & multi-touch interaction to tangible displays for augmented reality or entire rooms equipped with display walls and interactive tables.

The workshop “Blended Interaction – Envisioning Future Collaborative Interactive Spaces” at CHI 2013 in Paris on Apr 28 that was organized by Christian Jetter with colleagues from Konstanz, Dresden, Edinburgh, St. Andrews and Hagenberg (see http://hci.uni-konstanz.de/blendedinteraction2013/) established “Blended Interaction” as a novel concept for understanding what makes interactive technologies “natural” or “intuitive”. In brief, “Blended Interaction” combines the virtues of physical and digital artifacts, so that desired properties of each are preserved while integrating computing power in a considered manner. In a world of Blended Interaction, computing is woven into the fabric of our natural physical and social environment (e.g. our cities) without being too obtrusive or disruptive.

Keynote speakers Robert Jacob (Tufts University), Michel Beaudouin-Lafon (Université de Paris-Sud) and Andy Wilson (Microsoft Research) gave exciting and inspiring talks about the theory, technology and vision of Blended Interaction. During the workshop, Christian also presented his vision of future self-organizing user interfaces that would be particularly appropriate for interacting in rapidly changing physical and social environments and usage contexts such as cities (slides: http://hci.uni-konstanz.de/downloads/blend13_jetter_slides.pdf, paper: http://hci.uni-konstanz.de/downloads/blend13_jetter.pdf).

ws1 ws2

Johannes Schöning (ICRI Cities, UCL & Hasselt University) and Hans-Christian Jetter (ICRI Cities, UCL) were invited to the European Commission’s Joint Research Centre (JRC) at Ispra, Italy for attending a workshop in the European Crisis Management Laboratory of the Institute for the Protection and Security of the Citizen (http://old.gdacs.org/cr/ECMLWorkshops/BigWallHCI/JRC-ECML-BigWallHCI-Workshop-Programme_FINALv3LQ.pdf).

ispra

A view into the European Crisis Management Laboratory.

Johannes and Christian presented their vision of collaborative interactions in future crisis rooms together with Harald Reiterer and Simon Butscher from the University of Konstanz. In a joint presentation they showed how different designs and technologies from their research in the field of Human-Computer Interaction and Information Visualisation (e.g., collaborative tangible search on tabletops, folding views and lenses for collaborative geographical visualization, user identification and hand tracking using RGB and IR cameras, curved displays) can be used to enable multi-user geospatial analysis of real-time data, e.g., twitter feeds.

ispra2

Different components for future crisis rooms developed by the University of Konstanz, University of Hasselt and UCL.

There was a lively exchange with the 40 workshop attendants from civil protection agencies, industry and academia about ideas and visions for future multi-user interaction with large displays in different application scenarios. For the research at ICRI Cities the event was particularly interesting to learn more about the many different data sources, sensor networks and simulation tools that are currently in use by civil protection practitioners to monitor, analyse, make sense and report about critical incidents on a global, regional or urban scale. By letting the workshop take place in the actual crisis room that is used to provide the President of the European Commission with situation reports, there was the opportunity to get a first-hand experience of typical tasks, tools and challenges. The inspiring event that was organized by Markus Rester from the Crisis Monitoring and Response Technologies (CRITECH) group at the JRC helped us to generate novel ideas about how to make data and simulations from a connected city accessible to analysts, policy makers, practitioners or city dwellers in a future “urban observatory”.

 

Netmob, a small conference dedicated to the study of mobility and network datasets (primarily sourced from mobile phones, although a couple of Foursquare based studies managed to sneak in), took place at the beginning of May at the MIT Media Lab. These kind of massive behavioural datasets paint a picture of human dynamics at an unprecedented level of detail, at city scale and beyond, and have proven invaluable for research in social sciences, statistical physics (of human behaviour), urban planning and computer science alike. Despite the fact that no official proceedings are published, other than a book of abstracts (pdf 40MB) and the D4D book (pdf 122MB), the quality of work was high, perhaps due to the big names on the organising committee.

The first day was a special session for the Orange D4D challenge in which around 150 teams of researchers tackled a wide range of development related problems using the anonymised and aggregated digital traces of 5 million Orange mobile customers in Cote d’Ivoire. It was great to see that Orange seem genuinely excited to be taking a lead in the ‘data philanthropy’ agenda, although they were also clearly concerned that research based on large scale personal data be conducted in a responsible manner, lest data sharing and extracting its value become politically and commercially impossible. For example, they urged researchers to be careful not to attract ‘big brother’ accusations, however misguided they may be, and to be sure to understand the local context before reaching potentially sensitive conclusions (e.g., do community detection algorithms reveal tribal divisions or merely economic regions?). UN Global Pulse also had a large presence at the conference, and were extolling the huge potential of large scale data analysis to help tackle global development issues – exciting times for the data scientist. Below I summarise some personal highlights of the work presented at D4D and Netmob.

Data 4 Development

In “AllAboard: a system for exploring urban mobility and optimizing public transport using cellphone data” Berlingo et al. (prize winners) develop a number of techniques for inferring public transport trips from people’s movement between cell towers (passive crowdsourcing), which is then fed into a system which optimises the transit network. This kind of system could potentially be applied to any city in the world, even more easily in cities that already capture trips with automated fare collection systems, such as London. However, an important consideration is that public transport routes are often planned in order to encourage trips in order to bolster activity in certain regions. Indeed, the network structure and cost of public transport will largely determine the OD matrix, so optimising based solely on existing flow patterns may not be enough.

In “Crowdsourcing Physical Package Delivery Using the Existing Routine Mobility of a Local Population”, McInerney et al. propose a novel method of package delivery which exploits people’s regular travel patterns. Taking inspiration from work on mobile ad-hoc networks, mobility data is analysed to determine the best path between participants to get a package to its destination, with minimal disruption to the participants route.

Mao et al. present a number of interesting findings including discovering a number of features of call activity that correlate with socioeconomic indicators around economic centres. Particularly interesting are correlations with the Gini coefficient, a measure of income inequality. They also find that a community detection algorithm delineates customers along boundaries between wealthier and poorer areas. In my own submission with Afra Mashadi and Licia Capra (which was mentioned as a ‘significant project’) we took a similar approach to mining the call data, and found a number of features which correlated strongly with poverty levels in different parts of the country. We then demonstrated how this might be used to produce poverty maps at a finer level of granularity.

First prize winners Lima et al. made full use of both communication and mobility patterns in “Exploiting Cellular Data for Disease Containment and Information Campaigns Strategies in Country-Wide Epidemics”. By comparing simulations of information diffusion over the communication network and infection spread over a physical contact network, they showed that information campaigns may be a more effective means of disease containment and prevention than physical quarantine efforts.

Mobility Modelling

The next two days of Netmob contained a lot of other interesting work using different data sets. Yang and Gonzales presented an extension to the radiation model of human mobility which includes a parameter to account for different scales. They showed that it performs better than the original radiation model and the doubly constrained gravity model, which has many more parameters to fit. Deville et al. showed that a simple scaling relationship exists between the exponents of mobility patterns and spatial-social networks, meaning that we could estimate the flow of traffic between areas from the flow of communication. An example of GPS mobility traces was given by Horanont et al., this time looking at visitor numbers in Japan. A nice 3D visualisation shows the numbers of visitors per building in the Odaibo area of Tokyo, something that certainly couldn’t be measured from CDR alone. Most haunting however, was an animation depicting the effect on mobility of the 2011 earthquake. The familiar flashes of movement across the city abruptly halted as the quake struck. Then individuals could be seen making long and slow walks home before the public transport network finally came back online late at night.

Crowdsensing

Salnikov and Lambiotte presented a brilliant example of incentivised crowdsensing in “Late For Good”. Smart phone users are offered a convenient service which automatically fills and submits delay claim forms to the Belgian rail operator in return for passively submitting regular GPS coordinates. Thus, what users lose in battery life, they gain in compensation and satisfaction. So far they have received around 600k data points, far from the billions often found in call detail records (CDR) datasets but with substantially increased spatial and temporal resolution.

Well-being

Back to Cote d’Ivoire but using data from a mobile operator other than Orange (and therefore not part of the D4D challenge), Gutierrez et al. present what might be known as the ‘top-up’ model of wealth. The reasonable, but as yet unvalidated, hypothesis is that top-up behaviour reflects the wealth of the phone user (i.e., frequent small top-ups = poor, infrequent large top-ups = rich). Using the model to produce a proxy wealth indicator they map the average and diversity of wealth of different regions in the country. The final (and to my knowledge, original) example of using CDR to map wealth/poverty at the conference came from Frias-Martinez et al. who presented “CenCell”. This is a system that takes as input, for a sample of census areas, ground truth socioeconimic level indicators and numerous mobility, social, and usage features of CDR data, and applies some advance machine learning techniques to build a classification model which can produce estimates for the remaining census areas. Posed as a binary classification problem (i.e., predict high or low socioeconomic level) and tested on data from a Latin American country the system was able to achieve up to 76% accuracy. Blumenstock and Toomet present an interesting study of ethnic segregation in cities, in the specific case of Estonia, which has a long history of in- and out-migration of Russians. Using language as a reliable proxy for ethnicity they are able to look in detail at social homophily and physical segregation, and more importantly the effect of migration and urbanisation on community integration.

Connected by Location

Toole et al. looked at the similarity between people’s movement patterns and how this similarity relates to social relations. Among their findings are that tie strength correlates with movement similarity and that individuals tend to share their most important locations with their top four social contacts. This means that movement similarity could be used as a proxy for a social network. Along similar lines, but this time using Foursquare data, Brown et al. develop a place-based model of social network formulation in cities, inspired by Feld’s theory of focused organisation which says that friendships often form around common foci. Their model reproduces many structural characteristics of real social networks. It is nice to see that a model based on location preference has similar results to those using mechanisms such as preferential attachment. It would be interesting to see if this approach was better at reproducing spatial characteristics of social networks such as aggregated tie strength between neighbourhoods within cities. This may also be significant for urban planners who want to think about the effects of the physical structure on promoting healthy social relationships.

Temporal Dynamics

Finally, Miritello et al. looked at medium-term temporal dynamics of communication patterns and produced several important results (see this blog post with visualisations and link to full paper). Notably, they found a high rate of decay among social ties – only 60% of a month’s social ties are present in the next month – meaning that a long period of observation (at least 6 months they suggest) is needed to gain an accurate picture of social relationships. Likewise, this means that a long period will be required to gain an accurate picture of flows between different parts of the city. They also found that individuals have a limited and fixed capacity for communication (although it differs between people), and reveal a distinction between ‘social keepers’, who maintain a fixed set of contacts, ‘social explorers’, who replace old contacts with new over time, and ‘social balanced’, whos rate of link birth/death is proportional to their capacity. Unexpectedly, simulations show that social keepers receive information in the network first, and given the connection between information diffusion and economic advantage, these new temporal properties of social networks may prove useful in modelling the well-being of neighbourhoods.

CHI_2013_Personal_Informatics_workshop

Prior to the main ACM SIGCHI Conference on Human Factors in Computing Systems, more commonly known as CHI, a variety of related workshops were held in Paris. Lisa Koeman took part in the Personal Informatics in the Wild: Hacking Habits for Health & Happiness workshop after her submission ‘Enabling Foresight and Reflection: Interactive Simulations to Support Behaviour Change’ was accepted.

The two-day Personal Informatics workshop was held at the Université Paris Dauphine and started off with 4-minute presentations from all attendees. Here, I presented the concept of ‘foresight’: the explicit visualisation of predictions. Such foresight can offer people insight into the consequences behaviour changes will have in the future. Current personal informatics tools mainly focus on providing users with visualisations of their previously collected data. Combined with goal-setting, (dis)incentives, and competitions or comparisons with other users, it is hoped users will become or remain motivated to change. Supporting people by enabling them to explore the effects of theoretical or actual behaviour changes has so far remained unexplored. If and how foresight can help will be addressed in the project around creating Interactive Visualisations of Urban Lifestyles.

The topics addressed by the attendees varied greatly, ranging from discussions on the ethical issues around persuading users, to designing a well-being monitoring application for residents of an elderly home. A complete listing of accepted papers can be found on the workshop’s website.

Following the presentations, the remaining time was spent on a “hackathon”. In the weeks before the workshop, groups were formed, and equipment of choice could be ordered (Arduinos, heart rate monitors, Jawbones, etc.). Groups were free to focus on any topic around personal informatics and deliverables of the workshop could range from paper prototypes and presentations to working applications.

My group focused on the issues around the long term usage of personal informatics tools, and how systems should adapt to both the individual’s setting and the time course of use. After filling up whiteboard after whiteboard with ideas, and exhausting all available post-its, we ended up creating a ‘Family Informatics Model’. We are planning to continue working on this idea, as will no doubt many of the other groups. The final workshop day ended with group presentations and a meetup at Quantified Self Paris for those interested. Despite flaky WiFi connections and a power outage, the Personal Informatics workshop proved to be a productive and very inspiring get-together.

My workshop submission can be found here and my presentation here.