1 Introduction

The multi-projector-mapper (MPM) is an open-source software framework for 3D projection mapping using multiple projectors. It contains a basic rendering infrastructure, and interactive tools for projector calibration. For calibration, the method given in Oliver Bimber and Ramesh Raskar’s book Spatial Augmented Reality, Appendix A, is used.

The framework is the outcome of the “Projections of Reality” cluster at smartgeometry 2013, and is to be seen as a prototype that can be used for developing specialized projection mapping applications. Alternatively, the projector calibration method alone could also be used just to output the OpenGL projection and modelview matrices, which then can be used by other applications. In addition, the more generic code within the framework might as well serve as a starting point for those who want to dive into ‘pure’ Java / OpenGL coding (e.g. when coming from Processing).

Currently, at ETH Zurich’s Future Cities Laboratory we continue to work on the code. Among upcoming features will be the integration of the 3D scene analysis component, that was so far realised by a separate application. Your suggestions and feedback are welcome!

1 Source Code Repository @ GitHub

The framework is available as open-source (BSD licensed). Jump to GitHub to get the source code:

https://github.com/arisona/mpm

The repository contains an Eclipse project, including dependencies such as JOGL etc. Thus the code should run out of the box on Mac OS X, Windows and Linux.

2 Usage

The framework allows an arbitrary number of projectors - as many as your computer allows. At smartgeometry, we were using an AMD HD 7870 Eyefinity 6 with 6 mini-displayport outputs, where four outputs were used for projection mapping and one as control output:

2.1 Configuration

The code allows opening an OpenGL window for every output (for projection mapped scenes, windows without decorations are used, and they can be placed accordingly at full screen on the virtual desktop):

public MPM() {
  ICalibrationModel model = new SampleCalibrationModel();
  scene = new Scene();
  scene.setModel(model);
  scene.addView(new View(scene, 0, 10, 512, 512, "View 0", 0, 0.0, View.ViewType.CONTROL_VIEW));
  scene.addView(new View(scene, 530, 0, 512, 512, "View 1", 1, 0.0, View.ViewType.PROJECTION_VIEW));
  scene.addView(new View(scene, 530, 530, 512, 512, "View 2", 2, 90.0, View.ViewType.PROJECTION_VIEW));
  ...
}

Above code opens three windows: one control view (which contains window decorations), and two projection views (without decorations). The coordinates and window sizes in this example are just samples and need to be adjusted for a concrete case (i.e. depending on virtual desktop configuration).

2.2 Launching the Application & Calibration

Once the application launches, all views show the default scene. The control view in addition shows the available key strokes. Pressing “2” switches to calibration mode. The views will now show the calibration model, with calibration points. Note that in calibration mode, all projection views will blank, unless their window is activated (i.e. by clicking into the window).

For calibration, 6 circular calibration points in 3D space need to be matched to the their physical counterparts. Thus, when using the default calibration model, which is a cube, a physical calibration rig corresponding to the cube needs to be used:

The individual points can now be matched by dragging them with the mouse. For fine tuning, use the cursor keys. As soon as the 6th point is selected, the scene automatically adjusts.

For first time setup, there is also a ‘fill mode’ which basically projects a white filled rectangle (with a cross hair in the middle) for each projector. This allows for easy rough adjustment of each project. Hit “3” to activate fill mode.

Once calibration is complete, press “S” to save the configuration, and “1” to return to navigation / rendering mode. On the next restart, press “L” to load the previous configuration. When in rendering mode, the actual model is shown, which by default is just a fake model, thus a piece of code that is application specific. The renderer includes shadow volume rendering (press “0” to toggle shadows), however the code is not optimised at this point.

Note that it is not necessary to use a cube as calibration rig - basically any 3D shape can be used for calibration, as long as you have a matching 3D and physical model. Simply replace the initialisation of your ICalibrationModel with an instance of your custom model.

The following YouTube video provides a short overview of the calibration and projection procedure:

https://youtu.be/bWULiD3BWzY

3 Code Internals and Additional Features

The code is written in Java using the JOGL OpenGL bindings for rendering, and the Apache Commons Math3 Library for the matrix decomposition. Most of the it is rather straightforward, as it is intentionally kept clean and modular. Rendering to multiple windows makes use of OpenGL shared contexts. Currently, we’re working on the transition towards OpenGL 3.2 and will replace the fixed pipeline code.

In addition, the code also contains a simple geometry server, basically listening via UDP or UDP/OSC for lists of coloured triangles, which are then fed into the renderer. Using this mechanism, at smartgeometry, we build a system consisting of multiple machines doing 3D scanning, geometry analysis and rendering, by sending geometry data between them using the network. Note that this is prototype code and will be replaced with a more systematic approach in future.

4 Credits & Further Information

Concept & projection setup: Eva Friedrich & Stefan Arisona. Partially also based on earlier discussions and work of Christian Schneider & Stefan Arisona.

Code: MPM was written by Stefan Arisona, with contributions by Eva Friedrich (early prototyping, and shadow volumes) and Simon Schubiger (OSC).

Support: This software was developed in part at ETH Zurich’s Future Cities Laboratory in Singapore.

A general overview of the work at smartgeometry'13 is available at the “Projections of Reality” page.

Authors: Wei Zeng, Chi-Wing Fu, Stefan Arisona, Huamin Qu

Abstract: Massive amount of movement data, such as daily trips made by millions of passengers in a city, are widely avail- able nowadays. They are a highly valuable means not only for unveiling human mobility patterns, but also for assisting transportation planning, in particular for metropolises around the world. In this paper, we focus on a novel aspect of visualizing and analyzing massive movement data, i.e., the interchange pattern, aiming at re- vealing passenger redistribution in a traffic network. We first formulate a new model of circos figure, namely the interchange circos diagram, to present interchange patterns at a junction node in a bundled fashion, and optimize the color assignments to respect the connections within and between junction nodes. Based on this, we develop a family of visual analysis techniques to help users interactively study interchange patterns in a spatiotemporal manner: 1) multi-spatial scales: from network junctions such as train stations to people flow across and between larger spatial areas; and 2) temporal changes of patterns from different times of the day. Our techniques have been applied to real movement data consisting of hundred thousands of trips, and we present also two case studies on how transportation experts worked with our interface.

https://www.youtube.com/watch?v=_QWnA1k2ZrU

Title: Visualizing Interchange Patterns in Massive Movement Data
Authors: Wei Zeng, Chi-Wing Fu, Stefan Arisona, Huamin Qu
Journal: Computer Graphics Forum
Publisher: Wiley
Year: 2013
Volume: 32(3)
Pages: 271-280
DOI: 10.1111/cgf.12114
Link: https://onlinelibrary.wiley.com/doi/10.1111/cgf.12114/abstract

The energy scenario planning tool, developed by Eva Friedrich, has been presented at the ADDIS 2050 conference in Addis Ababa on October 9th and 10th 2012. The basic concept of the work is to create an intuitive and playful tool that allows for collaborative, interactive exploration of possible energy supply and consumption scenarios in Ethiopia. The users can move in time and compare different approaches to energy development in their effect: centralized hydro power, less-centralized wind power with local support, and completely decentralized solar power for rural, disconnected areas - thereby obtaining a better understanding of the effects of mid- to long-term energy planning.

The tool’s interface is built in a minimal style in order to keep the focus on the relevant points. It supports multi-touch surfaces, and borrows concepts from game engines, e.g. physics based animations for connecting networks.

https://www.youtube.com/watch?v=ymLicY8RHVY

The conference ADDIS 2050 – an alternative pathway into Ethiopia’s future – was held on October 9th and 10th 2012 at the campus of the Ethiopian Institute of Architecture, Building Construction and City Development EiABC in Addis Ababa. Addis Ababa belongs to the fastest growing urban centers in the world. Migration from the rural areas as well as massive redevelopment strategies of the City Government put the African capital under enormous pressure. Infrastructural deficiencies, water and energy shortages, environmental hazards and mobility challenges question the current modus operandi in place.

The Green Forum Ethiopia under the leadership of Heinrich Boell Foundation in Addis Ababa commissioned the Chair of Architecture and Construction Dirk E. Hebel at FCL Singapore in collaboration with the Ethiopian Institute of Architecture, Building Construction and City Development EiABC to invent an alternative “green” scenario for the city of Addis Ababa in the year 2050. The conference concentrated on the issues of Energy, Mobility, Cultural and Social Space, Housing and Information. The event was visited by more than 500 people and arose immense interest from the media as well as the City Administration. As guest of honor, the Swiss Ambassador H.E. Dominik Langenbacher attended the conference as well as delegates from several federal ministries and the UN-Habitat. In the meantime, the Department of Masterplanning and Vision of the City Adeministration invited the speakers to present the work in their offices.

Team FCL Singapore: Dirk Hebel, Felix Heisel, Marta Wisniewska, Alireza Javadian, Gerhard Schmitt, Stephen Cairns, Remo Burkhard, Eva-Maria Friedrich, Matthias Berger, Stefan Mueller Arisona, Ludger Hovestadt, Jorge Orozco, Alex Erath, Max Hirsh, Sonja Berthold, Ying Zhou, Edda Ostertag, Naomi Hanakata, Lindsey Ann Sawyer, Cheryl Song, Noor Faizah Binte Othman, Kevin Lim, Amanda Tan
Team EiABC: Joachim Dieter, Bisrat Kifle, Addis alem Fekele, Tewedaj Eshetu, Yosef Teferri, Eyob Wedesu
Team Green Forum/Heinrich Boell Foundation: Patrick Berg, Ayele Kebede, Jonas

Software Design and Development: Eva Friedrich
Energy Modelling: Matthias Berger
Principal Investigators: Gerhard Schmitt & Stefan Arisona
Institution: Future Cities Laboratory, ETH Zürich
Period: August / September 2012

Pascal Müller, Stefan Arisona, Simon Schubiger, Matthias Specht, since 2001

Corebounce is a collective of artists and scientists with the common goal of mediating between arts, science, and technology. We maintain a number of new media projects and our own multimedia software research platform, Soundium. We are organised as a non-profit association and collaborate with a number of partners from education, in particular with ETH Zürich, and industry.

We regularly tour as the Scheinwerfer Live Visuals Collective, where we create the visual experience at various electronic music events since 2001. Our live-composited and sound-driven visuals are designed to emphasise the theme of the event as well as taking into account the architectural framework. Not coincidentally, we have been labelled as “Club Scientists”: As stated above, our performances are deeply influenced by the momentary state of the Soundium research software platform. At the same time research is typically induced by specfic artistic performance goals.

Scheinwerfer has performed at some of the coolest locations around the globe, supporting world-class DJs and musicians like Jeff Mills, Rush, Miss Kittin, Dave Clarke, Josh Wink, Mouse on Marks, Jimi Tenor and many more.

Agent based simulation tools such as MATSim and MITSIM allow us to achieve efficient and accurate predictions of crowd behavior, thereby increasing our understanding of urban systems and assist in urban planning. However, output produced by such simulation platforms are difficult to communicate to stakeholders such as government agencies and the general public due to their technical nature.

In Project Metrobuzz we present a novel and visually striking method of displaying and interacting with large amounts of simulation data in a way that is both scientifically interesting as well as understandable for a broader audience. In order to achieve this, we generalised trip origin-destination information in terms of series of line segments in 3D space that allow spatiotemporal queries to quickly retrieve relevant data. In addition, we implemented interactive tools running on mobile devices (e.g. tablets) to define such queries in an intuitive manner.

W. Zeng, C. Zhong, A. Anwar, S. Arisona, and I. V. McLoughlin. 2012. MetroBuzz: Interactive 3D Visualization of Spatiotemporal Data. International Conference on Computer and Information Sciences (ICCIS), Kuala Lumpur, Turkey, June 12 – 14.

Project: Ongoing research, software prototype.
Institution: ETH Zurich’s Future Cities Laboratory
Location: Singapore
Period: Since 2011
Concept and programming: Stefan Arisona (ETH Zurich)
Programming: Zeng Wei (NTU), Afian Anwar (MIT), Christian Schneider (ETH Zurich)
Acknowledgments: Kay Axhausen and Alex Erath of the Future Cities Laboratory’s Mobility and Transportation group for providing the base data