Gaia finishes its RUN 1000 and celebrates the 10th anniversary of the Data processing and analysis Consortium (DPAC)
Credit: ESA/ATG medialab; background image: ESO/S. Brunier
The IDT software, coordinated and developed in part by the Institute of Cosmos Sciences (IEEC-UB), is executed on some 28 computing nodes at the Gaia Science Operations Centre (SOC, at ESAC, near Madrid), receiving raw data from the spacecraft through the Mission Operations Centre (MOC, in Darmstadt), the MIT (MOC-SOC Interface Task), and the DCS (Decompression and Calibration Service). It processes spacecraft attitude data, astrometric, photometric, and spectroscopic instrument data, Basic Angle Monitor data, Wave Front Sensor data, and a handful of ancillary data packet types. Additionally, it runs complex algorithms to determine “intermediate data”: refined attitude, basic angle variations, sky background, image parameters (fluxes, positions, quality, etc.) and a first cross-matching (identification of measurements in the reference on-ground catalogue). These algorithms require several instrumental calibrations determined either by IDT itself or by other DPAC systems.
All this must be done in real-time, on a daily basis throughout the whole mission, to feed the so-called First Look diagnostics and calibration system, as well as all the downstream Data Processing Centres (DPCs). Thus, IDT feeds the whole Gaia consortium.
This milestone has almost coincided in time with the 10th anniversary of the formation of the DPAC, on June 15. The Consortium currently consists of about 450 astronomers, engineers, programmers, and project management specialists distributed in about 25 countries (mostly European). Data processing is carried out in six European data processing centers.
Gaia_DPAC_map-20151116
The first CU meetings took place in early 2006 in anticipation of the formal start of DPAC. Figure 1 shows a group photo from the first meeting of the Coordination Unit in charge of the ‘Core Processing’. Well over 100 CU meetings have taken place since, as well as a similar number of smaller workshops, and countless telecons. Keeping a large consortium such as DPAC on track requires lots of communication!
The first DPAC consortium meeting, attended by about 200 DPAC members, took place last November in Leiden, The Netherlands. During the meeting the documentary ‘Gaia – Teamwork For a Billion Stars‘ was recorded, featuring interviews with a number of DPAC members. The next consortium meeting will be held in Sitges (Spain) in January 2017.
Institute of Cosmos Sciences (IEEC-UB) contribution
One of the most challenging aspect of Gaia is the data managing and reduction process to yield the final Gaia catalogue. Researchers at Institute of Cosmos Sciences (IEEC-UB) have been deeply involved since the very beginning (1998) in this task. Nowadays the ICCUB researchers have a leading role at the highest technological, scientific and management levels of the Gaia Data Processing & Analysis Consortium (DPAC).
Firstly, a significant part of the software system that performs the ingestion, data reduction, production and verification of the Gaia satellite data is being designed, implemented, tested and operated by our groups, using large supercomputing facilities such as CSUC, CNS/BSC and CESGA. Secondly, off-line processes for astrophysical parameters and photometric standardisation are now fully operational and will be adapted to the evolving behaviour of the instrument in terms of stray light, basic angle variations and contamination on the optics. Finally, the tasks around the Gaia archive will become critical in the next years to support the operations for four data releases, with increasing volume and complexity of the data, and the intensive associated scientific exploitation, including the implementation of several sophisticated archive tools. And last but not least, it is time for our own science exploitation of the real data that will be made public in the releases, because this is the primary goal of the participation in Gaia, to address the challenging science case.
More information at ESA website