The International Virtual Observatory Alliance plays a pivotal role in making astronomy data FAIR (Findable, Accessible, Interoperable, Reusable). Virtual Observatory standards are now mature and underpin data discovery, usage and interoperability from most major observatories around the world, including those managed by NASA, ESA, ESO and many others. New facilities such as Vera Rubin Observatory and SKAO are currently being built with these standards fully integrated, and they are central to their future operations. The VO is an enabling excellence through interoperability among both the service implementations and in the data exchange layer, and continues to demonstrate success year after year. In this talk I will give an overview of the importance of the VO to the modern observatory, highlighting its successes and discussing some of its upcoming challenges.
From the collection of proposals, telescope and instrument control, driving archives, and simulating and processing data, research software and data engineering underpins almost every process in the advancement of astronomy. And yet this has at times been an afterthought, receiving little attention or funding. Some institutes have always valued software engineering, and the community is slowly coming to realize that the discipline must be supported so that the best science can be carried out. We will discuss software engineering careers within astronomy, and the problems we must tackle if we wish to continue to attract excellent minds to our field from a diverse range of backgrounds. Not just attract but retain them, in an era where flexible working conditions are no longer a perk of academia, and salary disparity between our institutions and industry is larger than ever. We describe the AAO’s Research Data and Software section’s work to provide a stable career path for its engineers, and to attract a portfolio of work which both satisfies the requirements of the instrumentation and data projects, and the needs of our team to have a challenging, creative, and fulfilling work life.
KEYWORDS: Stars, Point spread functions, Sensors, Large Synoptic Survey Telescope, Calibration, Data modeling, Equipment, Signal to noise ratio, Modeling, Edge detection, Galaxy evolution, Galaxy groups and clusters
We present the phase one report of the Bright Star Subtraction (BSS) pipeline for the Vera C. Rubin Observatory’s Legacy Survey of Space and Time (LSST). This pipeline is designed to create an extended PSF model by utilizing observed stars, followed by subtracting this model from the bright stars present in LSST data. Running the pipeline on Hyper Suprime-Cam (HSC) data shows a correlation between the shape of the extended PSF model and the position of the detector within the camera’s focal plane. Specifically, detectors positioned closer to the focal plane’s edge exhibit reduced circular symmetry in the extended PSF model. To mitigate this effect, we present an algorithm that enables users to account for the location dependency of the model. Our analysis also indicates that the choice of normalization annulus is crucial for modeling the extended PSF. Smaller annuli can exclude stars due to overlap with saturated regions, while larger annuli may compromise data quality because of lower signal-to-noise ratios. This makes finding the optimal annulus size a challenging but essential task for the BSS pipeline. Applying the BSS pipeline to HSC exposures allows for the subtraction of, on average, 100 to 700 stars brighter than 12th magnitude measured in g-band across a full exposure, with a full HSC exposure comprising ≈100 detectors.
Despite Python being the preferred programming language of choice for most astronomers, building or extending data reduction pipelines in the language can be problematic. A common approach is to write Python functions or classes as wrappers, calling individual pipeline recipes underneath, but this does not scale well with increasing pipeline complexity. Data management is also fraught since housekeeping code must be written to carefully handle input and output products between recipes. We have addressed these issues by creating an extensible pipeline development framework that leverages the Python bindings for the ESO Common Pipeline Library (PyCPL) toolkit. Pipeline recipes can be defined in a regulated manner using existing ESO pipeline recipes or new Python recipes compliant with ESO standards. Users can easily build their own pipeline workflows for execution by the PyCPL companion package PyEsorex. The ability to define Python recipes offers a powerful means to extend existing ESO pipelines or develop entirely new pipelines. An overview of the framework is presented along with an illustrative MUSE pipeline workflow.
The Two-degree Field (2dF) facility of the Anglo-Australian Telescope (AAT) continues to take regular observations with millions of spectra collected over its lifetime. While individual projects have used the 2dFdr data reduction package to reduce and publish their own spectra, the majority of 2dF spectra are relatively inaccessible inside raw files located in the AAT archive. Here we introduce our 2dFdr Pipeline As a Web Service (PAWS) system that allows users to reduce 2dF-AAOmega observations on demand from the upgraded AAT archive. Without downloading data or installing 2dFdr, users can select science observations and reduction parameters before jobs are submitted for reduction. The system uses docker-py and Celery to robustly execute the reduction workflows, while a custom job tracking system keeps users informed of job progress. Data products may be downloaded and individual spectra can be viewed interactively. We intend to support additional instruments in the future.
Astronomers routinely have to collate heterogeneous observational data for one or several targets from a variety of online resources. Traditionally this process of data aggregation can be time consuming and error prone, especially if multiple telescope archives or data centres are searched individually. To streamline this task we have developed Data Central’s Data Aggregation Service (DAS), an interactive web application that leverages Aladin Lite to display images and catalogues resulting from multiple online service queries of a given target. The modern asynchronous Python design allows these queries to be sent simultaneously and individual results are quickly displayed as soon as they are received. The DAS also hosts Pipeline As a Web Service (PAWS) data reduction workflows that may be triggered on demand. The DAS can effectively unlock science from unreduced data in telescope archives and may help manage the massive data volumes expected from next generation facilities.
Data Central is the AAO's flagship virtual observatory service, providing a central repository for Anglo-Australian Telescope (AAT) and UK Schmidt observations, survey-team derived data products and documentation. The system brings together catalogues, imaging, spectra and data cubes from dozens of surveys, providing an intuitive interface to query, explore and cross-match data sets of national and international significance. In this presentation, we brie y introduce the current services Data Central offers including: a publication-quality image cutout service, SQL query, a Single Object Viewer bringing together data products (IFS cubes, spectra, catalogues, photometry) from crossmatched sources across multiple surveys, VO services, schema browser and team-curated documentation via an in-house CMS.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.