Mr. Arvind Gopu Profile

Arvind Gopu

SCA Group Manager at Pervasive Technology Institute

SPIE Involvement:

Author

Area of Expertise:

Interdisciplinary Computer Science , User Interface Design , Project Management , Software Design , High Performance Computing , System Administration

Websites:

Company Website

One of my objectives is to enable and foster collaboration between the Computer Science domain, and other sciences, and to enable more efficient scientific research in those domains. I've been heavily focused on design/development/deployment/maintenance of mostly self-contained software systems [sometimes referred to as science gateways] that use University hardware resources, business logic contributed by domain experts, and are accessible via web portals.

Thus far, I've been lucky enough to be able to work on general HPC/Grid Computing projects (biology, geology, etc.), then serve as a technical lead in the OSG Operations group (physics), and most recently to manage the ODI Portal, Pipeline, and Archive project (astronomy)!

Proceedings Article | 2 March 2020 Presentation + Paper

Scalable Quality Assurance for Neuroimaging (SQAN): automated quality control for medical imaging

Arvind Gopu, Michael Young, Andrea Avena Koenigsberger, Raymond Perigo, John West, Meenakshisundaram Paramasivam, Soichi Hayashi, Robert Henschel

Proceedings Volume 11318, 1131807 (2020) https://doi.org/10.1117/12.2549722

KEYWORDS: Databases, Imaging systems, Medical imaging, Scanners, Neuroimaging, Human-machine interfaces, Medical research, Quality testing methods

Read Abstract +

Proceedings Article | 6 July 2018 Paper

Toward sustainable deployment of distributed services on the cloud: dockerized ODI-PPA on Jetstream

Raymond Perigo, Arvind Gopu, Michael Young, Yuanzhi Bao

Proceedings Volume 10707, 107072X (2018) https://doi.org/10.1117/12.2313647

KEYWORDS: Clouds, Data archive systems, Web services, Imaging systems, Computer security, Software development, Sustainable technology

Read Abstract +

The One Degree Imager - Portal, Pipeline and Archive (ODI-PPA) - a mature and fully developed product - has been a workhorse for astronomers observing on the WIYN ODI. It not only provides access to data stored in a secure archive, it also has a rich search and visualization interface, as well as integrated pipeline capabilities connected with supercomputers at Indiana University in a manner transparent to the user. As part of our ongoing sustainability review process, and given the increasing age of the ODI-PPA codebase, we have considered various approaches to modernization. While industry currently trends toward Node.js based architectures, we concluded that porting an entire legacy PHP and Python-based system like ODI-PPA with its complex and distributed service stack would require too significant an amount of human development/testing/deployment hours. Aging deployment hardware with tight budgets is another issue we identified, a common one especially when deploying complex distributed service stacks. In this paper, we present DockStream (https://jsportal.odi.iu.edu), an elegant solution that addresses both of the aforementioned issues. Using ODI-PPA as a case study, we present a proof of concept solution combining a suite of Docker containers built for each PPA service and a mechanism to acquire cost-free computational and storage resources. The dockerized ODI-PPA services can be deployed on one Dockerenabled host or several depending on the availability of hardware resources and the expected levels of use. In this paper, we describe the process of designing, creating, and deploying such custom containers. The NSF-funded Jetstream led by the Indiana University Pervasive Technology Institute (PTI), provides cloud-based, on-demand computing and data analysis resources, and a pathway to tackle the issue of insufficient hardware refreshment funds. We briefly describe the process to acquiring computational and storage resources on Jetstream, and the use of the Atmosphere web interface to create and maintain virtual machines on Jetstream. Finally, we present a summary of security refinements to a dockerized service stack on the cloud using nginx, custom docker networks, and Linux firewalls that significant decrease the risk of security vulnerabilities and incidents while improving scalability.

Proceedings Article | 6 July 2018 Presentation + Paper

ImageX 3.0: a full stack imaging archive solution

Michael Young, Arvind Gopu, Raymond Perigo

Proceedings Volume 10707, 107071A (2018) https://doi.org/10.1117/12.2313684

KEYWORDS: Image processing, Image compression, Control systems, Astronomy, Human-machine interfaces, Interfaces, Visualization, Data archive systems, Image visualization, Observatories

Read Abstract +

Over the past several years we have faced the need to develop a number of solutions to address the challenge of archiving large-format scientific imaging data and seamlessly visualizing that data—irrespective of the image format—on a web browser. ImageX is a ground-up rewrite and synthesis of our solutions to this issue, with a goal of reducing the workload required to transition from simply storing vast amounts of scientific imaging data on disk to securely archiving and sharing that data with the world. The components that make up the ImageX service stack include a secure and scalable back-end data service optimized for providing imaging data, a pre-processor to harvest metadata and intelligently scale and store the imaging data, and a flexible and embeddable front-end visualization web application. Our latest version of the software suite called ImageX 3.0 has been designed to meet the needs of a single user running locally on their own personal computer or scaled up to provide support for the image storage and visualization needs of a modern observatory with the intention of providing a ’Push button’ solution to a fully deployed solution. Each ImageX 3.0 component is provided as a Docker container, and can be rapidly and seamlessly deployed to meet demand. In this paper, we describe the ImageX architecture while demonstrating many of its features, including intelligent image scaling with adaptive histograms, load-balancing, and administrative tools. On the user-facing side we demonstrate how the ImageX 3.0 viewer can be embedded into the content of any web application, and explore the astronomy-specific features and plugins we’ve written into it. The ImageX service stack is fully open-sourced, and is built upon widely-supported industry standards (Node.js, Angular, etc.). Apart from being deployed as a standalone service stack, ImageX components are currently in use or expected to be deployed on: (1) the ODI-PPA portal serving astronomical images taken at the WIYN Observatory in near real-time; (2) the web portal serving microscopy images taken at the IU Electron Microscopy Center; (3) the RADY-SCA portal supporting radiology and medical imaging as well as neuroscience researchers at IU.

Proceedings Article | 8 August 2016 Paper

RabbitQR: fast and flexible big data processing at LSST data rates using existing, shared-use hardware

Ralf Kotulla, Arvind Gopu, Soichi Hayashi

Proceedings Volume 9913, 99131U (2016) https://doi.org/10.1117/12.2233527

KEYWORDS: Data processing, Astronomy, Calibration, Large Synoptic Survey Telescope, Imaging systems, Image processing, Data storage, Data archive systems, Data acquisition, Sensors

Read Abstract +

Proceedings Article | 8 August 2016 Paper

StarDock: shipping customized computing environments to the data

Michael Young, Soichi Hayashi, Arvind Gopu

Proceedings Volume 9913, 991318 (2016) https://doi.org/10.1117/12.2233081

KEYWORDS: Data storage, Astronomy, Data processing, Imaging systems, Human-machine interfaces, Data archive systems, Phase modulation, Computing systems, Observational astronomy, Charge-coupled devices

Read Abstract +

Proceedings Article | 8 August 2016 Paper

ImageX: new and improved image explorer for astronomical images and beyond

Soichi Hayashi, Arvind Gopu, Ralf Kotulla, Michael Young

Proceedings Volume 9913, 99134E (2016) https://doi.org/10.1117/12.2233425

KEYWORDS: Image visualization, Image compression, Performance modeling, Astronomy, Imaging systems, Visualization, Electron microscopy, Image analysis, Radiology, Human-machine interfaces

Read Abstract +

The One Degree Imager - Portal, Pipeline, and Archive (ODI-PPA) has included the Image Explorer interactive image visualization tool since it went operational. Portal users were able to quickly open up several ODI images within any HTML5 capable web browser, adjust the scaling, apply color maps, and perform other basic image visualization steps typically done on a desktop client like DS9. However, the original design of the Image Explorer required lossless PNG tiles to be generated and stored for all raw and reduced ODI images thereby taking up tens of TB of spinning disk space even though a small fraction of those images were being accessed by portal users at any given time. It also caused significant overhead on the portal web application and the Apache webserver used by ODI-PPA. We found it hard to merge in improvements made to a similar deployment in another project's portal. To address these concerns, we re-architected Image Explorer from scratch and came up with ImageX, a set of microservices that are part of the IU Trident project software suite, with rapid interactive visualization capabilities useful for ODI data and beyond. We generate a full resolution JPEG image for each raw and reduced ODI FITS image before producing a JPG tileset, one that can be rendered using the ImageX frontend code at various locations as appropriate within a web portal (for example: on tabular image listings, views allowing quick perusal of a set of thumbnails or other image sifting activities). The new design has decreased spinning disk requirements, uses AngularJS for the client side Model/View code (instead of depending on backend PHP Model/View/Controller code previously used), OpenSeaDragon to render the tile images, and uses nginx and a lightweight NodeJS application to serve tile images thereby significantly decreasing the Time To First Byte latency by a few orders of magnitude. We plan to extend ImageX for non-FITS images including electron microscopy and radiology scan images, and its featureset to include basic functions like image overlay and colormaps. Users needing more advanced visualization and analysis capabilities could use a desktop tool like DS9+IRAF on another IU Trident project called StarDock, without having to download Gigabytes of FITS image data.

Proceedings Article | 8 August 2016 Paper

Trident: scalable compute archives: workflows, visualization, and analysis

Arvind Gopu, Soichi Hayashi, Michael Young, Ralf Kotulla, Robert Henschel, Daniel Harbeck

Proceedings Volume 9913, 99131H (2016) https://doi.org/10.1117/12.2233111

KEYWORDS: Visual analytics, Imaging systems, Human-machine interfaces, Data archive systems, Data processing, Visualization, Computing systems, Astronomy, Data archive systems, Computing systems, Prototyping, Databases

Read Abstract +

The Astronomy scientific community has embraced Big Data processing challenges, e.g. associated with time-domain astronomy, and come up with a variety of novel and efficient data processing solutions. However, data processing is only a small part of the Big Data challenge. Efficient knowledge discovery and scientific advancement in the Big Data era requires new and equally efficient tools: modern user interfaces for searching, identifying and viewing data online without direct access to the data; tracking of data provenance; searching, plotting and analyzing metadata; interactive visual analysis, especially of (time-dependent) image data; and the ability to execute pipelines on supercomputing and cloud resources with minimal user overhead or expertise even to novice computing users. The Trident project at Indiana University offers a comprehensive web and cloud-based microservice software suite that enables the straight forward deployment of highly customized Scalable Compute Archive (SCA) systems; including extensive visualization and analysis capabilities, with minimal amount of additional coding. Trident seamlessly scales up or down in terms of data volumes and computational needs, and allows feature sets within a web user interface to be quickly adapted to meet individual project requirements. Domain experts only have to provide code or business logic about handling/visualizing their domain's data products and about executing their pipelines and application work flows. Trident's microservices architecture is made up of light-weight services connected by a REST API and/or a message bus; a web interface elements are built using NodeJS, AngularJS, and HighCharts JavaScript libraries among others while backend services are written in NodeJS, PHP/Zend, and Python. The software suite currently consists of (1) a simple work flow execution framework to integrate, deploy, and execute pipelines and applications (2) a progress service to monitor work flows and sub-work flows (3) ImageX, an interactive image visualization service (3) an authentication and authorization service (4) a data service that handles archival, staging and serving of data products, and (5) a notification service that serves statistical collation and reporting needs of various projects. Several other additional components are under development. Trident is an umbrella project, that evolved from the One Degree Imager, Portal, Pipeline, and Archive (ODI-PPA) project which we had initially refactored toward (1) a powerful analysis/visualization portal for Globular Cluster System (GCS) survey data collected by IU researchers, 2) a data search and download portal for the IU Electron Microscopy Center's data (EMC-SCA), 3) a prototype archive for the Ludwig Maximilian University's Wide Field Imager. The new Trident software has been used to deploy (1) a metadata quality control and analytics portal (RADY-SCA) for DICOM formatted medical imaging data produced by the IU Radiology Center, 2) Several prototype work flows for different domains, 3) a snapshot tool within IU's Karst Desktop environment, 4) a limited component-set to serve GIS data within the IU GIS web portal. Trident SCA systems leverage supercomputing and storage resources at Indiana University but can be configured to make use of any cloud/grid resource, from local workstations/servers to (inter)national supercomputing facilities such as XSEDE.

Proceedings Article | 18 July 2014 Paper

ODI - Portal, Pipeline, and Archive (ODI-PPA): a web-based astronomical compute archive, visualization, and analysis service

Arvind Gopu, Soichi Hayashi, Michael Young, Daniel Harbeck, Todd Boroson, Wilson Liu, Ralf Kotulla, Richard Shaw, Robert Henschel, Jayadev Rajagopal, Elizabeth Stobie, Patricia Knezek, R. Pierre Martin, Kevin Archbold

Proceedings Volume 9152, 91520E (2014) https://doi.org/10.1117/12.2057123

KEYWORDS: Data archive systems, Calibration, Astronomy, Visualization, Data processing, Decision support systems, Visual analytics, Human-machine interfaces, Image processing, Image quality

Read Abstract +

Proceedings Article | 18 July 2014 Paper

Integrating the ODI-PPA scientific gateway with the QuickReduce pipeline for on-demand processing

Michael Young, Ralf Kotulla, Arvind Gopu, Wilson Liu

Proceedings Volume 9152, 91522U (2014) https://doi.org/10.1117/12.2056763

KEYWORDS: Calibration, Data archive systems, Image processing, Sensors, Imaging systems, Decision support systems, Astronomy, Data processing, Signal processing, Capacitors

Read Abstract +

Showing 5 of 9 publications

Keywords/Phrases

Search In:

Publication Years