This PDF file contains the front matter associated with SPIE Proceedings Volume 13137, including the Title Page, Copyright information, Table of Contents, and Conference Committee information.

Market-available automated microscopy systems are often unaffordable for research institutions, particularly in economically disadvantaged countries, limiting their access to advanced technologies. This work addresses this challenge by developing a cost-effective virtual microscopy and telemicroscopy system, aimed at creating a remote-controlled microscopy setup for analyzing digital samples with performance comparable to high-end equipment but at a reduced cost. The system includes a web platform for telemicroscopy, enabling remote control of the robotic stage and real-time viewing of the microscope camera. Additionally, a decision support system has been implemented, integrating AI-based models for identifying and classifying objects of interest in three histopathology use cases for breast cancer analysis: i) identification of tumor cells in H&E samples; ii) grading of Her2 samples; and iii) evaluation of Ki67 samples. These models enhance diagnostic capabilities, increasing productivity for experts and reducing manual workload. Sample virtualization and automatic processing simplify tasks for professionals, allowing remote participation in concurrent work sessions and streamlining processes for digital samples.

Multiple sclerosis is one of the critical autoimmune diseases that require early detection. The central nervous system (CNS) is demyelinated by multiple sclerosis (MS), which causes lesions in both the grey and white matter (GM and WM). However, the lesions in the WM are most frequently seen on routinely obtained clinical scans. Magnetic resonance imaging (MRI) is used to identify these lesions. As part of the clinical diagnostic process, the variation in lesion charge obtained in the various MRI sequences is an important criterion indicating disease progression in terms of volume and localization as well as providing information on the efficacy of therapy. Manual segmentation, while still being used, is not ideal due to its reliance on expert knowledge, time-consuming nature, and susceptibility to variations among different experts. To address these challenges, demonstrating a tremendous deal of promise to support the diagnostic procedure of professionals, deep learning is one such subject that focuses on developing sophisticated algorithms to automate diagnosis. This research compares two different deep learning techniques (PCAEClassifier, BetaVAEClassifier) for automatically analyzing brain MRI scans from multiple sclerosis (MS) patients. The goal is to identify these WMLs and classify them into two classes, the class that contains the MRIs with white matter lesions, and the other class is for MRIs that do not have these lesions, ultimately helping diagnose and assess this disease. The performance of each model was then evaluated using the metrics of accuracy, precision, recall, and F-score. It is evident that the choice of hyperparameters significantly impacts the model's performance. For the BetaVAEClassifier, the model's performance varied significantly with different beta values. With a beta (β) of 20, the model achieved an accuracy of 87.61%, maintaining a balanced performance with precision, recall, and F1-score hovering around 87%. Similarly, for the PCAEClassifier, the model's performance varied with different sparsity parameters (S). With S equal to 20, the model achieved the highest accuracy of 88.82%, accompanied by precision, recall, and F1-score of 90.38%, 86.40%, and 88.34%, respectively. These models' sensitivity to hyperparameters highlights the intricate balance required in model configuration, especially in tasks where precision and recall are crucial, such as anomaly detection. Particularly in medical diagnostics like multiple sclerosis (MS) classification, where accurately distinguishing between affected and unaffected individuals is paramount, the performance of these models holds significant promise. Leveraging their capabilities for anomaly detection in medical imaging, particularly in the context of MS classification, could lead to earlier and more accurate diagnoses, enabling timely intervention and improved patient outcomes.

This study builds upon our previous investigations into identifying and classifying white matter lesions on the brain surfaces of multiple sclerosis patients using both traditional machine learning and novel deep learning models. Our prior work demonstrated the effectiveness of these methods with high-resolution MRI modalities. In this paper, we evaluate the performance of these methodologies using a degraded dataset instead of high-resolution images. Specifically, we compared a traditional machine learning approach using K-Nearest Neighbors (KNN) with a deep learning approach employing ResNet-18 with transfer learning. The KNN-based approach achieved moderate performance metrics, with an accuracy of 71.15% for lesions and 72.22% for non-lesions. These results indicate a balanced performance; however, the algorithm's efficacy was significantly compromised by the compressed images, highlighting the limitations of traditional machine learning methods in handling lossy compression artifacts. In contrast, the ResNet-18 deep learning model demonstrated remarkable performance, achieving an overall accuracy of 94% on the compressed dataset. The model's success was further reflected in high sensitivity, specificity, precision, and F1-score metrics, all hovering around 94% to 95%. By leveraging transfer learning, the deep learning approach excelled even with compressed medical images, showcasing the potential of modern deep learning techniques in handling degraded datasets, and enhancing the disease diagnosis process.

This paper introduces an automated system comparing VGG16 and ResNet50 for dermatoscopic image processing and classification. This method utilized transfer learning and fine-tuning VGG16 and ResNet50 using the HAM10000 dataset. Random resampling balanced the dataset, optimizing models for accurate results with limited resources. We preprocessed images, performed data augmentation, modified the pre-existing models, and tuned the hyperparameters to increase the overall accuracy of both the models. Results demonstrate VGG16 and ResNet50 achieving 93.45% and 94.69% accuracy, respectively, showcasing the effectiveness of the proposed system in advancing early skin cancer intervention with deep learning techniques. Time comparison shows that VGG16 is 4 times faster than the other indicating the time-complexity of ResNet50 when skin lesion images are used for training.

This research introduces a novel approach to automate the classification of brain tumors in MRI images by leveraging three prominent deep learning models: VGG16, ResNet18, and DenseNet. The study harnesses datasets from Figshare, the SARTAJ, and Br35H, comprising 7023 human brain MRI images categorized into glioma, meningioma, no tumor, and pituitary classes. Data augmentation techniques are performed to improve the learning process, optimizing the models for discerning intricate patterns within the diverse tumor categories.

In contrast to conventional methods, the proposed methodology incorporates an advanced image enhancement algorithm. This algorithm involves a multi-step preprocessing approach, including applying a Kuwahara filter, a median filter, and a homomorphic sharpening filter. The primary purpose is to enhance the visibility of tumor structures in MRI images, effectively highlighting relevant features while mitigating unwanted details and noise.

This study adopts a comparative analysis strategy, independently employing VGG16, ResNet18, and DenseNet pretrained models to classify brain tumors. By evaluating and contrasting the performance of these models, we aim to identify the most adequate one in terms of accuracy and efficiency. Our findings demonstrate that implementing this methodology with deep learning architectures results in an outstanding classification accuracy of up to 95%.

This research brings significant progress to automated brain tumor classification by providing detailed insights into the strengths and limitations of various deep-learning models used in medical imaging. The results serve as a practical guide for future research, which could lead to the development of more accurate diagnostic tools. Specifically, the findings can help create targeted tools for diagnosing glioma, meningioma, no tumor, and pituitary classes, which could bring advancements in medical imaging technology, leading to more efficient neurology diagnostics.

While distributed version control systems offer a solid foundation for monitoring revision history, their effectiveness is hindered when dealing with digital media assets, which are often treated as opaque binary data. This makes it challenging to precisely track modifications and compromises storage efficiency. Despite this, a significant portion of embedded metadata within these files is actually textual in nature, though it remains unrecognized due to its integration into the binary structure. Moreover, alterations to the metadata and the underlying structure of metadata container formats, such as the JPEG Universal Metadata Box Format (JUMBF), go unnoticed during media rendering, further complicating the identification process. To address these issues, this paper proposes a solution that defines an asset decomposition and structured serialization scheme. This framework enables the individual tracking of subcomponents within media assets, facilitating more accurate version control and metadata management.

The proliferation of artificial intelligence (AI) has revolutionised various fields, including image processing and manipulation. While AI-driven image manipulation techniques such as DALL-E¹ and Stable Diffusion² offer unprecedented capabilities for creativity and visual enhancement, they also pose significant challenges in terms of authenticity, integrity, and misinformation. Advancements in AI Image modification have blurred the line between reality and fiction, raising concerns about the potential for misinformation, propaganda, and manipulation in various domains, including journalism, advertising, and social media. These techniques enable the creation of visually convincing yet falsified images which pose challenges for current state-of-the-art techniques such as early/late fusion³ which often struggle to discern subtle alterations made by AI algorithms and as such report poor detection results, necessitating the development of advanced detection methods capable of discerning AI manipulations.

This paper presents a dataset of images containing AI generated modifications and a method for the detection of AI manipulations in images. The dataset consists of over 14,000 AI manipulated images and their ground truth masks which indicate where on the images the manipulations take place. The modified images were created using a state-of-the-art text-guided generation method⁴ that can generate modifications within an image from a text prompt. This dataset facilitates the development of a new deep-learning based image manipulation detection model which can reliably determine the existence of edited content in an image and localise the area that has been modified.

The massive development of IoT, Big Data and other technologies has led to security concerns with respect to data protection. It has become imperative to develop solutions to protect our data, such as images, texts, and audios from unauthorized access. This work presents an encrypted image transmission scheme based on a chaotic dynamic configuration of two synchronized spherical chaotic attractors of 3 dimension in a parent-child topology. We synchronized the future evolution of the chaotic systems starting with different initial conditions using the Hamiltonian observer-based approach and then utilized the resulting phase space points as the pseudo-random numbers for securing image transmitted through the communication channel. The scheme designed is realized and implemented on the Multiprocessor System-on- Chip (MPSoC) platform by harnessing the easy and synthesizable programming features of Python with MPSoC. The image is transmitted through the state variables x1, x2, and x3, and analyzed using the two statistical techniques namely, information entropy and correlation analysis where the result shows the full recovery of the image that was transmitted through the state variables.

In recent years, significant advancements in face recognition have been witnessed thanks to the rapid development of artificial intelligence. Despite remarkable performance, predictions made by such techniques tend to be challenging to explain. Considering their wide applications in security-sensitive areas, it is essential to fully understand the decision-making process of AI-based face recognition techniques and make them more acceptable to society. Many studies have been dedicated to offering visual interpretation for face recognition systems’ decisions, such as generating similarity and dissimilarity saliency maps. One of the most promising approaches is based on the perturbation mechanism, which has demonstrated exceptional performance in highlighting similar regions between two matching face images. However, this type of method has shown to be less effective in identifying the dissimilar regions, which are particularly critical in the decision-making process for two nonmatching face images. Therefore, this study focuses on the specific problem of the perturbation-based mechanism when applied to the explainable face recognition task. In particular, we first thoroughly analyze the limitation of a perturbation-based method in generating dissimilarity saliency maps. Then, a new regularization technique is proposed to alleviate this problem, followed by experiments to validate its effectiveness.

JPEG Trust is a new international standard that responds to the pressing need of assessing trust in digital media assets. JPEG Trust provides a comprehensive framework addressing key elements such as provenance, authenticity, integrity, and copyright declaration. Built on top of established JPEG and other industry standards, the framework ensures compatibility across digital media ecosystems. This paper provides an overview of the JPEG Trust framework and demonstrates its potential to enhance the reliability and trustworthiness of digital media content.

There is little information in the literature about the specific impacts of image manipulation and misleading images in society. While some anecdotal references to qualitative factors exist, such factors are thinly covered in the literature, and estimates of quantitative, especially monetary, costs are even less available. That these costs are substantial is indicated by the 2019 study issued by cybersecurity firm CHEQ claiming fake news generally costs the global economy $78 billion annually, but the basis for such a figure has only been vaguely motivated. This paper explores potential impact factors relating to fake images that may provide a foundation for clarifying the quantitative impacts of misinformative online images in society. Models of quantitative costs are imperative to motivate governments and organisations to implement investment in new trust technologies as preventative measures, and give such organisations and institutions confidence that such expenditure is warranted and can deliver a beneficial return on investment. To overcome the sparsity of modelled information in this nascent field, a ‘scattergun’ approach was undertaken, in which computer science students were instructed to search online for examples of qualitative and quantitative costs of cybersecurity scam activities and fake images. 127 students responded, with a total of 341 examples, which were then grouped into broad categories of costs and separated into resultant and preventative costs. This paper further considers how such disparate information can inform quantitative modelling of costs to the global society and to individual organisations. Finally, the paper considers how adopting an authenticity framework such as the JPEG Trust standard might ameliorate such costs.

Mosquito-borne diseases affect over 3 billion people worldwide, resulting in more than 600,000 fatalities annually. The ability to precisely identify mosquito species is essential for predicting disease outbreaks, yet it is hindered by manual, subjective methods that demand specialized skills and hardware, restricting monitoring scalability. Climate change further aggravates this challenge by altering habitats. Citizen science, using smartphone-captured mosquito images, has the potential for scalable mosquito identification and tracking. However, smartphone images gathered by citizen scientists can contain varied backgrounds, multiple mosquitoes, or damaged mosquitoes, challenging classic image classification. Our approach employs object detection and classification to identify mosquito types from varied smartphone photos accurately. This study’s contributions include converting an image classification dataset into an object detection dataset with precise bounding boxes and breed classification, alongside utilizing two distinct datasets for model training and testing to establish the model’s generalizability. Our training dataset comprises 10,000 images, and we focus on the two most represented categories, Aedes albopictus and Culex quinquefasciatus, which are key vectors for diseases like Dengue, Zika, and West Nile virus. This dataset originates from the Mosquito Alert project, a collaborative citizen science initiative that engages volunteers to capture and submit mosquito photographs. These contributors come from a diverse geographical span, including Spain, the Netherlands, Italy, and Hungary, contributing to the dataset over a period from 2014 to 2022. The YOLOv8 model was optimized through hyperparameter tuning using the aforementioned dataset. This process resulted in the model achieving over 98% mean Average Precision (mAP) at 50% IoU (Intersection over Union) during validation. Subsequently, the refined model was evaluated using a secondary dataset (the Mosquito on Human Skin or MOHS) gathered by Malaysian researchers. In this testing phase, the model demonstrated a high accuracy, achieving a mAP50 value of 99%, indicating superior performance in identifying mosquito species from varied image inputs. Furthermore, cross-testing demonstrated that both models could generate mAP50 of over 98% on the other dataset, showing the generalizability of our approach. Comparisons with baselines using a ResNet101 model for image classification show that our approach outperforms a standard neural network approach for the real-world MosquitoAlert dataset and enables a generalization from the lab-based dataset not possible via the classification baseline.

The problem of detecting the vegetation index utilizing UAVs has been addressed in multiple articles in the literature, in which many special hardware and thermal or infrared cameras are adapted to improve its detection. The Vegetation Index is determined as a parameter calculated from the reflectance values at different wavelengths of the vegetation and is particularly sensitive to the vegetation cover. This article seeks to identify the vegetation index from its biophysical parameters. We help ourselves with artificial intelligence algorithms and machine learning algorithms. A semi-physical model was designed to estimate the ecosystem and establish the vegetation index correctly. The results will be validated by remote sensing. Finally, an ecological model will be developed to simulate the environmental impact on vegetation patterns and geographic plains. The proposed model successfully imitated the urban effect. Given these results, it was possible to predict better the impact of changing seasons in a defined geographic area.

Deep Neural Network (DNN) algorithms have become ubiquitous within the vision domain, encompassing various tasks, including object detection, segmentation, and classification. However, executing complex DNNs in realtime systems demands improved energy efficiency, runtime, and accuracy. Traditional embedded imaging designs, typically implemented on homogeneous architectures, face hardware limitations, prompting the need for heterogeneous computing architectures. These architectures combine CPUs, GPUs, FPGAs, and other accelerators, enabling applications to use the most efficient architecture for a given algorithm. The challenge lies in scheduling and partitioning algorithms across accelerators with different computing paradigms and tool-sets. This requires balancing computational power, memory bandwidth, and communication overhead. Effective scheduling involves considering task dependencies, resource availability, and synchronisation. Current deep learning libraries often target single architectures and lack mechanisms to intelligently partition sub-operations across the most suitable processors.

This paper introduces a scheduler for heterogeneous vision systems that finely partitions and maps suboperations of convolutional neural networks and image processing algorithms. Leveraging state-of-the-art compiler frameworks such as PyTorch, TVM, and ONNX, the proposed scheduler optimally distributes tasks across heterogeneous components. Experimental results show that the heterogeneous platform achieves on average 1.12× and 1.08× improvements in kernel runtime and energy consumption, compared to the best-performing discrete hardware counterparts, GPU and FPGA. The study demonstrates that partitioning algorithms based on their runtime and energy properties and optimally scheduling them improves energy and runtime efficiency compared to homogeneous components executing the complete algorithm.

The effectiveness of autonomous vehicles relies on clear visual input, which rain can significantly obstruct. Rain streaks degrade the quality of captured images and videos, affecting both user perception and the functionality of outdoor vision systems, such as those in autonomous vehicles. This visual degradation impacts the vehicle's ability to interpret its environment, increasing the risk of driving in rainy conditions. Researchers have responded to this challenge by developing various rain removal algorithms, ranging from single-image to video-based approaches, each with its own strengths and weaknesses. This research aims to develop two novel, efficient single-image rain removal algorithms that strike a balance between high performance and quick execution. The proposed algorithms will address the need for an effective de-raining technique suitable for real-time use in autonomous vehicles. By improving visibility in rainy conditions, this innovation will enhance the performance and safety of autonomous vehicles, contributing to advancements in the field. A survey involving ten image processing experts and professionals, who evaluated the results of both algorithms based on perceived quality and improvement, revealed that Algorithm 1 received a higher average rating (0.58) compared to Algorithm 2 (0.43). Although Algorithm 1 is slightly preferred based on average participant ratings, Algorithm 2's superior edge preservation and image sharpness make it more favorable for applications demanding high accuracy and detailed image retention. Overall, the project meets the demand for real-time rain removal in autonomous vehicles and provides valuable insights into the effectiveness of Algorithm 1 in de-raining images compared to Algorithm 2.

Food recognition plays a vital role in various domains including dietary monitoring, nutrition analysis and food service automation. Real-world food recognition is a challenging task, as the contents of a plate of food can be complex intermixed objects, making it difficult to define their individual structures. Currently, technology offers a wide range of feasible options for dietary assessment and image-based methods hold the capability of substituting for traditional methods such as food records, food frequency questionnaires, and 24-hour recalls, which can have low accuracy and provide unreliable results. Faced with these issues, deep learning methods have shown better accuracy and ability to identify ingredients and types of food compared to traditional approaches for image classification. However, many deep learning methods rely on powerful computational resources which have limitations in terms of cost, energy consumption, and size. The rapid evolution of embedded hardware systems has significantly influenced the domain of computer vision and offers promising solutions to these challenges.

This paper presents a method that utilises deep-learning methods for detection and segmentation that are optimised for resource-constrained embedded platforms. These networks are tailored to efficiently process food images while ensuring low latency and energy efficiency. Additionally, strategies such as model quantisation, pruning, and compression are employed to reduce the computational complexity and memory footprint, making them suitable for deployment on embedded devices with limited resources, such as a Raspberry Pi. The method consists of a custom recognition pipeline that makes use of YOLOv8¹ and EdgeSAM² approaches for detection and segmentation, which are trained on the foodSeg103³ dataset. The resulting system provides a fast, accurate way to recognise foods without requiring expensive, energy intensive hardware.

Despite tremendous advancement in computer vision, especially with deep learning, understanding scenes in the wild remains challenging. Even modern image classification models often misclassify when presented with out-of-distribution inputs despite having been trained on tens of millions of images or more. Moreover, training modern deep-learning classifiers requires a lot of energy due to the need to iterate many times over the training set, constantly updating billions of model parameters. Owing to problems with generalisability and robustness as well as efficiency, there is growing interest in computer vision to mimic biological vision (e.g., human vision) in the hope that doing so will require fewer resources for training both in terms of energy and in terms of data sets while increasing robustness and generalisability. This paper proposes a biologically plausible neuromorphic vision system that is based on a spiking neural network and is evaluated on the classification of hand-written digits from the MNIST dataset. The experimental outcome indicates improved robustness of the proposed approach over state-of-the-art considering non-digit detection.

While traditional on-orbit geometric calibration relies on comprehensive imaging parameters, such models are often unavailable for widely distributed remote sensing products. This limitation hinders the geometric accuracy of these images, impacting their usability for various applications. To address this challenge, we propose a novel approach that leverages rational polynomial coefficients (RPCs) to refine the geometric fidelity of remote sensing images. By employing RPCs, our method bypasses the need for a rigorous sensor model, making it applicable to a broader range of remote sensing data. This paper details the methodology and demonstrates its effectiveness in improving geometric accuracy.

Retinal Optical Coherence Tomography (OCT) plays a pivotal role in diagnosing ocular disorders by providing detailed imaging of retinal layers. However, the analysis process remains time-consuming, posing a challenge to its widespread use.

This study investigates the integration of Artificial Intelligence (AI) to streamline the analysis of OCT images. Employing Deep Learning (DL) models—VGG16, ResNet18, DenseNet—transfer learning, and data augmentation, the research aims to enhance OCT images, optimize disease recognition, and accurately classify CNV (Choroidal Neovascularization), DME (Diabetic Macular Edema), DRUSEN, and NORMAL pathologies.

The dataset undergoes preprocessing, resizing, and enhancement to refine the images. The DenseNet model achieved the highest test accuracy of 92.41% after 25 epochs, demonstrating its potential in efficiently diagnosing ocular pathologies through OCT images.

The advent of high-resolution satellite imagery has revolutionized remote sensing applications, offering unprecedented access and detail worldwide. However, the vast volume and complexity of this data pose significant challenges for traditional classification methods, necessitating advanced computational approaches. This research leverages the OpenEarthMap dataset, an extensive and diverse collection of high-resolution earth observation imagery, to improve landcover classification accuracy using deep learning techniques. We optimize a U-Net convolutional neural network architecture by analyzing different learning rates and variations on the OpenEarthMap dataset and choose this architecture because of its previous success in semantic segmentation tasks. Through a comprehensive methodology encompassing dataset preparation, preprocessing, network parameter experimentation, and model evaluation, this research seeks to push the boundaries of current landcover classification capabilities and contribute valuable insights to the field of remote sensing. Using a Unet on 3500 OpenEarthMap images, Adam optimizer, and image variation preprocessing, we found an f-score of 0.75, consistent with visual image interpretation. Future work involving additional images, a 4x image split implementation, and other models may improve this f-score and the associated semantic segmentation.

The image compression field is witnessing a shift in paradigm thanks to the rise of neural network-based models. In this context, the JPEG committee is in the process of standardizing the first learning-based image compression standard, known as JPEG AI. While most of the research to date has focused on image coding for humans, JPEG AI plans to address both human and machine vision by presenting several non-normative decoders addressing multiple image processing and computer vision tasks in addition to standard reconstruction. While the impact of conventional image compression on computer vision tasks has already been addressed, no study has been conducted to assess the impact of learning-based image compression on such tasks. In this paper, the impact of learning-based image compression, including JPEG AI, on computer vision tasks is reviewed and discussed, mainly focusing on the image classification task along with object detection and segmentation. This study reviews the impact of image compression with JPEG AI on various computer vision models. It shows the superiority of JPEG AI over other conventional and learning-based compression models.

Reference Picture Resampling (RPR) is a powerful tool that allows improving video coding efficiency of next generation codecs like Versatile Video Coding (VVC) or Enhanced Compression Model (ECM). This feature is well designed to support frame changing resolution without inserting an instantaneous decoder refresh (IDR) or intra random access picture (IRAP). Video streaming and low delay scenarios can take advantage of RPR to ensure a smooth frame-based bit-rate adaptation, compared to traditional techniques that can generate bitrate leaps.

This paper proposes an encoder method to select the picture resolution change parameters effectively depending on the video signal characteristics. The picture resolution change decision is based on a low complexity neural network, and it is performed before the encoding process without RD-score computations making this approach suitable for real time and low delay implementation.

The experiments under Random Access (RA) and All Intra (AI) configurations of the VVC Test Model (VTM-21.1) show that the proposed method can bring luma BD-rate gain improvement of 1.46% and 0.95% respectively compared to the VVC Test Model anchor.

JPEG XS is a lightweight, low-latency image coding standard for transmission of video streams over IP; sim- ilar to many other coding standards, it does only define the codestream syntax and the decoding algorithm, but not the transport mechanism. For this, IETF defines in RFC 9134 an RTP payload format for JPEG XS for transmission over UDP packets. As long as encoder and decoder are part of the same local area network, error-free transmission can be assumed, but this does not hold for wide-area or wireless networks were packet loss may disrupt the transmission.

While multiple protocols exist to ensure error-free transmission in such networks, many of them depend on packet retransmission which impacts the latency of the overall system and by that one of the unique design aspects of JPEG XS.

In this work, we equip a JPEG XS encoder and decoder with foreward error correction according to SMPTE ST 2022-5 which is able to mitigate such errors while also keeping the latency under control. We report on the outcome of an experiment where we measured image quality as a function of error rate and also provide estimates on the additional latency due to the error correction layer.

Evaluating the visual quality of distorted images is a crucial task in the field of image compression, as artifacts may significantly impair the appeal and fidelity of images, therefore reducing the user experience. As assessing the visual quality of images through subjective visual quality experiments is often not feasible, objective image quality metrics are considered to be very attractive alternatives. Traditional objective image quality metrics, such as the Peak Signal-to-Noise Ratio and Structural Similarity Index, have long been used to assess compression artifacts. However, due to the complexity of human perception, estimated objective visual quality scores often diverge from their subjective counterparts. Recent advancements in deep learning have led to the development of learning-based metrics that promise to estimate the perceived visual quality of images with better accuracy. While learning-based methods have demonstrated enhanced performance compared to conventional methods on a number of datasets, their generalization performance across different quality ranges and artifacts has not been assessed yet. This paper presents a benchmarking study of conventional and learning-based objective image quality metrics while focusing solely on image compression artifacts. The experimental framework includes five source images with various contents compressed with five legacy and recent compression algorithms at four different quality levels, specifically focusing on the high-quality range. Results indicate that, in many cases, learning-based metrics present a higher correlation with human visual perception when compared to conventional methods, highlighting the potential of integrating such metrics in the development and refinement of image compression techniques.

With the constant increase in video resolution and frame rate, notably those required for immersive applications, there is a need for efficient and reliable coding technologies suitable for very high visual quality delivery with very low latency. Immersive applications can use the low latency and high bandwidth throughput of 5G networks for increased mobility. JPEG XS is a low-complexity coding standard that can be implemented with very low latency. It is designed to provide visually lossless quality while offering compression efficiency, making it suitable for immersive applications that rely on video content. This paper reports a quality evaluation of omnidirectional video using 360° test sequences coded with JPEG XS. A subjective quality experiment was performed using an alternating double-stimulus method in a VR environment, where subjects could freely commute between the reference and distorted video. Test sequences were encoded at five different bitrates, ranging from 0.35 bpp to 2 bpp. These bitrates are suitable for real-time high-resolution video transmission over 5G networks. It was concluded that JPEG XS provides an effective low-latency solution suitable for high-quality immersive applications using 5G networks.

This paper introduces a methodology for evaluating quality of experience in mixed immersive communication systems. It focuses on assessing the impact of advanced 3D capture techniques, immersive eye-sensing light field displays, and efficient compression mechanisms in mixed setups where terminals offer different imaging modalities. The paper methodically investigates the influence of the above technologies on user experience in various communication scenarios. By employing a quantitative assessment, the primary objective is to develop methods for comprehensive evaluation of how such immersive technologies affect perceived visual quality, presence, engagement, and overall satisfaction and preference compared to traditional video communication techniques. The experimental design incorporates a series of tests where participants interact through state-of-the-art immersive communication setups, followed by detailed feedback sessions to gauge their experiences. Through this approach, the study seeks to uncover the nuances of user satisfaction in immersive environments and identify the key factors that enhance the overall quality of peer-to-peer communication.

Using DNA molecules for data storage presents a compelling solution to the ever-growing demand for efficient and sustainable data storage systems. DNA offers notable advantages in terms of storage density, longevity, and energy efficiency. This has made the development of effective coding and compression techniques for DNA-based storage a critical research area within signal processing. One particularly challenging aspect is the encoding of multimedia content, such as images, for storage in DNA. JPEG DNA, a recent standardization effort led by the JPEG Committee, addresses this challenge by integrating both source and channel coding. The source coding focuses on data compression, while the channel coding ensures error resilience and accommodates the biochemical constraints of the DNA medium. In this paper, the impact of integrating learning-based source coding into the JPEG DNA framework is explored. This study reveals promising improvements in performance by replacing traditional image compression techniques with a learning-based approach, highlighting the potential for further advancements in DNA-based data storage.

Emerging 5G technologies bring various new opportunities for the media sector. In particular, they allow the incorporation of ultra-high resolution video formats and immersive augmented, virtual, and extended reality content into low-latency streaming applications while providing a reliable and high-quality user experience. In this paper, we focus on streaming 8K immersive content in a transmedia scenario and validate the feasibility of efficient, cost-effective solutions by measuring the added value brought by the latter. This is done using various key performance indicators in the framework of a European innovation project called 5GMediaHUB, with a scenario focusing on Interactive Digital Narratives. The main story is presented linearly to the user on the first screen, relying on an encoded video stream with marked frames. Each marked frame prompts the user to interact with the story on a secondary screen. The user is finally immersed in a Virtual Reality experience by completing a quiz on the second screen. The quality of the transmitted video is a critical requirement in this scenario, as well as a very low jitter and packet loss. Overall, a very low latency is required to allow effective and satisfactory interaction.

The current scale of online video streaming requires hardware accelerated video transcoding solutions. Historically, hardware solutions have been excellent at offloading the computationally intensive tasks from CPUs, but often came with the penalty of being inflexible and not quickly adaptable to emerging market trends. We are presenting an architecture, which maintains all the benefits of hardware acceleration but also adds an unparalleled level of programmability and flexibility. This architecture supports a wide spectrum of markets ranging from ultra-low latency encoding all the way to high quality video on demand (VOD) markets with only firmware changes. These capabilities are achieved by a strategic combination of built-in hardware acceleration components and many embedded CPUs that have full control over the video encoding pipeline flow. This architecture not only provides deterministic timing, which is critical for ultra-low latency transcoding, but it also offers flexibility and programmability allowing robust product roadmaps through simple firmware updates.

Google is working on a new video codec standard with multiple other industry members within Alliance of Open Media (AOM). Within Google, especially YouTube is actively driving advancements in the AV1 and AVM video codecs to enhance streaming quality and efficiency for diverse video including live and high production value content as well as user-generated content (UGC). Efforts include customizing the AV1 codec for UGC, optimizing quality/bitrate/compute tradeoffs, and developing hardware encoding/decoding support within YouTube's data centers to support AV1 at scale. To accelerate adoption, YouTube works to increase AV1 corpus coverage, expand device compatibility, and contributes to the Alliance for Open Media (AOM) for the ongoing algorithmic improvement of AV1 and AVM and the complexity reduction in both encoder and decoder. To ensure the practical feasibility of AVM, we assess hardware complexity and propose methods to reduce it. YouTube also prioritizes reducing AV1/AVM encoder complexity using approaches like machine learning-based partition search pruning. YouTube led the collective efforts to modify existing tools by chairing HW Subgroup within the AOM group, and developed statistics (bitstream) analysis tools to expedite the development of AVM. Furthermore, YouTube developed a new flow which can enable practical implementation of ML based tools. There are few ML based tools proposed for AVM which includes CNN based loop restoration filter. YouTube’s ML implementation flow reduced the complexity of that tool dramatically. This multi-pronged strategy has led to the successful integration of AV1 within Google, and we strive for faster adoption of AVM in the ecosystem.

In this paper, we present a methodology for benchmarking the coding efficiency and energy efficiency of software and hardware video transcoding implementations. This study builds upon our previous work, which focused on software encoders such as x264, x265, libvpx, vvenc, and SVT-AV1. We have since added a closed-source video software encoder implementation, EVE-VP9, as well as Meta’s MSVP VP9 encoder as a hardware representative, and expanded the test set to include a wider variety of test content in our analysis. To ensure a fair comparison between software and hardware encoders, we normalize the video encoding efficiency to energy used in watt-hours. Our proposed test methodology includes a detailed description of the process for measuring compression efficiency and energy consumption. We summarize limitations of our methodology and identify future opportunities for improvement.

In the video streaming service for VOD use case, an important task is to transcode user uploaded videos into multiple encoded bitstreams at different encoding bitrates and encoding resolutions, which allows the client player to leverage ABR (adaptive bitrate) algorithm to select the bitstream segments based on its available bandwidth. In this workflow, the key decision needs to be made is to determine the optimal encoding bitrates and encoding resolutions for every video at each quality or bitrate target in a ABR ladder. To tackle this challenge, an efficient two-stage convex hull based dynamic optimization framework was recently proposed. In this two-stage processing flow, two different encoders, or encoder presets can be used to construct the convex hull to improve the computation efficiency. In this work, we study the cross codec encoding parameter prediction problem in the two-stage system. We first formulate the prediction into an optimization problem, then propose two methods towards this optimization with validation results. We also discuss some potential directions that can further improve the results.

Rapid development and deployment of GPU based computation has led to an improvement in diffusion generation of video and images. Further, a rapid reduction in the effective cost of compression using NNC techniques provides opportunities to compress images and videos in new ways. The overall structure of diffusion based generative video and images is leveraged to take advantage of the compressed latent to lower overall compression costs and latency. This paper presents an architecture to compress a latent for transmission and reduce overall latency and cost as compared to alternatives using traditional Codecs or NNC on the raw image. It explores a proof of concept based on image compression of a latent. It further presents computational cost, quantitative and perceptual quality, and latency for this architecture as compared to the alternatives.

This paper presents a method to simplify content-aware encoding for streaming services using AI, to enhance viewing experience and efficiency in bandwidth-limited environments. Unlike traditional encoding, which uses fixed bitrates, this AI-driven approach optimizes the bitrate based on the content's complexity, significantly reducing the necessary computational steps and bitrates. It is achieved by predicting an optimized Adaptive Bitrate (ABR) ladder through minimal encoding steps and lightweight analysis, resulting in substantial bitrate savings and streamlined workflow. The approach also fits well with the trend of adopting video ASICs in data centers, further enhancing cost-effectiveness and scalability.

Advanced AI and new compression standards are needed to improve the viewing experience and reduce service costs, but the explosion in computational complexity is a significant barrier to adoption. This paper proposes AI algorithms and corresponding hardware accelerators for super-resolution and perceptual quality optimization. Super-resolution is for video upscaling and visual quality enhancement, while perceptual quality optimization is a pre-process to improve the coding efficiency of the encoders. Video ASICs for data centers include hardware decoders and encoders with high throughput to handle large amounts of data. The proposed algorithm is designed with dedicated hardware accelerators to maximize the efficiency of on-chip resources. These advances are essential to balancing high-quality streaming services with operational efficiency.

The use of DNA molecules as a storage medium has been recently proposed as a solution to the exponentially increasing demand for data storage, achieving lower energy consumption while offering better lifespan and higher information density. The nucleotides composing the molecules can be regarded as quaternary symbols, but constraints are generally imposed to avoid sequences prone to errors during synthesis, archival, and sequencing. While the majority of previous works in the field have proposed methods for translating general binary data into nucleotides, others have presented algorithms tailored for specific data types such as images as well as joining source and channel coding into a single process. This paper proposes and evaluates a method that integrates DNA Fountain codes with state-of-the-art compression techniques, targeting the storage of images and three-dimensional point clouds. Results demonstrate that the proposed method outperforms previous techniques for coding images directly into DNA, putting forward a first benchmark for the coding of point clouds.

Karhuen-Loeve Transform (KLT) is a valuable tool in many applications, but its computation is not exactly trivial. Generally, it requires finding the solution of an eigenvector problem, and with general types of inputs, the typical path forward is to use iterative numerical methods. Such methods are usually complex. In some cases, KLTs allow approximations by sinusoidal transforms – DCT-II likely the best-known example, but the number of such cases is limited, and usually constrained to very simple (1-st order) processes. However, as we will show in this paper, for some short sizes, KLTs can still be computed analytically, with only mild assumptions about the structure of the covariance matrix. For example, we show analytic solutions for arbitrary real symmetric 3x3 covariance matrixes. With symmetric 3-diagonal and some special cases of 5-diagonal matrices the solutions can also be found. In the end, we discuss a few possible applications of such transforms for image and video coding.

We review the history of the development of one of the most iconic tools in image and video coding – the zigzag scan. Despite its simplicity, we will show that its development was a non-trivial process that took several years, multiple iterations, and multiple ideas that eventually led to the formation of its final "zigzag" shape. Remarkably, we also discover that early variants of the zigzag scan appeared before the invention of the DCT, intra-predictors, and many other techniques in image and video coding algorithms. It is one of the oldest and most fundamental techniques in this context. This paper also traces the evolution of image and video codec architectures over the last six decades and brings examples of uses of the zigzag scan in modern-era image and video coding standards.

We review the history of the development of transform-based image and video codecs and re-construct logical chains that led to the inventions of DCT, zigzag scan, adaptive coding, and hybrid DPCM + transform-based architecture. We also review the subsequent evolution of this architecture and explain the reasoning behind multiple transform choices in modern video codecs (HEVC, VVC, etc.), in-loop filters, etc. Finally, we also describe the role of fast transform algorithms in image and video codec evolution and will give an outlook on current developments, including increasing use of CNNs, learning methods, and performance-energy usage tradeoffs that may shape future architectures.

A matrix method for constructing a discrete cosine-sine transform of order N is proposed, upon which discrete and integer cosine-sine transforms of order 16 are constructed. A fast algorithm for computing an integer cosine-sine transform of order-16 with low multiplicative complexity, which is 8.25 times smaller and requires 115% more addition operations compared to the fast discrete cosine-sine transform algorithm, is presented. Additionally, a fast algorithm for computing 2D 16-point integer cosine and cosine-sine separable adaptive transforms with low multiplicative complexity for intra-prediction with 16x16 blocks has been developed. The proposed transforms are effective in the modes of 2D and separable adaptive transforms, providing high coding efficiency in terms of accuracy and compression ratio.

An integer cosine transforms of order -4 and -8 with fast algorithms has been constructed. The proposed transforms is orthogonal, maintains a one-norm, with the normalization error not exceeding 0.23%, and very close to DCT type II. A new quantization approach is proposed, using adaptive parameters that depend on the norm of the transform. The proposed integer cosine transform of order-8 for adaptive quantization parameters is 25% faster than the well-known algorithm from H.265, taking into account the computational complexity of quantization and normalization.

This paper presents cutting-edge design and application techniques for secondary transforms in the context of the next generation of video coding standardization beyond AV1 by the Alliance for Open Media (AOM). It thoroughly discusses methods for applying flexible secondary transform sets and kernels to both intra and inter coded blocks. The proposed methods enable the encoder to optimize transform sets for each intra prediction residual block and extend the use of secondary transforms for inter prediction residuals. Experimental results using the reference software show that the proposed approach enhances overall coding efficiency, measured in weighted YUV PSNR BD-Rates, by 3.40% for All Intra (AI), 1.70% for Random Access (RA), and 1.10% for Low Delay (LD) configurations according to AOM CTC (Common Test Condition) version 7.

With the advent of learned image compression, numerous models have been developed. These models make use of non-linear transforms that are learnt during the training process, where an image is transformed into a latent space, quantized and entropy coded. At the decoder, the quantized latent is recovered and transformed back to image space through a synthesis transform. In this work, we attempt to present an analysis of the energy distribution across channels. In our prior works, we demonstrated the features captured by the analysis transform, that can provide insights into the bitrate distribution across channels. Building on that, we extend our findings with quantitative measurements. We consider various learned image codecs that are based on the variational autoencoder framework and compare them with Karhunen Loeve Transform (KLT) in terms of energy compaction. We also compare the closeness of the learned transforms to KLT to study the relationship between the design of classical codecs and learned codecs.

The Karhunen-Loeve Transform (KLT) when applied to an AR (1) process with known block boundaries resembles the Discrete Sine Transform of type VII (DST-VII). Similarly, when both boundaries are known, the KLT becomes like the DST-I. In this paper, we will use the same methodology to suggest new shapes and forms of temporal prediction structures for video coding. Specifically, treat the Group of Pictures (GOP) as samples from some AR (1) process and interpret factorizations of the resulting DST-VII and DST-I transforms as sequences of temporal predictions with specific weight factors applied. Then, identify a subset of GOP lengths producing simple structures of short-length DST-VII and DST-I factorizations as candidates for practical implementations of temporal prediction algorithms. We will also analyze the coding gains achievable by traditional vs transform-based predictions considering single- and dual-side boundaries. These results may be of interest for future evolutions of video coding algorithms and architectures.

Matrix methods for constructing order-16 integer sine and cosine transforms of type IV (IST-IV/ICT-IV) are proposed. Based on the method two integer sine and cosine transforms of type IV are constructed and fast algorithms for computing of these transforms are developed, which require only integer operations. These algorithms have low multiplicative complexity, being 4.9 times less, and require 127,37% more addition operations compared to the well-known algorithms of discrete sine and cosine transforms type IV. The proposed fast IST-IV/ICT-IV, compared to the well-known fast DSTVII/ DCT-VIII in the VVC standard, have multiplicative complexity that is 12.7 times less and require 26.32% more addition operations.

Current video coding standards, including H.264/AVC, HEVC, and VVC, utilize discrete cosine transform (DCT), discrete sine transform (DST), to decorrelate the intra-prediction residuals. However, these transforms often face challenges in effectively decorrelating signals with complex, non-smooth, and non-periodic structures. Even in smooth areas, an abrupt transition (due to noise or prediction artifacts) can limit their effectiveness. This paper presents a novel block-adaptive separable path graph-based transform (GBT) that is particularly adept at handling such signals. This new method focuses on adaptively modifying the block size and learning GBT to enhance the performance. The GBT is learned in an online scenario using sequential K-means clustering, where each available block size has K clusters and K GBT kernels. This approach allows the GBT to be dynamically learned for the current block based on previously reconstructed areas with same block size and similar characteristics. Our evaluation, integrating this method with H.264/AVC intra-coding tools, shows significant improvement over the traditional H.264/AVC DCT in processing high-resolution natural images.

This work aims to show the latest scientific advances in digital image processing applied to wine grape characterization. The literature review highlights in existing studies about external and internal grape characteristics, but often rely on expensive equipment and lack of essential features for robust predictive results. Consequently, experts in the field advocate for new studies to widen the range of characteristics and economic viability for end-users. A systematic literature analysis of the Scopus database, using keywords like ”grape image processing”, identified 290 papers from 2012 to 2023. Additionally, advanced searches related to maturation, color, chemical analysis, phenolic composition, sugar content, prediction models, and correlation of physical and chemical attributes indicate an area of opportunity due to the decrease of works found on these specific topics. Nevertheless, bibliometric results reveal the evolving research landscape in these areas over the past decade, with notable authors such as Whitty, M., and Liu, S. Leading institutions and countries include China, India, the United States, and Spain. The VOSviewer software was used to confirm influential studies, prominent research groups, and emerging trends in the field. Finally, as a case study, the initial results of our work are presented, which aims to characterize the physical-chemical parameters of wine grapes in different stages of ripening, through digital processing of grape images.

Simultaneous Localization and Mapping (SLAM) is the task of reconstructing an environmental model passed using onboard sensors and at the same time maintaining an estimate of the mobile sensor location within the model. One of the known approaches to the SLAM problem is the Kalman filter. The Kalman filter efficiency is based on the fact that it contains a fully correlated posterior over feature maps and mobile sensor poses. The important element of the SLAM problem is the reconstruction of the environmental 3D scene. In this paper, we propose an algorithm to restore the 3D scene using consistent condition and a modified version of the Kalman filter. The reconstruction algorithm is noniterative. Computer simulation results are provided to illustrate the performance of the proposed method.

3D point cloud registration is of great importance in robotics and computer vision to find a rigid body transformation to align a pair of point clouds with unknown point correspondences. In recent years, the deep learning model has dominated the field of computer vision. The important part of registration is the estimation of correspondences between point clouds. The main idea of studying correspondences between point clouds is to establish correspondences through the multidimensional features of each point. In this paper, we propose a simple neural network algorithm to register incongruent point clouds. The proposed algorithm utilizes the virtual points and is partially based on the PointNet++ neural network. Computer simulation results are provided to illustrate the performance of the proposed method.

This study focuses on the problem of finding pathologies on mammogram screenings using computer vision-based systems. For that we use deep neural networks and use open-source datasets for our models training for finding segments of pathologies on screenings. These datasets are also used for further analyses of different supervised training techniques, functions and losses to achieve best results for generalization of data even when its’ domain changes. We don’t use screenings with pathologies during training while trying to achieve the results where our network would be able to still recognize these pathologies and embeddings (feature maps) extracted at certain layers of our network would allow to group up the samples where these pathologies are present by calculating either cosine or normalized L2 distances between samples. First, we train our network to recognize normal mammogram screenings and anything that is different. We create a vocabulary of known classes using extracted embeddings of samples from training subset to obtain centroids of embeddings for 2 separate classes – benign and malignant tumors. We then use a method known as CAM (Class Activation Mapping) along with Selective Search algorithm for segmentation task. In our work we use mainly dataset CBIS-DDSM [1] for validation which consists of a few hundreds of annotated images of digitalized mammogram screenings. The entire pipeline of the system has the following steps: first, extracting the embeddings from training subset and clustering them into vocabularies for classes of benign and malignant tumors; then, the clusters can be analyzed and regrouped manually depending on the situation to create sub-classes such as small or big benign tumors; afterwards, we use these existing vocabularies for classification task to detect screenings with tumors; and to obtain segments of the detected pathologies CAM along with Selective Search with carefully selected parameters is used.

Mammography screening also leads to a high rate of false positive results. This may lead to unnecessary worry, inconvenient follow-up care, additional imaging studies, and sometimes the need for tissue. blood draws (often a needle biopsy). Convolutional neural networks (CNN) are one of the most important networks in the field of deep learning. The neural networks form some feature vectors often contain weak features. There are known methods for eliminating weak features based on the mutual information. In this paper, we propose a convolutional neural network based to recognize local geometrical features. Computer simulation results are provided to illustrate the performance of the proposed method.

Mammography is a standardized imaging technique crucial for the early detection of breast cancer, primarily aimed at identifying abnormalities, or ’findings’ in the breasts that cannot be detected through palpation. This study proposes different models for classifying breast abnormalities, integrating machine learning and deep learning methods to improve classification rates. The proposed methodology involves several key steps: preprocessing of image datasets, training of base classification models, and construction of a meta-classifier. By enhancing the performance of individual classifiers, the model is benchmarked against various machine learning models. The evaluation of this method is conducted using the CBIS-DDSM mammography dataset, demonstrating its effectiveness in improving classification accuracy and reliability. The hybrid approach leverages convolutional neural networks (CNNs) such as VGG16, VGG19, and DenseNet121 for feature extraction, followed by machine learning algorithms for final classification. The VGG16 network, combined with machine learning techniques, aimed to surpass the results obtained by VGG19. Ensemble methods, particularly Voting and Stacking Classifiers, showed that combining VGG16 and DenseNet121 yielded the highest accuracy of 91.66%. These findings underscore the potential of hybrid models in breast cancer classification, offering significant improvements over single classifiers and providing valuable insights for future research in medical image analysis.

The article addresses the importance of timely detection of errors occurring during 3D printing. One of the 3D printing methods chosen is FFF (Fused Filament Fabrication). Sources are provided where one can familiarize themselves with the main errors that occur during 3D printing. A dataset dedicated to the detection and correction of 3D printing errors using neural networks has been found. The results of training neural networks with ResNet50 and EfficientNet architectures on the found dataset are presented.