Generating super resolution images using transformers

Editor's Notes

• #2: Good evening, neeraj baghel, enrollment no, my project topic is
• #4: Image super-resolution aims at recovering a high-resolution image from its low-resolution counterpart. In this example we can see there is low resolution image from an cctv camera. And the Objective is to read the registration no from its number plate.
• #5: So we have find the reason of interest. Cropped , zoomed , numberplate
• #6: Then we have reconstructed the high resolution image, where we can clearly see the registration no.
• #7: So, it is a task to generate the high resolution Image It is an still active area for offering the promise of overcoming resolution limitations in many applications, such as ……... The image super-resolution task has witnessed great strides with the development of deep learning.
• #8: Motivation: Identifying the person in a low resolution image. To achieve low storage memory in mobiles and transferring low resolution images. Ob: We want to propose a novel Super-Resolution Transformer for fast and accurate image super-resolution with comparatively less computational cost and memory storage. Recent years, Transformer has made great progress in computer vision task as its strong self attention mechanism. In SISR, similar image blocks within the image can be used as reference images to each other, so that the texture details of the current image block can be restored with reference to other image blocks, which is proper to use Transformer.
• #9: Motivation: Identifying the person in a low resolution image. To achieve low storage memory in mobiles and transferring low resolution images. Ob: We want to propose a novel Super-Resolution Transformer for fast and accurate image super-resolution with comparatively less computational cost and memory storage. Recent years, Transformer has made great progress in computer vision task as its strong self attention mechanism. In this similar image blocks can be used as reference images, so that the texture details of the current image block can be restored with reference to other image blocks, which is proper to use Transformer.
• #10: Motivation: Identifying the person in a low resolution image. To achieve low storage memory in mobiles and transferring low resolution images. Ob: We want to propose a novel Super-Resolution Transformer for fast and accurate image super-resolution with comparatively less computational cost and memory storage. Recent years, Transformer has made great progress in computer vision task as its strong self attention mechanism. In SISR, similar image blocks within the image can be used as reference images to each other, so that the texture details of the current image block can be restored with reference to other image blocks, which is proper to use Transformer.
• #12: In this, On top of the texture transformer, author propose a cross-scale feature integration module (CSFI) to further enhance model performances. The proposed texture transformer. Q, K and V are the texture features extracted from an up-sampled LR image, a sequentially down/up-sampled Ref image, and an original Ref image, respectively. H and S indicate the hard/soft attention map, calculated from relevance embedding. F is the LR features extracted from a DNN backbone, and is further fused with the transferred texture features T for generating the SR output.
• #14: absolute differences between the true value and the predicted value. L1 Loss (Least Absolute Deviations) L2 Loss (Least Square Errors)
• #15: It present the overall architecture of the proposed model “Efficient SR Transformer” (ESRT). In this author present the LCB (lightweight CNN backbone) with novel high preserving block (HPB) and high-frequency filtering module (HFM). Next, author present the LTB (lightweight Transformer backbone) with an efficient Transformer (ET). Finally, it present the difference between our ESRT and other SR methods
• #16: Lightweight CNN Backbone (LCB) is built like other SR models, which served as the front part of ESRT. The function of LCB is to extract the latent SR features in advance so that the model has the initial ability of super-resolution. HPB first isolates the high-frequency information with the help of HFM. In HPB, an adaptive residual feature block (ARFB) is also introduced as the basic feature extraction unit. Firstly, an ARFB is adopted to extract the input features Fn 1 for HFM. HFM then calculates the high-frequency information (marked as Phigh) of the features. In this paper, we propose a lightweight Transformer backbone (LTB), which is composed of efficient Transformer (ET), to capture the long-term dependence of similar local regions in the image at a low computational cost.
• #18: However, previous variants of vision Transformer commonly need heavy GPU memory cost, which hinders the development of Transformer in the vision area. Extensive benchmark and real-world datasets demonstrate that our ESRT achieves the best trade-off between model performance and computation cost.
• #20: Our model is trained with the DIV2K [13] dataset, which is widely used in SISR task. DIV2K contains 800 RGB training images and 100 validation images with rich textures (2K resolution). Common evaluation dataset f for Super Resolution of images. contains various images of buildings to animal faces. The Set14 dataset is a dataset consisting of 14 images commonly used for testing performance of Image Super-Resolution models. Visual comparison for 4 scale factor on urban100 dataset and B100 dataset. For evaluation, we use five benchmark datasets to validate the effectiveness of our method, including Set5 [38], Set14 [39], B100 [40], Urban100 [41], and Manga109 [42]. Meanwhile, Peak signal-to-noise ratio (PSNR) and structure similarity index (SSIM) are used to evaluate the performance of the reconstructed SR images.
• #21: are used to evaluate the performance of the reconstructed SR images. PSNR: ratio between the maximum possible power of a signal and the power of corrupting noise