Official pytorch implementation of OAIV-Coherence(Enhancing Seam Carving with Object Awareness for Improved Visual Coherence)
Seam carving is a classic topic in computer graphics, playing a crucial role in reducing misalignment caused by differences in captured perspectives and object motion in image stitching. Traditional seam carving methods fail to consider semantic information, resulting in disrupted foreground continuity. We propose a deep learning-based framework that leverages semantic priors of foreground objects, introducing a novel loss function to preserve semantic integrity and enhance visual coherence. Additionally, We propose two specialized real-world datasets to evaluate our method. Experimental results demonstrate significant improvements in image quality, addressing traditional technique limitations and providing robust support for practical applications.
We implement this work with UWSL2(Ubuntu24.04), 3090, and CUDA11.
- Git clone this repository.
git clone https://github.com/Pokerman8/OAIV-Coherence.git
- Create a new conda environment
conda env create -f environment.yml
We trained our network using the unsupervised UDIS-D dataset [1], which consists of 10,440 cases derived from various moving videos, including those from [2] and independently captured scenes.
Download the training set from Onedrive.
we developed two specialized datasets: DAVISProcessed10 and RealWorld400. DAVISProcessed10, generated by selecting the first and last frames from every ten-frame sequence in DAVIS, contains 1,770 image pairs, while RealWorld400 comprises 400 pairs of real-world data(details are shown in the figure below).
A portion of the test set can be downloaded from this link, and we will release all the test data soon.
To train the mdoel, set the path to the training dataset in train.py. Please follow the instructions in the code comments to adjust the paths accordingly.
python train.py
You can download the pre-trained model from this link. To run the tests, set the path to the testing dataset in test.py. Please follow the instructions in the code comments to adjust the paths accordingly.
python test.py
We plan to release the code for multi-image stitching and performance evaluation in the near future.
-
Lang Nie, Chunyu Lin, Kang Liao, Shuaicheng Liu, and Yao Zhao. "Unsupervised Deep Image Stitching: Reconstructing Stitched Features to Images." IEEE Transactions on Image Processing, vol. 30, pp. 6184-6197, 2021. doi:10.1109/TIP.2021.3092828.
-
Jirong Zhang, Chuan Wang, Shuaicheng Liu, Lanpeng Jia, Nianjin Ye, Jue Wang, Ji Zhou, and Jian Sun. "Content-aware unsupervised deep homography estimation." In Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part I 16, pp. 653-669. Springer, 2020.