Dr Shanxin Yuan
Lecturer in Digital Environment
School of Electronic Engineering and Computer Science
Queen Mary University of London
Queen Mary University of London
Research
3D Computer Vision, Reconstruction and Rendering, Multimodal Learning, Low-level Vision, Music Understanding
Interests
The research focuses on computer vision and machine learning, with a strong philosophy of transferring research to significant real-world applications. I work on several lines of research, especially 3D digital humans and computational photography. The recent topics include hand/head/body pose estimation and reconstruction, neural rendering for deformable objects, pose retargeting, immersive gaming, music understanding, and fashion AI. My previous research has been successfully shipped to several products that are being used by millions of people worldwide.Publications
Publications of specific relevance to the Centre for Multimodal AI2024
Steinmetz C, Singh S, Comunit� M, Ibnyahya I, Yuan S, Benetos E and Reiss J (2024). ST-ITO: Controlling audio effects for style transfer with inference-time optimization. 25th International Society for Music Information Retrieval Conference (ISMIR) San Francisco, CA, USA 10 Nov 2024 - 14 Nov 2024. 10-11-2024
Zhang Q, Zheng B, Li Z, Liu Y, Zhu Z, Slabaugh G and Yuan S (2024). Non-local degradation modeling for spatially adaptive single image super-resolution. Neural Networks, Elsevier vol. 175, 106293-106293. 10-04-2024
Chen Q, Zheng B, Yan C, Zhu Z, Wang T, Slabaugh G and Yuan S (2024). GoLDFormer: A global–local deformable window transformer for efficient image restoration. Journal of Visual Communication and Image Representation, Elsevier vol. 100, 104117-104117. 01-04-2024
Senior H, Slabaugh G, Yuan S and Rossi L (2024). Graph neural networks in vision-language image understanding: a survey. The Visual Computer, Springer Science and Business Media LLC 29-03-2024
Liu L, An J, Yuan S, Zhou W, Li H, Wang Y and Tian Q (2024). Video Demoiring With Deep Temporal Color Embedding and Video-Image Invertible Consistency. IEEE Transactions on Multimedia, Institute of Electrical and Electronics Engineers (IEEE) vol. 26, 7386-7397. 26-02-2024
Dai T, Li W, Cao X, Liu J, Jia X, Leonardis A, Yan Y and Yuan S (2024). Wavelet-based network for high dynamic range imaging. Computer Vision and Image Understanding, Elsevier vol. 238 01-01-2024
2023
Chen Q, Zheng B, Zhou X, Huang A, Sun Y, Chen C, Yan C and Yuan S (2023). Depth-guided deep filtering network for efficient single image bokeh rendering. Neural Computing and Applications, Springer vol. 35 (28), 20869-20887. 26-07-2023
Liu L, An J, Liu J, Yuan S, Chen X, Zhou W, Li H, Wang YF and Tian Q (2023). Low-Light Video Enhancement with Synthetic Event Guidance. 26-06-2023
Yang C, Li P, Zhou Z, Yuan S, Liu B, Yang X, Qiu W and Shen W (2023). NeRFVS: Neural Radiance Fields for Free View Synthesis via Geometry Scaffolds. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 24-06-2023
Chen R, Zheng B, Zhang H, Chen Q, Yan C, Slabaugh G and Yuan S (2023). Improving Dynamic HDR Imaging with Fusion Transformer. Annual AAAI Conference on Artificial Intelligence. 07-02-2023
2022
Zheng B, Pan X, Zhang H, Zhou X, Slabaugh G, Yan C and Yuan S (2022). DomainPlus: Cross Transform Domain Learning towards High Dynamic Range Imaging. Proceedings of the 30th ACM International Conference on Multimedia. 10-10-2022
Zheng B, Yuan S, Yan C, Tian X, Zhang J, Sun Y, Liu L, Leonardis A and Slabaugh G (2022). Learning Frequency Domain Priors for Image Demoireing. IEEE Transactions on Pattern Analysis and Machine Intelligence, Institute of Electrical and Electronics Engineers (IEEE) vol. 44 (11), 7705-7717. 04-10-2022
Liu L, Yuan S, Liu J, Guo X, Yan Y and Tian Q (2022). SiamTrans: Zero-Shot Multi-Frame Image Restoration with Pre-trained Siamese Transformers. 30-06-2022
Zheng B, Chen Q, Yuan S, Zhou X, Zhang H, Zhang J, Yan C and Slabaugh G (2022). Constrained Predictive Filters for Single Image Bokeh Rendering. IEEE Transactions on Computational Imaging, Institute of Electrical and Electronics Engineers (IEEE) vol. 8, 346-357. 03-05-2022
Hu X, Li X, Busam B, Zhou Y, Leonardis A and Yuan S (2022). Disentangling 3D Attributes from a Single 2D Image: Human Pose, Shape and Garment. 01-01-2022
Liu L, Xie L, Zhang X, Yuan S, Chen X, Zhou W, Li H and Tian Q (2022). TAPE: Task-Agnostic Prior Embedding for Image Restoration. 01-01-2022
2021
Zhao H, Zheng B, Yuan S, Zhang H, Yan C, Li L and Slabaugh G (2021). CBREN: Convolutional Neural Networks for Constant Bit Rate Video Quality Enhancement. IEEE Transactions on Circuits and Systems for Video Technology, Institute of Electrical and Electronics Engineers (IEEE) vol. 32 (7), 4138-4149. 27-10-2021
2020
Yuan S, Timofte R, Leonardis A, Slabaugh G, Luo X, Zhang J, Qu Y, Hong M, Xie Y, Li C, Xu D, Chu Y, Sun Q, Liu S, Zong Z, Nan N, , Kim S, Nam H, Kim J, et al. (2020). NTIRE 2020 Challenge on Image Demoireing: Methods and Results. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 19-06-2020
Isobe T, Li S, Jia X, Yuan S, Slabaugh G, Xu C, Li Y-L, Wang S and Tian Q (2020). Video Super-resolution with Temporal Group Attention. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 13-06-2020
Zheng B, Yuan S, Slabaugh G and Leonardis A (2020). Image Demoireing with Learnable Bandpass Filters. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 13-06-2020
Liu L, Yuan S, Liu J, Bao L, Slabaugh G and Tian Q (2020). Self-adaptively learning to Demoiré from focused and defocused image pairs. 01-01-2020
Liu L, Liu J, Yuan S, Slabaugh G, Leonardis A, Zhou W and Tian Q (2020). Wavelet-Based Dual-Branch Network for Image Demoiréing. 01-01-2020
2019
Yuan S, Timofte R, Slabaugh G and Leonardis A (2019). AIM 2019 Challenge on Image Demoireing: Dataset and Study. 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW). 27-10-2019
Yuan S, Stenger B and Kim T-K (2019). 3D Hand Pose Estimation from RGB Using Privileged Learning with Depth Data. 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW). 27-10-2019
Yuan S, Timofte R, Slabaugh G, Leonardis A, Zheng B, Ye X, Tian X, Chen Y, Cheng X, Fu Z, Yang J, Hong M, Lin W, Yang W, Qu Y, Shin H-K, Kim J-Y, Ko S-J, Dong H, Guo Y, et al. (2019). AIM 2019 Challenge on Image Demoireing: Methods and Results. 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW). 27-10-2019
2018
Tang D, Ye Q, Yuan S, Taylor J, Kohli P, Keskin C, Kim T-K and Shotton J (2018). Opening the Black Box: Hierarchical Sampling Optimization for Hand Pose Estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, Institute of Electrical and Electronics Engineers (IEEE) vol. 41 (9), 2161-2175. 15-06-2018
Garcia-Hernando G, Yuan S, Baek S and Kim T-K (2018). First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotations. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 01-06-2018
Yuan S, Hernando GG, Stenger B, Moon G, Chang JY, Lee KM, Molchanov P, Kautz J, Honari S, Ge L, Yuan J, Chen X, Wang G, Yang F, Akiyama K, Wu Y, Wan Q, Madadi M, Escalera S, Li S, et al. (2018). Depth-Based 3D Hand Pose Estimation: From Current Achievements to Future Goals. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 01-06-2018
2017
Yuan S, Ye Q, Stenger B, Jain S and Kim T-K (2017). BigHand2.2M Benchmark: Hand Pose Dataset and State of the Art Analysis. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 01-07-2017
2016
Ye Q, Yuan S and Kim T-K (2016). Spatial Attention Deep Net with Partial PSO for Hierarchical Hybrid Hand Pose Estimation. 01-01-2016