Dr Shanxin Yuan

Shanxin Yuan

Lecturer in Digital Environment

School of Electronic Engineering and Computer Science
Queen Mary University of London
ORCID Google Scholar LinkedIn

Research

3D Computer Vision, Reconstruction and Rendering, Multimodal Learning, Low-level Vision, Music Understanding

Interests

The research focuses on computer vision and machine learning, with a strong philosophy of transferring research to significant real-world applications. I work on several lines of research, especially 3D digital humans and computational photography. The recent topics include hand/head/body pose estimation and reconstruction, neural rendering for deformable objects, pose retargeting, immersive gaming, music understanding, and fashion AI. My previous research has been successfully shipped to several products that are being used by millions of people worldwide.

Publications

Relevant PublicationPublications of specific relevance to the Centre for Multimodal AI

2024

Relevant PublicationSteinmetz C, Singh S, Comunit� M, Ibnyahya I, Yuan S, Benetos E and Reiss J (2024). ST-ITO: Controlling audio effects for style transfer with inference-time optimization. 25th International Society for Music Information Retrieval Conference (ISMIR) San Francisco, CA, USA 10 Nov 2024 - 14 Nov 2024
10-11-2024
bullet iconZhang Q, Zheng B, Li Z, Liu Y, Zhu Z, Slabaugh G and Yuan S (2024). Non-local degradation modeling for spatially adaptive single image super-resolution. Neural Networks, Elsevier vol. 175, 106293-106293.  
10-04-2024
bullet iconChen Q, Zheng B, Yan C, Zhu Z, Wang T, Slabaugh G and Yuan S (2024). GoLDFormer: A global–local deformable window transformer for efficient image restoration. Journal of Visual Communication and Image Representation, Elsevier vol. 100, 104117-104117.  
01-04-2024
bullet iconSenior H, Slabaugh G, Yuan S and Rossi L (2024). Graph neural networks in vision-language image understanding: a survey. The Visual Computer, Springer Science and Business Media LLC 
29-03-2024
bullet iconLiu L, An J, Yuan S, Zhou W, Li H, Wang Y and Tian Q (2024). Video Demoiring With Deep Temporal Color Embedding and Video-Image Invertible Consistency. IEEE Transactions on Multimedia, Institute of Electrical and Electronics Engineers (IEEE) vol. 26, 7386-7397.  
26-02-2024
Relevant PublicationDai T, Li W, Cao X, Liu J, Jia X, Leonardis A, Yan Y and Yuan S (2024). Wavelet-based network for high dynamic range imaging. Computer Vision and Image Understanding, Elsevier vol. 238 
01-01-2024

2023

Relevant PublicationChen Q, Zheng B, Zhou X, Huang A, Sun Y, Chen C, Yan C and Yuan S (2023). Depth-guided deep filtering network for efficient single image bokeh rendering. Neural Computing and Applications, Springer vol. 35 (28), 20869-20887.  
26-07-2023
Relevant PublicationLiu L, An J, Liu J, Yuan S, Chen X, Zhou W, Li H, Wang YF and Tian Q (2023). Low-Light Video Enhancement with Synthetic Event Guidance. 
26-06-2023
Relevant PublicationYang C, Li P, Zhou Z, Yuan S, Liu B, Yang X, Qiu W and Shen W (2023). NeRFVS: Neural Radiance Fields for Free View Synthesis via Geometry Scaffolds. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
24-06-2023
Relevant PublicationChen R, Zheng B, Zhang H, Chen Q, Yan C, Slabaugh G and Yuan S (2023). Improving Dynamic HDR Imaging with Fusion Transformer. Annual AAAI Conference on Artificial Intelligence
07-02-2023

2022

Relevant PublicationZheng B, Pan X, Zhang H, Zhou X, Slabaugh G, Yan C and Yuan S (2022). DomainPlus: Cross Transform Domain Learning towards High Dynamic Range Imaging. Proceedings of the 30th ACM International Conference on Multimedia
10-10-2022
Relevant PublicationZheng B, Yuan S, Yan C, Tian X, Zhang J, Sun Y, Liu L, Leonardis A and Slabaugh G (2022). Learning Frequency Domain Priors for Image Demoireing. IEEE Transactions on Pattern Analysis and Machine Intelligence, Institute of Electrical and Electronics Engineers (IEEE) vol. 44 (11), 7705-7717.  
04-10-2022
Relevant PublicationLiu L, Yuan S, Liu J, Guo X, Yan Y and Tian Q (2022). SiamTrans: Zero-Shot Multi-Frame Image Restoration with Pre-trained Siamese Transformers. 
30-06-2022
Relevant PublicationZheng B, Chen Q, Yuan S, Zhou X, Zhang H, Zhang J, Yan C and Slabaugh G (2022). Constrained Predictive Filters for Single Image Bokeh Rendering. IEEE Transactions on Computational Imaging, Institute of Electrical and Electronics Engineers (IEEE) vol. 8, 346-357.  
03-05-2022
Relevant PublicationHu X, Li X, Busam B, Zhou Y, Leonardis A and Yuan S (2022). Disentangling 3D Attributes from a Single 2D Image: Human Pose, Shape and Garment. 
01-01-2022
Relevant PublicationLiu L, Xie L, Zhang X, Yuan S, Chen X, Zhou W, Li H and Tian Q (2022). TAPE: Task-Agnostic Prior Embedding for Image Restoration. 
01-01-2022

2021

Relevant PublicationZhao H, Zheng B, Yuan S, Zhang H, Yan C, Li L and Slabaugh G (2021). CBREN: Convolutional Neural Networks for Constant Bit Rate Video Quality Enhancement. IEEE Transactions on Circuits and Systems for Video Technology, Institute of Electrical and Electronics Engineers (IEEE) vol. 32 (7), 4138-4149.  
27-10-2021

2020

Relevant PublicationYuan S, Timofte R, Leonardis A, Slabaugh G, Luo X, Zhang J, Qu Y, Hong M, Xie Y, Li C, Xu D, Chu Y, Sun Q, Liu S, Zong Z, Nan N, , Kim S, Nam H, Kim J, et al. (2020). NTIRE 2020 Challenge on Image Demoireing: Methods and Results. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
19-06-2020
Relevant PublicationIsobe T, Li S, Jia X, Yuan S, Slabaugh G, Xu C, Li Y-L, Wang S and Tian Q (2020). Video Super-resolution with Temporal Group Attention. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
13-06-2020
Relevant PublicationZheng B, Yuan S, Slabaugh G and Leonardis A (2020). Image Demoireing with Learnable Bandpass Filters. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
13-06-2020
Relevant PublicationLiu L, Yuan S, Liu J, Bao L, Slabaugh G and Tian Q (2020). Self-adaptively learning to Demoiré from focused and defocused image pairs. 
01-01-2020
Relevant PublicationLiu L, Liu J, Yuan S, Slabaugh G, Leonardis A, Zhou W and Tian Q (2020). Wavelet-Based Dual-Branch Network for Image Demoiréing. 
01-01-2020

2019

Relevant PublicationYuan S, Timofte R, Slabaugh G and Leonardis A (2019). AIM 2019 Challenge on Image Demoireing: Dataset and Study. 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)
27-10-2019
Relevant PublicationYuan S, Stenger B and Kim T-K (2019). 3D Hand Pose Estimation from RGB Using Privileged Learning with Depth Data. 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)
27-10-2019
Relevant PublicationYuan S, Timofte R, Slabaugh G, Leonardis A, Zheng B, Ye X, Tian X, Chen Y, Cheng X, Fu Z, Yang J, Hong M, Lin W, Yang W, Qu Y, Shin H-K, Kim J-Y, Ko S-J, Dong H, Guo Y, et al. (2019). AIM 2019 Challenge on Image Demoireing: Methods and Results. 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)
27-10-2019

2018

Relevant PublicationTang D, Ye Q, Yuan S, Taylor J, Kohli P, Keskin C, Kim T-K and Shotton J (2018). Opening the Black Box: Hierarchical Sampling Optimization for Hand Pose Estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, Institute of Electrical and Electronics Engineers (IEEE) vol. 41 (9), 2161-2175.  
15-06-2018
Relevant PublicationGarcia-Hernando G, Yuan S, Baek S and Kim T-K (2018). First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotations. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
01-06-2018
Relevant PublicationYuan S, Hernando GG, Stenger B, Moon G, Chang JY, Lee KM, Molchanov P, Kautz J, Honari S, Ge L, Yuan J, Chen X, Wang G, Yang F, Akiyama K, Wu Y, Wan Q, Madadi M, Escalera S, Li S, et al. (2018). Depth-Based 3D Hand Pose Estimation: From Current Achievements to Future Goals. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
01-06-2018

2017

Relevant PublicationYuan S, Ye Q, Stenger B, Jain S and Kim T-K (2017). BigHand2.2M Benchmark: Hand Pose Dataset and State of the Art Analysis. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
01-07-2017

2016

Relevant PublicationYe Q, Yuan S and Kim T-K (2016). Spatial Attention Deep Net with Partial PSO for Hierarchical Hybrid Hand Pose Estimation. 
01-01-2016