Dr Shanxin Yuan
Lecturer in Digital Environment
School of Electronic Engineering and Computer Science
Queen Mary University of London
Queen Mary University of London
Research
3D Computer Vision, Reconstruction and Rendering, Multimodal Learning, Low-level Vision, Music Understanding
Interests
The research focuses on computer vision and machine learning, with a strong philosophy of transferring research to significant real-world applications. I work on several lines of research, especially 3D digital humans and computational photography. The recent topics include hand/head/body pose estimation and reconstruction, neural rendering for deformable objects, pose retargeting, immersive gaming, music understanding, and fashion AI. My previous research has been successfully shipped to several products that are being used by millions of people worldwide.Publications
Publications of specific relevance to the Centre for Multimodal AI2025
SuperCap: Multi-resolution Superpixel-based Image CaptioningSenior H Rossi L Slabaugh G Yuan S
In Arxiv
11-03-2025
2024
ST-ITO: Controlling audio effects for style transfer with inference-time optimizationSteinmetz C Singh S Comunit� M Ibnyahya I Yuan S Benetos E Reiss J
25th International Society for Music Information Retrieval Conference (ISMIR) San Francisco, CA, USA 10 Nov 2024 - 14 Nov 2024.
10-11-2024
Non-local degradation modeling for spatially adaptive single image super-resolutionZhang Q Zheng B Li Z Liu Y Zhu Z Slabaugh G Yuan S
Neural Networks, Elsevier vol. 175, 106293-106293.
10-04-2024
GoLDFormer: A global–local deformable window transformer for efficient image restorationChen Q Zheng B Yan C Zhu Z Wang T Slabaugh G Yuan S
Journal of Visual Communication and Image Representation, Elsevier vol. 100, 104117-104117.
01-04-2024
Graph neural networks in vision-language image understanding: a surveySenior H Slabaugh G Yuan S Rossi L
The Visual Computer, Springer Science and Business Media Llc
29-03-2024
Video Demoiring With Deep Temporal Color Embedding and Video-Image Invertible ConsistencyLiu L An J Yuan S Zhou W Li H Wang Y Tian Q
IEEE Transactions on Multimedia, Institute of Electrical and Electronics Engineers (IEEE) vol. 26, 7386-7397.
26-02-2024
ST-ITO: CONTROLLING AUDIO EFFECTS FOR STYLE TRANSFER WITH INFERENCE-TIME OPTIMIZATIONSteinmetz CJ Singh S Comunità M Ibnyahya I Yuan S Benetos E Reiss JD
vol. 2024, 661-668.
01-01-2024
Wavelet-based network for high dynamic range imagingDai T Li W Cao X Liu J Jia X Leonardis A Yan Y Yuan S
Computer Vision and Image Understanding, Elsevier vol. 238
01-01-2024
2023
Depth-guided deep filtering network for efficient single image bokeh renderingChen Q Zheng B Zhou X Huang A Sun Y Chen C Yan C Yuan S
Neural Computing and Applications, Springer vol. 35 (28), 20869-20887.
26-07-2023
Low-Light Video Enhancement with Synthetic Event GuidanceLiu L An J Liu J Yuan S Chen X Zhou W Li H Wang YF et al.
Proceedings of the AAAI Conference on Artificial Intelligence. vol. 37 (2), 1692-1700.
26-06-2023
NeRFVS: Neural Radiance Fields for Free View Synthesis via Geometry ScaffoldsYang C Li P Zhou Z Yuan S Liu B Yang X Qiu W Shen W
2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). vol. 00, 16549-16558.
24-06-2023
Graph Neural Networks in Vision-Language Image Understanding: A SurveySenior H Slabaugh G Yuan S Rossi L
In Arxiv
07-03-2023
Improving Dynamic HDR Imaging with Fusion TransformerChen R Zheng B Zhang H Chen Q Yan C Slabaugh G Yuan S
Annual AAAI Conference on Artificial Intelligence. vol. 37 (1), 340-349.
07-02-2023
2022
DomainPlus: Cross Transform Domain Learning towards High Dynamic Range ImagingZheng B Pan X Zhang H Zhou X Slabaugh G Yan C Yuan S
Proceedings of the 30th ACM International Conference on Multimedia., 1954-1963.
10-10-2022
Learning Frequency Domain Priors for Image DemoireingZheng B Yuan S Yan C Tian X Zhang J Sun Y Liu L Leonardis A et al.
IEEE Transactions on Pattern Analysis and Machine Intelligence, Institute of Electrical and Electronics Engineers (IEEE) vol. 44 (11), 7705-7717.
04-10-2022
Disentangling 3D Attributes from a Single 2D Image: Human Pose, Shape and GarmentHu X Li X Busam B Zhou Y Leonardis A Yuan S
In Arxiv
05-08-2022
SiamTrans: Zero-Shot Multi-Frame Image Restoration with Pre-trained Siamese TransformersLiu L Yuan S Liu J Guo X Yan Y Tian Q
Proceedings of the AAAI Conference on Artificial Intelligence. vol. 36 (2), 1747-1755.
28-06-2022
Constrained Predictive Filters for Single Image Bokeh RenderingZheng B Chen Q Yuan S Zhou X Zhang H Zhang J Yan C Slabaugh G
IEEE Transactions on Computational Imaging, Institute of Electrical and Electronics Engineers (IEEE) vol. 8, 346-357.
03-05-2022
TAPE: Task-Agnostic Prior Embedding for Image RestorationLiu L Xie L Zhang X Yuan S Chen X Zhou W Li H Tian Q
In Arxiv
11-03-2022
Disentangling 3D Attributes from a Single 2D Image: Human Pose, Shape and GarmentHu X Li X Busam B Zhou Y Leonardis A Yuan S
Bmvc 2022 33rd British Machine Vision Conference Proceedings.
01-01-2022
TAPE: Task-Agnostic Prior Embedding for Image RestorationLiu L Xie L Zhang X Yuan S Chen X Zhou W Li H Tian Q
Lecture Notes in Computer Science. vol. 13678, 447-464.
01-01-2022
2021
CBREN: Convolutional Neural Networks for Constant Bit Rate Video Quality EnhancementZhao H Zheng B Yuan S Zhang H Yan C Li L Slabaugh G
IEEE Transactions on Circuits and Systems For Video Technology, Institute of Electrical and Electronics Engineers (IEEE) vol. 32 (7), 4138-4149.
27-10-2021
2020
Self-Adaptively Learning to Demoire from Focused and Defocused Image PairsLiu L Yuan S Liu J Bao L Slabaugh G Tian Q
In Arxiv
03-11-2020
Video Super-resolution with Temporal Group AttentionIsobe T Li S Jia X Yuan S Slabaugh G Xu C Li Y-L Wang S et al.
In Arxiv
21-07-2020
Wavelet-Based Dual-Branch Network for Image DemoireingLiu L Liu J Yuan S Slabaugh G Leonardis A Zhou W Tian Q
In Arxiv
14-07-2020
NTIRE 2020 Challenge on Image Demoireing: Methods and ResultsYuan S Timofte R Leonardis A Slabaugh G Luo X Zhang J Qu Y Hong M et al.
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). vol. 00, 1882-1893.
19-06-2020
Video Super-resolution with Temporal Group AttentionIsobe T Li S Jia X Yuan S Slabaugh G Xu C Li Y-L Wang S et al.
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). vol. 00, 8005-8014.
13-06-2020
Image Demoireing with Learnable Bandpass FiltersZheng B Yuan S Slabaugh G Leonardis A
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). vol. 00, 3633-3642.
13-06-2020
Image Demoireing with Learnable Bandpass FiltersZheng B Yuan S Slabaugh G Leonardis A
In Arxiv
01-04-2020
Self-adaptively learning to Demoiré from focused and defocused image pairsLiu L Yuan S Liu J Bao L Slabaugh G Tian Q
Advances in Neural Information Processing Systems. vol. 2020-December
01-01-2020
Wavelet-Based Dual-Branch Network for Image DemoiréingLiu L Liu J Yuan S Slabaugh G Leonardis A Zhou W Tian Q
Lecture Notes in Computer Science. vol. 12358, 86-102.
01-01-2020
2019
AIM 2019 Challenge on Image Demoireing: Dataset and StudyYuan S Timofte R Slabaugh G Leonardis A
In Arxiv
06-11-2019
AIM 2019 Challenge on Image Demoireing: Dataset and StudyYuan S Timofte R Slabaugh G Leonardis A
2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW). vol. 00, 3526-3533.
27-10-2019
AIM 2019 Challenge on Image Demoireing: Methods and ResultsYuan S Timofte R Slabaugh G Leonardis A Zheng B Ye X Tian X Chen Y et al.
2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW). vol. 00, 3534-3545.
27-10-2019
3D Hand Pose Estimation from RGB Using Privileged Learning with Depth DataYuan S Stenger B Kim T-K
2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW). vol. 00, 2866-2873.
27-10-2019
2018
RGB-based 3D Hand Pose Estimation via Privileged Learning with Depth ImagesYuan S Stenger B Kim T-K
In Arxiv
18-11-2018
Opening the Black Box: Hierarchical Sampling Optimization for Hand Pose EstimationTang D Ye Q Yuan S Taylor J Kohli P Keskin C Kim T-K Shotton J
IEEE Transactions on Pattern Analysis and Machine Intelligence, Institute of Electrical and Electronics Engineers (IEEE) vol. 41 (9), 2161-2175.
15-06-2018
Depth-Based 3D Hand Pose Estimation: From Current Achievements to Future GoalsYuan S Hernando GG Stenger B Moon G Chang JY Lee KM Molchanov P Kautz J et al.
2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition., 2636-2645.
01-06-2018
First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose AnnotationsGarcia-Hernando G Yuan S Baek S Kim T-K
2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition., 409-419.
01-06-2018
2017
Depth-Based 3D Hand Pose Estimation: From Current Achievements to Future GoalsYuan S Garcia-Hernando G Stenger B Moon G Chang JY Lee KM Molchanov P Kautz J et al.
In Arxiv
11-12-2017
BigHand2.2M Benchmark: Hand Pose Dataset and State of the Art AnalysisYuan S Ye Q Stenger B Jain S Kim T-K
2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)., 2605-2613.
01-07-2017
BigHand2.2M Benchmark: Hand Pose Dataset and State of the Art AnalysisYuan S Ye Q Stenger B Jain S Kim T-K
In Arxiv
09-04-2017
2016
Spatial Attention Deep Net with Partial PSO for Hierarchical Hybrid Hand Pose EstimationYe Q Yuan S Kim T-K
Lecture Notes in Computer Science. vol. 9912, 346-361.
01-01-2016
Research Group
PhD Students
- Hong Sen Cao
CSR - Marco Comunità
Understanding and Modelling Audio Effects With Differentiable Methods - Yilan Dong
MMV - Luca Forneris
Closing The Loop in Automated Driving: Dataset Design, Scenario Synthesis, and Deployed Perception - Nicolas Guo
Towards Tonality-Aware Music Understanding: Modeling Complex Tonal Harmony - Alessandro Pighetti
Adversarial Policies For Automated Driving - Damith Senadeera
MMV - Christian Steinmetz
Controlling Audio Effects With Deep Learning - Kaiwei Wang
A Gan-Based Framework For Robust Traffic Data Imputation in Sparse Traffic Environments - Qing Wang
Multi-Modal Learning For Music Understanding - Jiahao Yang
MMV - Xiaohang Yang
MMV
News
No news items found.