Loading [MathJax]/extensions/MathMenu.js
Simon Niklaus - IEEE Xplore Author Profile
Subscribe
ADVANCED SEARCH

Author details

< Back

Simon Niklaus

Publications
13
Citations
2,015
Publications by Year
20172024
Co-Authors:
Show All Co-Authors (24)
Image of author Simon Niklaus

Simon Niklaus

Affiliation

Adobe
San Jose
CA, USA

Publication Topics

Biography

Simon Niklaus received the PhD degree from Portland State University, in 2020 with his dissertation topic covering novel view synthesis in time and space. He is a research scientist with Adobe where he is working on video processing and novel view synthesis.(Based on document published on 24 October 2023).
Publications
13
Citations
2,015
Publications by Year
20172024
Co-Authors:
Show All Co-Authors (24)
Author's Published Works

Showing 1-13 of 13 results

  • Sort

Filter Results

Show

Sort By

Results

Video Frame Interpolation With Many-to-Many Splatting and Spatial Selective Refinement

Ping Hu;Simon Niklaus;Lu Zhang;Stan Sclaroff;Kate Saenko

IEEE Transactions on Pattern Analysis and Machine Intelligence
Year: 2024 | Volume: 46, Issue: 2 | Journal Article |
Cited by: Papers (3)
In this work, we first propose a fully differentiable Many-to-Many (M2M) splatting framework to interpolate frames efficiently. Given a frame pair, we estimate multiple bidirectional flows to directly forward warp the pixels to the desired time step before fusing any overlapping pixels. In doing so, each source pixel renders multiple target pixels and each target pixel can be synthesized from a la...Show More

Video Frame Interpolation With Many-to-Many Splatting and Spatial Selective Refinement

Ping Hu;Simon Niklaus;Lu Zhang;Stan Sclaroff;Kate Saenko

Year: 2024 | Volume: 46, Issue: 2 | Journal Article |

Splatting-based Synthesis for Video Frame Interpolation

Simon Niklaus;Ping Hu;Jiawen Chen

2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
Year: 2023 | Conference Paper |
Cited by: Papers (11)
Frame interpolation is an essential video processing technique that adjusts the temporal resolution of an image sequence. While deep learning has brought great improvements to the area of video frame interpolation, techniques that make use of neural networks can typically not easily be deployed in practical applications like a video editor since they are either computationally too demanding or fai...Show More

Splatting-based Synthesis for Video Frame Interpolation

Simon Niklaus;Ping Hu;Jiawen Chen

Year: 2023 | Conference Paper |

Towards Accurate Reconstruction of 3D Scene Shape From A Single Monocular Image

Wei Yin;Jianming Zhang;Oliver Wang;Simon Niklaus;Simon Chen;Yifan Liu;Chunhua Shen

IEEE Transactions on Pattern Analysis and Machine Intelligence
Year: 2023 | Volume: 45, Issue: 5 | Journal Article |
Cited by: Papers (26)
Despite significant progress made in the past few years, challenges remain for depth estimation using a single monocular image. First, it is nontrivial to train a metric-depth prediction model that can generalize well to diverse scenes mainly due to limited training data. Thus, researchers have built large-scale relative depth datasets that are much easier to collect. However, existing relative de...Show More

Towards Accurate Reconstruction of 3D Scene Shape From A Single Monocular Image

Wei Yin;Jianming Zhang;Oliver Wang;Simon Niklaus;Simon Chen;Yifan Liu;Chunhua Shen

Year: 2023 | Volume: 45, Issue: 5 | Journal Article |

Many-to-many Splatting for Efficient Video Frame Interpolation

Ping Hu;Simon Niklaus;Stan Sclaroff;Kate Saenko

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Year: 2022 | Conference Paper |
Cited by: Papers (46)
Motion-based video frame interpolation commonly relies on optical flow to warp pixels from the inputs to the desired interpolation instant. Yet due to the inherent challenges of motion estimation (e.g. occlusions and discontinuities), most state-of-the-art interpolation approaches require subsequent refinement of the warped result to generate satisfying outputs, which drastically decreases the eff...Show More

Many-to-many Splatting for Efficient Video Frame Interpolation

Ping Hu;Simon Niklaus;Stan Sclaroff;Kate Saenko

Year: 2022 | Conference Paper |

Layered Depth Refinement with Mask Guidance

Soo Ye Kim;Jianming Zhang;Simon Niklaus;Yifei Fan;Simon Chen;Zhe Lin;Munchurl Kim

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Year: 2022 | Conference Paper |
Cited by: Papers (5)
Depth maps are used in a wide range of applications from 3D rendering to 2D image effects such as Bokeh. However, those predicted by single image depth estimation (SIDE) models often fail to capture isolated holes in objects and/or have inaccurate boundary regions. Meanwhile, high-quality masks are much easier to obtain, using commercial auto-masking tools or off-the-shelf methods of segmentation ...Show More

Layered Depth Refinement with Mask Guidance

Soo Ye Kim;Jianming Zhang;Simon Niklaus;Yifei Fan;Simon Chen;Zhe Lin;Munchurl Kim

Year: 2022 | Conference Paper |

Learning to Recover 3D Scene Shape from a Single Image

Wei Yin;Jianming Zhang;Oliver Wang;Simon Niklaus;Long Mai;Simon Chen;Chunhua Shen

2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Year: 2021 | Conference Paper |
Cited by: Papers (123)
Despite significant progress in monocular depth estimation in the wild, recent state-of-the-art methods cannot be used to recover accurate 3D scene shape due to an unknown depth shift induced by shift-invariant reconstruction losses used in mixed-data depth prediction training, and possible unknown camera focal length. We investigate this problem in detail, and propose a two-stage framework that f...Show More

Learning to Recover 3D Scene Shape from a Single Image

Wei Yin;Jianming Zhang;Oliver Wang;Simon Niklaus;Long Mai;Simon Chen;Chunhua Shen

Year: 2021 | Conference Paper |

Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes

Zhengqi Li;Simon Niklaus;Noah Snavely;Oliver Wang

2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Year: 2021 | Conference Paper |
Cited by: Papers (376)
We present a method to perform novel view and time synthesis of dynamic scenes, requiring only a monocular video with known camera poses as input. To do this, we introduce Neural Scene Flow Fields, a new representation that models the dynamic scene as a time-variant continuous function of appearance, geometry, and 3D scene motion. Our representation is optimized through a neural network to fit the...Show More

Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes

Zhengqi Li;Simon Niklaus;Noah Snavely;Oliver Wang

Year: 2021 | Conference Paper |

Revisiting Adaptive Convolutions for Video Frame Interpolation

Simon Niklaus;Long Mai;Oliver Wang

2021 IEEE Winter Conference on Applications of Computer Vision (WACV)
Year: 2021 | Conference Paper |
Cited by: Papers (35)
Video frame interpolation, the synthesis of novel views in time, is an increasingly popular research direction with many new papers further advancing the state of the art. But as each new method comes with a host of variables that affect the interpolation quality, it can be hard to tell what is actually important for this task. In this work, we show, somewhat surprisingly, that it is possible to a...Show More

Revisiting Adaptive Convolutions for Video Frame Interpolation

Simon Niklaus;Long Mai;Oliver Wang

Year: 2021 | Conference Paper |

Learned Dual-View Reflection Removal

Simon Niklaus;Xuaner Cecilia Zhang;Jonathan T. Barron;Neal Wadhwa;Rahul Garg;Feng Liu;Tianfan Xue

2021 IEEE Winter Conference on Applications of Computer Vision (WACV)
Year: 2021 | Conference Paper |
Cited by: Papers (10)
Traditional reflection removal algorithms either use a single image as input, which suffers from intrinsic ambiguities, or use multiple images from a moving camera, which is inconvenient for users. We instead propose a learning-based dereflection algorithm that uses stereo images as input. This is an effective trade-off between the two extremes: the parallax between two views provides cues to remo...Show More

Learned Dual-View Reflection Removal

Simon Niklaus;Xuaner Cecilia Zhang;Jonathan T. Barron;Neal Wadhwa;Rahul Garg;Feng Liu;Tianfan Xue

Year: 2021 | Conference Paper |

Softmax Splatting for Video Frame Interpolation

Simon Niklaus;Feng Liu

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Year: 2020 | Conference Paper |
Cited by: Papers (255)
Differentiable image sampling in the form of backward warping has seen broad adoption in tasks like depth estimation and optical flow prediction. In contrast, how to perform forward warping has seen less attention, partly due to additional challenges such as resolving the conflict of mapping multiple pixels to the same target location in a differentiable way. We propose softmax splatting to addres...Show More

Softmax Splatting for Video Frame Interpolation

Simon Niklaus;Feng Liu

Year: 2020 | Conference Paper |

Context-Aware Synthesis for Video Frame Interpolation

Simon Niklaus;Feng Liu

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
Year: 2018 | Conference Paper |
Video frame interpolation algorithms typically estimate optical flow or its variations and then use it to guide the synthesis of an intermediate frame between two consecutive original frames. To handle challenges like occlusion, bidirectional flow between the two input frames is often estimated and used to warp and blend the input frames. However, how to effectively blend the two warped frames sti...Show More

Context-Aware Synthesis for Video Frame Interpolation

Simon Niklaus;Feng Liu

Year: 2018 | Conference Paper |

Video Frame Interpolation via Adaptive Separable Convolution

Simon Niklaus;Long Mai;Feng Liu

2017 IEEE International Conference on Computer Vision (ICCV)
Year: 2017 | Conference Paper |
Cited by: Papers (484)
Standard video frame interpolation methods first estimate optical flow between input frames and then synthesize an intermediate frame guided by motion. Recent approaches merge these two steps into a single convolution process by convolving input frames with spatially adaptive kernels that account for motion and re-sampling simultaneously. These methods require large kernels to handle large motion,...Show More

Video Frame Interpolation via Adaptive Separable Convolution

Simon Niklaus;Long Mai;Feng Liu

Year: 2017 | Conference Paper |

Video Frame Interpolation via Adaptive Convolution

Simon Niklaus;Long Mai;Feng Liu

2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Year: 2017 | Conference Paper |
Video frame interpolation typically involves two steps: motion estimation and pixel synthesis. Such a two-step approach heavily depends on the quality of motion estimation. This paper presents a robust video frame interpolation method that combines these two steps into a single process. Specifically, our method considers pixel synthesis for the interpolated frame as local convolution over two inpu...Show More

Video Frame Interpolation via Adaptive Convolution

Simon Niklaus;Long Mai;Feng Liu

Year: 2017 | Conference Paper |

IEEE Account

Purchase Details

Profile Information

Need Help?

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity.
© Copyright 2025 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.