Skip to content

Add smoothed TV inpainting demo#321

Open
jb2178-star wants to merge 2 commits into
jorgensd:mainfrom
jb2178-star:tv-inpainting-demo
Open

Add smoothed TV inpainting demo#321
jb2178-star wants to merge 2 commits into
jorgensd:mainfrom
jb2178-star:tv-inpainting-demo

Conversation

@jb2178-star
Copy link
Copy Markdown

@jb2178-star jb2178-star commented Apr 24, 2026

Summary

This demo was originally proposed for the DOLFINx main demo repository, but following maintainer feedback, it is submitted here as a more application-focused example. This PR adds a demo illustrating the smoothed total variation (TV) image inpainting using DOLFINx. The example consideres a sythnetic image defined on the unit square with an irregular interior mask, and reconstructs missing data using a nonlinear variational formulation.

Starting from a variational inpainting model from Chan and Shen, we define the energy functional:

$$ J(u)={1\over 2 }\beta \int_{\Omega}m(u-f)^2\mathrm{d}x+\alpha \int_{\Omega}\sqrt{||\nabla u ||^2 +\varepsilon^2}\mathrm{d}x $$

Where the first term enforces data fidelity on know regions, weighted with $\beta$, on the know regions with mask $m$, and the second term is a smoothed total variation regularization inspired by Rudin, Osher, Fatemi, weight with $\alpha$:

$$ ||\nabla u||\to \sqrt{||\nabla u||^2 +\varepsilon^2} $$

The smoothing parameter $\varepsilon>0$ in introduced to make the TV term differentiable, enabling the use of Newton methods (second order).

Taking the Euler-Lagrange equation associated with $J(u)$ yields the nonlinear PDE:

$$ \beta m(u-f)+\alpha {\nabla \cdot {\nabla u}\over \sqrt{||\nabla u||^2+\varepsilon^2}}=0 $$

Which leads to the weak formulation: find $u\in V$ such that:

$$ \beta \int_{\Omega} m(u-f)v\mathrm{d}x + \alpha \int_{\Omega}{\nabla u \cdot \nabla v \over \sqrt{ ||\nabla u||^2 +\varepsilon^2} }\mathrm{d}x=0 \quad \forall v$$

The demo includes:

  • A unit square mesh generated with DOLFINx
  • A synthetic ground truth image and irregular mask
  • A nonlinear variational formulation in UFL
  • A residual and Jacobian via ufl.derivative
  • A nonlinear solve using dolfinx.fem.petcs.NonlinearProblem
  • Visualization of the reconstruction
  • Evaluation metrics to measure convergence and reconstruction quality

Notes

The example is intended as a tutorial style demonstration of using DOLFINx for nonlinear variational problems in imaging. Further extensions could be added such as solving for F, the weak form with ufl.derivative, although I'd prefer it remain explicitly in terms of the weak form to emphasize the connection between the mathematical model and its implementation.

@jb2178-star
Copy link
Copy Markdown
Author

Hi @jorgensd,

It seemed like chapter 3 would be the best place for this?

Thanks again.

@jb2178-star
Copy link
Copy Markdown
Author

Hi again @jorgensd ,

Was there anything else that you though was need or need to be changed?

Thanks

@jorgensd
Copy link
Copy Markdown
Owner

jorgensd commented May 5, 2026

Hi again @jorgensd ,

Was there anything else that you though was need or need to be changed?

Thanks

Hi @jb2178-star, I've been quite busy lately and will hopefully have time to review this during this week.

@jb2178-star
Copy link
Copy Markdown
Author

@jorgensd
Thank you again for your time and effort, I understand. Hopefully your work goes well.

Copy link
Copy Markdown
Owner

@jorgensd jorgensd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments regarding both the variational formulation and the plotting.
I'll also push some general changes.

Comment thread chapter3/demo_smoothed_tv_inpainting.py Outdated
grad_u = ufl.grad(u)
tv_denom = ufl.sqrt(ufl.inner(grad_u, grad_u) + eps**2)

F = beta * m * (u - f) * v * ufl.dx + alpha * ufl.inner(grad_u, ufl.grad(v)) / tv_denom * ufl.dx
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't we want to just define J and then use ufl.derivative(J, u, dv) to get F?

Comment on lines +449 to +453

msh.topology.create_connectivity(msh.topology.dim, 0)
cells = msh.topology.connectivity(msh.topology.dim, 0)
triangles = np.array(cells.array, dtype=np.int32).reshape(-1, 3)
triang = mtri.Triangulation(x, y, triangles)
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not safe (or correct), as x and y stems from

coords = V.tabulate_dof_coordinates()
x, y = coords[:, 0], coords[:, 1]

If your coordinates x and y is based on this you should use triangles = V.dofmap.list.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @jorgensd ,
Thank you so much for review and suggesting changes. Im working on the revisions, will update this.

Copy link
Copy Markdown
Author

@jb2178-star jb2178-star May 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @jorgensd,

Thanks for the review. I pushed a commit addressing the requested changes.

I defined the smoothed TV energy functional first and used ufl.derivative to obtain the residual form; updated the plotting triangulation to use V.dofmap.list with coordinates from V.tabulate_dof_coordinates; updated the notebook and script together; removed some outputs that I had used to tune alpha.

I also ran the demo successfully after the changes.

I again really appreciate the time you've taken with me during this process.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants