On a Rayleigh-Faber-Krahn Inequality for the Regional Fractional Laplacian

Abstract

We study a Rayleigh-Faber-Krahn inequality for regional fractional Laplacian operators. In particular, we show that there exists a compactly supported nonnegative Sobolev function $u_0$ that attains the infimum (which will be a positive real number) of the set$$\left\{\iint_{\{u>0\}\times\{u>0\}}\frac{|u(x)-u(y)|^2}{|x-y|^{n+2\sigma}}dxdy:u\in H^{\sigma}(\mathbb{R}^n), \int_{\mathbb{R}^n}u^2=1, |\{u>0\}|\le 1\right\}$$Unlike the corresponding problem for the usual fractional Laplacian, where the domain of the integration is $\mathbb{R}^n \times \mathbb{R}^n$, symmetrization techniques may not apply here. Our approach is instead based on the direct method and new a priori diameter estimates. We also present several remaining open questions concerning the regularity and shape of the minimizers, and the form of the Euler-Lagrange equations.