I'm learning to use some methods of complex analysis, solving some problems.
Could you give me a hint to solve the following problem?
$f$ is holomorphic in $D^2=\{z: |z|<1\}$ and continious in $\partial D^2\cup D^2$. Also, there is an open subset $U$ of $\partial D^2$ such as $f|_U=0.$ I am to prove $f|_U=0$.
Unfortunately, I have a lack of techniques, but I think somethin like maximum modulus principle would be useful. Perhaps there is some general result?
The maximum modulus principle is indeed useful to prove that under the assumptions (including $U \neq \varnothing$), you have $f \equiv 0$.
An open subset $U$ of the boundary contains the image of an interval $[a, a + 2\pi/n]$ under $t \mapsto e^{it}$, and with $\zeta = e^{2\pi i/n}$, consider
$$g(z) = \prod_{k=0}^{n-1} f(z\cdot \zeta^k).$$
$g$ is (easily seen to be) holomorphic in the unit disk, and continuous on the boundary. Its boundary values are, by construction, very simple.
An alternative way to prove it:
Since $f$ is continuous on $D^2 \cup \partial D^2$, it is the Cauchy integral of its boundary values,
$$f(z) = C_f(z) = \frac{1}{2\pi i} \int_{\lvert \zeta\rvert = 1} \frac{f(\zeta)}{\zeta - z}\, d\zeta.$$
Since $f$ vanishes on $U$ (which we may assume relatively open in $\partial D^2$), the Cauchy integral defines a holomorphic function on $D^2 \cup U \cup \left(\mathbb{C}\setminus \overline{D^2}\right)$ that vanishes identically on a non-discrete set (namely $U$).
