Let $(M,g)$ be a Riemannian manifold and $S\subset M $ open and strongly convex, i.e. any two points $p,q\in \bar S$ are connected by a unique minimizing geodesic with interior completely lying in $S$.
Conjecture: There exists an open neighbourhood $S'$ of $\bar S$ which is also strongly convex.
New question (March 20): It seems like the conjecture is false in general and user Dap has suggested a counterexample as an answer. Can you find another counterexample, maybe a simpler one?
Some remarks and proof attempts:
For sets which are merely convex this is not true. For example an open hemisphere of $S^2$ cannot be enlarged without losing convexity. The strongly convex sets of $S^2$ on the other hand have their closure strictly contained in a hemisphere and thus can obviously be enlarged to a new strongly convex set. This picture (respectively the ones that I can image on other surfaces) leads me to the assumption that the question has a positive answer.
Strongly convex sets are totally normal, (i.e. for every point $p\in S$ there is an open set $V\subset T_pM$ such that $\exp_p\colon V\rightarrow S$ is a diffeomorphism). My first idea was to first find an $S''\supset \bar S$ totally normal and then try to restict it to a strictly convex set. But just working with total normality cannot work: $S^2 \backslash p_0$ is totally normal, but cannot be extended to a larger totally normal set. Unfortunately I don't know how strong convexity can be incorporated from the start.
Edit (March 17): Here is a false counterexample, that was posted as an (now deleted) answer. I find it interesting neverletheless: Let $M=\{(x,y)\vert y>-e^x\}\subset \mathbb{R}^2$ and $S$ the open upper half plane (which is only convex and not strongly convex). If $S'$ is a open set containing $\bar S$, then $(0,-\epsilon)\in S'$ for some $\epsilon >0$. Take $x> 0$ so large that $\epsilon> xe^{-x+1}$, then the straight line between $(-x,0)$ and $(0,-\epsilon)$ contains the point $(-x+1,-\epsilon/x)$, which lies outside of $M$. In particular $S'$ cannot be convex.