Assume, as in the question, that $A$ is symmetric (real) positive definite and $B$ is symmetric. All matrices are real. All numbers are real. The set of all matrices on the form
$A+\epsilon B$ is called a matrix pencil, there is a wikipedia aricle on that (too short).
This can be used to define a generalized eigenvalue problem:
$$
Bv=\lambda A v
$$
solutions $\lambda $ are called generalized eigenvalues and solutions $v$ are called generalized eigenvectors (note that this terms also has other meanings!). We can define a
generalized Rayleigh ratio as
$$
R(B,A,x) = \frac{x^T Bx}{x^T Ax}
$$
Since A is positive definite, this will always be defined when $x \not=0$. Let $\delta$ be the minimum generalized eigenvalue. Then we have, in analogy with the situation for the ordinary Rayleigh quotient (there is a Wikipedia article), that
$$
R(B,A,x) \ge \delta \text{~~when $x\not= 0$.}
$$
What does $\delta$ say about $B$? First, note that $\delta=0$ then $B$ is positive semidefinite (and singular). In that case we can see that
$$
x^T(A+\epsilon B)x >0
$$ as long as $\epsilon \ge 0$.
Then we can look at the case $\delta >0$: Let $\mu_0>0$ be the smallest eigenvalue of $A$.
Vi har
$$
x^T Bx \ge \delta x^T Ax \ge \delta \mu_0 x^Tx
$$
slik at
$\begin{multline}
\frac{x^T (A+\epsilon B) x}{x^T x} = (x^T Ax+\epsilon x^T Bx )/x^Tx \\
\ge (x^T Ax +\epsilon \delta \mu_0 x^Tx)/x^T x \\
\ge (\mu_0 x^Tx + \epsilon \delta \mu_0 x^T x)/x^T x \\
\ge \mu_0 (1+\epsilon \delta) > 0
\end{multline} $
and solving the inequality for $\epsilon$ gives
$$
\epsilon > -\frac{1}{\delta}
$$
so we can conclude that $A+\epsilon B$ is positive definite if that inequality is fulfilled.
You can solve for the case $\delta <0$ in like manner!
