I would highly suggest reading Andre's book.
Let us say that a sheaf $\mathcal{O}$ on a topological space $X$ is non-degenerate if whenever $U\subseteq X$ is a non-empty open subset, one has that $\mathcal{O}(U)$ is non-zero. This assumption is completely reasonable if, for instance, you desire that $\mathcal{O}(U)$ should contain 'all constant functions on $U$' (where this is intentionally vague, but is clear in all examples).
So, let us make the following trivial observation
Claim: If $\mathcal{O}$ is non-degenerate and $\mathcal{O}(X)$ is an integral domain (or more specifically 'connected' -- i.e. that $\mathrm{Spec}(\mathcal{O}(X))$ is connected) then $X$ is connected.
Proof: Indeed, suppose that $X=U\sqcup V$ where $U,V\subseteq X$ are open. Then, $\mathcal{O}(X)=\mathcal{O}(U)\times\mathcal{O}(V)$ and by our assumption on $\mathcal{O}(X)$ we see that $\mathcal{O}(U)$ or $\mathcal{O}(V)$ is zero, and so one of $U$ or $V$ is empty. $\blacksquare$
So then, where is the issue with disconnectedness in rigid geometry? Let $K$ be a non-archimedean field (maybe algebraically closed for total realism) and let us denote
$$\mathbb{B}^1(K):=\{x\in K: |x|\leqslant 1\},$$
endowed with the subspace topology of $K$ (where $K$ itself is given the topology associated to $|\cdot|$). One then wants to define a sheaf of analytic functions $\mathcal{O}$ on $\mathbb{B}^1(K)$ and one expects that
one has the equality $$\mathcal{O}(\mathbb{B}^1(K))=K\langle x\rangle:=\left\{\sum_n a_n x^n : \lim |a_n|=0\right\},$$
the ring of convergent (on the unit ball) power series,
for all non-empty open subsets $U$ of $\mathbb{B}^1(K)$ one has that $\mathcal{O}(U)$ contains the set of constant functions $U\to K$ (and in-particular is non-zero).
From 2. we see that $\mathcal{O}$ is non-degenerate, but as $\mathcal{O}(\mathbb{B}^1(K))$ is an integral domain by 1. this is a contradiction as $\mathbb{B}^1(K)$ is highly disconnected as $K$ is non-archimedean.
PS: At this point there are many flavors of rigid geometry: Tate, Raynaud, Berkovich, Huber, Fujiwara--Kato,... and all of them aim to 'fix' this problem in one way or another. Saying that Berkovich was the first to do so is anachronistic, although Berkovich's approach was the first to fix this problem by modifying the space $\mathbb{B}^1(K)$ (to what is called $\mathbb{B}^{1,\mathrm{Berk}}$) in a literal sense opposed to the more indirect sense of Tate (i.e. having $\mathcal{O}$ only be defined on certain 'admissible opens' in $\mathbb{B}^1(K)$).