There are some important theorems about compact sets that can simplify your work:
If $A$ is closed, $A\subseteq B$, and $B$ is compact, then $A$ is compact
If $A$ is compact and $f:A\to B$ is continuous, then $f(A)$ is compact.
If $A$ is compact, $A\subseteq B$, and $B$ is Hausdorff, then $A$ is closed.
The complete proof can be given using these things, but I will just prove one direction.
A closed cell is compact. To see this, note that the map used for attachment goes from $D^n$ to $X$ is continuous. Since images of compact sets under continuous maps are compact, and $D^n$ is compact, we get that a closed cell is compact.
Any finite subcomplex of $X$ is compact. Since closed cells are closed and compact, the disjoint union of finitely-many of them is closed and compact. This means that any finite subcomplex of $X$ is comapct. Since any closed subset of a compact set is comapct$-$and $A$ is closed$-$it suffices to show that $A$ is contained in a finite subcomplex of $X$.
This is immediate from the hypotheses that $A$ intersects only finitely-many open cells. Simply take the subcomplex formed by using the closures of these open cells, of which there are finitely-many.
For the other direction, you can try to prove $X$ is Hausdorff by showing each of its $n-$skeletons is Hausdorff (use induction) and then use the fact that $A$ is compact to show $A$ is closed. To show that $A$ intersects finitely-many open cells, the argument is less intuitive, but you can pick a point in each open cell that $A$ intersects, and show the collection of all such points will be closed, and that any subset of this collection is closed. This makes the collection of these points discrete and, since it is closed, it will be compact because $A$ is assumed to be compact in this case. But any discrete compact space must be finite by the open cover definition you give above.