I highly recommend Introduction to Manifolds by Loring Tu. The reason is that he specifically delays the need for topology so as to get into the meat of the subject quicker. This means that he develops the material on $\mathbb{R}^n$ first, and then uses these results plus minimal topology to generalize to manifolds. Tu's book really is the best to get into the subject quickly and concisely, particularly someone without a course in topology. I learned the material from a course that used Lee but used Tu as a reference for qualifying exams and also as a reference since then.
Introduction to Smooth Manifolds by John M. Lee is good, but it is more verbose, less focused, and requires a lot more topology to get through. Tu's book covers the same core material.
Given your background, you have the minimal prequisities for Tu, but it might be a little tough at first recalling a lot of your vector calculus and linear algebra. You will have to pick up pieces of algebra, analysis, and topology along the way, but I don't see a reason why you need to learn all of that before trying out Tu. Just go through it and read up on subjects when you need to.
If you truly find yourself stuck on Tu, then I recommend the following two books to help out:
- Advanced Calculus: A Geometric View by James Callahan
- Vector Calculus, Linear Algebra, and Differential Forms: A Unified Approach by Hubbard and Hubbard
Both of these books should get you back on track regarding any missing material from your background. They are both excellent.
Other books to be aware of regarding supplemental material are:
- The Geometry of Physics by Theodore Frankel - Really fantastic book, but I haven't gone through all of it as I have Tu and Lee. It is quite comprehensive, and there isn't a heavy emphasis (from what I have read) on proofs. Thus there are a ton of examples and applications.
- Applied Differential Geometry by William Burke - Lots of great pictures and insight. It's too quirky to use as your main learning source though, but it's great for supplemental insight.
- Geometrical Vectors by Gabriel Weinreich - This is a very short but insightful read that would help put vectors, covectors, etc. into context.
Three other books on the subject that I am aware of but have no experience with:
- First Steps in Differential Geometry by Andrew McInerney
- Manifolds and Differential Geometry by Jeffrey M. Lee
- Differential Geometry: Curves - Surfaces - Manifolds by Wolfgang Kuhnel