Don't be scared :)
It's normal not to understand everything at first, Machine Learning (ML) is a vast domain and nobody knows everything about it.
My advice: pace yourself, don't try to learn everything at once. start from the most simple concepts and methods and understand them really well before going to more advanced topics. There are plenty of resources available for starting with ML, including maths courses for people with a Computer Science background. Generally it's recommended to have a decent level in linear algebra, statistics and probabilities.
But also don't underestimate the general concepts which are specific to ML, it's important to have a really good understanding of concepts such as supervised vs. unsupervised, classification vs. regression, training/testing/evaluation, etc.
Finally keep in mind that ML (and data science in general) is a highly experimental domain, so hands-on experience is key: reproduce a tutorial or code for a problem that you like, and play with it: see what happens if you change the parameters, the data, the evaluation measure...