A parameter is simply an element of a structure.
Usually, we use the word "parameter" in a context where we are intepreting some (but not necessarily all) of the free variables in a formula. For example, let $\varphi(x,y)$ be a formula with free variables $x$ and $y$ (or let $x$ and $y$ be finite tuples of free variables). Let $M$ be a structure, and $b\in M$ an element (or a tuple of the same length as $y$). Then $\varphi(x,b)$ is a formula with parameter $b$. Note that $\varphi(x,b)$ is no longer a purely syntactic object. But this is no problem: formally, we can view it as the formula $\varphi(x,y)$ together with a partial interpretation of the free variables by elements of $M$ (namely, $y$ is interpreted as $b$, but $x$ is left uninterpreted).
What do we do with formulas with parameters? We ask whether elements/tuples from $M$ satisfy them. And this question makes sense: Let $a\in M$ be an element (or a tuple of the same length as $x$). Then we say $a$ satisfies $\varphi(x,b)$ if and only if $M\models \varphi(a,b)$, i.e., if and only if $(a,b)$ satisfies $\varphi(x,y)$. Formally, we interpret the remaining free variables $x$ as $a$, and we apply the definition of satisfaction in $M$. The set defined by $\varphi(x,b)$ is $\varphi(M,b) = \{a\in M\mid M\models \varphi(a,b)\}$.
Now there are situations in which it can be conceptually or technically useful to view a formula with parameters as a purely syntactic object. For example, usually the compactness theorem is proven for theories (sets of sentences), and then we want to apply it to realize types (sets of formulas with parameters). In this sort of situation, we can (1) introduce a new constant symbol $c_b$ (or a tuple of constant symbols of the same length as $y$) to get a language $L(b)$, (2) expand $M$ to an $L(b)$-structure $M'$ by interpreting the new constant as $b$, and (3) substitute $c_b$ for $y$ in $\varphi(x,y)$, to get a new formula $\varphi(x,c_b)$, which is an $L(b)$-formula without parameters.
Some people might prefer to always do this, and to define a parameter to be a constant symbol which is interpreted as a designated element in some structure. It doesn't matter so much: these two points of view are essentially interchangeable, since for all $a\in M$, $$M\models \varphi(a,b)\iff M'\models \varphi(a,c_b).$$ I just find it easier to think directly about elements of structures, rather than changing the language all the time.