I'll try to sketch the connections between your bullet points with an easy example of a Coxeter group. You probably want to study some basic algebraic knowledge to follow this.
We examine the dihedral group of the hexagon $W=A_2$. This is a hexagon:

We pick two opposed vertices and consider the reflection given by the line through those two vertices, call it $s$. Take an "adjacent" reflection, call it $t$. Now $s$ and $t$ generate a group of order $6$ (you should convice yourself that this is true). This group is a Coxeter group.
- It has something to do with reflections.
- The Coxeter matrix and the Coxeter diagram are a way to encode the properties of the Coxeter group. In our case, we have the Coxeter matrix
$$M=\begin{pmatrix}2&3\\3&2\end{pmatrix}$$ and the Coxeter diagram $\circ \overset{3}{-} \circ$. The $2$'s in the matrix tell you that $s\circ s=t\circ t=\text{id}$ and the $3$'s in the matrix and in the diagram tell you that $(s\circ t)^3=\text{id}$.
- This is made precise with the presentation of the Coxeter group. In our case, the presentation is $W \cong \langle s,t \mid s^2=t^2=ststst=1\rangle$. This presentation is an abstract way to define the Coxeter group: Take all finite words in the letters $s$ and $t$. This would be set contaning elements as $s$, $sts$, $tttss$, $ttssttttsst$ or even the empty word $\varepsilon$. Now we introduce an equivalence relation: We call two words equivalent, if one can be obtained from the other by deleting or inserting $ss$, $tt$ or $ststst$. Concatenation of two words gives a multiplication: $st*sst=stsst \sim stt \sim s$. It turns out that, modulo this equivalence relation, words in $s$ and $t$ are a group. And it is isomorphic to the reflection group we defined above.
To summerize: Coxeter diagram and Coxeter matrix are a tool to encode the presentation of the Coxeter group. Each Coxeter group has such a special representation. Each Coxeter group can be realized geometrically as a group generated by reflection of "something".
Edit: To answer the question how the "non-reflection" look like: The element $st\in A_2$ is not a reflection, it is a rotation by $120^\circ$(the top vertex at $0^\circ$ gets mapped to $240^\circ$, the $60^\circ$-vertex gets mapped to $300^\circ$). In general, the reflections of a Coxeter group (often denoted by $T$, in contrast to the set of generating, simple reflections $S$) are precisely the conjugates of the generators, i.e. $s$, $t$ and $sts=tst$ in our example.