I quote a few excerpts from the foreword (by C. P. Snow) to A Mathematician's Apology (by G. H. Hardy) (emphasis in italics mine) :
One morning early in 1913, he found among the
letters on his breakfast table, a large untidy envelope decorated with Indian stamps. When he
opened it, he found sheets of paper by no means fresh, on which, in a non-English holograph, were line after line of symbols. Hardy glanced at them without enthusiasm.
So Hardy felt, more than anything, bored. He glanced at the letter, written in halting English,
signed by an unknown Indian, asking him to give an opinion of these mathematical discoveries.
The script appeared to consist of theorems, most of them wild or fantastic looking, one or two
already well-known, laid out as though they were original. There were no proofs of any kind.
Hardy was not only bored, but irritated. It seemed like a curious kind of fraud.
That particular day, though, while the timetable wasn’t altered, internally the things were not going according to plan. At the back of his mind, getting in the way of his complete pleasure in his game, the Indian manuscript nagged away. Wild
theorems. Theorems such as he had never seen before, nor imagined. A fraud of genius? A
question was forming in his mind. As it was Hardy’s mind, the question was forming itself with
epigrammatic clarity: is a fraud of genius more probable than an unknown mathematician of
genius? Clearly the answer was no. Back in his rooms in Trinity, he had another look at the script.
He sent word to Littlewood that they must have a
discussion after hall.
Anyway, by nine o’clock or so they were in one of Hardy’s rooms, with the manuscript stretched out in front of them. Apparently it did not take them long. Before midnight they knew, and knew for certain. The writer of these manuscripts was a man of genius. That was as much as they could judge, that night.
Fron the above it should be clear that at first Hardy got really bored and irritated with Ramanujan's letter. But somehow the theorems of the letter got registered in his mind and this was probably due to extreme nature of the theorems (wild, not seen before, couldn't be imagined by Hardy).
When Hardy and Littlewood met together they probably tried to verify some of the theorems and concluded that there was some genuine mathematics involved. In particular the theorems on Rogers-Ramanujan continued fraction were the most difficult. Hardy remarked that "they defeated him completely". The quote by Hardy in your question is also related to the same Rogers-Ramanujan continued fraction formulas.
I have shared some of my thoughts on Ramanujan in a blog post and I have mentioned there something about the nature of Ramanujan's formulas:
- meaning of the formula could be understood by anyone with basic knowledge of algebra and calculus
- focus was on special cases of general formulas with actual numbers rather than general formula itself
- minimal use of symbolism and wherever possible indicate a pattern by exhibiting it numerically or by writing about the pattern in English rather than describing pattern via a formula
- each formula had a certain unexpectedness providing a shock treatment to the reader
- most of the formulas had deep theories behind them
- avoiding use of $\sum$ and $\prod$ symbols to represent infinite series and products
I think these qualities are reasonable enough to impress some people. It is also well known that Ramanujan sent the same letter to H. F. Baker and E. W. Hobson and they did not give any feedback. So probably the formulas couldn't impress everyone.