I don't think the phone will have enough power to handle the speakers.
One question:
Do you need batteries to operate the speakers ?
If yes, most probably your connections are not correct.
It will be helpful if you can post a drawing of what you have made.
Added after 3 minutes:
Oh, my mistake.
I did not understand your post the first time.
If you say there is sound but not too loud, then GrandAlf is correct (Sorry)
Anyway it will be helpful if you post a picture of your connections.
Cheers.