Caution is called for. Li-ion are the type in news reports for causing fires and explosions.
You ought to find out what is a safe maximum voltage and safe maximum charging current for your battery.
It appears you want to have simultaneous current and voltage control. You may need to choose which one takes precedence over the other.
For a battery under 4.2v you can charge it by applying 4.2v (and wise to put a safety resistor inline). Now imagine the battery has reached 4.2V. How many Amperes can 4.2v supply push into the battery? None. A constant current control cannot come into play then. And we aren't sure if the battery is fully charged at that time.
As for charging ordinary nicads and NIMH type (nominal 1.25v), a voltage reading can go higher than 1.25v as you charge the cell (maybe as high as 2v for an aging cell). Then you stop after a certain amount of time. Eventually its voltage settles to 1.25-1.45v.
Try experimenting with nicads or nimh first, since they can tolerate abuse better than Li-on type. Your home-brew charge circuit should be designed with an aim to duplicate 'smart' chargers according to Li-ion safety practices.