Hi,
Yes, unclear where this unsymmetry in the joystick signal is coming from.
In values:
* center is 2.5V.
* most low is 0.5V, which is 2.0V from the center
* most high is 4.98V which is 2.48V from center.
The 2.0V vs 2.48V is the problem a simple Opamp circuit can not accomplish.
So now it's on the OP to decide what to do.
* leave it as is, then 0.5V --> 4.5V, 2.5V --> 2.5V, 4.98V --> 0.02V (with a suitable RR out Opamp)
* adjust the center to (0.5 + 4.98)/2 = 2.79V, to make 0.5V --> 4.98V, 4.98V --> 0.5V, ... but 2.5V --> 2.21V
* or a more advance solution to "switch between two different gain" maybe with the use of a comparator and an analog switch.
Klaus