The stack is not a standard TCP one, but it is a reduced customized stack that only sends packets blindly, without checksum.
That means the FPGA transmits TCP packets but does not wait for a reply from the sender....ok I take that with a pinch of salt!
One question why didn't you use UDP stack?
Back to the main question regarding RGMII transmit interface debug....
My guess is that the RGMII clock and data transmit signals are not always properly aligned leading to a 10% loss.
You must be aware about the RGMII clock skew that can be added to the rgmii_tx_clk at 3 different stages.
1. Added by the FPGA
2. Added via PCB traces
3. Added at the PHY by configuring a register commonly called RGMII Transmit Delay Register.
Now you must tell us how in your design is this delay being added. Also is the question as to whether you are adding the correct delay value in nano-seconds?
Then have your TCP stack send out frames one by one and I guess at the receive side you are using something like Wireshark to verify the frames where you observe a 10% loss?
Basically out of the 3 delay sources you must turn off 2 sources and rely only on one. Taking the hypothetical case that 1. and 2. are adding no delays, you do the delay setting at the PHY register and observe the Wireshark data. If you still see some % error, change the delay value again at the PHY and observe the Wireshark results again. At some point of time you will get 0% packet loss.
When I was working with RGMII interface, I kept the RGMII delay value fixed at the PHY. I did not care what amount of delay the PCB traces were adding to the rgmii_tx_* signals (as I had no way of finding it out). I was playing around with my FPGA design constraints file such that I was shifting all the non clock RGMII signals to get the required oversall shift relative to the clock.
Other people might suggest you other methods, but the basic thing is, you must at some part of your design add the skew between the RGMII transmit clock and the data signals. Then to get 0% packet loss you have to
find out experimentally (because you have no idea what delay is inserted by the PCB traces) which delay value suits the best!
My assumption for this reply - You have verified through simulation that your RGMII interface is working as expected. If not done, do it as a 1st priority without even thinking of anything else.