Dynamic Reconfiguration Port in Virtex 4 Transceivers.

[kaiserschmarren87]

Hi,

I have a DRP design for Virtex-4 transceiver. I would like to know is it possible to have a single .bit file and use it for several different bit rates (i.e. 3.125 Gbps, 4.8 Gbps, 6.4 Gbps etc.) ?

How to modify DRP to handle this ? Since I use rocketIO wizard generated .vhd files for transceiver configuration, I have enabled DRP in the wizard. So if I generate rocketIO wizard for 6.4 Gbps and then enable DRP in it, what changes in DRP module or which parameter in it needs to be changed so that I can use only one .bit file for several bit rates as mentioned above.

Thank you.

 

[ads-ee]

Read chapter 6 for information on how to control the DRP interface: https://www.xilinx.com/support/documentation/user_guides/ug071.pdf
Chapter 6 has some timing diagrams of how you will send data to it.

Read Appendix C for the registers accessible in the MGT and their DRP address: https://www.xilinx.com/support/documentation/user_guides/ug076.pdf

As there are such a large number of parameters in the V4 MGTs you might want to create the transceiver core(s) with each of the possible rates and take a look at the changes to the values for each of the parameters. Then it should be a lot easier to correlate that with the appropriate DRP registers you need to modify.


Regards

 

[kaiserschmarren87]

Read chapter 6 for information on how to control the DRP interface: https://www.xilinx.com/support/documentation/user_guides/ug071.pdf
Chapter 6 has some timing diagrams of how you will send data to it.

Read Appendix C for the registers accessible in the MGT and their DRP address: https://www.xilinx.com/support/documentation/user_guides/ug076.pdf

As there are such a large number of parameters in the V4 MGTs you might want to create the transceiver core(s) with each of the possible rates and take a look at the changes to the values for each of the parameters. Then it should be a lot easier to correlate that with the appropriate DRP registers you need to modify.


Regards

Those documents were helpful. From the first document, I could understand what those timing diagrams are.
The second document shows the memory map of DRP.

When I generate rocketIO wizard, I omit many parameters since they are not at all used. Hence many paramters get eliminated.

When you say "take a look at the changes to the values for each of the parameters", you mean the simulation output ? I have added the rocketIO wizard configuration of my project, I hope what I did is right in that. I am not using TX since I am receiving data from ADC.

 

[ads-ee]

When you say "take a look at the changes to the values for each of the parameters", you mean the simulation output ? I have added the rocketIO wizard configuration of my project, I hope what I did is right in that. I am not using TX since I am receiving data from ADC.

You take a look in the directory where you generate the various versions of the same MGT core. In that directory should be a file that has the definition of all the settings to generate that core again (I've forgotten the file extension for ISE). The file is text so you should be able to easily diff the files for each version of the core against whatever base version you use. Then you'll know exactly which settings get changed for each version.

Regards

 

[kaiserschmarren87]

You take a look in the directory where you generate the various versions of the same MGT core. In that directory should be a file that has the definition of all the settings to generate that core again (I've forgotten the file extension for ISE). The file is text so you should be able to easily diff the files for each version of the core against whatever base version you use. Then you'll know exactly which settings get changed for each version.

Regards

I have generated various versions of rocketIO file for the same MGT core. I have cmpared those 3 i.e. the 'rocketIO_wrapper.vhd' file and found only 4 parameters, which change for various input bitstreams.

They are:
1. RXOUTDIV2SEL - A:0x7D (DRP Address)
2. RXPLLNDIVSEL - A:0x7D (DRP Address)
3. RXRCPADJ - A:0x56 (DRP Address)
4. RXVCODAC_INIT - A/B: 0x68 (DRP Address)

I hope these are the parameters which needs to be handled by DRP to work for all the 3 versions. But UG076 Appendix C which you mentioned says that for 3rd and 4th parameters " This attribute should never be changed from the default setting. Otherwise the MGT can operate below optimum levels, compromising overall performance.". So these two parameters should not be changed at all ?

Also when I generate a rocketIO wrapper for 6.4 Gbps and synthesize the design and get a .bit file. Then the same bit file can be used to work for other versions too by using these DRP settings. Am I right ?

 

[ads-ee]

I'm not sure but I suspect you'll need to change them.

RXRCPADJ looks like it might be Receive recovered clock phase adjust. Check to see if the difference between the values is large or small, if it's small you can probably get away without changing it. I suspect it's probably not that small a difference between the values.

RXVCODAC_INIT seems to be part of the PLL adjustments based on the line rate and the reference clock, which probably needs to change if the line rate changes (which it does in your case)

Also when I generate a rocketIO wrapper for 6.4 Gbps and synthesize the design and get a .bit file. Then the same bit file can be used to work for other versions too by using these DRP settings. Am I right ?

That was the whole point of this, isn't it? I'm assuming you couldn't afford to download different bit files for the design or perform partial reconfiguration.

Regards

 

[kaiserschmarren87]

I'm not sure but I suspect you'll need to change them.

RXRCPADJ looks like it might be Receive recovered clock phase adjust. Check to see if the difference between the values is large or small, if it's small you can probably get away without changing it. I suspect it's probably not that small a difference between the values.

RXVCODAC_INIT seems to be part of the PLL adjustments based on the line rate and the reference clock, which probably needs to change if the line rate changes (which it does in your case)

Regards

RXRCPADJ value is small for different line rate and RXVCODAC_INIT varies too. But from the document, it says not to change these values. So I hope it is not needed as you suspected. But the other two parameters are important. So I will use them in the protocol.

The current design reads the DRP data and then writes the data back to FPGA. When I change the line rate and reset, DRP value doesn't change. So I guess the protocol from PC is not getting the changed line rate value ? Does it mean I have to give manual line rate value that will choose proper DRP data which will be sent to FPGA ?

I have written something like this in MatLab:

%read the frequency or line rate from the user
frequnecy = input(prompt);
if (frequency == 6.4)

   drp_data = 81;          %DRP Address 7D for MGTA and 59 for MGTB

elseif (frequency == 4.8)

   drp_data = A2;          %DRP Address 7D for MGTA and 59 for MGTB

elseif (frequency == 3.125)

   drp_data = A2;          %DRP Address 7D for MGTA and 59 for MGTB

end

I hope this way it will keep track of change in line rate.

 

[ads-ee]

RXRCPADJ value is small for different line rate and RXVCODAC_INIT varies too. But from the document, it says not to change these values. So I hope it is not needed as you suspected. But the other two parameters are important. So I will use them in the protocol.

I recommend they be changed. The point of the note is that the settings selected by the wizard shouldn't be changed manually from the defaults selected by the tool for the requested transceiver implementation. You actually want to use different implementations for different line rates, therefore you aren't manually changing them to something different but to the wizard values for those other implementations.

The current design reads the DRP data and then writes the data back to FPGA. When I change the line rate and reset, DRP value doesn't change. So I guess the protocol from PC is not getting the changed line rate value ? Does it mean I have to give manual line rate value that will choose proper DRP data which will be sent to FPGA ?
I have written something like this in MatLab:

I don't follow this. Are you saying that you read the DRP (how are you reading this with a HW circuit you added to the design that interfaces to a microblaze/eternal_UC/FSM/something_else?) then write the new values back to the DRP through a PC from Matlab? How do you do this? I have no idea how you expect to control the DRP through Matlab.

Perhaps you need to post a block diagram of the architecture of your system that interfaces Matlab to the DRP port.

Regards

 

[kaiserschmarren87]

I recommend they be changed. The point of the note is that the settings selected by the wizard shouldn't be changed manually from the defaults selected by the tool for the requested transceiver implementation. You actually want to use different implementations for different line rates, therefore you aren't manually changing them to something different but to the wizard values for those other implementations.

Oh k. Thank you for that info. I thought it should not be changed. But I observe that, those two parameters haven't been used in the current design. So I will try to implement those two parameters into the protocol and see the changes.

I don't follow this. Are you saying that you read the DRP (how are you reading this with a HW circuit you added to the design that interfaces to a microblaze/eternal_UC/FSM/something_else?) then write the new values back to the DRP through a PC from Matlab? How do you do this? I have no idea how you expect to control the DRP through Matlab.

Perhaps you need to post a block diagram of the architecture of your system that interfaces Matlab to the DRP port.

Regards

Here is the block diagram of the logic for DRP the data flow in the design.


In brief, PC sends a protocol command which requests DRP data from FPGA. In return FPGA sends the data back to PC. Then this data is modified(right now I do not see the value getting changed) and sent back to DRP module for controlling the transceiver module.

So with that in mind I am trying to include the MatLab code for varying line rates which will inform DRP module with the specified values. I hope what I have thought makes sense.

 

[ads-ee]

Okay now that I have a picture of the design it's clear.

So your serial interface (SPI?) is communicating to the DRP. You've run simulations on that circuit for both the read and write? The model you use to drive the serial interface was verified to be producing the correct protocol?

If you have access to chipscope you might want to connect the DRP to that. Otherwise another option is to create your own chipscope by connecting a FIFO to the DRP port and writing to the FIFO on every clock cycle after the start of a read or write transaction. You can connect the read side of the FIFO to your serial interface so you can read back the waveform in your Matlab user interface.

 

[kaiserschmarren87]

Okay now that I have a picture of the design it's clear.

So your serial interface (SPI?) is communicating to the DRP. You've run simulations on that circuit for both the read and write? The model you use to drive the serial interface was verified to be producing the correct protocol?

If you have access to chipscope you might want to connect the DRP to that. Otherwise another option is to create your own chipscope by connecting a FIFO to the DRP port and writing to the FIFO on every clock cycle after the start of a read or write transaction. You can connect the read side of the FIFO to your serial interface so you can read back the waveform in your Matlab user interface.

PC to FPGA interface is RS232. MatLab is used to send address and data information using a protocol. This protocol carries DRP address and data which it will write into the DRP fo the transceiver.

I reprogrammed the MatLab which has all 3 different line rate DRP config values. I used ChipScope to check the value in DO[15:0] and found that they change as expected. But now I don't get the synchronized output values even if I am changing the line rate and the DRP values. Now I am guessing there may be some other issues in the design apart from this which I expected.

 

[kaiserschmarren87]

PC to FPGA interface is RS232. MatLab is used to send address and data information using a protocol. This protocol carries DRP address and data which it will write into the DRP fo the transceiver.

I reprogrammed the MatLab which has all 3 different line rate DRP config values. I used ChipScope to check the value in DO[15:0] and found that they change as expected. But now I don't get the synchronized output values even if I am changing the line rate and the DRP values. Now I am guessing there may be some other issues in the design apart from this which I expected.

After working on changing the DRP value from PC using MatLab, now I am able to change the DRP values in the transceiver. The same changes are observed in the ChipScope tool also at DO_OUT[15:0]. I am using single bit file for the all the 3 different line rates which I am interested in at the moment(This bit file uses 6.4Gbps rocketIo wrapper with 400MHz REFCLK).

But after doing proper DRP changes, design works for 6.4Gbps correctly but adoesn't work for other 2 line rates. Has it got something to do with the REFCLK ? Since different line rates have different REFCLK, is it necessary to have specific REFCLK for those line rates or a +/- change in REFCLK is negligible ?

I have used:

for 3.2Gbps - REFCLK=400 MHz. -> RXRCPADJ="011".

for 4.2Gbps - REFCLK=420 MHz. -> RXRCPADJ="110".

for 3.2Gbps - REFCLK=400 MHz. -> RXRCPADJ="011"(?).

Also about RXRCPADJ parameter which is restricted and advised not to change it in the transceiver. So it should never be changes or can we modify it ? Since this parameter's value has specific value for specific frequency range, what if it has different value than what is expected ? Xilinx has provided RXRCPADJ value for range of input frequency values. But I got only till 5GHz so what about 5GHz or more, say for e.g. 6.4GHz ?

I am just putting the Xilinx info on RXRCPADJ here:

"

RXRCPADJ

This attribute affects only the Analog CDR.

2.488 GHz < Fvco <= 2.7 GHz - RXRCPADJ = 010.

2.7 GHz < Fvco <= 4.0 GHz - RXRCPADJ = 011.

4.0 GHz < Fvco <= 5 GHz - RXRCPADJ = 110.

Virtex-4 RocketIO Wizard v1.5 and later will set these attributes correctly."

 

[ads-ee]

In #8,I already recommended that any value that was different in the wizard created transceiver should be modified.

Regards

 

[kaiserschmarren87]

In #8,I already recommended that any value that was different in the wizard created transceiver should be modified.

Regards

I made the changes for parameters like RXRCPADJ and RXVCODAC_INT for corresponding line rates. The values are being loaded from MatLab correctly. Now it works fine for 3.2 Gbps and 6.4 Gbps but nothing is working on 4.2 Gbps line rate.

3.2 Gbps and 6.4 Gbps have same REFCLK and same RXRCPADJ and RXVCODAC_INT values. But 4.2 differs in REFCLK rate as well those two parameters. I don't know what went wrong. I made separate bit files for all the 3 line rates and noted the required parameter value. Then I used the bit file having 6.4 Gbps line rate and sending the modified DRP settings for the line rates.

Is BER (Bit Error Ratio) affecting this ?(I am speculating though it seems to be working fine).
RXUSRCLK and RXCLKSTABLE are the parameters which I do not use in the wizard. How can I make use of these parameters to make stable system configuration ?

 

[ads-ee]

You've tried running the FPGA with the 4.2Gbps line rate bit file directly? You're give a reset to the transceivers after DRP update?

Based on the diagram I assume you are changing the REFCLK using the PLL? When do you change the value before or after you perform the DRP update? I would probably put the transceiver in reset, change the PLL settings for the 4.2Gbps REFCLK, reprogram the transceiver settings with DRP, and then release the reset. Are you doing something different and or have you tried different ordering of programming/reset/REFCLK changes?

RXUSRCLK is a version of the recovered clock that is produced from the core. It's there so you can run the user interface side of your logic synchronously to the core. Don't know about RXCLKSTABLE but I suspect it's used like TXCLKSTABLE. They are probably used to keep things in reset until the clock goes stable. What does the datasheet say?

BER shouldn't affect anything. It should be something that tells you the amount of bit errors you are getting on the link. (I admit I haven't looked it up).

Regards

 

[kaiserschmarren87]

You've tried running the FPGA with the 4.2Gbps line rate bit file directly? You're give a reset to the transceivers after DRP update?

I have tried bit file with 4.2 Gbps separately and it works perfectly for the wizard's default values. No issues. I give a reset to the Transceiver, after the DRP update.

Based on the diagram I assume you are changing the REFCLK using the PLL? When do you change the value before or after you perform the DRP update? I would probably put the transceiver in reset, change the PLL settings for the 4.2Gbps REFCLK, reprogram the transceiver settings with DRP, and then release the reset. Are you doing something different and or have you tried different ordering of programming/reset/REFCLK changes?

I change the REFCLK value from the signal generator for corresponding line rates. My design doesn't use DCM. So I was planning to use DCM so that I can make use of RXCLKSTABLE and LOCKED combination(This is what I thought). The order of working is like this:
1. Change the line rate to 3.2 Gbps and change the REFCLK to 400 MHz.
2. Change the DRP values in the design(loading from PC).
3. Then make Transceiver reset. After this I get all channels synchronized.
Same procedure for other line rates too but 4.2 Gbps output doesn't get synchronized at all.

RXUSRCLK is a version of the recovered clock that is produced from the core. It's there so you can run the user interface side of your logic synchronously to the core. Don't know about RXCLKSTABLE but I suspect it's used like TXCLKSTABLE. They are probably used to keep things in reset until the clock goes stable. What does the datasheet say?

This is what user guide says about RXCLKSTABLE:
"Power-up or reset sequences affect simulation.
After reset or power-up, the inputs TX/RXCLKSTABLE going into the MGT must be
asserted. (These can be driven by a DCM locked signal, for example.) The TX/RXLOCK signals do not assert if the TX/RXCLKSTABLE signals do not go High. The TX/RXCLKSTABLE signals start the frequency calibration process whereby the PLL lock is determined."
I can choose it from the wizard and connect it to the DCM, if used which, I think might help but not sure.

BER shouldn't affect anything. It should be something that tells you the amount of bit errors you are getting on the link. (I admit I haven't looked it up).

No issue with BER but I just wanted to confirm.

 

[kaiserschmarren87]

Apart from things which I said in the previous comment, there is one more parameter called RXDIGRX which can be used according to our requirement. According to the manual it says as shown in the image.

In the current design it is chosen Automatic or Default (i.e. 0). With this, the design is working now perfectly for 6.4 Gbps and 3.2 Gbps (Thanks to the Transceiver 'RESET' which made it work finally!). Sometimes 6.4 line rate needs several transceiver resets but apart from that it works better than earlier.

Now the issue is only with 4.2 Gbps to 4.8 Gbps. All the parameters are changed as per the manual guidelines but it seems not working even after correct sequence of programming and resets ! Also REFCLK is changed according to its requirement.
I don't see any other DRP parameters to be changed to make it work !

 

[ads-ee]

It probably has something to do with the REFCLK changing. Have you tried chipscoping the transceiver a bit and determine if any of the error status is reporting anything?

I haven't worked with the V4 transceiver for a long time, so I don't remember all the status that you can observe from the IP.

If it takes several resets to get 6.4 to sync to the input, then you might want to carefully verify there isn't some reset syncrhonization required and/or reset ordering required. I remember having to change the reset ordering based on suggestions from our FAE.

Regards

 

[kaiserschmarren87]

It probably has something to do with the REFCLK changing. Have you tried chipscoping the transceiver a bit and determine if any of the error status is reporting anything?

I haven't worked with the V4 transceiver for a long time, so I don't remember all the status that you can observe from the IP.

If it takes several resets to get 6.4 to sync to the input, then you might want to carefully verify there isn't some reset syncrhonization required and/or reset ordering required. I remember having to change the reset ordering based on suggestions from our FAE.

Regards

The transceiver has no issues. 3.2, 4.2 and 6.4 line rates work perfectly when they are worked with separate .bit files. But I still could not figure out why 4.2 line rate is not working when the same .bit file works for 6.4 and 3.2 line rate ! I load correct DRP values from PC and Reset the transceiver as I did for other line rates.

6.4 and 3.2 have same values for DRP parameters under consideration except for RXOUTDIV2SEL.
Whereas for 4.2, all the parameter values are different and even the REFCLK. I tried various combinations of REFCLKs for 4.2 Gbps but ended up with no change in the output. Not sure whether I am missing something here but I am not going to giveup now....

 

[ads-ee]

The transceiver has no issues. 3.2, 4.2 and 6.4 line rates work perfectly when they are worked with separate .bit files. But I still could not figure out why 4.2 line rate is not working when the same .bit file works for 6.4 and 3.2 line rate ! I load correct DRP values from PC and Reset the transceiver as I did for other line rates.

6.4 and 3.2 have same values for DRP parameters under consideration except for RXOUTDIV2SEL.
Whereas for 4.2, all the parameter values are different and even the REFCLK. I tried various combinations of REFCLKs for 4.2 Gbps but ended up with no change in the output. Not sure whether I am missing something here but I am not going to giveup now....

I wasn't suggesting that the transceivers don't work at 4.2 line rate. My guess it has something to do with when you perform the reference clock frequency switch. That is why in one suggestion I made early you might want to try different reset sequencing. It seems to me that the transceiver may not be starting up correctly so can never synchronize to the input.

When you've stated it doesn't synchronize does this mean it can't recover the clock or it's not getting the correct data? I'm asking because these are two entirely different things. If it's not recovering the clock that would indicated problems with reset sequencing and the loop isn't locking to the embedded clock (could be stuck in some bad state). If we're talking about you don't get the correct data, are you sure it's not an alignment issue with the data? I really think it would be helpful to capture data at the transceiver's receive port along with the status and see what the raw deserialized data and the K-code status for each word received.

Never thought to ask but what is the protocol you are using for the links? Is it 8-bit data or 16-bit data, if you are using 16-bit data you should use two different K-codes for idle and alternate them. You should also make sure that the alignment never changes and that any insertion or deletion of symbols is done as pairs.

You know all these problems you are contending with is the reason Xilinx only used this version of transceiver in a single product. The V5 has a different transceiver that has a much simpler configuration, without all the extra options that the V4 transceivers had. Both V6 & V7 have continued that trend so they are all much easier to use.

Regards