Trouble getting started with a Spartan-6

[mrflibble]

I've only evaluated the gigabit Ethernet cores on Opencores and was pretty underwhelmed by the poor documentation. I'm not sure about 10/100, sorry!

I second that sentiment. Opened opencores documentation, became underwhelmed, closed documentation.

I've published a working example design that uses the USRP2's GPL-ed gigabit MAC on a Spartan-6 (should work with any board using the Marvell 88E1111), but it would take a significant redesign to get this core working at other speeds.

Yup, EnderW4785 is probably better of using your code. I did that some time ago and got it running in no time. Slapped on fifo + did a few tweaks and it's merrily sending udp packets at about 500 Mbps max. Of course it did help that I was using the exact same atlys board. Not sure how easy it will be for a random other PHY. But hey, can't be worse than the opencores stuff.

 

[joelby]

I'm glad it's working well for you!

The USRP2 code should work with any GMII PHY if you set the registers appropriately. It'll probably be easier to get a new development board with a gigabit PHY than trying to add one to the Drigmorn3, but if you only need 100 Mbit I would recommend getting a license for the Xilinx core. It's not that expensive (I think the EDK is about $500) compared to the amount of time it takes to figure out one of the open source cores - it took me about a month of part time hacking to get the USRP2 code working well.

 

[mrflibble]

I'm glad it's working well for you!

The USRP2 code should work with any GMII PHY if you set the registers appropriately. It'll probably be easier to get a new development board with a gigabit PHY than trying to add one to the Drigmorn3, but if you only need 100 Mbit I would recommend getting a license for the Xilinx core. It's not that expensive (I think the EDK is about $500) compared to the amount of time it takes to figure out one of the open source cores - it took me about a month of part time hacking to get the USRP2 code working well.

Yeah, it worked pretty well. After I went over my stuff (your code + some tweaks) I could definitely see the stuff based on other peoples stuff based on other peoples stuff based on ... signs. No complaints at all, more of a .. funny how these things work. I for one am glad with some an ethernet module that just works without too much effort. Effort on my part that is.

Oh yeah, I don't know how much further your own code evolved, but I did encounter one bug. It was related to a 2-bit vector only being connected by 1-bit somewhere. I believe it was a flag or mode or something. In your FSM code the upper bit was always 0, so you didn't notice the difference. But when I used 2'b10 or 2'b11 values instead of only 2'b01 and 2'00, strange things occured. Mmmh, what was it ... I believe it was something related to the packet length being odd or something. Or something where the packet was not a multiple of 4 bytes, something like that. If you're interested I can look it up what the hell it was. It's been a "few" months since I last looked at that code.

 

[EnderW4785]

I'm looking for implement gigabit ethernet on my Spartan-6. What do you mean by "USRP2's GPL-ed gigabit MAC on a Spartan-6." Is this functionally different than what I'm looking to do?

---------- Post added at 16:48 ---------- Previous post was at 16:44 ----------

Sorry, didn't see the most recent responses before posting that. I see, the Drigmorn3 is limited to 10/100. What would it take to upgrade my board to something more suitable for my application?

 

[EnderW4785]

Another, slghtly related question. When clocking an FPGA, is there a good reason to not clock on both the rising and falling edges of an incoming clock as opposed to generating a clock at twice the frequency? This would allow me to do a 4:1 deserialization as opposed to an 8:1 which requires cascaded ISERDES

 

[barry]

Another, slghtly related question. When clocking an FPGA, is there a good reason to not clock on both the rising and falling edges of an incoming clock as opposed to generating a clock at twice the frequency? This would allow me to do a 4:1 deserialization as opposed to an 8:1 which requires cascaded ISERDES

I think it depends. The tools will take care of timing constraints in either case, but if you need to control other parameters like phase offset, you'll need to use a DLL/DCM

 

[EnderW4785]

Thank you for all of the help. Another pretty simple issue thats unclear to me. When using serdes_1_to_n_clk_ddr_s8_diff to deserialize 8:1, is the "Input from LVDS receiver pin" the bit frame clock or the bit rate clock. More clearly, is the input clock (clkin_p and clkin_n) the "slow clock" and the module creates the 8x clock as an output on the rxioclkp and rxioclkn ports? Or is the input clock to the module the fast clock and the output clock is (1/8)x?

Furthermore I'm a little concerned about latency in my system. Say I'm clocking an ADC at X MHz. The ADC outputs serialized LVDS at a bit rate of 8x as well as an output frame clock at X Mhz and an output bit clock at 8X MHz. Both of these output clocks of the ADC are slightly delayed as compared to my original clock. I am currently clocking the FPGA with the original clock at X MHz. To do the deserialization do I need to use the output clocks of the ADC (part of this will be answered in the above question by whether I need to input the 8X clock to serdes_1_to_n_clk_ddr_s8_diff)? Should I really be clocking the FPGA with the delayed X clock or does this not matter?

Thanks,

Scott

 

[mrflibble]

Another, slghtly related question. When clocking an FPGA, is there a good reason to not clock on both the rising and falling edges of an incoming clock as opposed to generating a clock at twice the frequency? This would allow me to do a 4:1 deserialization as opposed to an 8:1 which requires cascaded ISERDES

Yep, there's a good enoug reason for that. First, the fabric flipflops can only clock on 1 particular edge. So either posedge or negedge, but not both. Generally only the IO flipflops will be doing any DDR.

Incidentally, another reason is duty cycle.... If you have a clock that is not perfectly 50% duty cycle, then your timing for the posedge and negedge are going to be a bit off. For serdes fun it's probably still not too much of an issue, but for timing applications DDR can be tricky.

And in general timing analysis will be a bit easier with only one clock edge to worry about.

All that said, why worry about having to cascade ISERDES? If I recall correctly, you can just have coregen do that for you...

I think it depends. The tools will take care of timing constraints in either case, but if you need to control other parameters like phase offset, you'll need to use a DLL/DCM

Also, see this one: DIVCLK synchronization between BUFIO2 - Xilinx User Community Forums

 

[FvM]

When using serdes_1_to_n_clk_ddr_s8_diff to deserialize 8:1, is the "Input from LVDS receiver pin" the bit frame clock or the bit rate clock. More clearly, is the input clock (clkin_p and clkin_n) the "slow clock" and the module creates the 8x clock as an output on the rxioclkp and rxioclkn ports? Or is the input clock to the module the fast clock and the output clock is (1/8)x?

You'll need at least the frame clock for frame synchronization. A common approach is to generate the (dual edge) bit clock from the frame clock by a PLL. It will also allow you a fine adjustment of the sampling window which may be reasonable when working with bit rates near the FPGA limits. Alternatively, you can use both frame and bit clock from the ADC and no PLL at all. Finally there's is principle option to connect no source side clock and generate the timing by a PLL from the master clock. But it will require individual timing adjustment due to the large clock skew of typical LVDS ADCs. It's particular interesting with FPGAs that have an automatic DPA (dynamic phase align) feature, Spartan family hasn't.

I think most of your questions, e.g. about IO standards and termination or SERDES operation are answered by reference designs and application notes. But it takes some time to learn about state of the art.

 

[EnderW4785]

Thank you FvM and mrflibble, those responses were very helpful. I've spent some time reading the APP notes and familiarizing myself with their code. One of the aspects thats not clear to me is the bitslip operation. I understand the concept but I do not understand what causes the need for bitslip? Why would you need to reorder the output bits of the LVDS deserialization? Depending on the reason for the need of reordering I may not need to worry about this.

 

[FvM]

I'm not familiar with the Xilinx SERDES details, but bitslip is generally necessary for those cases, where no unequivocal frame sync signal is available and you need the sync the receiver to a data pattern. Doing this with a LVDS ADC would require to set the device into test mode, sending a defined pattern.

 

[EnderW4785]

Thank you that was very helpful.

Can someone point me at a document that explains what pins are capable of driving what sort of output levels (specifically in a Spartan-6 if possible) and what terminations should be used? I'm currently using LVCMOS_33 levels to drive output pins to monitor with an oscilloscope / control my VXCO. However, some pins seems to be more stable than others and it seems that they are very sensitive to load capacitance and even sometimes my scope probes. I'm sure I'm doing something against the rules but I haven't been able to locate the appropriate documentation. What does FP_VTT_50 mean? Does that mean it has an internal 50 Ohm ternination or is expecting an external termination?

Thanks

Scott

 

[joelby]

What do you mean by stability in a pin? The two most common problems with using oscilloscopes with FPGAs are poor grounding and insufficient bandwidth. Don't use the 10cm ground wire - use the spring-loaded ground wire that attaches right at the probe tip.

The document you're after is UG381 - Spartan-6 FPGA SelectIO Resources. Terminations are described in figure 1-18. FP_VTT_50 is "50R Far-end Parallel Termination to VTT"

Some ADCs provide a framing signal that you can use to control the bitslip state machine, and others let you enable or program a test pattern (and some will do both) - which ADC did you say you were using?

With regard to DDR vs. double frequency SDR - on the Spartan-6, both are equivalent and the latter is arguably easier and better documented, unless you exceed the maximum frequency or are using an unsupported deserialisation factor. There are more than a few threads on the Xilinx forum about these issues, e.g. this one.

It would be difficult to upgrade the PHY on the Drigmorn3 to something that supports gigabit Ethernet unless there's a drop-in pin-compatible replacement (somewhat unlikely). Your best bet would be to contact the very friendly people at Enterpoint. Otherwise, there are no high speed interface options conducive for attaching an external PHY. It will be much easier and cheaper to buy a different development board!

mrflibble: I've since cleaned up and rewritten my part of the Ethernet code and plan to release it as soon as I've documented it! I wouldn't be surprised if there were issues with odd-sized packets as the old example was basically the bare minimum to get things going before I moved on to adding the functionality to my work project. The new one implements UDP reception as well - let me know if you're interested in testing out a pre-release.

 

[EnderW4785]

I'm using the ADS5263 - it does provide a framing signal. I still don't completely understand the different clocking signals within the LVDS deserialization procedure but hopefully on Monday when I try to implement it it will become clear.

Thank you for the responses. This has been very helpful. I figured out my termination issues and looks like at least for now I'll be implementing 100M ethernet.

 

[joelby]

If you've got the I/Os to spare, the 2-wire and 8 x serialisation mode is easy to implement with DDR (figure 65). Just combine the two halves of the sample word together in fabric. Actually, all of the 2-wire modes look like they might work.

The only problem you might have is with spanning half-bank clocking regions on the Spartan-6, unless you are incredibly careful with your pin selection. Unless you're pushing the frequency limits of ISERDES, it's safe to use an extra clock buffer though. If you get stuck with this (errors about clocking regions and placement), let me know and I'll dig up the details.

According to figure 65, the frame clock is synchronous to the output data signals, so you can connect this to an ISERDES block as well. Just bitslip until frame clock is all zeros (grey cells, in figure 65) or all ones (white cells). Alternatively you could use the ADC's test pattern modes if you wanted to save two I/Os. I figure that it's best to connect absolutely every available signal to the FPGA in case you decide you need it later - it's no work to ignore a signal, but it might be impossible to wire it up if you decide you need it.

 

[mrflibble]

mrflibble: I've since cleaned up and rewritten my part of the Ethernet code and plan to release it as soon as I've documented it! I wouldn't be surprised if there were issues with odd-sized packets as the old example was basically the bare minimum to get things going before I moved on to adding the functionality to my work project. The new one implements UDP reception as well - let me know if you're interested in testing out a pre-release.

Cool! So far all I have is UDP send, because it's all I really needed. I couldn't even be bothered to implement arp. My thinking was that if I need that sort of thing, then I am going to need some microblaze-ish solution anyways. For now it is basically a fifo + your statemachine with a few extra states to assemble the UDP packets. But UDP receive would be a handy for future projects. So yes, I'd be interested in trying it out.

 

[EnderW4785]

Can you explain why using the ADC's test pattern would save me two I/O's? Wouldn't I need the frame clock anyways?

Scott

 

[joelby]

Not really. The frame clock is only used for the bitslip state machine, so if you can get the data lanes to output a fixed, known pattern, you can use those instead, though you may need to periodically check that you haven't lost synchronisation.

 

[EnderW4785]

I'm clearly missing something key or doing something incorrectly. Let me break this down a little.

In xapp1064 they provide the deserialization code and protocols. From the ADS5263 I have three clocks (the clock signal thats clocking the ADC, output frame clock of the ADC and the output bit rate clock of the ADC). Using 2 wire, 8x serialization the output frame clock is .5x the input clock and the output bit rate clock is 4x the input clock.

The code block, serdes_1_to_n_clk_pll_s8_diff, has the following inputs and outputs.:

parameter integer S = 8 ; // Parameter to set the serdes factor 2..8
parameter BS = "FALSE" ; // Parameter to enable bitslip TRUE or FALSE
parameter PLLX = 7 ; // Parameter to set multiplier for PLL (7 for video links, 2 for DDR etc)
parameter PLLD = 1 ; // Parameter to set divider for PLL
parameter real CLKIN_PERIOD = 6.000 ; // clock period (ns) of input clock on clkin_p
parameter DIFF_TERM = "FALSE" ; // Parameter to enable internal differential termination

input clkin_p ; // Input from LVDS receiver pin
input clkin_n ; // Input from LVDS receiver pin
input reset ; // Reset line
input [S-1:0] pattern1 ; // Data to define pattern that bitslip should search for
input [S-1:0] pattern2 ; // Data to define alternate pattern that bitslip should search for
output rxioclk ; // IO Clock network
output rx_serdesstrobe ; // Parallel data capture strobe
output rx_bufg_pll_x1 ; // Global clock output
output rx_pll_lckd ; // PLL locked - only used if a 2nd BUFPLL is required
output rx_pllout_xs ; // Multiplied PLL clock - only used if a 2nd BUFPLL is required
output bitslip ; // Bitslip control line
output rx_bufpll_lckd ; // BUFPLL locked
output [S-1:0] datain ; // Data


The code block, serdes_1_to_n_clk_pll_s8_diff, has the following inputs and outputs.:

input use_phase_detector ; // '1' enables the phase detector logic
input [D-1:0] datain_p ; // Input from LVDS receiver pin
input [D-1:0] datain_n ; // Input from LVDS receiver pin
input rxioclk ; // IO Clock network
input rxserdesstrobe ; // Parallel data capture strobe
input reset ; // Reset line
input gclk ; // Global clock
input bitslip ; // Bitslip control line
input [1:0] debug_in ; // Debug Inputs
output [(D*S)-1:0] data_out ; // Output data
output [2*D+6:0] debug ; // Debug bus, 2D+6 = 2 lines per input (from mux and ce) + 7, leave nc if debug not required

Which clock should clkin be? How do you use the "frame clock" to control the bitslip state machine? Where is the frame-clock used as an input? Also, what exactly is the serdes factor? It seems like it should be the ratio of number of bits to input clock cycle but in some of the examples that doesn't line up. In my case should PLLX be 4 and PLLD be 2?

Scott

 

[joelby]

Clkin should be LCLK. Don't think of the frame clock as a 'clock' - just hook it up as another ISERDES input (but with a single lane) and use its deserialised output as the control for the bitslip state machine. Basically, you assert bitslip until the frame output is all zeros or all ones, if you're using the mode indicated in figure 65. See the reference design for an example state machine. It makes more sense when you simulate it

If you're doing DDR, PLLX=1 and PLLD=1 because LCLK is running at the same rate as the output data (well, half the rate, but it's DDR). Yes, it's four times the sampling clock CLKP, but the ISERDES side doesn't care about that - especially if your FPGA isn't outputting the sample clock.

I would use 8:1 deserialisation. Figure 65 matches XAPP1064's Figure 2 (8:1 Data Stream Using DDR) exactly.