FAST DMA transfer + slow file saving problem...any solution

[syedshan]

hello every one

I am transferring DMA data from FPGA using PCIe which is VERY fast(using c++), and I need to take them and store in .text file, now if I save them normally it takes LOADS of time, hence the benefit of DMA dies completely.

Hence then the solution that comes to my mind is the threads...but I dont have much idea about how to use the threads such that I can save files simultaneously + take new data. I think I need to open and close many files at a time, allocate memory locations etc. and then free them and close files so that PC does not get hanged.

Example tested CASE

When I save files in binary the time it takes is 0.5 seconds for 512 DMA transfers (hence 512 files) while when I do same with the .txt file (without threading), the time it takes for complete process is 15 seconds. Hence I am able to completely draw data from FPGA, but need to make the system faster now.

Any idea would be appreciable. Please see the image for what I am trying to tell


thanks
Shan

 

[bsprajc]

Try saving all of your data in binary, then when all transfers are complete, then convert the binary files to .txt files...
Not sure why it takes so much time to convert binary to text...

 

[syedshan]

:O

this is what I am doing now. and my limitation is to reduce time at all cost hence I need time reduction technique and also I cannot stop data coming from the back (from DMA)