Data lost in streaming application using Win32IO ( NI PXI-6562)

[Phone Thet Khaing]

Hi,
   I have a problem in losing data for both generation and acquisition,using (1042Q chassis, NI PXI 8196 embedded controller(2.0GHz  & 512MB RAM) & NI PXI 6562).
For generation, first and last part of data is lost, for example if i send 0,1,2,3,4,5,6,7 , only received 1,2,3,4,5,6,so the application is continuously reading from Harddrive,transferring to controller RAM, then transfer to device onboard memory through PCI Bus,  so where could the data sending be lost?  How can I prevent it ? The file used is in binary format 32 bits,
for acquisiton, it is too the same as above case, lost first and last part of the data, so this is only happening in streaming applications,
For example, if i generate the waveform with .hws file whose file size has to be lower than onboard memory of 16MB of PXI-6562,I have no problem receiving data i am expected,no data is lost, so this application is reading .hws file and transfer to device memory,then generate so this is the case of one time read/write, and I do not any data lost with this,
since we aim to generate file size larger than onboard memory , we use some examples found in NI examples for streaming ,so could anyone suggest me a pointer ?
 
Attached are streaming VIs and binary file used for generation,and .hws VI
Thanks & Best Regards,
 
Phone TK
Thanks,
Phone TK


phone1.zip:
http://forums.ni.com/attachments/ni/70/8291/1/phone1.zip

[samantham]

Hello, 

I have taken a look at your files and have not been able to reproduce the issue.  Can you please provide further details about your application setup and how you are physically testing when there is data loss?  Are you solely using the two examples as a pair ? acquiring data/.bin file with the ?to disk? example and then generating the .bin file with the ?from disk? example VI?  I am mainly looking for information about your starting point.  Are you starting with a specific (all ready created) binary file of interest or acquiring data present on the digital lines? 

I am also interested to know the purpose of the loop around the acquisition start.vi.  The loop will only cycle if there is an error. 

I see that you are already using Strobe as your clock source on the acquisition side and On Board for the generation.  The default setting for the Strobe source is 6.25 M Hz = 12.5M S/s.  If your application is in fact at the 12.5 M S/s, you might want to consider sticking with the HWS file mode.  The benefits of using Win32 are smaller at lower rates.  What is the desired sampling rate of the application?  Are you trying to synchronize acquisition and generation by using the Strobe clock source or is it just an external clock?

[Phone Thet Khaing]

Thanks for your reply,Generation set-up : Ch 0 : clk ,Ch 1 : sync,Ch2 : Data 1,Ch3 : Data 2As you can see in screen capture, it is the beginning of the file, fs is a synchronization pulse, so data will be saved in receiving once it sees the sync(fs) pulse,  so fs is defined as starting of 1st data, for next group of data it would be started with another sync pulse,    Our binary file is created using waveform generator, so we use it for example of generation, so at the receiving side, when we analyze the data received, a few data is lost, for example let say we send 10 groups of data, so there will be 10 sync pulses, but received file only has information with 4,5,6,7,8 sync pulses , data with sync pulses 1,2 and 9,10 are lost ,see attached data format
So for our case: Data A,B and 4,5 are lost at the receiving side,Physical set-up : for acquisition : Strobe : input clkChannel : 4 : sync           5 : data 1          6 : data 2 for acquisition,You also mentioned there is data loss from acquisition, how was it determined?  As mentioned above as in generation, data is lost at the front part and last part of data. Is the "receiving" and "acquisition" execution using the same program and same hardware?Yes, acquisition and generation are done with same hardware with different VI program as you can see that examples are written in Single Data Rate mode: which means during generation, all 16 channels are used, so doing acquisition is impossible, and vice versa.

Are you solely using the two examples as a pair ? acquiring data/.bin file with the ?to disk? example and then generating the .bin file with the ?from disk? example VI?   : Yes but one at a time,  I am mainly looking for information about your starting point.  Are you starting with a specific (all ready created) binary file of interest or acquiring data present on the digital lines?  : Already created binary file as mentioned above and attached screen shot,.I am also interested to know the purpose of the loop around the acquisition start.vi.  The loop will only cycle if there is an error.  : That loop is mainly for waiting for the clk signal to come in at strobe terminal, as we are sending data in a burst,  so while loop will be executing until it receives clk signal at strobe and then starts acquisition.I see that you are already using Strobe as your clock source on the acquisition side and On Board for the generation.  The default setting for the Strobe source is 6.25 M Hz = 12.5M S/s.  If your application is in fact at the 12.5 M S/s, you might want to consider sticking with the HWS file mode.  The benefits of using Win32 are smaller at lower rates.  What is the desired sampling rate of the application?Desired sampling rates are 3.125MHz,6.25MHz, 25MHz and 50MHz, as we have PCI bus and HD read write speed limitation in higher clk rate for so we are trying out low freq first for large data  and low data for high frequency,  Are you trying to synchronize acquisition and generation by using the Strobe clock source or is it just an external clock?Nope as examples I used us written in SDR mode, so only one application at a timePls let me know if you have other questions or unclear about what I explained.Thanks Phone TKMessage Edited by Phone Thet Khaing on 03-04-2008 11:37 PM


Data Format Example.JPG:
http://forums.ni.com/attachments/ni/70/8335/1/Data Format Example.JPG


Data Mapping to binary file ( jpeg)3.JPG:
http://forums.ni.com/attachments/ni/70/8335/2/Data Mapping to binary file ( jpeg)3.JPG

[samantham]

Hello Phone,

Sorry for the delay ? I recently found another forum post tread you created that has more details about your application.  I have a little better insight into your application now.  Since there has been a delay / gap in our contact, please update me with any new information.

<a href="http://forums.ni.com/ni/board/message?board.id=70&amp;thread.id=8168&amp;view=by_date_ascending&amp;page=1" target="_blank">Discussion Forum:&nbsp; NI PXI 6562 Streaming (continuous generating and acquisition)</a>&nbsp; &nbsp;&nbsp;

Are you still interested in using Scripting to reduce the generation streaming of 100 MB/s?&nbsp; It looks like you were interested in Scripting with a binary file.&nbsp; I also understand that 50 MHz is your maximum rate, but you are still interested in 25, 6.25, and 3.125 MHz.&nbsp; Thus are testing at the 6.25 MHz rate currently.

I noted that you are using SDR and thus can only generate or acquire at one time due to the fixed data width for generation being 2 bytes (equal to 16 channels on the 6562).&nbsp; I assume you are using one example to generate and then waiting for generation to finish / un-reserve the channels to then run the acquisition example.&nbsp; You also mentioned that you are using the same hardware.&nbsp; I am assuming that the hardware setup is a loop back test ? only using the NI-6562.&nbsp; Please clarify if there is a device under test (DUT) you are sending the generated date to, storing data, and receiving acquisition data from.&nbsp; &nbsp;

If the hardware setup is a loop back test ? You could be missing the first part of your data because it was present at the end of the transmission line when the hardware was not ready for acquisition.&nbsp; The hardware could still be in generation mode or switching to&nbsp;run the acquisition example when data is present at the end of the transmission line and thus data is missing.&nbsp;

Please give me some feedback and an update and I will reply with a recommendation.&nbsp; It would be helpful to understand the big picture of the application in further detail. &nbsp;For example, what is the goal to be achieved and what is the hardware setup?&nbsp; I realize you want to use the streaming examples to generate and acquire, but a bigger picture with more details should help narrow in on the application.&nbsp; Are you interested in burst handshaking?&nbsp; I noticed you mentioned that your data was being sent in a burst.&nbsp; The four signals that you described seem to relate to a burst handshaking application.

[Phone Thet Khaing]

Hi Samantha,
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Thanks a lot for your supply, let focus our attention on streaming VIs that I will attach togehter with this reply,Data transfer timing is of synchronous transfer, and for generation there are 4 channels(clk,sync,data&nbsp;0, data1)&nbsp;are used as inputs to DUT and for acquisition strobe is used as clk and 3 channels (sync,data 0,data1) are used as inputs from DUT to NI PXI 6562. We are doing generation and acquisition seperately as we are making use of SDR mode. so basically there is&nbsp;a DUT(hardware device) that is receiving the generated data and will be transmitting data for NI PXI6562 to acquire.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Interest of clk rates are 3.125MHz,6.25MHz,25MHz and 50MHz, therefore BW requirements are of 6.25MB/s,12.5MB/s,50MB/s and 100MB/s, as we use PCI bus, BW is therorectically 130MB/s. Our findings are that for clk speed of 25MHz &amp; 50MHz would make PCI bus unstable &amp; for clk speed of 3.125MHz &amp; 6.25MHz, PCI bus rather stable, so for 25MHz and 50MHz we are generating and acquiring data&nbsp;right before PCI bus unstable&nbsp;resluting in limited amount of data being generated or acquired, for the case of 3.125MHz &amp; 6.25MHz,&nbsp;as PCI bus is stable for quite a long time, so we&nbsp;are able to generate and acquire large amount of data. So we are here being able to generate and acquire data to &amp; from DUT using Streaming examples VIs according to our optimized data and clk speeds.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;When we analyze the acquired data(from DUT to PXI 6562),&nbsp;a few data at the beginning are lost ( e.g, if DUT sends data of 1,2,3,4,5,6,7,8,9,10, NI only received 3,4,5,6,7,8,9,10, losing data of 1 &amp; 2), when we analyze data at DUT received from 6562, a few data at the beginning and the last are lost ( e.g, if&nbsp;PXI 6562 sends data of 1,2,3,4,5,6,7,8,9,10, DUT only received data of 3,4,5,6,7,8, losing data 1,2,9,10).&nbsp;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Our goal&nbsp;is to resolve the&nbsp;lost data issue.Attached is the block daigram,straming VIs and test binary file&nbsp;used for generation,
If you are still unclearm, pls let me know,
Thanks,
Phone TK&nbsp;&nbsp;


Streaming VIs.zip:
http://forums.ni.com/attachments/ni/70/8508/1/Streaming VIs.zip

[samantham]

Hello,
I have a couple question I just want to get clarified.&nbsp; For acquisition I understand that you are using strobe as the clock,&nbsp;where is this signal coming from?&nbsp; I did not want to assuse it was coming from the DUT since you specificly&nbsp;state the other three channels are from the DUT.&nbsp;
You mentioned "so for 25MHz and 50MHz we are generating and acquiring data&nbsp;right before PCI bus unstable&nbsp;resulting in limited amount of data being generated or acquired".&nbsp; I understand that you are able to see what rates make the PCI bus stable or unstable, however I am confused that after you realize 25 and 50 MHz rate are unstable ... are you still trying to generate and acquire at these rates?&nbsp; And if so you are doing it in short bursts?&nbsp; Please clarify the statement quoted above.&nbsp; I am assuming the data loss happens at all four rates regardless of stable or unstable.
With this said, I am assuming you are just trying to generate and acquire at the two lower rates, however data loss is present at the two lower rates.
Thank you for taking the time to explain the application in further detail.&nbsp; I really appreciate it.&nbsp; Please get back with me about the few questioned I posed.&nbsp; I am already further looking into this issue.&nbsp;
&nbsp;

[samantham]

Hello,

I have a few more secondary questions, because the setup is not adding up.&nbsp; What does your system clock setup look like on both the acquisition and generation side?&nbsp; How are you synchronizing the 6562 and the DUT and vise versa?&nbsp; I am interested to learn about the physical setup.&nbsp;

I am also interested to understand what your DUT actual is and how are you acquiring the data from the 6562. &nbsp;I see the one of your data lines is actually your clock.&nbsp; Is there a reason you are not using one of the clock lines from the 6562 for your clock signal?

[Phone Thet Khaing]

Hi Samantha,
&nbsp;Sorry about the confusion, our design is that whoever generating will&nbsp;provide the clock signal&nbsp;to destination HW,so that both will have shared-clk line to sample.
&nbsp;There is no specific reason that we do not use clk-out from NI, since we are creating binary file using Digital waveform editor where we create four digital lines namely clk,sync,data0 &amp; data1,
for generation,sample clk source is 'OnBoard Clock' and
for acquisition 'Strobe' which is provided by DUT, .I believe this is where you got confused, yes,DUT is supplying clk to 'Strobe' because in this case DUT is generating data.&nbsp;
Synchronization is maily on&nbsp;clk singal lines,so &nbsp;if you have any better idea or ways, pls share it with me.
For clk rate of 50MHz &amp; 25MHz, it could generate or acquire 100MSamples before it is unstable, so for these clk rates we aregenerating and acquiring for a limited amount of 100MSamples, I am assuming the data loss happens at all four rates regardless of stable or unstable.Data lost occured at all four rates regardless of stable or unstable.
AS for the lower clk rates,of 3.125MHz &amp; 6.25MHz we could generate and acquired up to 2GSamples which is our maximum size of our Solid State Recorder.
actual DUT is transferring data to different destination using different clk rates, so for our application 3.125MHz,6.25MHz,25MHZ &amp; 50MHz are the clk rates that we would like to run the test without any data lost.
Thanks
Phone TK

[samantham]

Hello,
Thanks for the information.&nbsp; I have not been able to reproduce the issue yet.&nbsp; I am going to test one more setup that should mimic your application a little better.&nbsp; Please note I am still interested to know if the win32 examples posted for HSDIO at the link below worked in their original unmodified state.

<a href="http://zone.ni.com/devzone/cda/tut/p/id/6102#toc2" target="_blank">Developer Zone Tutorial:&nbsp; Stream to and from Disk with Many NI Products</a>

[Phone Thet Khaing]

Hi,
&nbsp;&nbsp;&nbsp; Generation and acquisition examples were downloaded from that link, and there is no modification to generation example and only for that&nbsp; acquisition, a few modification was done to acquisition ( a simple while loop to detect strobe clk signal to come in).
&nbsp;&nbsp;&nbsp; Pls let me know if you need anything to clarify.
We only have data lost problem in streaming application,
in the case of single data transfer ( less than on-board memory of 16MB) using HWS file, there is no data lost.
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; only with the case of binary file streaming then we got the data lost problem,
&nbsp;
Thanks
Phone TK