NI PXI 6562 Streaming ( Continuous generating and acquisition)

[Phone Thet Khaing]

Hi,
    We would like to generate and acquire data using NI PXI 6562 for 2GB of data with 50MHz for generation and 25MHz for acquisition, we have 1042Q chassis, NI PXI 8196 embedded controller(2.0GHz  & 512MB RAM) & NI PXI 6562, so PCI bus is only used by 6562.
I am using examples "niHSDIO (U32) Stream from Disk Queues Win32 File IO".
I am now able to generate or acquire samples of 100MB , beyong that I got an error "data overwritten" for generation and "lower sample rate or write more data to streaming waveform" , how can i do to achieve 2GB of data generation and acquiring?
 
For PCI BW calculation,


for NI PXI 6562, for acquisition we can select data width of 1byte or 2 bytes for efficient onboard memory usage, so if we assign 1 byte, only 8 lines are for acquisition, therefore

Required Bandwidth = Acquisition Rate * Number of Channels * Memory per sample (Each line is 1 bit so 1/8 of a byte)

Using that formula, acquiring at 25 MHz we get 25MHz * 8 * 1/8 = 25 MB/s

So, there will be 25MB of samples will transfer through PCI Bus to controller memory and then write to HD, if my HD Write speed is 25MB/s average, streaming acquisition should work out smoothly, now the problem is that fetched data have been piling up in controller memory and subsequently it is full, unable to perform any more when it reach 100MSamples, How can I solve this problem?

 

for generation, data width is fixed at 2bytes for 16 channels so 1 bit for each channel,

Required Bandwidth = generation Rate * Number of Channels * Memory per sample (Each line is 1 bit so 1/16 of 2 bytes)

Using that formula, generating at 50 MHz we get 50MHz * 16 * 1/16 = 50 MB/s

Since 50MB/s is required, I tried to solve it by transferring most of my data to controller memory, I got an error message asking to reduce sample rate or write data to streaming waveform more frequently,

I would like to the bottleneck here, if I reduce the sample rate, data throughput is lower, I can do the streaming properly at 6MHz clk speed for 1GB of data, so is this the case of PCI Bus limitation for my 50MHz since it is required BW of 50MBps while 6MHz requires only 12.12MBps?

As you can see in picture attached, there is an increase in modular device on-board memory (space available in streaming waveform), as it increases, it will stop the application, we have 16MB of on-board memory, is on-board memory the bottleneck? How can I solve this?

One of my intention is to create some waveform in .hws format , write to onboard memory and repeat it, my only concern is that discontinuity between repition steps, can it happen ?

Thanks,


Storing data in controller memory.JPG:
http://forums.ni.com/attachments/ni/70/8168/1/Storing data in controller memory.JPG


Generation error.JPG:
http://forums.ni.com/attachments/ni/70/8168/2/Generation error.JPG

[ryanm]

Hi Thet Khaing,

 

You are correct that the acquisition data width can be set.  If you are acquiring data at 25MHz with a datawidth of 1 byte, the data will be acquired at 25 MB/s.  For generation, again you are correct that the data width is fixed.  However, there seems to be some confusion.  The datawidth determines the sample size.  It will not change based on the number of channels used.  Therefore, at 50 MHz and a datawidth of 2 bytes, you have 100 MB/s.

 

Keep in mind that PCI/PXI is a shared bus.  In order to sustain the generation and acquisition rates above, you will need to transfer data at 125 MB/s.  Although this is below the maximum theoretical transfer rate of PCI/PXI (132 MB/s), in reality you are doing very well to get 110-115 MB/s.  Furthermore, the PXI-8196 has a laptop hard drive.  Laptop hard drives typically transfer data at a maximum rate of 25-30 MB/s.  This makes the hard drive the bottle neck of the system.

 

From here there are several options:

1)  If the generation waveform repeats, you could use scripting to link and loop waveforms.  Unless you explicitly add a delay, there is no delay from on waveform to the next.  In the script below, the first sample of wfm2 will occur on the next active sample clock edge following the last sample of wfm1.

script myScript1  Generate wfm1       Generate wfm2     end script

By using scripting, all the waveforms are loaded before the board starts generating data.  This would reduce the needed streaming rate to 25 MB/s.

 

2) Adding hard disks in a raid 0 configuration would greatly increase the maximum transfer rate of the hard drives.  It is important to note that hard drives perform best when read or writing from one file.  If you are reading and writing to one disk at the same time you will see poor performance.  In your case you may be able to use a raid disk for generation and the PXI-8196 hard disk for acquisition.  Keep in mind you will not be able to transfer the full 125 MB/s, so you will slowly fall behind.  Since you only need 2 GB of data the onboard memory should not overflow in the short amount of time you are streaming data.

 

3)  There are other options if you need to indefinitely stream data at 125 MB/s, but they will be significantly more expensive.  I will be happy to address them if needed.

 

Please let me know if you have any questions.

 

Ryan

[Phone Thet Khaing]

Hi,
    Thanks a lot for the reply, I had managed to run with the script and it is running fine.in this case I am making use of .hws digital waveform file to generate data,but I am thinking of creating a binary file so as to be able to use with the examples that I found in NI website, but I have no idea how to create a binary file for digital waveforms, anyone can help me ? If possible, can point me out or guide me how to create a binary file to generate and how to analyze the data acquired ??Attached are examples
I am needed to generate 4 digital lines with maximum clk of 50MHz,
Thanks ,
Phone Thet Khaing


niHSDIO (U32) Stream from Disk Queues Win32 File IO.vi:
http://forums.ni.com/attachments/ni/70/8198/1/niHSDIO (U32) Stream from Disk Queues Win32 File IO.vi


niHSDIO (U32) Stream to Disk Queues Win32 File IO.vi:
http://forums.ni.com/attachments/ni/70/8198/2/niHSDIO (U32) Stream to Disk Queues Win32 File IO.vi

[Phone Thet Khaing]

Hi,
   Can u pls show me a few steps to create a binary file in either using U8 array or U16array of data, as attached provided files do not have such examples for win32VIOs and using labview
Thanks,
Phone TK,

[ryanm]

As stated in the last post, you could use either the LabVIEW or Win32 File I/O VIs.  I have attached a simple example showing how to use the LabVIEW File I/O VIs to create a binary file.
 
Ryan


Write Binary.vi:
http://forums.ni.com/attachments/ni/70/8202/1/Write Binary.vi

[Phone Thet Khaing]

Hi,
   thanks a lot for the help, now i managed to create an binary file for 4 channels for generation,
however i got problem in acquisition, at the starting of my acquisition , a few data are lost but the rest is ok, for acquisition I used strobe for my trigger, can you pls help me how to prevent from losing data for acquisition?
 i do not have any problem with acquisition storing the data as .hws file but we do not use it as it is device-memory dependent, I just used it to test of my lost data, very small file i send out , first i stored as .hws file and nothing is lost , same I did for my win32io VIs(e.g you can see in previous attachment), & still lost a few data at the beginning , so can you pls help me ?
Thanks,
Phone TK

[Phone Thet Khaing]

Hi,
 we calculated the lost data size , and it happened to be 1Mbytes of data at the beginning and in addition we lostt 1Mbytes of data at the end of our data send(generated),
so now we are sure that we lost 1Mbytes of our data( it is consistent of losing 1Mbytes of data at front part of data and 1 Mbytes of data at the last part of data, in between data is correct,) happening  for both generation and acquisition,
may I know how to solve this ?
attached is the screen-shot of generation session,it could be seen that Coerced # of Samples to Generate is smaller than Total Samples in File, could it be the reason that we lost our last part of data ? and pls help me advise on the data-lost at the begginning of my data .
thanks
Phone TK


Generation.JPG:
Loading Image...

[Phone Thet Khaing]

Hi,
  Another scrreen-shot, as you can see attached picture, "Space Available in Streaming Waveform (Samples)" indicator is in negative value(-1153434), before i run the test, it is zero, may I know what does it refers to ? it too happens the same for sample written as showing -1 sometimes ,
Thanks & Best Regards,
Phone TK


generation.JPG:
Loading Image...

[Phone Thet Khaing]

Hi Ken,
 This is the test binary file,
Phone TK


Test binary file.zip:
http://forums.ni.com/attachments/ni/70/8492/1/Test binary file.zip

[Phone Thet Khaing]

Hi Joe,
 Sorry for the late reply, if i fetch large amount of blk form controller to onboard memory, for example if i fetched blk size of 1048576, less data are lost compared to blk size of 655360, for your understanding I could explain our application again.
let focus our attention on streaming VIs that I will attach togehter with this reply,Data transfer timing is of synchronous transfer, and for generation there are 4 channels(clk,sync,data 0, data1) are used as inputs to DUT and for acquisition strobe is used as clk and 3 channels (sync,data 0,data1) are used as inputs from DUT to NI PXI 6562. We are doing generation and acquisition seperately as we are making use of SDR mode. so basically there is a DUT(hardware device) that is receiving the generated data and will be transmitting data for NI PXI6562 to acquire.                      Interest of clk rates are 3.125MHz,6.25MHz,25MHz and 50MHz, therefore BW requirements are of 6.25MB/s,12.5MB/s,50MB/s and 100MB/s, as we use PCI bus, BW is therorectically 130MB/s. Our findings are that for clk speed of 25MHz & 50MHz would make PCI bus unstable & for clk speed of 3.125MHz & 6.25MHz, PCI bus rather stable, so for 25MHz and 50MHz we are generating and acquiring data right before PCI bus unstable resluting in limited amount of data being generated or acquired, for the case of 3.125MHz & 6.25MHz, as PCI bus is stable for quite a long time, so we are able to generate and acquire large amount of data. So we are here being able to generate and acquire data to & from DUT using Streaming examples VIs according to our optimized data and clk speeds.                      When we analyze the acquired data(from DUT to PXI 6562), a few data at the beginning are lost ( e.g, if DUT sends data of 1,2,3,4,5,6,7,8,9,10, NI only received 3,4,5,6,7,8,9,10, losing data of 1 & 2), when we analyze data at DUT received from 6562, a few data at the beginning and the last are lost ( e.g, if PXI 6562 sends data of 1,2,3,4,5,6,7,8,9,10, DUT only received data of 3,4,5,6,7,8, losing data 1,2,9,10).                        Our goal is to resolve the lost data issue.Attached is the block daigram,straming VIs and test binary file used for generation,If you are still unclearm, pls let me know,Thanks,Phone TK 


Streaming VIs1.zip:
http://forums.ni.com/attachments/ni/70/8509/1/Streaming VIs1.zip

[Joe F.]

Hi Phone TK,I see that you've opened a new thread on this problem, and it looks like Samantha is getting to the bottom of the issue. Let's keep it simple and jump to the new thread and not have the information in two places.Data lost in streaming application using Win32IO ( NI PXI-6562)<a href="http://forums.ni.com/ni/board/message?board.id=70&amp;message.id=8508#M8508" target="_blank">http://forums.ni.com/ni/board/message?board.id=70&amp;message.id=8508#M8508</a>

[Phone Thet Khaing]

Hi Joe,
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Thanks for your help, yeah now I will continue with Samantha resloving the issue, ,
Phone TK
Hi Ken,
Attached is the NI LVDS acquired binary file, pls note that data representation is of 3 lines that is acquired.
Thanks,
Phone TK&nbsp;&nbsp;


5framesincreaseReceivedFromPPU.zip:
http://forums.ni.com/attachments/ni/70/8547/1/5framesincreaseReceivedFromPPU.zip