DIG_BLOCK_IN memory usage. for 6533 PCMCIA card

[Buffning]

I'm trying to take in large amounts of data (100mB) very quickly
(>100kHz) using a 6533 PCMCIA card and c++.  The card/computer
can't keep up with the speed if the data is written to a file during
input.  THerefore, I'm trying to keep all the data in RAM until it
is done transferring data and then save it to the harddrive.  I
have >200 MB of RAM free (should be enough?), yet it DIG_BLOCK_IN is
giving me error 10444, a memory full error when I pass it an array of
only 32 MB.  The error doesn't come when I allocate the memory,
but many lines of code later when I call DIG_BLOCK_IN.  Does
DIG_BLOCK_iN allocate more memory?  How much more?  Or is
there a memory limit to the function?  The documentation makes it
seem that you can pass any array of length less than 2^32-1 or the size
of  count. 

[Buffning]

It gives out of memory error when I pass DB_BLOCK_IN a count value greater than 2^25 despite the documentation showing

"count is the number of items (for example, 8-bit
items for a group of size 1, 16-bit items for a group of size 2, and 32-bit
items for a group of size 4) to be transferred to the area of memory specified
by buffer from the group indicated by
group.


Range: 2 through 232 - 1"


Is there a lower range on count than shown in the documentation?

[Travis W]

Hi Buffning,

After looking into this issue more, I've found varying information
about what Windows XP can and can't do in terms of buffer allocation
with this function.  Can you try changing the size of your buffer
to find out at what point this error occurs?  Is it exactly 32MB?

Also, do you have access to a Windows 98 machine?  I'm interested
to see if this application will run any differently on another OS as
per <a href="http://forums.ni.com/ni/board/message?board.id=170&amp;message.id=41975&amp;requireLogin=False" target="_blank">this post</a>.

Have a great day!
Travis W

[Buffning]

The error occurs when I pass ulCount &gt;= 2^25 to Block_In.&nbsp; It
doesn't seem to be effected by the size of the buffer I create or pass
to Block_In. I don't have access to any other OS systems unfortunately.&nbsp;

Thanks a lot for your help,
Mark

[Travis W]

Hi Mark,

It seems that this may be a limitation of your particular
computer.&nbsp; Do you have any extra RAM that you could borrow from
another machine to see if the 2^25 limit can be extended?

Also, stepping back to your original code where you were calling
DIG_BLOCK_IN with less data, I'm curious why streaming the data to disk
is too slow at acquisition rates as low as 100k.&nbsp; Even reading all
32 lines this should only be 4 Bytes * 100kHz = 400kB/s.&nbsp; Most
laptops should be able to write to disk at least at a few MB/s.&nbsp;
How are you writing to file?&nbsp; Are you using binary files or ASCII
or other?&nbsp; How many data points were you attempting to write each
time?&nbsp; Were you opening the file each time you attempted to write,
or did you leave the file open and simply add data each iteration of
your acquisition loop?

Thanks for working with me Mark.

Have a great day!
Travis W

[Buffning]

I don't, unfortunately, have any more ram I can add to try it.&nbsp;
The reason why the older version, where I take in less data with
block_in, doesn't work must be a limitation with the card and not with
the writing to disk because as you say it should write to the disk fast
enough.&nbsp; To save to the disk, I created a pointer to the file by
mapping a view of the file and filled in the file through the pointer
as I got the data, which means the file was binary and stayed open the
entire write process, it would write as many as the buffer, I guess, to
the file at a time (not sure how it handles that part).&nbsp; I tried
keeping everything in RAM and it still wasn't fast enough.&nbsp; I've
been talking to tech support and it seems like the card (6533PCMCIA
card) is limited to less than 10KHz.

I'm still confused as to why 2^25 is a limit for the count value in
Block_In, but I my card doesn't seem to be up to my specifications
anyways, so it's not too important.

Thanks for the help,
Mark

[Travis W]

Hi Buffning,

What operating system are you working with?&nbsp; Different versions of
Windows can have differnet limitations with memory allocation.

Are you using a double-buffered acquisition?&nbsp; <a href="http://forums.ni.com/ni/board/message?board.id=70&amp;message.id=680&amp;requireLogin=False" target="_blank">This</a> post describes how this can be a possible solution to the issue, and <a href="http://forums.ni.com/ni/board/message?board.id=70&amp;message.id=59&amp;requireLogin=False" target="_blank">this</a> post mentions the location of a good example.

If this doesn't help out, please let me know, as I want to make sure that you get up and running.

Have a great day!
Travis W


Message Edited by Travis W on 08-15-2006 11:19 PM

[Buffning]

Hello,

I'm working on windows xp pro.&nbsp; I have over 200
MB of ram free when I try to allocate the 64 MB.&nbsp; I tried to find
a
limit on the amount of memory that it allowed me to allocate in C++ and
it let me
allocate and work with 300MB of Ram, which I would assume isn't all
physical ram, like Block_In seems to require.&nbsp; It seems though
that it would give a pagelock error if it couldn't get the memory to
the physical ram and not a error 10444.&nbsp; I allocate all of the
memory it should need before calling Block_in.&nbsp;

I tried double buffering, but it wouldn't run fast enough to keep up
with the speeds I need.&nbsp; Actually, it wouldn't even run at 10KHz,
which seems very slow.

My code is based on the example you mentioned.&nbsp; I simply changed some variables to create a larger buffer.