C# exception when converting >400MB stream to byte array

Anzar Khatri

After searching a lot and becoming confused and disapointed I want to post my question here. I had encountered problem of OutofMemoryException when using MemoryStream and finally got around with the help of MemoryTributary. Another problem still remains. I want to convert that stream to byte array but again get OutofMemoryException error when converting MemoryTributary (or even MemoryStream. I do not use MemoryStream now because it cant handle large memory) stream to byte array on windows XP while same code is running without thowing that error on Windwos 7. Follwoing is code:

DataContractSerializer ds = new DataContractSerializer(typeof(List));
MemoryTributary stream2 = new MemoryTributary();
ds.WriteObject(stream2, tempdataitems);
byte[] data = stream2.ToArray(); // OutofMemoryException

My machine is core2 with 4GB RAM. I know this is not problem with RAM. Help is appreciated in solving this problem. I have to run that code on WinXP.

4 Replies

Know the answer? Post it — somebody with the same question will find it here.

Anzar Khatri12y ago
Thnx a lot for reply and sorry from my side for late reply. As I noted I need to run code on WinXP but documentation of BufferManager says it is not supported on WinXP. Truly speaking I've not checked this code on WinXP cos I read docs for the class and left it and also bcz of UploadFile method. This method seems to copy chunk to stream but where is the code of that method.

If any1 has checked and it is working on XP plz let me know or any other solution.

+1
Jaganathan Bantheswaran12y ago
I suggest you to use BufferManager or BufferedReader.

convert the stream chunks by chunks...

BufferManager bm = BufferManager.CreateBufferManager(chunkSize * 10, chunkSize);

for (int i = resumeChunk; i < chunks; i++)
{
byte[] buffer = bm.TakeBuffer(chunkSize);
try
{
fileStream.Position = i * chunkSize;
int actualLength = fileStream.Read(buffer, 0, (int)chunkSize);
if (actualLength == 0) break;
//Array.Resize(ref buffer, actualLength);
using (MemoryStream stream = new MemoryStream(buffer))
{
UploadFile(stream, actualLength);
}
}
finally
{
bm.ReturnBuffer(buffer);
}
}
+1
Anzar Khatri12y ago
If u r asking for .ToArray method, then following is code from MemoryTributary.cs. I have also attached the file.

public byte[] ToArray()
        {
            long firstposition = Position;
            Position = 0;
            byte[] destination = new byte[Length];
            Read(destination, 0, (int)Length);
            Position = firstposition;
            return destination;
        }

Please suggest a solution I'm stuck here.

If still anything u need plz ask me I want this problem to be solved asap.
+1
Jaganathan Bantheswaran12y ago
Can you post the conversion code?
+1