High ram consumption

Jul 16, 2012 at 12:11 PM

Hello,

when I try to open a file which has a size of 200mb, the applicaion needs more than 1GB ram. How can I change this? Because more than 970MB ram consumption for 200MB text is too much, if you compare it with the consumption of notepad++ or even the normal windows notepad. How can I reduce the ram usage?

Thanks again for your help.

Coordinator
Jul 17, 2012 at 5:21 PM

I'd have to see the loading code to know for sure. But in the past when I've seen this the memory consumption can be greatly reduced by loading into Scintilla this way:

  • Declare a byte[] buffer.
  • Read a chunk from your file directly into your buffer.
  • Directly call the native Scintilla's AppendText function passing a pointer to the buffer.

 

byte[] buffer;
// Fill the buffer, record how many bytes are actually used in variable "length". 
// Don't use buffer.Length unless the entire array is filled.
fixed (byte* bp = buffer)
    scintilla.NativeInterface.SendMessageDirect(ScintillaNET.Constants.SCI_APPENDTEXT, (IntPtr)length, (IntPtr)bp);

This avoids the extra memory of converting the binary data (from source) into a .NET string, ScintillaNet adding a null terminator (for c based string handling) which of course allocates another string, converting that string back to a byte[], then passing the string onto Scintilla. Additionally most of the higher level Scintilla APIs will create a range object that you may not need.

Jul 19, 2012 at 4:55 PM

I did this, but never the less Scintilla throws a "std::bad_alloc" exception when I try to set a text which has a size of 200 MB... Is it possible, that native .dlls called from managed code are not allowed to use as much ram, as when they were called from a unmanaged application?

Coordinator
Jul 20, 2012 at 2:51 AM

Before we go any further, it might be helpful for you to read up on how .NET garbage collection works. Depending on how you're loading your string it might be landing in the Large Object Heap which is seldom collected and might throw off you memory profile. Also, prior to doing any memory profiling make sure you are performing a garbage collection so you are getting accurate numbers. Lastly, there are many tools and some APIs within the framework itself that will help you know how much of your memory consumption is managed vs unmanaged memory and I bet you'll be surprised how it breaks down.

 

Jacob