for (int i =0; i

Ex. 1
for (int i = 0; i

But - just call Close() and Dispose() as soon as you can on EVERY .Net object that offeres these methods, and sleep easy at night.

1) Are you creating 10,000 objects for every user that hits your web application?

No, this is an admin application where a few users are occasionally performing work on ~10000 records. For example, generating PDF documents for records, or exporting to a CSV file, or performing some sort of mass update.

2) How are you storing the data in the MyObject class? What kind of data types are you
using?

Actually, the MyObject class is typically a class representing an entity. For example, a Student record let’s say. The student record is for the most part pretty basic, with simple short string properties. But, it does have a IDictionary property that has approximately 300 items. This is a custom field -> value dict that allows us to store arbitrarily defined data for the student. That is the big item for these objects. And, because this is actually stored as XML in the DB, I deserialize it into the dictionary when it is loaded from the DB. I perform this deserialization using an XmlReader and Serialize using XmlWriter. Certainly, this could be causing an issue, but I thought using the stream approach would be better than loading an XML document and looping through it.

3) I noticed you are using the StringBuilder class.

Yes, and I also think this might be an issue. In the example I presented, I am basically storing a CSV record for each record in the string builder, then after the loop completes, I create and write the contents to a CSV file. I have already started modifying this approach to stream to the file during the processing, rather than creating the x-large SB object.

I have also seen some others discuss issues where you do not set an initial size for the SB, and the SB has to dynamically allocate more memory.

Thanks!

Kevin

While it is true that the objects you are instantiating in the loop will go out of scope
when the loop completes, that does NOT mean that the memory allocated and used for those
objects will be released back to the ASP.NET worker process at that time. That release
of the memory will only occur during the next Garbage Collection execution. So rather
than trying to cleanup the memory, which you cannot control, you have to examine your
algorithm and data types to find the best solution to resolve the memory problem. In
other words, you have to change the way that you designed the code. That is how you can
solve this memory usage problem.

There could be many reasons that you are getting the Out of Memory error. Also, it could
be a combination of of things. Here are some things to consider.

1) Are you creating 10,000 objects for every user that hits your web application? That
appears to be a lot of memory usage if the size of MyObject is large as you say below in
your comment. The way you estimate how much available ASP.NET RAM you need for that
block of code to run successfully is by using this equation: (Est. size of MyObject) * (#
of objects created in your loop) * (Max # of concurrent users). For example, if the
estimated size of MyObject is 10K bytes, you are creating 10,000 MyObjects in the loop
and your application has max 50 concurrent users, you would need about 10k * 10,000 * 50
= 5000000K bytes of memory available. If your object is large, it is probably much more
than 10k in my example.

Is there a need to loop and create 10,000 objects for every user that hits the
application? Why do you need so many? Can you reduce the loop iteration count to much
less? This of course requires analysis of your overall algorithm and maybe even business
requirements.

2) How are you storing the data in the MyObject class? What kind of data types are you
using? Are those the most efficient data types with regards to memory? You can e-mail
me the code for that class and I can look at it if you want.

3) I noticed you are using the StringBuilder class. Although StringBuilder is generally
much better for performance versus using String concatenation, you can try this to see if
less memory is consumed. Try to give the StringBuilder a size when you instantiate it.
How big do you think the string will be once you have appended everything to it? If you
think the average largest size of the string is going to be 5Megabytes, then you can
instantiate the StringBuilder like this:
StringBuilder mybuilder = new StringBuilder(5242880);

This tells ASP.NET to allocate 5Meg worth of space right away and it will not require any
further allocation whenever you append data to string using the Append method. What this
does is ensure that you are not allocating any new blocks of memory each time in the loop
with regards to the StringBuilder-this does not affect the memory needed for the MyObject
class instantiations.

Yours,
Ted Kolovos

StringBuilds output = new StringBuilder();
IList myIDs = GetMyIDs(); // Returns ~10,000 ids
foreach (int id in myIDs)
{
MyObject myObj = GetMyObject(id); // Returns a new object. Very large object (holds alot of XML data in a property for example.)
output.Append(myObj.Render()); // Just appends some text to the SB.
}

I am getting System.OutOfMemory exceptions.

Based on your tutorial, each of the 10,000 objects instantiated will hold a place in memory until they go out of scope. I think that this means the scope only ends after the loop completes processing. So, is there a way to release these resources earlier than the end of the loop?

Thanks!

Kevin