Advise GC to cleanup object

[msedi]

Just an idea that came up lately. Does it make sense or is it even possible to advise the GC to cleanup object in the LOH? The idea arose from another question regarding the need for enforcing a LOH compaction.

My question is if you know that you need to cleanup an object, does it help to assist the GC to prioritize these objects?

[Xyncgas]

GC.Collect()

[StellarWolfEntertainment]

If I understand the OPs question correctly, this doesn't do what they want. Assuming that I do understand, they want something more akin to a GC.Prioritize(obj) so that when an object goes out of scope it's memory is more quickly handled than it otherwise would have been. I could be wrong about what their asking but that is how I understood it.

[msedi]

@StellarWolfEntertainment. Thanks for clarifying. I think yes, thats what I assume could help in the management of LOH objects.

[danmoseley]

What value would it have to hint to the GC (if you could somehow)? It will collect garbage when it thinks it needs to in order to satisfy an allocation request.

[StellarWolfEntertainment]

The only issue that I see with it is that it doesn't allow me to specify the object that has allocated said memory, but I'm not certain as to exactly how it works, so it make have the context required to know that the 'previously allocated type' is the bearer of the memory, or it may not know which doesn't really help anything because while 'AddMemoryPressure' may cause GC to be more active, it wouldn't change the icebergs priority

[sfiruch]

There appears to be a misunderstanding: GC is not focussing on certain objects to collect. It can only either do nothing, or collect all unreferenced objects. There's no way to prioritize a specific "iceberg" to collect only that early. If the iceberg should be collected, the GC has to do a pass over all objects. Therefore, there's no point trying to "connect" the memory pressure to a specific object.

[StellarWolfEntertainment]

GC prioritizes objects based on age and size, the older and larger it is the more likely that when GC does a pass that it will be cleaned up, it doesn't [and shouldn't] clean up all 'unreferenced' objects in a single pass because that would be a serious performance problem, so if GC doesn't know the 'true' size of a specific object it will keep getting 'left' due to 'small size'.

[sfiruch]

As stated, I don't think this is correct. A GC might focus on certain generations, or perhaps even certain regions - but I don't know of any GCs that (can) focus on specific objects. A GC also doesn't "leave" objects due to small size. Referenced objects are either moved, or overwritten. It doesn't make sense to keep "small" objects around for a GC.

[StellarWolfEntertainment]

It's not that the GC is 'deliberately' leaving 'small objects' around (if it were I would agree that, that makes no sense), it's that in order to minimize runtime disruption GC will clean up the largest blocks of memory first before cleaning up the smaller objects, yes it is 'generational' [in that the older an object is the higher it's cleanup priority] and I do suppose it's not so much about 'specific' objects, but it is aware of the regions of memory where those objects are and how big those regions are, GC does (and should) prioritize the larger blocks because that means overall it has to do less work to keep memory low, and GC doing less work means less disruption to the given application, and there is nothing wrong with that except that GC has no context which is all that is being requested by OP, the ability to provide GC with context about the true size of an object [even if you don't actually have to provide it's size, and can only flag the object as 'priority cleanup', this wouldn't change when GC cleans up just what it cleans up first]

[msedi]

@danmoseley. This for sure no common scenario. I was only thinking about my current scenario where I'm working with many GB/TB of data where I'm allocating a lot in the LOH. Almost 90% of the data are in the LOH. The amount of data I need to allocate during one run is dependent on the input data which range from 5GB to 120GB). During processing multiple intermediate datasets of the same size are allocated (50-100, so for the extreme case it could happend that I allocate 120TB during one run), not all at once but during the whole processing chain. Which means the required amount of memory goes way beyond what is available in the system memory (max. 256GB but it's able to scale) so I crceated some sort of memory mapped files without using them for many reasons:

1. Since the windows memory manager is running concurrently I don't know when to throttle my processing pipeline which leads to the problem that my processing pipeline is creating new memory while the windows memory manager is trying to page out the dirty and unused pages. The windows memory manager (or better the page disk) is slower than I create data. which brings the system to a condition where everything is at 100% (100% CPU, 100% RAM and 100% disk usage). Ultimately the system halts and I need to restart. Since there is not information from windows that the memory manager is currently paging I have no knowledge of when to throttle my pipeline.

2. Another thing is with memory mapped files that I do not know which page is dirty. Windows does not provide any method to get the information. QueryWorkingSetEx, VirtualQueryXX or QueryVirtualMemoryInformation do not return the full information of the page table entry) especially the dirty/modified bit, but a lot of information that doesn't help in my case. Additionally I cannot install a write watch that I is available when creating memory using VirtualAlloc. I could of course track it myself (which I do now) but then I need to work with additional info structures that track the dirty state which makes it necessary that some has to mark the page as dirty which is as additional overhead (and windows also does it, so I have to create such structures twice for no use).

3. The memory manager does not know about my access patterns which I know much better (many of them are sequentially, other are random), but here I at least have some possibilities with Virtualock, OfferVirtualMemory, DiscardVirtualMemory or PrefetchVirtualMemory.

Due to this problems I need to manage things myself and created my own memory mapped files (although I find this stupid and would prefer using memory mapped files - but as shown above a few links are missing, especially the dirty bit and the information when paging is running).

The reason why I'm asking if the GC can be given some additional information (in LOH only) if it would help to coordinate the cleanup for the LOH better. Still, of course the GC has to check for references to the data but if the GC would know better it could prioritize the memory that has been marked with this "special" advise method.

I also thought about having some sort of ArrayPool, but I don't know the size of the data in advance and there are many different ranges of data sizes (from 32x32 to 4096x4096) and additionally the ArrayPools need to cooperate and lose memory in favor of other pools that currently need memory pressure is higher than on others. Thats why I have some sort of master control unit that coordinates the memory among the pools .

Its not that I urgently need such an Advise method and of course its a very uniqe scenario, it's just a thought experiment if it could potentially improve GC performance in LOH cleanups.

Does that make sense?

[Xyncgas]

1. Did you know you can use async to yield processor clocks while waiting for your IO operations that's slower (including memory de/allocations)
2. GC detects memory pressure, so if your memory pressure is high .NET will look for unused objects and free them from the memory.

[danmoseley]

@mangod9 another GC q.

[msedi]

@Xyncgas: Thanks for the info. I'm not sure if I get the first point. How do I do that?

For the second point. All my objects are in use and I don't want to given them up top early. So what my memory is currently doing is:

1. look for objects I can quickly recover (e.g. either by calculating them or by reading them again) => Fast path
2. Looking for older objects that I have to write first => Slow path
3. After that I have to call a hard GC with compacting sometimes also call EmptyWorkingSet so that I know how much memory I can fully use to reallocate new objects.

I wouldn't need this if the GC could tell how much memory is free for me so that I can calculate how many objects can be allocated until the next cleanup needs to be done. I do have to experiment with GCMemoryInfo.TotalAvailableMemoryBytes.

The question is though will I always need to put this much pressure on the GC or would it help if I could tell him some more info about the cleanup. I I wouldn't interfere it currently a back and forth, I create, the GC destroys and if the GC is not fast enough the game is over and I get an OOM.

As said above its nothing that I really need (if it even helps at all and its just nonsense what I ask) its more to have some discussion about this.

[Xyncgas]

you can buy intel's optane and use NVME backed storage.
Meanwhile for example you can await memory operations by :

        await using var input = new MemoryStream(bytes);

[msedi]

See also: #7933

[sfiruch]

The idea is a good idea, but such a hint would not be actionable for the GC. In order to collect memory, the GC has to check whether the memory is reachable from anywhere. This means that it has to scan all the stacks, all pointers from there, all pointers from there, etc. until it has found all reachable objects. That's the expensive part of GC. Even if the GC got the hint to collect a single object, it would have to determine its reachability - and for that it has to scan the "world". Now that the GC scanned the world, it might have detected a lot of unreachable objects. It makes sense to collect all these objects in one swoop.

In short: Getting a hint for a single object doesn't improve the situation over GC.Collect.

[msedi]

@sfiruch: Thanks for the comment. That really makes sense. Today @StellarWolfEntertainment brought up a very good point to prioritize LOH objects. Currently LOH objects are left alone because the GC does not take care of them in the first place. You have to explicitly force a GCSettings.LargeObjectHeapCompactionMode as @AustinWise brought up.

GCSettings.LargeObjectHeapCompactionMode seems a biut weird, though since you have to set it once, call GC.Collect(), afterwards and then the flag flips back.

Its interesting why there is no method to have a GC.Collect for the LOH.

Nevertheless, I like the idea of @StellarWolfEntertainment to somehow prioritize, but I don't know if it helps.

[elgonzo]

Its interesting why there is no method to have a GC.Collect for the LOH.

GCSettings.LargeObjectHeapCompactionMode with GC.Collect() makes more sense in my opinion.

Because consider this: There's an object in the normal (small object) managed heap subject to collection. It holds a reference to another large object on the LOH. Now, calling a special collect method for the LOH only wouldn't be able to collect the small object (because it's not on the LOH), and therefore the large object on the LOH wouldn't be collected either.

(Also note that GCSettings.LargeObjectHeapCompactionMode is effecting the next blocking collection, whether this is triggered by an explicit call to GC.Collect or by some other circumstance. I fail to imagine a use case where you would want to compact the LOH but leave the time when that occurs indeterminate and uncontrolled by the application, but it's technically possible to do so...)

[msedi]

@elgonzo.

and therefore the large object on the LOH wouldn't be collected either.

That makes sense. Our situation is though, that we are dealing with Terabytes of LOH objects where we currently do not much have control over it. Having a better way (I don't know which) to clean up LOH objects more efficiently would help.

But your concerns are also true.

[AustinWise]

Is GCSettings.LargeObjectHeapCompactionMode what you are looking for?

[msedi]

@AustinWise. Thanks for the hint, I already do it. Do you know the compaction is not requested, does the GC automatically do it when it's needed before an OutOfMemoryException occurs?

[AustinWise]

Maybe not by default. There are some settings you change to cause the GC to compact the large object heap automatically. For example, the documentation for the Conserve memory setting says:

If the value is non-zero, the large object heap will be compacted automatically if it has too much fragmentation.

Searching around the GC source code for loh_compaction = TRUE suggests that setting the heap hard limit setting may also compact the LOH under certain conditions, include when a OOM is imminent. This is not mentioned in this settings documentation however.

[danmoseley]

@msedi do you think you can mark one of the responses as answer? feels like we've covered everything