I understood wyldfire's post to mean counting & imposing your own artificial memory limit, no mlock involved. But I wouldn't know how to pick that limit :/.
Oh I see. Well I'm not trying to add a limit to my RSS, I just want to be notified when allocation fail in my process on my machine where overcommit is enabled.
Maybe what I'm asking for makes no sense though, because even if my process handles out of memory errors gracefully, it might still get OOM-killed when another process allocates some more.
Yes that is what I meant. And it would avoid the latency hit of a system call to satisfy an allocation. However doing an mlock would avoid the latency hit of paging, so that's good too.