Как запретить OOM киллеру мочить процессы кроме того, которые пытается выделить лишнего?

Вот такой вот опцией ядра, установив ее в единицу:

cat /proc/sys/vm/oom_kill_allocating_task

Подробнее:

/proc/sys/vm/oom_kill_allocating_task (since Linux 2.6.24)
This enables or disables killing the OOM-triggering task in out-of-
memory situations.

If this is set to zero, the OOM-killer will scan through the entire
tasklist and select a task based on heuristics to kill. This normally
selects a rogue memory-hogging task that frees up a large amount of
memory when killed.

If this is set to nonzero, the OOM-killer simply kills the task that
triggered the out-of-memory condition. This avoids a possibly
expensive tasklist scan.

If /proc/sys/vm/panic_on_oom is nonzero, it takes precedence over
whatever value is used in /proc/sys/vm/oom_kill_allocating_task.

The default value is 0.

Для sysctl.conf это будет строчка:

vm.oom_kill_allocating_task = 1