Ignore the problem

Individual users can prevent core dumps from filling up their disk by setting the core dump size limit to zero. If you're using the bash shell (other shells have similar commands), type this to set the core dump size limit to 0, effectively disabling them:

$ ulimit -c 0

Without the 0 argument, you can print the current limit:

$ ulimit -c
unlimited

You can re-enable core dumps with this command:

$ ulimit -c unlimited

You can also disable core dumps system-wide by setting the following kernel parameter:

$ sudo sysctl kern.coredump=0
kern.coredump: 1 -> 0

Note that neither approach prevents the child processes from crashing, it just prevents your disk from getting filled. So it's not really a solution, because the code in your child process will still not get executed.

Band-aid the problem

You see lots of recommendations to set the following environment variable:

$ export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES

This does prevent the child process from crashing, but it comes with an important caveat: You must set this outside of the forking process. It is not good enough to do something like this in your Python code:

os.environ['OBJC_DISABLE_INITIALIZE_FORK_SAFETY'] = 'YES'
pid = os.fork()
if pid == 0:
    # I am the child process.
    requests.get('https://www.python.org')
    os._exit(0)
else:
    # I am the parent process.
    do_something()

I haven't positively confirmed this with outside references, but my experimentation has shown that this environment variable is only consulted when the parent process first starts, so if you set it in the parent before the fork(), it's ignored. Using os.putenv() doesn't help either.

Maybe this is good enough for you. At work, it's not, because we can't change the environment for every possible process. We use both shiv and pex as Python zip application formats (transitioning to all-shiv for Python >= 3.6) for command line tools (CLIs). For CLIs with multiple entry points, we do use a wrapper script that forks-and-execs the zip application, and we can set the environment variable there to prevent crashes. But for zip applications with a single entry point, we don't use the wrapper, so there's no universally good place to set this, and asking all the users to update their various shell initialization scripts isn't viable.

Other possible solutions

You'll see these other recommendations out on the 'net:

1. Use NSTask or posix_spawn() instead of fork() and exec().
2. Do nothing between fork() and exec().
3. Use only async-signal-safe operations between fork() and exec().
4. Use ObjC classes with no +initialize overrides between fork() and exec().
5. Use pthread_atfork() to force your +initialize methods to run before fork().
6. Define environment variable OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES, or add a __DATA,__objc_fork_ok section, or build using an SDK older than macOS 10.13. Then cross your fingers.

Let's look at each of these in turn.

#1 and #2 are the real solutions because they use safer mechanisms to spawn a child process, but both have the disadvantage of requiring you to implement your child process in a separate file. This can be quite a serious drawback. For us at work, it means designing a protocol to pass the required information between the parent and the child, where with the old idiom, this data was just maintained in local variables which get copied to the child upon fork()-ing. Still, this is guaranteed to be safe, so it's what we'll implement.

#3 and #4 are not, IMHO, practically effective because as I've mentioned, it can be surprising what gets called in a scripting language like Python. Before I debugged the problem, I never expected the calls from requests into _scproxy into the Objective-C runtime.

#5 sounds promising, and I actually experimented with using the prepare handler of pthread_atfork(), but I could never actually get it to work. Another recommendation I found suggests that simply loading a framework library will invoke its +initialize method. I wrote some code to implement this. While it was fun to play with ctypes, this never actually worked for me:

import ctypes
from ctypes.util import find_library

@ctypes.CFUNCTYPE(None)
def prefork_callback():
    ctypes.CDLL('/System/Library/Frameworks/Foundation.framework/Foundation')
    ctypes.CDLL('/System/Library/Frameworks/CoreFoundation.framework/CoreFoundation')
    ctypes.CDLL('/System/Library/Frameworks/SystemConfiguration.framework/SystemConfiguration')

libc = ctypes.CDLL(find_library('c'))
code = libc.pthread_atfork(prefork_callback, None, None)
if code != 0:
    print('Could not install the pthread_atfork() prepare handler')

The list of frameworks I load was found by inspecting the core dumps with lldb but try as I might, I could not find the right combination of loads to prevent the crashes. It seems as if loading the framework doesn't actually guarantee that its +initialize method gets called.

As I thought about it more, I'm not even sure a prepare handler and calling pthread_atfork() is necessary. If this technique worked, why not just call it inline before the call to os.fork()? Suggestions are welcome!

I have not played with the other suggestions of #6, namely adding a __DATA__,__objc_fork_ok section, but I'm skeptical that those will solve the problem. And using an older SDK is also not a viable option.