source: http://bugs.python.org/issue5162

classificationTitle: multiprocessing cannot spawn child from a Windows service  
Type: behavior Stage: test needed
Components: Library (Lib) Versions: Python 2.6
processStatus: open Resolution: 
Dependencies:   Superseder: 
Assigned To: jnoller  Nosy List:  jnoller, orlenko (2) 
Priority:  normal Keywords patch
 
Created on 2009-02-06 02:00 by orlenko, last changed 2009-03-29 15:44 by jnoller.

Files
File name Uploaded Description Edit Remove
forking-patch  orlenko, 2009-02-06 02:00  Patch of the forking module  
Messages (1)
msg81247 - (view) Author: Volodymyr Orlenko (orlenko) Date: 2009-02-06 02:00 
I think I've found a small bug with multiprocessing package on
Windows. If you try to start a multiprocessing.Process from a Python-
based Windows service, the child process will fail to run. When
running the parent process as a regular Python program, everything
works as expected.
I've tracked the problem down to how main_path is prepared in
multiprocessing.forking.get_preparation_data() (lines 370-377):
def get_preparation_data(name):
    [...skipped a few lines...]
    if not WINEXE:
        main_path = getattr(sys.modules['__main__'], '__file__', None)
        if not main_path and sys.argv[0] not in ('', '-c'):
            main_path = sys.argv[0]
        if main_path is not None:
            if not os.path.isabs(main_path) and \
                                      process.ORIGINAL_DIR is not
None:
                main_path = os.path.join(process.ORIGINAL_DIR,
main_path)
            d['main_path'] = os.path.normpath(main_path)
    return d
When the program is running as a Windows service, but is not packaged
into a single executable, main_path will become the path to the
service executable (typically, pythonservice.exe). When this data
makes it to the child process, the prepare() function will treat
main_path as a path to a python module, and will try to import it.
This causes it to fail.
My quick-and-dirty solution was to check in get_preparation_data() if
main_path ends with '.exe', and if it does, to not pass it at all.
This solves the problem in my case, but perhaps there's a better way
to fix this? Here is my version of get_preparation_data():
def get_preparation_data(name):
    '''
    Return info about parent needed by child to unpickle process
object
    '''
    from .util import _logger, _log_to_stderr
    d = dict(
        name=name,
        sys_path=sys.path,
        sys_argv=sys.argv,
        log_to_stderr=_log_to_stderr,
        orig_dir=process.ORIGINAL_DIR,
        authkey=process.current_process().authkey,
        )
    if _logger is not None:
        d['log_level'] = _logger.getEffectiveLevel()
    if not WINEXE:
        main_path = getattr(sys.modules['__main__'], '__file__', None)
        if not main_path and sys.argv[0] not in ('', '-c'):
            main_path = sys.argv[0]
        if main_path is not None:
            if not os.path.isabs(main_path) and \
                                      process.ORIGINAL_DIR is not
None:
                main_path = os.path.join(process.ORIGINAL_DIR,
main_path)
            if not main_path.endswith('.exe'):
                d['main_path'] = os.path.normpath(main_path)
    return d