Include .pyd module files in py2exe compilation

Question

I'm trying to compile a python script. On executing the exe I got:-

C:\Python27\dist>visualn.exe
Traceback (most recent call last):
  File "visualn.py", line 19, in <module>
  File "MMTK\__init__.pyc", line 39, in <module>
  File "Scientific\Geometry\__init__.pyc", line 30, in <module>
  File "Scientific\Geometry\VectorModule.pyc", line 9, in <module>
  File "Scientific\N.pyc", line 1, in <module>
ImportError: No module named Scientific_numerics_package_id

I can see the file Scientific_numerics_package_id.pyd at the location "C:\Python27\Lib\site-packages\Scientific\win32". I want to include this module file into the compilation. I tried to copy the above file in the "dist" folder but no good. Any idea?

Update: Here is the script:

from MMTK import *
from MMTK.Proteins import Protein
from Scientific.Visualization import VRML2; visualization_module = VRML2
protein = Protein('3CLN.pdb')
center, inertia = protein.centerAndMomentOfInertia()
distance_away = 8.0
front_cam = visualization_module.Camera(position= [center[0],center[1],center[2]+distance_away],description="Front")
right_cam = visualization_module.Camera(position=[center[0]+distance_away,center[1],center[2]],orientation=(Vector(0, 1, 0),3.14159*0.5),description="Right")
back_cam = visualization_module.Camera(position=[center[0],center[1],center[2]-distance_away],orientation=(Vector(0, 1, 0),3.14159),description="Back")
left_cam = visualization_module.Camera(position=[center[0]-distance_away,center[1],center[2]],orientation=(Vector(0, 1, 0),3.14159*1.5),description="Left")
model_name = 'vdw'
graphics = protein.graphicsObjects(graphics_module = visualization_module,model=model_name)
visualization_module.Scene(graphics, cameras=[front_cam,right_cam,back_cam,left_cam]).view()

Janne Karila · Answer 1 · 2012-03-08T10:27:45.317

3

Py2exe lets you specify additional Python modules (both .py and .pyd) via the includes option:

setup(
    ...
    options={"py2exe": {"includes": ["Scientific.win32.Scientific_numerics_package_id"]}}
)

EDIT. The above should work if Python is able to

import Scientific.win32.Scientific_numerics_package_id

edited Mar 08 '12 at 10:27

answered Mar 06 '12 at 11:09

Janne Karila

24,266
6
53
94

It gave this error - `ImportError: No module named Scientific.win32` – user1144004 Mar 07 '12 at 19:21

score 1 · Answer 2 · edited May 23 '17 at 12:23

1

I encountered similar probelm with py2exe and the only solution I can find ,is to use another tool to convert python to exe - pyinstaller

Its very easy tool to use and more important , it works!

UPDATE

As I understood from your comments below , running your script from command line is not working also , due to import error (My recommendation is to first check your code from command line ,and than try to convert it to EXE)

It looks like PYTHONPATH problem.
PYTHONPATH is list of paths (similar of Windows PATH) that python programs use to find import modules. If your script run from your IDE , that means the PYTHONPATH is set correctly in the IDE ,so all imported modules are found.

In order to set PYTHONPATH you can use :

import sys|
sys.path.append(pathname)

or use the following code that add the all folders under path parameter to PYTHONPATH:

import os
import sys

def add_tree_to_pythonpath(path):
    """
    Function:       add_tree_to_pythonpath
    Description:      Go over each directory in path and add it to PYTHONPATH
    Parameters:     path - Parent path to start from
    Return:         None
    """    
    # Go over each directory and file in path
    for f in os.listdir(path):
        if f ==  ".bzr" or  f.lower() ==  "dll":
            # Ignore bzr and dll directories (optional to NOT include specific folders)
            continue
        pathname = os.path.join(path, f)
        if os.path.isdir(pathname) ==  True:
            # Add path to PYTHONPATH
            sys.path.append(pathname)

            # It is a directory, recurse into it
            add_tree_to_pythonpath(pathname)
        else:
            continue

def startup():
    """
    Function:       startup
    Description:      Startup actions needed before call to main function 
    Parameters:     None
    Return:         None
    """    

    parent_path = os.path.normpath(os.path.join(os.getcwd(), ".."))
    parent_path = os.path.normpath(os.path.join(parent_path, ".."))

    # Go over each directory in parent_path and add it to PYTHONPATH
    add_tree_to_pythonpath(parent_path)

    # Start the program
    main()

startup()

edited May 23 '17 at 12:23

Community

1
1

answered Mar 06 '12 at 08:23

Gil.I

895
12
23

yeah, I tried that too but compiling by puinstaller is very complex and gave me other error [PyInstaller: IOError: No such file or directory](http://stackoverflow.com/questions/9553262/pyinstaller-ioerror-errno-2-no-such-file-or-directory) – user1144004 Mar 06 '12 at 09:09
In my computer the exe gave the error even when python is present. It doesn't execute at all. – user1144004 Mar 06 '12 at 09:14
Maybe its PYTHONPATH related problem. Is your script run without problems as python code in command line (and not from your IDE?) I.E > python script.py – Gil.I Mar 06 '12 at 10:59
no it didn't run. It gave me this error- `from MMTK.win32 import *` `ImportError: No module named win32` – user1144004 Mar 07 '12 at 18:13
@user1144004 See my updated answer for PYTHONPATH related error – Gil.I Mar 07 '12 at 20:21
Your answer solved my most of the problem. now python is including that pyd module and some more pyd files in the same folder but not all of them. I used `sys.path.append("C:\Python27\Lib\site-packages\MMTK\win32")` to include all pyd files in this folder but on executing the compiled exe, it gave me this error - `ImportError: No module named MMTK_trajectory_action` although the `MMTK_trajectory_action.pyd` is present in the same folder. – user1144004 Mar 08 '12 at 06:37
Now if you run your script from the command line ,(not the exe file) , it works? Please attach your source code to your question if you want me (or other pepole) to find the rest of the problem. Try to attach only the relevant part – Gil.I Mar 08 '12 at 08:09
Yes the script runs from the command line. I've updated the question. – user1144004 Mar 08 '12 at 09:05
Where in the attached code ,you try to import MMTK_trajectory_action (your ImportError) ? and what is the ; after import VRML2 ? and maybe try to use my original suggestion , pyinstaller , now after the script run from command line – Gil.I Mar 08 '12 at 09:09
I got the code from [here](http://www2.warwick.ac.uk/fac/sci/moac/people/students/peter_cock/python/mmtk_vrml_cameras/). I'm just trying to compile it. – user1144004 Mar 08 '12 at 09:32
I looked in the code in the link ,but in order to run it I need to install many packages that some of them dont support my python version. So I have 3 suggestions: 1. Now, after the script run from the command line , try to use pyinstaller or other python to exe tools. 2. are you sure you include all the packages you need in python path? nympy,ScientificPython,MMTK.... 3. Use Janne Karila suggestion above – Gil.I Mar 09 '12 at 23:00

score 1 · Answer 3 · answered Mar 06 '12 at 10:28

There is a way to work around this types of issues that I have used a number of times. In order to add extra files to the py2exe result you can extend the media collector in order to have a custom version of it. The following code is an example:

import glob
from py2exe.build_exe import py2exe as build_exe

def get_py2exe_extension():
    """Return an extension class of py2exe."""

    class MediaCollector(build_exe):
        """Extension that copies  Scientific_numerics_package_id missing data."""

        def _add_module_data(self, module_name):
            """Add the data from a given path."""
            # Create the media subdir where the
            # Python files are collected.
            media = module_name.replace('.', os.path.sep)
            full = os.path.join(self.collect_dir, media)
            if not os.path.exists(full):
               self.mkpath(full)

            # Copy the media files to the collection dir.
            # Also add the copied file to the list of compiled
            # files so it will be included in zipfile.
            module = __import__(module_name, None, None, [''])
            for path in module.__path__:
                for f in glob.glob(path + '/*'):  # does not like os.path.sep
                    log.info('Copying file %s', f)
                    name = os.path.basename(f)
                    if not os.path.isdir(f):
                        self.copy_file(f, os.path.join(full, name))
                        self.compiled_files.append(os.path.join(media, name))
                    else:
                        self.copy_tree(f, os.path.join(full, name))

        def copy_extensions(self, extensions):
            """Copy the missing extensions."""
            build_exe.copy_extensions(self, extensions)
            for module in ['Scientific_numerics_package_id',]:
                self._add_module_data(module)

    return MediaCollector

I'm not sure which is the Scientific_numerics_package_id module so I've assumed that you can import it like that. The copy extensions method will get a the different module names that you are having problems with and will copy all their data into the dir folder for you. Once you have that, in order to use the new Media collector you just have to do something like the following:

cmdclass['py2exe'] = get_py2exe_extension()

So that the correct extension is used. You might need to touch the code a little but this should be a good starting point for what you need.

score 1 · Accepted Answer · edited May 23 '17 at 12:31

1

The ImportError is rectified by using "Gil.I" and "Janne Karila" suggestion by setting pythonpath and by using include function. But before this I had to create __init__.py file in the win32 folder of both the modules. BTW I still got another error for the above script - link

edited May 23 '17 at 12:31

Community

1
1

answered Mar 10 '12 at 07:47

user1144004

183
3
4
21

Include .pyd module files in py2exe compilation

4 Answers4