This article is again about programming language, so if you are not interested in this area, see you next time.
My first python project is a test suite. I like Martin Fowler's articles, so, naturally, my first program is a test. My project is usually started with test, documentation, and interface design, yet incremental.
This is about python's os.environ and LD_LIBRARY_PATH. This is a follow up of the boost.python's story. My development environment is Linux, but later I have a plan to go to Windows also.
In python interpreter, we can change the environment variables through os.environ. For example, if you want to change the LD_LIBRARY_PATH:
os.environ['LD_LIBRARY_PATH'] = '/some/directory/lib'
I thought this is all for LD_LIBRARY_PATH, but I hit a wall. My department policy doesn't allow to have administrator right and Internet access to the developers. Therefore, if I want to use something not in my computer, for example, boost or python, I first need to ask to get the source code, and need to think how to install the software locally. Under this environment, the setting of shared library path is important.
My project has the following directory structure.
test_x +--+ pymodule +--+ mypython_binding_module.so
+--+ testbase +--+ test_run.py
+--+ boost144 +--+ lib +--+ ...
+--+ libboost_python.so
The problem here is mypython_binding_module.so depends on libboost_python.so. LD_LIBRARY_PATH is an environment variable. Since environment variable is program global and has implicit effect, I usually try to avoid to use them. But this time I need it and I try to set this in the python interpreter only.
Python interpreter can recognize mypython_binding_module.so as a python module, so I can set sys.path to import this module. But, mypython_binding_module.so depends on libboost_python.so, this can not be done by sys.path. dlopen() does this task. Therefore, I need to set LD_LIBRARY_PATH to load libboost_python.so.
I set LD_LIBRARY_PATH by os.environ, however, this does not work. I first suspect, python's os module only tells the current environment variables, but actually did not change them. However, manual said python changed them.
The problem was at the dlopen(). dlopen()'s manual said, this only looks up the LD_LIBRARY_PATH when the program starts. The process changes the LD_LIBRARY_PATH, it doesn't affect the dlopen()'s behavior. This is understandable behavior considering the security.
Once I understood this, I changed the LD_LIBRARY_PATH through os.environ, then I created a child process. In this child process, LD_LIBRARY_PATH's change is effective as expected.
Here, os.environ can change the environment variables, but the function takes it or not is depends on the function. This is a simple thing, but, I don't realize for two days.
Yet, the problem of setting library path seems a common problem. I looked into the documentation (e.g., site module, etc.), I might miss something since I am only a beginner of python.
My first python project is a test suite. I like Martin Fowler's articles, so, naturally, my first program is a test. My project is usually started with test, documentation, and interface design, yet incremental.
This is about python's os.environ and LD_LIBRARY_PATH. This is a follow up of the boost.python's story. My development environment is Linux, but later I have a plan to go to Windows also.
In python interpreter, we can change the environment variables through os.environ. For example, if you want to change the LD_LIBRARY_PATH:
os.environ['LD_LIBRARY_PATH'] = '/some/directory/lib'
I thought this is all for LD_LIBRARY_PATH, but I hit a wall. My department policy doesn't allow to have administrator right and Internet access to the developers. Therefore, if I want to use something not in my computer, for example, boost or python, I first need to ask to get the source code, and need to think how to install the software locally. Under this environment, the setting of shared library path is important.
My project has the following directory structure.
test_x +--+ pymodule +--+ mypython_binding_module.so
+--+ testbase +--+ test_run.py
+--+ boost144 +--+ lib +--+ ...
+--+ libboost_python.so
The problem here is mypython_binding_module.so depends on libboost_python.so. LD_LIBRARY_PATH is an environment variable. Since environment variable is program global and has implicit effect, I usually try to avoid to use them. But this time I need it and I try to set this in the python interpreter only.
Python interpreter can recognize mypython_binding_module.so as a python module, so I can set sys.path to import this module. But, mypython_binding_module.so depends on libboost_python.so, this can not be done by sys.path. dlopen() does this task. Therefore, I need to set LD_LIBRARY_PATH to load libboost_python.so.
I set LD_LIBRARY_PATH by os.environ, however, this does not work. I first suspect, python's os module only tells the current environment variables, but actually did not change them. However, manual said python changed them.
The problem was at the dlopen(). dlopen()'s manual said, this only looks up the LD_LIBRARY_PATH when the program starts. The process changes the LD_LIBRARY_PATH, it doesn't affect the dlopen()'s behavior. This is understandable behavior considering the security.
Once I understood this, I changed the LD_LIBRARY_PATH through os.environ, then I created a child process. In this child process, LD_LIBRARY_PATH's change is effective as expected.
Here, os.environ can change the environment variables, but the function takes it or not is depends on the function. This is a simple thing, but, I don't realize for two days.
Yet, the problem of setting library path seems a common problem. I looked into the documentation (e.g., site module, etc.), I might miss something since I am only a beginner of python.
Comments