Update the Readme file.
epsi-builds/msspec_python3/pipeline/head This commit looks good Details

I updated the information in the Readme file to provide
detailed steps to build msspec with GNU make.
This commit is contained in:
Sylvain Tricot 2021-02-22 15:49:57 +01:00
parent ebb450024b
commit a4c13d3bdb
5 changed files with 96 additions and 296 deletions

126
README.md
View File

@ -4,68 +4,134 @@ Introduction
This is the Python MsSpec version with support for Python 3.x and dynamic memory allocation for Phagen and Spec This is the Python MsSpec version with support for Python 3.x and dynamic memory allocation for Phagen and Spec
Installation of the devel package
=================================
To work with the git repository Requirements
=============================== ------------
You can clone this repository onto your local computer using git To compile MsSpec shared libraries, you need:
- Python >= 3.6
- gfortran compiler
- cairo library (runtime files and header files)
Depending on your OS and Python version, you may need all the requirements (runtime libraries
and header files) to build wxPython (please consult your distribution's package list for the
appropriate package names) and their dependencies:
- python-dev (for your version of python)
- gtk (preferably version 3, but depends on your needs)
- gstreamer
- gstreamer-plugins-base
- glut
- libwebkitgtk (matching your gtk version)
- libjpeg
- libpng
- libtiff
- libsdl
- libnotify
- libsm
You also need a tool to create and manage Python virtual environments. For now only
*virtualenv* is supported. So if you plan to install msspec with the one-step command,
you need to install it.
Fetching the code
-----------------
You first need to clone the devel branch with either 2 commands below:
```Bash ```Bash
git clone https://git.ipr.univ-rennes1.fr/epsi/msspec_python3.git git clone https://git.ipr.univ-rennes1.fr/epsi/msspec_python3.git
git checkout devel
``` ```
This will create a folder named "msspec_python3" with all the source code inside.
You can also clone the development branch:
```Bash ```Bash
git clone --branch devel https://git.ipr.univ-rennes1.fr/epsi/msspec_python3.git git clone --branch devel https://git.ipr.univ-rennes1.fr/epsi/msspec_python3.git
``` ```
This will create a msspec\_python3 folder with a local copy of the repository.
It is higly recommended to work in a Python virtual environment.
To create one, you can use:
One step install
----------------
The command below will do all the stuff to install msspec in its dedicated Python
virtual environment provided all the above requirements are fullfilled
```Bash ```Bash
virtualenv --python=python3 msspec_venv cd msspec_python3
make devel
``` ```
or This will:
- Create a Python virtual environment in the "\_venv" folder inside the msspec\_python3 folder.
- Install all the required Python packages in this virtual environment.
- Install (or even maybe build) wxPython for your OS and Python version in this virtual environment.
- Build the msspec shared librarires.
- Install msspec in *edit mode* in this virtual environment.
You can tweak the process by specifying space separated key=value pairs on the command line.
Allowed keys are:
- PYTHON, to give the executable name of the python interpreter to use rather than the default (python)
- VERBOSE, set to 1 if you want the install process to be more verbose
- NO\_VENV, set to 1 to not create a virtual environment
- VENV\_PATH, specify the path to a virtual environment root folder
- DEBUG, set to 1 to add debugging symbols
- BUILDDIR, to set a different name than the default to store building artifacts (./build)
- FC, the Fortran compiler to use (gfortran)
- F2PY
Work with msspec
----------------
All you need to do each time you want to work with msspec is activating the virtual environment.
For example, if *virtualenv* was used to create the virtual environment:
```Bash ```Bash
virtualenv --python=python3 --system-site-packages msspec_venv source msspec_python3/_venv/bin/activate
``` ```
to activate the virtual environment (if you are in the folder where you created "msspec\_venv"): Then you can launch your script with
```Bash ```Bash
source ./msspec_venv/bin/activate python myscript.py
``` ```
Now you can build the program, go in the source folder See the online [tutorials](https://msspec.cnrs.fr/tutorials/index.html) for more information on how to use msspec.
When you're done, you can deactivate the virtual environment with
```Bash ```Bash
cd msspec_python3/src deactivate
``` ```
Install the python package dependencies. Uninstall MsSpec
----------------
```Bash When Installed in "devel mode", nothing is created outside the msspec\_python3 folder, so you just have to remove
pip install -r setup_requirements.txt that folder to get rid of msspec on your computer
pip install -r requirements.txt
```
You need to compile the fortran libs. This is done by the scons (Software Construction) program
```Bash Manual control over the install
scons -------------------------------
```
Finally, install the package in develop mode: If for any reason you want to control each step of the building process (for example you do not have *virtualenv* but you
are using *conda*), here is how:
```Bash - Create a virtual environment or use an existing one and activate it
pip install -e . - Install the list of requirements that are listed in the src/pip.freeze file
``` - Install wxPython
- Build the msspec shared librarires with the command "make pybinding"
- Install the msspec Python package with "pip install -e src/"
That's all. All you need to do each time you want to work with msspec is activating the virtual environment The last step can be replaced by adding the src/ path to your PYTHONPATH, but bear in mind that the package will then
be also visible to other virtual environments no matter they fit the msspec requirements.

View File

@ -1,60 +0,0 @@
VERSION:=$(shell python -c "import msspec; print(msspec.__version__)")
VERBOSE:=0
ifeq ($(VERBOSE),0)
SUPPRESS_OUPUT:=1>/dev/null 2>/dev/null
MAKEFLAGS += --no-print-directory --silent
else
SUPPRESS_OUPUT:=
endif
install: sdist
@echo "Installing the newly created Python package..."
@pip install dist/msspec-*.tar.gz $(SUPPRESS_OUPUT)
sdist: install_deps
@echo "Creating Python source distribution..."
# Create the source distrib
@python setup.py sdist $(SUPPRESS_OUPUT)
install_deps: pybinding
@echo "Installing Python dependencies..."
# Install dependencies with pip
@pip install --upgrade -r requirements.txt $(SUPPRESS_OUPUT)
pybinding:
@echo "Building Python binding for phagen and spec..."
@+$(MAKE) -C msspec/spec/fortran pybinding $(SUPPRESS_OUPUT)
@+$(MAKE) -C msspec/phagen/fortran pybinding $(SUPPRESS_OUPUT)
results: msspec/results.txt
msspec/results.txt: pybinding
@echo "Generating results for unittests"
@coverage run --source=./ --omit=msspec/es/*,msspec/msspecgui/* msspec/create_tests_results.py 1>/dev/null
# create the html coverage report
@coverage html -d ../doc/source/htmlcov
@rm .coverage
tests: pybinding
@echo "Runing unittests"
@python -m msspec.tests 1>/dev/null
clean:
@echo "Cleaning all..."
@find ./ -type f -name '*.pyc' -exec rm -f {} +
@find ./ -type d -name '__pycache__' -exec rm -rf {} +
@+$(MAKE) -C msspec/spec/fortran clean $(SUPPRESS_OUPUT)
@+$(MAKE) -C msspec/phagen/fortran clean $(SUPPRESS_OUPUT)
# remove previous sdist
@rm -rf dist
@rm -rf *.egg*
purge: clean
@echo "Removing also shared objects..."
@find ./ -type f -name '*.so' -exec rm -f {} +
help:
@echo "help message"

View File

@ -1,125 +0,0 @@
import os
import subprocess
import datetime
from pkg_resources import parse_version
from setuptools_scm import get_version
###############################################################################
def filtered_glob(env, pattern, omit=[],
ondisk=True, source=False, strings=False):
return list(filter(
lambda f: os.path.basename(f.path) not in omit,
env.Glob(pattern)))
# Create a builder for f2py
def f2py_generator(source, target, env, for_signature):
suffix = ".so"
main = source[0]
objects = " ".join(o.get_path() for o in source[1:] if str(o).endswith('.o'))
moduledir = os.path.dirname(target[0].abspath)
modulefile = str(target[0])
modulename = modulefile.replace(suffix, '').replace('/', '.')
compiler = env['FORTRAN']
cmd = f"$F2PY --fcompiler={compiler} "
cmd += f" $F2PY_OPTS -m {modulename} -c {objects} {main}"
cmd += f" && cp {'/'.join(modulename.split('.'))}.*.so {modulefile}"
#cmd += " 1>/dev/null 2>/dev/null"
return cmd
def install_module(env, module):
destdir = os.path.dirname(module[0].srcnode().abspath)
env.Install(destdir, module)
env.Alias('install', destdir)
return None
def version_action(target, source, env):
with open(str(target[0]), 'w') as fd:
v = get_version(root='../', relative_to='./', version_scheme="post-release")
v = parse_version(v)
if v._version.post[-1] == 0:
version = v.base_version
else:
version = v.public
fd.write(version + '\n')
now = datetime.datetime.now().isoformat()
fd.write(f'compiled on {now}\n')
p = subprocess.run([env['FORTRAN'], "--version"], stdout=subprocess.PIPE)
fd.write(p.stdout.decode('utf8'))
f2py_bld = Builder(generator=f2py_generator)
version_bld = Builder(action=version_action)
# define the default build environment
std = Environment(tools=['default', 'fortran'], F2PY_OPTS=[], LIBS=[])
std.AddMethod(filtered_glob, "FilteredGlob")
std.AddMethod(install_module, "InstallModule")
std['BUILDERS']['F2py'] = f2py_bld
std['BUILDERS']['Version'] = version_bld
###############################################################################
# Define the command line options
AddOption('--dbg',
dest='dbg',
action='store_true',
help='add debugging symbols')
AddOption('--verbose',
dest='verbose',
action='store_true',
help='add debugging symbols')
AddOption('--compiler',
dest='compiler',
default='gfortran',
choices=['gfortran', 'ifort'],
help='The Fortran compiler to use')
AddOption('--f2py_path',
dest='f2py_path',
default='f2py3',
help='The full path to the f2py compiler')
# define environments
gfortran_env = std.Clone(tools=['gfortran'])
#gfortran_env.Replace(FORTRANFLAGS=['-fPIC', '-O2', '-ffast-math', '-mcmodel=large', '-fdefault-real-8', '-fdefault-double-8'])
gfortran_env.Replace(FORTRANFLAGS=['-fPIC', '-mcmodel=large'])
ifort_env = std.Clone(tools=['ifort'])
# parse options
if GetOption('compiler') == 'gfortran':
env = gfortran_env
#env.Append(F2PY_OPTS='--opt=-O2')
elif GetOption('compiler') == 'ifort':
env = ifort_env
f2py_path = GetOption('f2py_path')
p = subprocess.run(['which', f2py_path], stdout=subprocess.PIPE)
if p.returncode != 0:
raise Exception('Invalid path for f2py compiler!')
else:
env.Append(F2PY=p.stdout.decode('utf8').strip())
if GetOption('dbg'):
flags = ['-g', '-Wall', '-Wextra', '-Warray-temporaries',
'-Wconversion', '-fbacktrace', '-ffree-line-length-0',
'-fcheck=all', '-ffpe-trap=zero,overflow,underflow,invalid,denormal',
'-finit-real=nan']
gfortran_env.Append(FORTRANFLAGS = flags,
F2PY_OPTS = ['--noopt', '--debug-capi', '--debug', '--f77flags=-g -fcheck=all -fbacktrace -ffpe-trap=zero,overflow,underflow,invalid,denormal'])
if GetOption('verbose'):
env.Replace(FORTRANCOMSTR = "")
Export('env')
SConscript('msspec/spec/fortran/SConstruct', variant_dir='build/build_spec')
SConscript('msspec/phagen/fortran/SConstruct', variant_dir='build/build_phagen')
version_file = env.Version('VERSION', [])
Depends(version_file, "install")

View File

@ -1,10 +0,0 @@
Import('env')
import os
objects_src= ['phagen_scf_2.1_dp.f']
objects = env.Object(objects_src)
module = env.F2py('libphagen.so', ['main.f'] + objects, F2PYCOMSTR=">>")
env.InstallModule(module)
env.Alias('phagen', module)

View File

@ -1,71 +0,0 @@
Import('env')
import os
# Configure the environment
env_spec = env.Clone()
cwd = Dir('.').get_abspath()
env_spec.Append(FORTRANFLAGS=['-I' + cwd,'-J' + cwd])
env_spec.Append(F2PY_OPTS=['-I' + cwd])
# define sources
dim_mod = ['memalloc/dim_mod.f']
memalloc = ['memalloc/modules.f', 'memalloc/allocation.f']
cluster_gen = Glob('cluster_gen/*.f')
common_sub = Glob('common_sub/*.f')
renormalization = Glob('renormalization/*.f')
phd_se_noso_nosp_nosym = env_spec.FilteredGlob('phd_se_noso_nosp_nosym/*.f', omit=['main.f'])
phd_mi_noso_nosp_nosym = env_spec.FilteredGlob('phd_mi_noso_nosp_nosym/*.f', omit=['main.f'])
eig_common = Glob('eig/common/*.f')
eig_mi = env_spec.FilteredGlob('eig/mi/*.f', omit=['main.f'])
eig_pw = env_spec.FilteredGlob('eig/pw/*.f', omit=['main.f'])
comp_curves = ['treatment/comp_curves.f']
conf = Configure(env_spec, log_file='./config.log')
if conf.CheckLib('lapack'):
env_spec.Append(F2PY_OPTS = "-llapack")
eig_mi = [f for f in eig_mi if str(f).find('lapack') == -1]
phd_mi_noso_nosp_nosym = [f for f in phd_mi_noso_nosp_nosym if str(f).find('lapack') == -1]
env_spec = conf.Finish()
env_spec.Append(FORTRANFLAGS='-fno-automatic')
# define objects
dim_mod_obj = env_spec.Object(dim_mod)
memalloc_obj = env_spec.Object(memalloc)
cluster_gen_obj = env_spec.Object(cluster_gen)
common_sub_obj = env_spec.Object(common_sub)
renormalization_obj = env_spec.Object(renormalization)
phd_se_noso_nosp_nosym_obj = env_spec.Object(phd_se_noso_nosp_nosym)
phd_mi_noso_nosp_nosym_obj = env_spec.Object(phd_mi_noso_nosp_nosym)
eig_common_obj = env_spec.Object(eig_common)
eig_pw_obj = env_spec.Object(eig_pw)
eig_mi_obj = env_spec.Object(eig_mi)
comp_curves_obj = env_spec.Object(comp_curves)
Requires(memalloc_obj, dim_mod_obj)
# define Python extensions
common_deps = dim_mod_obj + memalloc_obj + cluster_gen_obj + common_sub_obj
deps = common_deps + renormalization_obj + phd_se_noso_nosp_nosym_obj
phd_se_mod = env_spec.F2py('_phd_se_noso_nosp_nosym.so', ['phd_se_noso_nosp_nosym/main.f'] + deps)
env_spec.InstallModule(phd_se_mod)
deps = common_deps + renormalization_obj + phd_mi_noso_nosp_nosym_obj
phd_mi_mod = env_spec.F2py('_phd_mi_noso_nosp_nosym.so', ['phd_mi_noso_nosp_nosym/main.f'] + deps)
env_spec.InstallModule(phd_mi_mod)
deps = common_deps + renormalization_obj + eig_common_obj + eig_mi_obj
eig_mi_mod = env_spec.F2py('_eig_mi.so', ['eig/mi/main.f'] + deps)
env_spec.InstallModule(eig_mi_mod)
deps = common_deps + renormalization_obj + eig_common_obj + eig_pw_obj
eig_pw_mod = env_spec.F2py('_eig_pw.so', ['eig/pw/main.f'] + deps)
env_spec.InstallModule(eig_pw_mod)
deps = comp_curves_obj
comp_curve_mod = env_spec.F2py('_comp_curves.so', ['treatment/main.f'] + deps)
env_spec.InstallModule(comp_curve_mod)
# Alias
env_spec.Alias('spec', [phd_se_mod, phd_mi_mod, eig_pw_mod, eig_mi_mod])