4. Developers Installation

4.1. Python Installation

If you already have a suitable python-3 installation with all scientific packages updated to their latest versions, you may skip this 1st stage.

Note

Installing Python under Windows:

The program requires CPython-3, and depends on numpy, scipy, pandas, sklearn and matplotlib packages, which depend on C-native backends and need a C-compiler to install from sources.

In Windows it is strongly suggested NOT to install the standard CPython distribution that comes up first(!) when you google for “python windows”, unless you are an experienced python-developer, and you know how to hunt down pre-compiled dependencies from the PyPi repository and/or from the Unofficial Windows Binaries for Python Extension Packages.

Therefore we suggest that you download one of the following two scientific-python distributions:

  1. WinPython python-3 (64 bit)
  2. Anaconda python-3 (64 bit)

4.1.1. Install WinPython

The WinPython distribution is just a collection of the standard pre-compiled binaries for Windows containing all the scientific packages, and much more. It is not update-able, and has a quasi-regular release-cycle of 3 months.

  1. Install the latest python-3.4+ 64 bit from https://winpython.github.io/. Prefer an installation-folder without any spaces leading to it.

  2. Open the WinPython’s command-prompt console, by locating the folder where you just installed it and run (double-click) the following file:

    <winpython-folder>\"WinPython Command Prompt.exe"
    
  3. In the console-window check that you have the correct version of WinPython installed, and expect a similar response:

    > python -V
    Python 3.4.3
    
    REM Check your python is indeed where you installed it.
    > where python
    ....
    
  4. Use this console and follow Install CO2MPAS package instructions, below.

4.1.2. Install Anaconda

The Anaconda distribution is a non-standard Python environment that for Windows containing all the scientific packages we need, and much more. It is not update-able, and has a semi-regular release-cycle of 3 months.

  1. Install Anaconda python-3.4+ 64 bit from http://continuum.io/downloads. Prefer an installation-folder without any spaces leading to it.

    Note

    When asked by the installation wizard, ensure that Anaconda gets to be registered as the default python-environment for the user’s account.

  2. Open a Windows command-prompt console:

    "windows start button" --> `cmd.exe`
    
  3. In the console-window check that you have the correct version of Anaconda-python installed, by typing:

    > python -V
    Python 3.4.3 :: Anaconda 2.3.0 (64-bit)
    
    REM Check your python is indeed where you installed it.
    > where python
    ....
    
  4. Use this console and follow Install CO2MPAS package instructions, below.

4.2. Install CO2MPAS package

  1. Install CO2MPAS executable internally into your python-environment with the following console-commands (there is no prob if the 1st uninstall command fails):

    > pip uninstall co2mpas
    > pip install co2mpas
    Collecting co2mpas
    Downloading http://pypi.co2mpas.io/packages/co2mpas-...
    ...
    Installing collected packages: co2mpas
    Successfully installed co2mpas-1.5.5
    

    Warning

    Installation failures:

    The previous step require http-connectivity for pip command to Python’s “standard” repository (https://pypi.python.org/). In case you are behind a corporate proxy, you may try one of the methods described in section Alternative installation methods, below.

    If all methods to install CO2MPAS fail, re-run pip command adding extra verbose flags -vv, copy-paste the console-output, and report it to JRC.

  2. Check that when you run CO2MPAS, the version executed is indeed the one installed above (check both version-identifiers and paths):

    > co2mpas -vV
    co2mpas_version: 1.5.5
    co2mpas_rel_date: 2017-02-10 18:43:44
    co2mpas_path: d:\co2mpas_ALLINONE-64bit-v1.4.1\Apps\WinPython\python-3.4.3\lib\site-packages\co2mpas
    python_path: D:\co2mpas_ALLINONE-64bit-v1.4.1\WinPython\python-3.4.3
    python_version: 3.4.3 (v3.4.3:9b73f1c3e601, Feb 24 2015, 22:44:40) [MSC v.1600 XXX]
    PATH: D:\co2mpas_ALLINONE-64bit-v1.4.1\WinPython...
    

    Note

    The above procedure installs the latest CO2MPAS, which might be more up-to-date than the version described here!

    In that case you can either:

    1. Visit the documents for the newer version actually installed.
    2. “Pin” the exact version you wish to install with a pip command (see section below).

4.2.1. Install extras

Internally CO2MPAS uses an algorithmic scheduler to execute model functions. In order to visualize the design-time models and run-time workflows you need to install the Graphviz visualization library from: http://www.graphviz.org/.

If you skip this step, the modelgraph sub-command and the --plot-workflow option would both fail to run (see Debugging and investigating results).

4.2.2. Upgrade CO2MPAS (with internet connectivity)

  1. Uninstall (see below) and re-install it.

4.2.3. Uninstall CO2MPAS

To uninstall CO2MPAS type the following command, and confirm it with y:

> pip uninstall co2mpas
Uninstalling co2mpas-<installed-version>
...
Proceed (y/n)?

Re-run the command again, to make sure that no dangling installations are left over; disregard any errors this time.

4.2.4. Alternative installation methods

You may get multiple versions of CO2MPAS, from various places, but all require the use of pip command from a console to install:

Tip

In all cases below, remember to uninstall CO2MPAS if it’s already installed.

Remember also to store the installation logs with the -v --log options, particularly if you install a specific version from GitHub

  • Latest STABLE: use the default pip described command above.

  • Latest PRE-RELEASE: append the --pre option in the pip command.

  • Specific version: modify the pip command like that, with optionally appending --pre:

    pip install co2mpas==1.0.1 --process-dependency-links ... # Other options, like above.
    
  • Specific branch from the GitHub-sources:

    pip install -v log pip.log git+https://github.com/JRCSTU/co2mpas.git@dev --process-dependency-links
    
  • Specific commit from the GitHub-sources:

    pip install -v log pip.log git+https://github.com/JRCSTU/co2mpas.git@2927346f4c513a --process-dependency-links
    
  • Speed-up download: append the --use-mirrors option in the pip command.

  • (for all of the above) When you are behind an http-proxy: append an appropriately adapted option --proxy http://user:password@yourProxyUrl:yourProxyPort.

    Important

    To avert any security deliberations for this http-proxy “tunnel”, JRC cryptographically signs all final releases with one of those keys: - GPG key ID: 9CF277C40A8A1B08 form @ankostis - GPG key ID: 1831F9C2294A33CC for @vinci1it2000

    Your IT staff may validate their authenticity and detect man-in-the-middle attacks, however impossible.

  • (for all of the above) Without internet connectivity or when the above proxy cmd fails:

    1. Use an existing Python-3.5 environment; that might be an older ALLINONE, WinPython, Anaconda or Linux’s standard python environment.

    2. With with a “regular” browser and when connected to the Internet, pre-download locally and unzip the respective co2mpas_DEPENDENCIES-vX.X.XXX.7z file from the latest ALLINONE release (e.g. http://github.com/JRCSTU/CO2MPAS-TA/releases/). This archive contains all the dependent packages of CO2MPAS.

    3. Install CO2MPAS, referencing the above folder. Assuming that you unzipped the packages in the folder path/to/co2mpas_packages, use a console-command like this:

      pip install co2mpas  --no-index  -f path/to/co2mpas_packages --process-dependency-links
      

4.2.5. Install Multiple versions in parallel

In order to run and compare results from different CO2MPAS versions, you may use virtualenv command.

The virtualenv command creates isolated python-environments (“children-venvs”) where in each one you can install a different versions of CO2MPAS.

Note

The virtualenv command does NOT run under the “conda” python-environment. Use the conda command in similar manner to create child conda-environments instead.

  1. Ensure virtualenv command installed in your “parent” python-environment, i.e the “WinPython” you use:

    > pip install virtualenv
    

    Note

    The pip command above has to run only once for each parent python-env. If virtualenv is already installed, pip will exit gracefully.

  2. Ensure CO2MPAS uninstalled in your parent-env:

    > pip uninstall co2mpas
    

    Warning

    It is important for the “parent” python-env NOT to have CO2MPAS installed! The reason is that you must set “children venvs” to inherit all packages installed on their “parent” (i.e. numpy and pandas), and you cannot update any inherited package from within a child-env.

  3. Move to the folder where you want your “venvs” to reside and create the “venv” with this command:

    > virtualenv --system-site-packages co2mpas_v1.0.1.venv
    

    The --system-site-packages option instructs the child-venv to inherit all “parent” packages (numpy, pandas).

    Select a venv’s name to signify the version it will contains, e.g. co2mpas_v1.0.1.venv. The .venv at the end is not required, it is just for tagging the venv folders.

  4. “Activate” the new “venv” by running the following command (notice the dot(.) at the begining of the command):

    > .\co2mpas_v1.0.1.venv\Scripts\activate.bat
    

    Or type this in bash:

    $ source co2mpas_v1.0.1.venv\Scripts\activate.bat
    

    You must now see that your prompt has been prefixed with the venv’s name.

  1. Install the CO2MPAS version you want inside the activated venv. See the Install CO2MPAS package section, above.

    Tip

    Always store the installation logs, particularly if you install a specific version from GitHub:

    pip install -v log pip.log ...  ## co2mpas specifiers follows here...
    

    Don’t forget afterwards to check that what you get when running CO2MPAS is what you installed.

  2. To “deactivate” the active venv, type:

    > deactivate
    

    The prompt-prefix with the venv-name should now dissappear. And if you try to invoke CO2MPAS, it should fail.

Tip

  • Repeat steps 2–>5 to create venvs for different versions of co2mpas.
  • Use steps (6: Activate) and (9: Deactivate) to switch between different venvs.

4.3. Autocompletion

In order to press [Tab] and get completions, do the following in your environment (ALLINONE is pre-configured with them):

  • For the Clink environment, on cmd.exe, add the following lua script inside clink’s profile folder: clink/profile/co2mpas_autocompletion.lua

    --[[ clink-autocompletion for CO2MPAS
    --]]
    local handle = io.popen('co2mpas-autocompletions')
    words_str = handle:read("*a")
    handle:close()
    
    function words_generator(prefix, first, last)
        local cmd = 'co2mpas'
        local prefix_len = #prefix
    
        --print('P:'..prefix..', F:'..first..', L:'..last..', l:'..rl_state.line_buffer)
        if prefix_len == 0 or rl_state.line_buffer:sub(1, cmd:len()) ~= cmd then
            return false
        end
    
        for w in string.gmatch(words_str, "%S+") do
            -- Add matching app-words.
            --
            if w:sub(1, prefix_len) == prefix then
                clink.add_match(w)
            end
    
            -- Add matching files & dirs.
            --
            full_path = true
            nf = clink.match_files(prefix..'*', full_path)
            if nf > 0 then
                clink.matches_are_files()
            end
        end
        return clink.match_count() > 0
    end
    
    sort_id = 100
    clink.register_match_generator(words_generator)
    
  • For the bash shell just add this command in your ~/.bashrc (or type it every time you open a new console):

    complete -fdev -W "`co2mpas-autocompletions`" co2mpas