Installing Python versions with pyenv

Hi all

I’m trying to use Clear Linux on my main dev machine, and I’m having trouble installing different versions of Python with pyenv. Here are the steps to repro:

  1. Install Pyenv using the pyenv-installer script:
     curl -L https://github.com/pyenv/pyenv-installer/raw/master/bin/pyenv-installer | bash
    
  2. Try installing a new Python version (for example, 3.8.0):
    ~/.pyenv/bin/pyenv install 3.8.0
    

Expected: a new version of Python should be installed inside the ~/.pyenv folder
Actual: I get the following dump on the terminal

$ ./pyenv install 3.8.0                                                                                                                                                       
Downloading Python-3.8.0.tar.xz...                                                                                                                                            
-> https://www.python.org/ftp/python/3.8.0/Python-3.8.0.tar.xz                                                                                                                
Installing Python-3.8.0...                                                                                                                                                    
WARNING: The Python readline extension was not compiled. Missing the GNU readline lib?                                                                                        
ERROR: The Python ssl extension was not compiled. Missing the OpenSSL lib?                                                                                                    
                                                                                       
Please consult to the Wiki page to fix the problem.                                    
https://github.com/pyenv/pyenv/wiki/Common-build-problems                              
                                                                                       
                                                                                       
BUILD FAILED (Clear Linux OS 31350 using python-build 20180424)                        
                                                                                       
Inspect or clean up the working tree at /tmp/python-build.20191019091848.10384        
Results logged to /tmp/python-build.20191019091848.10384.log                           
                                                                                       
Last 10 log lines:                                                                     
         ./python -E -m ensurepip \                                                    
                $ensurepip --root=/ ; \                                                
fi                                                                                     
/tmp/tmpbwhkn7d1/pip-19.2.3-py2.py3-none-any.whl/pip/_vendor/ipaddress.py:1106: SyntaxWarning: 'str' object is not callable; perhaps you missed a comma?
/tmp/tmpbwhkn7d1/pip-19.2.3-py2.py3-none-any.whl/pip/_vendor/ipaddress.py:1106: SyntaxWarning: 'str' object is not callable; perhaps you missed a comma?
Looking in links: /tmp/tmpbwhkn7d1                                                     
Collecting setuptools                                                                  
Collecting pip                                                                         
Installing collected packages: setuptools, pip                                         
Successfully installed pip-19.2.3 setuptools-41.2.0

And these are the only errors I’m seeing in the log file mentioned in the dump above:

INFO: Can't locate Tcl/Tk libs and/or headers

Python build finished successfully!
The necessary bits to build these optional modules were not found:
_curses               _curses_panel         _dbm               
_gdbm                 _hashlib              _lzma              
_sqlite3              _ssl                  _tkinter           
readline                                                       
To find the necessary bits, look in setup.py in detect_modules() for the module's name.


The following modules found by detect_modules() in setup.py, have been
built by the Makefile instead, as configured by the Setup files:
_abc                  atexit                pwd                
time                                                           


Could not build the ssl module!
Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl with X509_VERIFY_PARAM_set1_host().
LibreSSL 2.6.4 and earlier do not provide the necessary APIs, https://github.com/libressl-portable/portable/issues/381

Clearly I’m missing some dev libraries, but after trying installing a mish mash of different bundles to fix this nothing works. So now at a loss what else to try…

Does anyone know where to find these libraries?

Most of these are in various *-dev bundles. Likely dev-basic-dev will get you most of them. sudo swupd search may help you find some of them as well.

FWIW python-3.8 will be coming very soon in Clear Linux OS. Maybe waiting is an option.

Thanks for the response @ahkok!

Most of these are in various *-dev bundles. Likely dev-basic-dev will get you most of them. sudo swupd search may help you find some of them as well.

Right, the trouble is finding the right ones! Just wondering if anyone here had already done the hard work :laughing:

…but I just found the missing packages. This seems to have done the trick:

sudo swupd bundle-add devpkg-curses sqlite devpkg-openssl tcl-basic devpkg-readline

(Plus probably some other packages I installed throughout the day, LOL)

FWIW python-3.8 will be coming very soon in Clear Linux OS. Maybe waiting is an option.

Thanks, but that wouldn’t help me. I’m developing a Python library and need to be able to test it in arbitrary Python versions, not just 3.8. I may just have to keep an Ubuntu Docker container around for this, but I’d rather avoid that as much as possible…

That’s a good point, yes, this should be possible - but it’s certainly going to be tricky. If you do manage to get it to work natively on Clear Linux, please do share :smiley:

Actually I did figure it out in the end, and updated my post above to match. But then I forgot to fully update it and now half the post looks like I solved my problem and half looks like I didn’t :rofl:

So the solution was to install these:

sudo swupd bundle-add devpkg-curses sqlite devpkg-openssl tcl-basic devpkg-readline

(Plus probably some other packages I installed throughout the day. Can’t be sure!)

And then you can just install your new Pythons normally with pyenv:

pyenv install 3.8.0

And use them with pipenv, virtualenv, etc. I use pipenv so that’s:

pipenv --python ~/.pyenv/versions/3.8.0/bin/python

…which tells pipenv where to find 3.8.0.

From now on you can use pipenv normally:

pipenv --python=3.8.0
pipenv shell

Hmm… turns out this doesn’t install tkinter properly, so matplotlib.pyplot doesn’t import.

If you know the name, or at least part of the name, of specific files of a library, try swups search-file. You can also specify whether it’s binary or library, or give longer filepath to have more accurate results.

Thanks @doct0rHu

That’s actually what I did. But I just ended up littering my system with a bunch of bundles that never solved the problem :confused:

At this point I’ve actually switched back to Ubuntu since I don’t have time to research this right now. I’ll try CL again in a few months – other than the package system it’s by far my favorite distro nowadays!

1 Like

Ok I’m back on ClearLinux :smiley:

I think I just figured this out. These are the required packages:

sudo swupd bundle-add devpkg-ncurses sqlite devpkg-openssl tcl-basic devpkg-readline devpkg-bzip2 devpkg-sqlite-autoconf devpkg-libffi devpkg-xz
4 Likes

Thanks so much. I’m needing to run Python 3.7 (for google colab) yet the CL server is on Python 3.8.1. Pyenv should be a bundle.