Products Resources Support About Us

Rocket Software

Python installation on z/OS filesystem

I ran into problems downloading the .Tar file to the file system i created on z/OS. I created a file
system with cyl(5000 150) When i TAR python-2017-04-12-py27.tar it run a long time and eventually
runs out of space. I cant believe this file is actually this big. How big should my filesystem be?
I followed your directions on installation of Rocket z/OS open source tools product. I appreciate your assistance.

1 Like

This release of Python is, in fact, quite large. The size of download itself (a compressed tar file) is 485,749,868 bytes (call it half a gigabyte…) and the uncompressed, restored file tree’s size is 2,458,780K (call it 2.5GB). Here’s the output of my uncompress/restore:

bash-2.03$ time gzip -d -c python-2017-04-12-py27.tar.gz  | tar xoUXf -
real  2m30.392s
user  0m24.329s
sys 0m8.110s
bash-2.03$ du -sk python-2017-04-12-py27
   2458780 python-2017-04-12-py27
bash-2.03$

The installation instructions are (somewhat unfortunately) contained in a README file in the restored file tree. I’m including them here for completeness.


This is a release of Python 2.7.13 and Python 3.6.1 for z/OS. There are separate downloads for python 2 and 3, but they were both built in the same process, with the same sources.

This release of Python contain the packages required for NumPy, SciPy, Pandas, and Jupyter, as well as some add-ons for Jupyter, and several other packages as well.

The conda packages are not currently usable.

JupyterHub is not available for Python 2.7.

StatsModels has not yet been built for Python 3.6.1.

All other Python packages is this release are built for both 2.7.13 and 3.6.1.

  • This release was built on a z/OS 2.2 system.
  • Everything is built 64 bit and in ASCII mode.
  • This distribution was created on “2017-04-12-12-30”.

Note! If you did not use xoUXf options in your tar command then installation will fail and you must start over.

Note! Environment variable _BPXK_AUTOCVT must be set to ON

Note! If you use tcsh, the export commands below must be edited by replacing every “export” with “setenv”, and replacing every “=” with " ", in your file ~/.tcshrc.

DOWNLOAD AND UNPACK INSTRUCTIONS

Choose whether you want to install only Python 2.7.13, only Python 3.6.1, or both. Extra space savings arise if you download the combined release, and your users will be able to choose whether to use an environment that contains python2, python3, or both. The extra savings results because the install procedure uses hard links between files when possible.

After being unpacked, this release will occupy about 2.4 GB, if you chose either Python 2 or Python 3, or 3.6 GB if you chose both.

Create a directory for the install, and cd into it.

Unpack using “gzip -d -c downloadfile.tar.gz | tar xoUXf -”.

PREREQUISITES

Before proceeding with installation instructions below you should install and set up the following:

  • bash 4.3 - Used by the utilities for the gzip and xz packages.
  • perl 5.24 - Used by two openssl scripts.

INSTALLATION INSTRUCTIONS

The following lines must be edited and executed. You can either execute them in order from the command line or paste them into a script and execute the script.

export BASH_PREFIX=<the_directory_containing_bin_bash>
export PERL_PREFIX=<the_directory_containing_bin_perl>
INSTALL=<your_install_directory>
export RELEASE_NAME=python-2017-04-12
export RELEASE_TYPE=py27 # or py36 or both, to match the download you chose
export RELEASE_DIR=$INSTALL/${RELEASE_NAME}-${RELEASE_TYPE}
export PKGS_BASE=$RELEASE_DIR/pkgs
cd $RELEASE_DIR; ls -ld python27 python36 both
# based on the results of the ls, run one or more of these commands
cd $RELEASE_DIR/python27; bin/install_all_packages
cd $RELEASE_DIR/python36; bin/install_all_packages
cd $RELEASE_DIR/both; bin/install_all_packages

USAGE

Every user of this distribution should put the following lines at the end of your shell init file:

If you use sh, put these lines at the end of your ~/.profile:
If you use bash, put these lines at the end of your ~/.bashrc:

export _BPXK_AUTOCVT=ON
INSTALL=<your_install_directory>
export RELEASE_NAME=python-2017-04-12
export RELEASE_TYPE=py27 # or py36 or both, to match the download
export RELEASE_DIR=$INSTALL/${RELEASE_NAME}-${RELEASE_TYPE}
export PYTHON_ENV=python27 # pr python36 or both, to match your desired environment

export PYTHON_HOME=$RELEASE_DIR/$PYTHON_ENV
export PATH=$PYTHON_HOME/bin:$PATH
export LIBPATH=$PYTHON_HOME/lib:$LIBPATH
export FFI_LIB=$PYTHON_HOME/lib/ffi
export TERMINFO=$PYTHON_HOME/share/terminfo
export PKG_CONFIG_PATH=$PYTHON_HOME/lib/pkgconfig:$PYTHON_HOME/share/pkgconfig
export CURL_CA_BUNDLE=$PYTHON_HOME/etc/ssl/cacert.pem

If you use sh, re-run your init file with:

. ~/.profile

If you use bash, re-run your init file with:

. ~/.bashrc

Note: There is a problem with this TERMINFO if you use /bin/vi. You will need to choose one of these approaches:

  • Comment out the line above that sets TERMINFO, and start python with: env TERMINFO=$PYTHON_HOME/share/terminfo python
  • Keep the line above that sets TERMINFO, start vi with: env TERMINFO= vi

Note: There is code in Python (and elsewhere) that uses /dev/urandom. This code will not work unless the ICSF service is started. If it has not been started, you will get results similar to this:

head -c40 /dev/urandom
head: /dev/urandom: EDC5157I An internal error has occurred.
1 Like

Hi, Am having trouble with the install. Hoping someone can help.

I get to the part where you enter bin/install_all_packages
At this point it returns lines like:
installing 0 files from /…py36/pkgs/bzip2-1.0.6-1 to /…py36/python36
pax: ÂÑ>ÂÍ>:Ñø–ÂÑ>Â:Ä/È-ÂÑ>Â:Ä_ø-ÂÑ>Â:ÀÑÃÃ-ÂÑ>Â:ÁÅÊÁø-ÂÑ>Â:ÃÅÊÁø-ÂÑ>Â:ÅÊÁø-ÂÑ>Â

I saw the notes that say “Note! If you did not use xoUXf options in your tar command then installation will fail and you must start over.”, I had actually not used those options, so I started again and used them. Same result.

Saw the “Note! Environment variable _BPXK_AUTOCVT must be set to ON”, so I set that, and tried again. Same result.

I did some digging and see that there is also the “install_package” which does a single package. Do you have the syntax for issuing this on its own, so that I can test my process on a single package rather than the complete list?

Thanks

I got past my problem above. The trick is the _BPXK_AUTOCVT parameter.

In your notes it says “Note! Environment variable _BPXK_AUTOCVT must be set to ON”. Well you are dealing with people who may not know the difference between “EXPORT _BPXK_AUTOCVT=ON” and “_BPXK_AUTOCVT=ON”. Both set the variable, but the scope is different. For example you say “INSTALL=<your_install_directory>” but your other lines say “EXPORT RELEASE…”. As a guide to noobs like me, you should make your installation documentation as idiot proof as possible.

On the other hand, I have learnt a lot about USS and shell scripts. I also found a program to convert between ASCII and EBCDIC which is “dd conv=ebcdic if={input file} of={output file}” IBM supplies it with the standard distribution.

Good luck to other installers of Python 3.6.1. Can’t wait to start using it.

2 Likes

Hi,

Thank you for feedback. We will try to make our guides more understandable.
Also you can use iconv to convert between different code set.

Hi

I have made some progress with my installation.

First I got error message: Fatal Python error: failed to get random numbers to initialize Python

I did a lot of work to start ICSF (we do not have the hardware installed, but it seems that just running the STC is sufficient).

When I start again I got the following:
ModuleNotFoundError: No module named 'encodings’
initstdio: failed to import module encodings.utf_8
Fatal Python error: Py_Initialize: can’t initialize sys standard streams

Current thread 0x174ce00000000000 (most recent call first):
äáá----á-èÇÁ-ËÑÅ>/%-ëñå âêè-Ï/Ë-ÊÁÄÁÑÎÁÀ–[1] + Done(131) python 33555310 Abort
/usr/lpp/python/python-2017-04-12-py36/python36/bin/python

I did some work on the rubbish characters and assuming they were ASCII the EBCDIC equivalent is “CEE____E_The_signal_SIGABRT_was_received”

Any ideas anyone?

1 Like

Hi,

Could you give more details about this issue? When does this error occur:

Fatal Python error: failed to get random numbers to initialize Python

What is next?
How do you get this error? Do you run “python” (interactive mode) or “python your_file”:

ModuleNotFoundError: No module named 'encodings’
initstdio: failed to import module encodings.utf_8
Fatal Python error: Py_Initialize: can’t initialize sys standard streams

Hi Tatyana,

Glad to share with you some more information…

I get into the OMVS shell and this conversation follows
# echo $PATH
/usr/lpp/python/python-2017-04-12-py36/python36/bin:/usr/lpp/java/current/bin:/bin
# head -c8 /dev/urandom
head: /dev/urandom: EDC5157I An internal error has occurred.
# python
Fatal Python error: failed to get random numbers to initialize Python
äáá----á-èÇÁ-ËÑÅ>/%-ëñå âêè-Ï/Ë-ÊÁÄÁÑÎÁÀ–[1] + Done(131) python
33554739Abort /usr/lpp/python/python-2017-04-12-py36/python36/bin/python
So at this point I issue “S CSF” on the master console and get message "CSFM126I CRYPTOGRAPHY - FULL CPU-BASED SERVICES ARE AVAILABLE. "

Back into OMVS and
# head -c8 /dev/urandom
Ê -¸-pó,
# python
ModuleNotFoundError: No module named 'encodings’
initstdio: failed to import module encodings.utf_8
Fatal Python error: Py_Initialize: can’t initialize sys standard streams

Current thread 0x174cca0000000000 (most recent call first):
äáá----á-èÇÁ-ËÑÅ>/%-ëñå âêè-Ï/Ë-ÊÁÄÁÑÎÁÀ–[1] + Done(131) python
50331957Abort /usr/lpp/python/python-2017-04-12-py36/python36/bin/python

I have the following settings in my environment …
# env
FFI_LIB=/usr/lpp/python/python-2017-04-12-py36/python36/lib/ffi
PYTHON_ENV=python36
MAIL=/usr/mail/DXC00
_BPX_TERMPATH=OMVS
PATH=/usr/lpp/python/python-2017-04-12-py36/python36/bin:/usr/lpp/java/current/bin:/bin
_BPXK_AUTOCVT=ON
SHELL=/bin/sh
CURL_CA_BUNDLE=/usr/lpp/python/python-2017-04-12-py36/python36/etc/ssl/cacert.pem
PYTHON_HOME=/usr/lpp/python/python-2017-04-12-py36/python36
RELEASE_NAME=python-2017-04-12
COLUMNS=80
PYTHONHOME=/usr/lpp/python/bin
_=/bin/env
CLASSPATH=.:/usr/lpp/smp/classes:/usr/lpp/java/current:
LOGNAME=DXC00
STEPLIB=none
LANG=C
PKG_CONFIG_PATH=/usr/lpp/python/python-2017-04-12-py36/python36/lib/pkgconfig:/usr/lpp/python/python-2017-04-12-py36/python36/share/pkgconfig
RELEASE_DIR=/usr/lpp/python/python-2017-04-12-py36
LIBPATH=/usr/lpp/python/python-2017-04-12-py36/python36/lib:/lib:/usr/lib:.
RELEASE_TYPE=py36
_OPTIONS=
TERM=dumb
HOME=/
PYTHONPATH=/usr/lpp/python/lib/python2.7
LINES=20
JAVA_HOME=/usr/lpp/java/current
TZ=UTC-10 UTC-11,M10.1.0,M4.1.0
MANPATH=/usr/man/%L:/usr/lpp/python/man/
NLSPATH=/usr/lib/nls/msg/%L/%N
TERMINFO=/usr/lpp/python/python-2017-04-12-py36/python36/share/terminfo

When I did this print for you I saw that my PYTHONPATH, MANPATH and PYTHONHOME were still pointing to python 2.7, I hadn’t touched them because the new release came with PYTHON_HOME which was a different variable.

I tried removing the PYTHONPATH, MANPATH and PYTHONHOME and now it looks a lot healthier…
# python
Python 3.6.1 (default, Apr 7 2017, 13:59:34) [C] on zos
Type “help”, “copyright”, “credits” or “license” for more information.
>>>

So it looks like I have pretty much solved my own problem.
If I could offer a suggestion or two:

  1. a note on running ICSF to get around the /dev/urandom would be nice
  2. a mention of the difference between PYTHONHOME and PYTHON_HOME environment variables would be nice
  3. a sample “hello World” type program with instructions on how to run it would be nice.
  4. The error messages print in ASCII not EBCDIC (the CEEnnnn which appear as äáá----á-èÇÁ-ËÑÅ>/%-ëñå âêè-Ï/Ë-ÊÁÄÁÑÎÁÀ–) Looks like something isn’t setting the ASCII/EBCDIC flag properly.

Cheers

1 Like

Thank you for your detailed description of your problems, and for the suggested improvements. Python for z/OS is currently under very active development, and your feedback is very helpful.

1 Like

Hey there. I’m very new to installing this kind of product so please be gentle :slight_smile:
I was having problems with trying to do the gzip and tar in one command using “gzip -d -c downloadfile.tar.gz | tar xoUXf -”. I was able to gzip the file but now I’m not sure of what the tar command should look like. Currently, I have the python-2017-04-12-py27.tar file in /u/ROCKET/python/py27. When all is said and done, that is where I want the product to live.
Any guidance would be greatly appreciated!
Thanks,
Debbi

Ok I have managed to get the combined gzip and tar to work…yay! Now a couple of question on the list of commands for the script…
export BASH_PREFIX=<the_directory_containing_bin_bash> Got it…
export PERL_PREFIX=<the_directory_containing_bin_perl> Got it…
INSTALL=<your_install_directory>
Here is where I have a question. I have /u/ROCKET/python created and I’m wanting to install both 27 and 36. So, should the /u/ROCKET/python be my install dir? Will I end up with a python27 and python36 dirs under that?
export RELEASE_NAME=python-2017-04-12
export RELEASE_TYPE=py27 # or py36 or both, to match the download you chose
Then here…do I make the value here actually “both” in order to get both releases or something like py27, run the script, then py36 and run the script a 2nd time?
export RELEASE_DIR=INSTALL/{RELEASE_NAME}-${RELEASE_TYPE}
export PKGS_BASE=$RELEASE_DIR/pkgs
cd $RELEASE_DIR; ls -ld python27 python36 both

based on the results of the ls, run one or more of these commands

cd $RELEASE_DIR/python27; bin/install_all_packages
cd $RELEASE_DIR/python36; bin/install_all_packages
cd $RELEASE_DIR/both; bin/install_all_packages

Again, any guidance is appreciated!
Deb

Hi Deb,
Please, read this topic: