Products Resources Support About Us

Rocket Software

Using git for z/OS with GitHub

In this post I will walk through the steps necessary to clone a repository from GitHub using Rocket’s port of git for z/OS.

Much of the content in this post is contained in the release notes for git for z/OS.

In a nutshell

These instructions will suffice for readers already familiar with git, ssh, GitHub, and Unix System Services on z/OS.

  1. Install git for z/OS, including its dependencies (bash and perl).
  2. Set up your environment on z/OS for git, as per the git release notes.
  3. Create an account on GitHub (if you don’t already have one).
  4. Generate an ssh key pair on z/OS.
  5. Add the public key to your account on on GitHub.
  6. Clone the repository using the ssh access method for git. A suggested starter repo is https://github.com/zorts/hello_world ;this repo contains an example of a .gitattributes file that arranges for text files to be encoded and tagged as EBCDIC.

A more detailed account

What, that wasn’t enough? :slight_smile: OK, here’s a more detailed description.

Install git for z/OS, including its dependencies (bash and perl).

Download the latest version of git for z/OS from the Rocket Open Source download page. You will also need the latest versions of bash and perl. If you don’t already have gzip on z/OS, you will need that, too.

Typically people download the software to a Windows PC and then transfer it to z/OS for installation. Be sure that the transfer is a binary transfer; FTP is a good choice. Note that some transfer methods (notably scp) will unconditionally convert an incoming (to z/OS) file from ASCII to EBCDIC, rendering the file unusable.

Once the files have been downloaded, install them into “an appropriate install location.” If you are just testing the software and don’t have the necessary permissions to install it in a location such as /usr/local, you can just install them into a directory in your home directory.

For more information on unpacking files for installation, see this post on the Rocket Community Forum.

Update: See this forum posting for more detailed installation instructions.

Set up your environment on z/OS for git, as per the release notes.

The git release notes contain detailed information on the necessary environment variables to set for using git.

That said - this script can be put into a file and sourced as needed, or invoked from your .bash_profile at login. Change the value for GIT_ROOT to point to the directory in which you installed git and its prerequisites.

# setup script for bash
#
# NOTE: this can either be added to your .bash_profile OR
#       sourced when needed:
#
#         . <this file name>
#
GIT_ROOT=/path/to/git/installation

export GIT_SHELL=$GIT_ROOT/bin/bash
export GIT_EXEC_PATH=$GIT_ROOT/libexec/git-core
export GIT_TEMPLATE_DIR=$GIT_ROOT/share/git-core/templates

export PATH=$GIT_ROOT/bin:$PATH
export MANPATH=$MANPATH:$GIT_ROOT/man
export PERL5LIB=$PERL5LIB:$GIT_ROOT/lib/perl5

# These enable enhanced ASCII support
export _BPXK_AUTOCVT=ON
export _CEE_RUNOPTS="FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)"
export _TAG_REDIR_ERR=txt
export _TAG_REDIR_IN=txt
export _TAG_REDIR_OUT=txt

# optional (do once): set git editor to create comments on encoding ISO8859-1
# git config --global core.editor "/bin/vi -W filecodeset=ISO8859-1"

Create an account on GitHub (if you don’t already have one).

Go to GitHub and follow the instructions there for creating an account. This is necessary because the only git access protocol currently supported by git for z/OS is the ssh protocol; the “git” and “http” protocols are not supported.

Generate an ssh key pair on z/OS.

See the instructions on GitHub for generating an ssh key pair more detail. There are a few things not mentioned there that are important:

You need to first create a .ssh directory in your home directory, and make sure that the path to that directory is not writable by anyone but you. Here are a typical set of commands to do this. Do NOT type the comments at the end of each command line.

$ cd ~           # cd to your home directory
$ chmod 755 .    # allow others to read and traverse your home directory,
                 # but not write into it. If you don't want ANYONE to be
                 # able to read your home directory, change to 700.
$ mkdir .ssh     # create the .ssh directory
$ chmod 755      # allow others to read this directory but not write into it.

Now you can create an ssh key pair. A description of public key encryption is beyond the scope of this article, but the important concept is that a key pair consists of a public key and a private key. You share the public key with other servers you wish to access, and keep the private key carefully protected on your own system. ssh will use them both to establish a secure communication channel between the local system and the remote (and untrusted) server.

You can generate an appropriate key pair with this command:

$ ssh-keygen -t rsa -b 4096 -f ~/.ssh/id_rsa -C "yourid@yourdomain.com" -P ""
Generating public/private rsa key pair.
Your identification has been saved in /u/yourid/.ssh/id_rsa.
Your public key has been saved in /u/yourid/.ssh/id_rsa.pub.
The key fingerprint is:
02:63:a9:bd:23:d0:61:0b:85:36:0b:af:86:71:54:75 yourid@yourdomain.com
$ 

This will create a public key pair and store both pieces in your .ssh directory.

Add the public key to your account on on GitHub.

Follow the instructions on GitHub for adding a new SSH keypair to your account. You will be copying the contents of the newly created public key to GitHub.

Clone the repository using the ssh access method for git.

Go to the repository on GitHub that you wish to clone and click on the green “Clone or download” button towards the right side of the page. A suggested starter repository is https://github.com/zorts/hello_world; this repo contains an example of a .gitattributes file that arranges for text files to be encoded and tagged as EBCDIC. Make sure you chose the SSH URL. Copy the URL and then insert it into this command on z/OS:

$ git clone <URL from GitHub>

This will clone the repository on your local z/OS machine.

What’s next?

Once you have successfully cloned a repository you can begin working with its files using git.

An important issue when using source from git is handling the ASCII/EBCDIC encoding issue. There are detailed instructions on this in the git release notes, but to summarize, you will need to create a .gitattributes file that identifies how to encode the files in the working directory. Some files must be encoded using ASCII, most notably the .gitattributes and .gitignore files. A good starting point for a .gitattributes file is:

# encode all files as EBCDIC unless mentioned elsewhere
*       git-encoding=iso8859-1 working-tree-encoding=ibm-1047

# encode selected files as ASCII
.gitattributes   git-encoding=iso8859-1 working-tree-encoding=iso8859-1
.gitignore       git-encoding=iso8859-1 working-tree-encoding=iso8859-1

You will need to become familiar with the features of IBM’s enhanced ASCII support, including file tagging and automatic conversion.

Why is perl a prereq for git?

Because git ships with many perl scripts (many are in contrib, but a few are part of the core of git). You can probably use git without perl, but some things will not work.

Hello,
I have follow the steps & success install the git, thanks so much!
I have a doubt Is it possible to manage z/OS datasets ( PDS, PS) via git? we have test git clone from github, but can’t find way to access datasets, nor move them to remote repository. I also read your the other topic, still didn’t find anything relate to z/OS datasets.
Would you please advise?

Hello,
The “cp” command can do everything you need, but you will need to read the manual carefully, because there are a lot of different options. Also, if any of your datasets have recfm=v, you may want to consider setting the environment variable _EDC_ZERO_RECLEN. Again, you will need to read the manual, and consider whether you want to use Y or N.

For a few very simple examples, see the to_pds and from_pds scripts in this github repo: https://github.com/zorts/hello_world

thanks, I will start from “cp” with the manual.

yes, it’s very helpful. thanks again.

I tried utility OPUTX/OGETX to transfer between datasets and zFS files, most of time i’m a zos guy and mainly works on it. it works quite good, but just some times the OPUTX will ignore some file without any notice, second run can copy the rest of the datasets. Now we can use it to pull/push large repo to github.
This is really a great work, Thanks for your effort!

i installed github on my z/§OS and did the initial setup. i also tried the first steps with the sample hello_world repo successfully.
However i’m having an issue when i issue git pull or git push commands, as described below:
When i issue git push or git pull commands, i get

git pull origin mybranch /usr/local/libexec/git-core/git-sh-setup: line 342: cd: ÅÑÈ: EDC5129I No such file or directory.
line 342 in the script shows:
BROWSE /Z22R01/usr/local/libexec/git-core/git-s Line 0000000342 Col
.test -n "$GIT_DIR" && GIT_DIR=$(cd "$GIT_DIR" && pwd) || {
..echo >&2 "Unable to determine absolute path of git directory"

is there any issue with $GIT_DIR ?

Can you please run the following commands in the working directory of your git repository (in the case of the hello_world repo, the directory containing the source files) and provide the output?

  • pwd
  • uname -aI
  • env | sort
  • git rev-parse --git-dir

GIT_DIR is not a variable you should set yourself; it is determined by the code in git-sh-setup and will be the path (either relative or absolute) of the .git directory in your working directory.

Rocket issue USSP-850 created.

pwd
SPARKID:/u/sparkid/hello_world/hello_world/hello_world: >pwd
/u/sparkid/hello_world/hello_world/hello_world

===========================

git rev-parse --git-dir
.git
===========================
uname -aI
z/OS X9 02.00 02 2827

================================
SPARKID:/u/sparkid/hello_world/hello_world/hello_world: >env | sort
A2E=-f ISO8859-1 -t IBM-1047
ATOE=-f ISO8859-1 -t IBM-1047
CLASSPATH=:/usr/lpp/hpj/lib:/usr/lpp/java/J7.1_64/lib/RAWTApplHost.zip:/usr/lpp/java/J7.1_64/lib/classes.zip:/usr/lpp/java/J7.1_64/lib/ext/recordio.jar:/usr/local/NetRexx/lib/NetRexxC.zip:/usr/lpp/fcgi-devel-kit/fcgi/java/classes:/usr/lpp/ctgv5/ctg/classes/ctgserver.jar:/usr/lpp/ctgv5/ctg/classes/ctgclient.jar:/usr/lpp/ctgv5/ctg/classes:/usr/lpp/ctgv5/ctg/classes/connector.jar:/usr/lpp/ctgv5/ctg/classes/cicsj2ee.jar:/usr/lpp/ctgv5/ctg/classes/ccf2.jar:/usr/lpp/ctgv5/ctg/samples/java:/usr/lpp/mqm/java/lib:/usr/lpp/mqm/java/lib/com.ibm.mq.jar:/usr/lpp/mqm/samp/mqjava:/usr/lpp/ldap/sbin/servertools.jar:/usr/lpp/db2v11_jdbc/classes/db2jcc.jar:/usr/lpp/db2v11_jdbc/classes/db2jcc_license_cisuz.jar:/usr/lpp/zWebSphere/Liberty/V8R5/clients/restConnector.jar:/usr/lpp/zWebSphere/V8R5M0/optionalLibraries/jython/jython.jar:.:/was85config/Jython_2.5.3/jython.jar
DEFAULT_CLIENT_XML_PATH=/usr/lpp/WebSphere/samples/smapi
DFHJVSYSTEM_00=A6PCIX9 CICS
DISPLAY=:0
DSNAOINI=SYS1.DSNDBO0.DSNAOINI
E2A=-f IBM-1047 -t ISO8859-1
GIT_EXEC_PATH=/usr/local/libexec/git-core
GIT_SHELL=/usr/local/bin/bash
GIT_TEMPLATE_DIR=/usr/local/share/git-core/templates
HOME=/u/sparkid
IBMHPJ_HOME=/usr/lpp/hpj
IBMHPJ_RTL=CEE.SCEELKED:CEE.SCEELKEX:CEE.SCEEOBJ:CEE.SCEECPP
IBM_JAVA_OPTIONS=-Dfile.encoding=ISO8859-1
ICU_DATA=/usr/lpp/ixm/IBM/xml4c-5_6/lib
IMOCONFIGCL=/etc/TextTools/TextSearch
IMOCONFIGSRV=/etc/TextTools/TextSearch
IMOINSTANCE=inqsrch
IMOSEARCHSERVICE=SERVER
IMYBASE=/etc/TextTools/Spidey
IMYINSTALLDIR=/usr/lpp/TextTools
IMZCFG=/usr/lpp/TextTools/TATools
IMZROOT=/usr/lpp/TextTools/TATools
IWV_CONFIG_DIR=/etc/TextTools/NetQWeb
IZU_CODE_ROOT=/usr/lpp/zosmf/V2R1
IZU_CONFIG_DIR=/etc/zosmf
IZU_ENV_FILE=/usr/lpp/zosmf/V2R1/defaults/izu_env.sh
IZU_LOGFILE_DIR=/var/zosmf/configuration/logs
JAVA_HOME=/usr/lpp/java/J8.0_64
JYTHONPATH=/usr/lpp/zWebSphere/Liberty/V8R5/clients/jython/:/was85config/liberty/scripts
LANG=C
LD_LIBRARY_PATH=/usr/lpp/db2810/lib:.
LIBPATH=/usr/lpp/ixm/IBM/xslt4c-1_7/lib:/usr/lpp/ixm/IBM/xml4c-5_6/lib::/usr/lpp/internet/bin:/usr/lpp/internet/sbin:/usr/lpp/db2810/lib:/usr/lpp/NetQ/nqproc:/usr/lpp/java/J7.1_64/lib/mvs/native_threads:/usr/lpp/java/J7.1_64/bin:/usr/lpp/java/J7.1_64/bin/classic:/usr/lpp/WebSphere/lib:/usr/lpp/gskssl/bin:/usr/lpp/ctg/bin:/usr/lpp/HOD/lib:/usr/lpp/mqm/java/lib:/usr/lpp/ocsf/lib:/usr/lpp/Printsrv/lib:/lib:/usr/lib:.:/usr/lpp/wbem/lib:/usr/lpp/wbem/provider:/usr/lpp/pkiserv/lib:/usr/lpp/hpj/lib:/usr/lpp/TextTools/lib
LOGNAME=SPARKID
MAIL=/usr/mail/SPARKID
MANPATH=/usr/lpp/Printsrv/man/En_US:/usr/man/%L:/usr/lpp/tcpip/X11R6/Xamples/man:/usr/lpp/tcpip/man/%L:/usr/oetools/man/%L:/usr/lpp/ported/man:/usr/local/man
MVSINST=/etc/NetQ
NLSPATH=:/usr/lib/nls/msg/%L/%N:/usr/lib/nls/msg/%L:/usr/lpp/internet/%L/%N:/usr/lpp/tcpip/lib/nls/msg/%L:/usr/lpp/Printsrv/En_US/%N:/usr/lib/nls/msg/%L/%N.cat:/usr/lpp/pkiserv/lib/nls/msg/%L/%N:/usr/lpp/TextTools/nls/En_US/%N
Notes_SHARED_DPOOLSIZE=32000000
OLDPWD=/u/sparkid/hello_world/hello_world
PATH=/usr/lpp/java/J8.0_64/bin:/usr/local/bin:/usr/lpp/ported/bin:.:/usr/lpp/zosmf/V2R1/bin:/usr/lpp/zWebSphere/Liberty/V8R5/bin:/usr/lpp/ixm/IBM/xml4c-5_6/bin:/bin:/usr/lpp/ported/bin:/bin:/usr/lpp/Printsrv/bin:/bin:/usr/sbin:/usr/lpp/tcpip/bin:/usr/lpp/NFS:/usr/lpp/tcpip/sbin:/usr/lpp/java/J7.1_64/lib/mvs/native_threads:/usr/lpp/java/J7.1_64/bin:/usr/lpp/db2810/bin:/usr/lpp/internet/bin:/usr/lpp/NetQ/nqdrv:/usr/lpp/lotus/bin:/usr/lpp/lotus/bin/tools:/usr/lpp/gskssl/bin:/usr/lpp/internet/sbin:/usr/lpp/ocsf/bin:/usr/lpp/HOD/bin:/usr/lpp/ocsf/ivp:/usr/local/coz/bin:/usr/local/bin:/usr/lpp/iwl/bin:/usr/lpp/dfsms/bin:/usr/lpp/zWebSphereOEM/V7R0/bin:/usr/lpp/pkiserv/bin:/usr/lpp/perl/bin:/var/zosmf/zospt/bin:/usr/local/nodejs/node-v6.11.2-os390-s390x/bin:.:/usr/lpp/hpj/bin:/usr/lpp/TextTools/bin:/u/sparkid::/usr/lpp/IBM/Spark/bin:/usr/lpp/IBM/Spark/sbin
PEGASUS_HOME=/usr/lpp/wbem
PRINTER=psc3130
PS1=$LOGNAME:$PWD: >
PWD=/u/sparkid/hello_world/hello_world/hello_world
PYTHONHOME=/usr/lpp/ported/bin
PYTHONPATH=/usr/lpp/ported/lib/python2.7
SHELL=/usr/local/bin/bash
SHLVL=1
SPARK_CONF_DIR=/u/sparkid/conf
SPARK_HOME=/usr/lpp/IBM/Spark
SSHD=YES
SSH_CLIENT=9.101.129.38 60852 22
SSH_CONNECTION=9.101.129.38 60852 9.212.128.238 22
SSH_TTY=/dev/ttyp0000
STEPLIB=PRICHAR.CEEZ230.D170510.RT22.SCEERUN2
TERM=xterm
TZ=CET-1CEST,M3.5.0/02:00:00,M10.5.0/03:00:00
USER=SPARKID
WASOEM_CONFIG_WORKDIR=/etc/
WLP_INSTALL_DIR=/usr/lpp/zWebSphere/Liberty/V8R5
WLP_USER_DIR=/was85config/liberty
XALANCROOT=/usr/lpp/ixm/IBM/xslt4c-1_7
XERCESCROOT=/usr/lpp/ixm/IBM/xml4c-5_6
_=/bin/env
_BPXK_AUTOCVT=ON
_BPX_SHAREAS=NO
_BPX_SPAWN_SCRIPT=YES
_C89_CCMODE=1
_C89_CLIB_PREFIX=CBC
_C89_INCDIRS=/usr/include /usr/include/sys /usr/lpp/ioclib/include /usr/include/netinet /usr/lpp/tcpip/samples
_C89_LIBDIRS=/lib /usr/lib
_C89_PLIB_PREFIX=CEE
_C89_SLIB_PREFIX=SYS1
_C89_WORK_UNIT=SYSALLDA
_CC_CCMODE=1
_CC_CLIB_PREFIX=CBC
_CC_INCDIRS=/usr/include /usr/include/sys /usr/lpp/ioclib/include /usr/include/netinet /usr/lpp/tcpip/samples
_CC_LIBDIRS=/lib /usr/lib
_CC_PLIB_PREFIX=CEE
_CC_SLIB_PREFIX=SYS1
_CC_WORK_UNIT=SYSALLDA
_CEE_RUNOPTS=FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)
_CXX_CCMODE=1
_CXX_CLIB_PREFIX=CBC
_CXX_INCDIRS=/usr/include /usr/include/sys /usr/lpp/ioclib/include /usr/include/netinet /usr/lpp/tcpip/samples
_CXX_LIBDIRS=/lib /usr/lib
_CXX_PLIB_PREFIX=CEE
_CXX_SLIB_PREFIX=SYS1
_CXX_WORK_UNIT=SYSALLDA
_EDC_IP_CACHE_ENTRIES=0
_TAG_REDIR_ERR=txt
_TAG_REDIR_IN=txt
_TAG_REDIR_OUT=txt
SPARKID:/u/sparkid/hello_world/hello_world/hello_world: >

git remote
origin
=====================================
git status
On branch master
Your branch is up-to-date with ‘origin/master’.
Changes not staged for commit:
(use “git add …” to update what will be committed)
(use “git checkout – …” to discard changes in working directory)

    modified:   hello.c

Untracked files:
(use “git add …” to include in what will be committed)

    TESTS.md
    env
    git
    pwd

no changes added to commit (use “git add” and/or “git commit -a”)

====================================

git commit -m "update newfile on master"
On branch master
Your branch is up-to-date with ‘origin/master’.
Changes not staged for commit:
modified: hello.c

Untracked files:
TESTS.md

no changes added to commit

git push origin master
Everything up-to-date
===========================
git pull origin master
/usr/local/libexec/git-core/git-sh-setup: line 342: cd: ÅÑÈ: EDC5129I No such file or directory.
Unable to determine absolute path of git directory

Can you please put this into a file (let’s call it test1.sh) in the working directory, source it (like this: . test1.sh), and then provide the output?

Then invoke bash again (just type bash -i) and source it again. Thanks.

test1.sh

 test_setting_gitdir () {
  GIT_DIR=$(git rev-parse --git-dir)
   status=$?
   if [[ $status == 0 ]] ; then
     echo "GIT_DIR is initially ${GIT_DIR}"
    if test -n "$GIT_DIR"; then
       GIT_DIR=$(cd "$GIT_DIR" && pwd)
       echo "GIT_DIR was changed to $GIT_DIR"
     else
       echo "GIT_DIR was empty"
     fi
   else
     echo "It looks like git rev-parse --git-dir failed"
   fi
 }

 test_setting_gitdir

can’t reply on the forum; it keeps saying ‘new user can’t define 2 links in a post’…

i replied with the tes1.sh output by mail

The message from Philippe:

#!/bin/bash 
set -x 
test_setting_gitdir () { 
GIT_DIR=$(git rev-parse --git-dir) 
status=$? 
if [[ $status == 0 ]] ; then 
echo "GIT_DIR is initially ${GIT_DIR}" 
if test -n "$GIT_DIR"; then 
GIT_DIR=$(cd "$GIT_DIR" && pwd) 
echo "GIT_DIR was changed to $GIT_DIR" 
else 
echo "GIT_DIR was empty" 
fi 
else 
echo "It looks like git rev-parse --git-dir failed" 
fi 
} 
test_setting_gitdir 

. test1.sh
. test1.sh
+ . test1.sh
++ set -x
++ test_setting_gitdir
+++ git rev-parse --git-dir
++ GIT_DIR=$'\056\147\151\164\012'
++ status=0
++ [[ 0 == 0 ]]
++ echo 'GIT_DIR is initially ÅÑÈ'
GIT_DIR is initially ÅÑÈ
++ test -n $'\056\147\151\164\012'
+++ cd $'\056\147\151\164\012'
bash: cd: ÅÑÈ: EDC5129I No such file or directory.
++ GIT_DIR=
++ echo 'GIT_DIR was changed to '
GIT_DIR was changed to

but if i enter git rev-parse --git-dir , then i get a good result back

+ git rev-parse --git-dir
.git

env | grep GIT
+ env
+ grep GIT
GIT_SHELL=/usr/local/bin/bash
GIT_TEMPLATE_DIR=/usr/local/share/git-core/templates
GIT_EXEC_PATH=/usr/local/libexec/git-core
SPARKID:/u/sparkid/hello_world/hello_world/hello_world: >env | grep CVT
+ env
+ grep CVT
_BPXK_AUTOCVT=ON
_CEE_RUNOPTS=FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)

do we have any sort of translation problem ?


I suspect that the version of bash may be the problem, as there have been some translation issues with some versions. Are you using the version specified in the release notes, and in this forum posting?

thanks for the reply Jerry
looks like i am using bash 4.2
_ File 740 2015-03-26 07:51 SAMBA 3563520 bash
Type Perm Changed-GMT-1 Owner ------Size Filename Row 1 of 10
_ Dir 755 2015-03-26 07:55 SAMBA 8192 .
_ Dir 777 2017-09-01 17:20 SAMBA 237568 …
_ Dir 740 2014-11-25 09:17 SAMBA 24576 bash-4.2
_ File 740 2015-06-29 08:53 SAMBA 3973120 bash-4.2.tar
_ File 740 2015-06-29 08:43 SAMBA 24842240 bash-4.2-src.tar
_ Dir 740 2013-11-12 16:51 SAMBA 8192 bin
_ Dir 740 2013-11-12 16:51 SAMBA 8192 man
_ File 640 2013-11-18 09:33 SAMBA 4459 LICENSE
_ File 640 2013-07-03 10:04 SAMBA 70826 README
_ File 640 2015-03-26 07:55 SAMBA 670 README.ZOS

i will try to download the latest version, as instructed.
sorry for not checking that myself ;-(

i don’t seem to be able to download bash 4.3 from https://my.rocketsoftware.com/RocketCommunity#/downloads. it is not in the list of products that are available for download. Can you help ?

Hi,

What’s wrong? USSP-711, bash-4.3_b018.170518.tar.gz
I am able to download it.