Products Resources Support About Us

Rocket Software

cURL to STDOUT - wrong data conversion of ZIP-File?

Hello Forum,
may I ask a question for this issue?

I’m recently installed cURL for z/OS.
During tests I’ve realized that the ZIP-File we’ve loaded from a website was converted to a wrong format if I omit the --output option and the data will be written to STDOUT. The zip-file can’t be processed afterwards!

curl --insecure --proxy1.0 webgwooe.rbgooe.at:8080 https://geolite.maxmind.com/download/geoip/database/GeoLite2-City-CSV.zip >geo.zip

Browsed ZIP-File looks like this - Maybe the data was converted to EBCDIC? :
BROWSE /ugrz/lrzspfo/geo.zip Line 0000000000 Col 001 132
Command ===> Scroll ===> CSR
*********************************************************** Top of Data ************************************************************
PK…y.O…<…GeoLite2-City-CSV_20191210/GeoLite2-City-Locations-zh-CN.csvUT…Ôµï¨Ôµï¨ux…Z²v…Z²v´ýY…Å./.~¿.#¯

Please notice:
Data transfer works fine using the --output geo2.zip option! In this case it is possible to process the data afterwards.
BROWSE /ugrz/lrzspfo/geo2.zip Line 0000000000 Col 001 132
Command ===> Scroll ===> CSR
*********************************************************** Top of Data ************************************************************
&…««!................åÁ?<ÑÈÁ..{ÑÈ.{ëî^…åÁ?<ÑÈÁ…{ÑÈ`.<?[/ÈÑ?>Ë.:Ç.{+.[ËÎíè…M@Õ)M@Õ)ÍÌ…ÆÜ¥Î.ÆܥΩÙ~pæEn.ðÿ×…®

If you wonder at this usage with STDOUT…
Aim is to execute cURL within a BPXBATCH-Job writing the data directly to an MVS-Dataset.
//LRZSPFOU JOB ,USS-TEST,NOTIFY=LRCUGAC J
//* $ACFJ219 ACF2 ACTIVE EXJS2
//BPXBATCH EXEC PGM=BPXBATCH,
// PARM=’’,
// REGION=8M
//STDPARM DD *
sh export _BPXK_AUTOCVT=OFF &&
curl --insecure --proxy1.0 webgwooe.rbgooe.at:8080
https://geolite.maxmind.com/download/geoip/database/GeoLite2-City-CSV.zip
//STDIN DD DUMMY
//STDOUT DD DISP=SHR,DSN=LRZSPFO.ZIP.OMVSOUT
//TDOUT DD DSN=LRZSPFO.ZIP.OMVSOUT2,
//
DISP=(,CATLG,DELETE),
//* UNIT=DISK,SPACE=(CYL,(100,200)),
//* DCB=(RECFM=VB,LRECL=32756,BLKSIZE=0)
//STDERR DD SYSOUT=*,OUTLIM=39999

Hi,
Ported tools (incl. curl) are designed to work from bash. You can see an example how to run curl from bash within BPXBATCH here.

Hello Tatyana,
In the meanwhile I’ve installed your ported bash Version.

I had success calling curl within bash and writing to STDOUT. Thank you for your Suggestion.

But now a new Problem came up - It didn’t work from BATCH if STDOUT was a MVS-Dataset.
*//USSBATCH EXEC PGM=BPXBATSL,PARM='PGM /bin/sh -c $STDIN!ICONV!sh' * *//STDIN DD DATA,DLM=## * *bash * *curl --insecure --proxy1.0 webgwooe.rbgooe.at:8080 \ * *"https://download.maxmind.com/app/geoip_download?edition_id=GeoLite2-City-C\ * *SV&license_key=Gx0aXLbbWDtWDeTq&suffix=zip" * *exit * *RC=?; if [ $RC -ne 0 ]; then; exit 1; else; exit 0; fi *
*## *
*//STDENV DD DATA,DLM=## *
*STDIN=/bin/cat //dd:STDIN *
*ICONV=iconv -f1141 -t1047 *
*SH=sh -L *
*_BPX_BATCH_UMASK=0022 *
*_BPX_SHAREAS=YES *
*_BPXK_AUTOCVT=ON *
*_CEE_RUNOPTS=FILETAG(AUTOCVT,AUTOTAG) POSIX(ON) *
PATH=/bin:/shr/lppgrz/ported/curl/bin:/shr/lppgrz/ported/bash/bin
*## *
*//STDOUT DD DISP=SHR,DSN=LRZSPFO.ZIP.OMVSOUT2 *
//STDERR DD SYSOUT=,OUTLIM=39999 *
//
I’ve called IBM and they told me that there is problably a Problem with this usage “STDOUT to MVS dataset”.
IBM has send me a Workaround which works fine. In this Workaround STDOUT is piped to a “cp -B”-command, which is able to write to a mvs-dataset.
https://download.maxmind.com/app/geoip_download?edition_id=GeoLite2-City-C
SV&license_key=Gx0aXLbbWDtWDeTq&suffix=zip” |\
cp -B /dev/fd0 “//‘LRZSPFO.ZIP.OMVSOUT’”

And now my question:
May I ask you to think about an enhancement of curl:
It should be possible to reference/code under Option --outfile a mvs-dataset similar to the cp-command: “//‘LRZSPFO.ZIP.OMVSOUT’”

What do you think about this Suggestion?

Regards, Helmut Pfoser