# Script containing Curl doesn't run with crontab

**URL:** <https://community.unix.com/t/script-containing-curl-doesnt-run-with-crontab/369179>\
**Category:** Shell Programming and Scripting\
**Created:** [November 8, 2017, 4:37am UTC](https://community.unix.com/t/script-containing-curl-doesnt-run-with-crontab/369179 "2017-11-08T04:37:08Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![beautymind](https://community.unix.com/letter_avatar/beautymind/32/6_22734026db12803ebb3e059622ac3534.png) [@beautymind](https://community.unix.com/u/beautymind)\
**Post date:** [November 8, 2017, 4:37am UTC](https://community.unix.com/t/script-containing-curl-doesnt-run-with-crontab/369179/1 "2017-11-08T04:37:08Z")

</div>

Hello  
I have a problem with the crontab command  
when I run a code containing Curl on the command line it runs without fail but as soon as I program it with crontab it executes everything except the curl returns fail  
thank you for helping me to resolve this problem because since Monday I look without finding an issue

ps. I got this message: `curl: (7) Failed connect to apirest.atinternet-solutions.com:443; Connection timed out`

thank you in advance

---

<div class="post-metadata">

**Author:** ![rbatte1](https://community.unix.com/letter_avatar/rbatte1/32/6_22734026db12803ebb3e059622ac3534.png) [@rbatte1](https://community.unix.com/u/rbatte1)\
**Post date:** [November 8, 2017, 6:08am UTC](https://community.unix.com/t/script-containing-curl-doesnt-run-with-crontab/369179/2 "2017-11-08T06:08:31Z")

</div>

This is most likely because the environment for the process set up by cron is very basic. You may need to include all sorts of things such as proxy variables in your called script.

Can you share the code and details of what OS/version you are using? Sanitised output from `uname -a` should be fine.

Kind regards,  
Robin

---

<div class="post-metadata">

**Author:** ![beautymind](https://community.unix.com/letter_avatar/beautymind/32/6_22734026db12803ebb3e059622ac3534.png) [@beautymind](https://community.unix.com/u/beautymind)\
**Post date:** [November 8, 2017, 6:16am UTC](https://community.unix.com/t/script-containing-curl-doesnt-run-with-crontab/369179/3 "2017-11-08T06:16:45Z")

</div>

Thank you Robin for your reply,  
`uname -a` returns :

```nohighlight
Linux aocsv155bed0p.cloudwatt.saint-gobain.net 3.10.0-327.36.3.el7.x86_64 #1 SMP Mon Oct 24 16:09:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

```

and this piece of code returns fail thoug it works fine in th command line :

```nohighlight
if /usr/bin/curl -LS -D- -X GET -H "Authorization:Basic $password" -H 'Content-Type: text/csv' -L -o $OUTPUT_FILENAME $url; then
                       # if wget GET password=$password -H 'Content-Type: text/csv' password=$password -o $OUTPUT_FILENAME $url; then
                            echo "success"
                        else
                            echo "fail"
                        fi

```

---

<div class="post-metadata">

**Author:** ![RudiC](https://community.unix.com/letter_avatar/rudic/32/6_22734026db12803ebb3e059622ac3534.png) [@RudiC](https://community.unix.com/u/RudiC)\
**Post date:** [November 8, 2017, 6:36am UTC](https://community.unix.com/t/script-containing-curl-doesnt-run-with-crontab/369179/4 "2017-11-08T06:36:33Z")

</div>

What be the contents ot the variables `password` , `OUTPUT_FILENAME` , `url` when run from `cron` ?

---

<div class="post-metadata">

**Author:** ![rbatte1](https://community.unix.com/letter_avatar/rbatte1/32/6_22734026db12803ebb3e059622ac3534.png) [@rbatte1](https://community.unix.com/u/rbatte1)\
**Post date:** [November 8, 2017, 6:43am UTC](https://community.unix.com/t/script-containing-curl-doesnt-run-with-crontab/369179/5 "2017-11-08T06:43:39Z")

</div>

What are your values for the following variables in your interactive shell and what are they set to in your script?

```nohighlight
http_proxy
HTTPS_PROXY
FTP_PROXY
ALL_PROXY
NO_PROXY

```

Thanks again,  
Robin

---

<div class="post-metadata">

**Author:** ![beautymind](https://community.unix.com/letter_avatar/beautymind/32/6_22734026db12803ebb3e059622ac3534.png) [@beautymind](https://community.unix.com/u/beautymind)\
**Post date:** [November 8, 2017, 6:51am UTC](https://community.unix.com/t/script-containing-curl-doesnt-run-with-crontab/369179/6 "2017-11-08T06:51:45Z")

</div>

excuse me Robin , I'm beginner in shell so I don't how to get theses informations http\_proxy etc

---

<div class="post-metadata">

**Author:** ![RudiC](https://community.unix.com/letter_avatar/rudic/32/6_22734026db12803ebb3e059622ac3534.png) [@RudiC](https://community.unix.com/u/RudiC)\
**Post date:** [November 8, 2017, 6:59am UTC](https://community.unix.com/t/script-containing-curl-doesnt-run-with-crontab/369179/7 "2017-11-08T06:59:02Z")

</div>

put

```nohighlight
echo $http_proxy
echo $HTTPS_PROXY
echo $FTP_PROXY
echo $ALL_PROXY
echo $NO_PROXY

```

just in front of the `curl` command.

---

<div class="post-metadata">

**Author:** ![beautymind](https://community.unix.com/letter_avatar/beautymind/32/6_22734026db12803ebb3e059622ac3534.png) [@beautymind](https://community.unix.com/u/beautymind)\
**Post date:** [November 8, 2017, 7:40am UTC](https://community.unix.com/t/script-containing-curl-doesnt-run-with-crontab/369179/8 "2017-11-08T07:40:02Z")

</div>

for ftp , all and no proxy : no display

when I run the script on the command line I get the two addresses for http proxy and https proxy and nothing else

```nohighlight
proxy.speria.saint-trop.net:3128 
proxy.speria.saint-trop.net:3128

```

but when I run it with crontab I get nothing at all

---

<div class="post-metadata">

**Author:** ![RudiC](https://community.unix.com/letter_avatar/rudic/32/6_22734026db12803ebb3e059622ac3534.png) [@RudiC](https://community.unix.com/u/RudiC)\
**Post date:** [November 8, 2017, 7:45am UTC](https://community.unix.com/t/script-containing-curl-doesnt-run-with-crontab/369179/9 "2017-11-08T07:45:40Z")

</div>

Give it a try setting the two in your `cron` script.

---

<div class="post-metadata">

**Author:** ![beautymind](https://community.unix.com/letter_avatar/beautymind/32/6_22734026db12803ebb3e059622ac3534.png) [@beautymind](https://community.unix.com/u/beautymind)\
**Post date:** [November 8, 2017, 8:19am UTC](https://community.unix.com/t/script-containing-curl-doesnt-run-with-crontab/369179/10 "2017-11-08T08:19:10Z")

</div>

how can I do that plz ?

---

<div class="post-metadata">

**Author:** ![RudiC](https://community.unix.com/letter_avatar/rudic/32/6_22734026db12803ebb3e059622ac3534.png) [@RudiC](https://community.unix.com/u/RudiC)\
**Post date:** [November 8, 2017, 8:23am UTC](https://community.unix.com/t/script-containing-curl-doesnt-run-with-crontab/369179/11 "2017-11-08T08:23:27Z")

</div>

Try

```nohighlight
http_proxy="proxy.speria.saint-trop.net:3128"
HTTPS_PROXY="proxy.speria.saint-trop.net:3128"

```

---

<div class="post-metadata">

**Author:** ![rbatte1](https://community.unix.com/letter_avatar/rbatte1/32/6_22734026db12803ebb3e059622ac3534.png) [@rbatte1](https://community.unix.com/u/rbatte1)\
**Post date:** [November 8, 2017, 8:23am UTC](https://community.unix.com/t/script-containing-curl-doesnt-run-with-crontab/369179/12 "2017-11-08T08:23:55Z")

</div>

Without these variables, the cron-run script will not know how to get out of your network. Why would it? These are probably in a profile for you somewhere.

You can explicitly set them in your cron-run script, or if it might be called from the command line and people may want to use their own, you could:-

```nohighlight
http_proxy="${http_proxy:-proxy.speria.saint-trop.net:3128}"

```

This will use the value `proxy.speria.saint-trop.net:3128` for http\_proxy if the variable has not been set yet, i.e. this is a default but an existing setting will be retained.

Obviously do the same thing for the https\_proxy variable too.

I hope that this helps,  
Robin

---

<div class="post-metadata">

**Author:** ![beautymind](https://community.unix.com/letter_avatar/beautymind/32/6_22734026db12803ebb3e059622ac3534.png) [@beautymind](https://community.unix.com/u/beautymind)\
**Post date:** [November 8, 2017, 10:21am UTC](https://community.unix.com/t/script-containing-curl-doesnt-run-with-crontab/369179/13 "2017-11-08T10:21:29Z")

</div>

Thank you guys , I'll test this solution

---------- Post updated at 09:57 AM ---------- Previous update was at 09:13 AM ----------

I got this error

```nohighlight
0curl: (5) Could not resolve proxy: ${https_proxy; Name or service not known

```

Is this the right syntax :

```nohighlight
 http_proxy="${http_proxy:-proxy.speria.saint-trop.net:3128}"

```

??

---------- Post updated at 10:21 AM ---------- Previous update was at 09:57 AM ----------

Thank you sooooooo much guys , it works 🙂

---

<div class="post-metadata">

**Author:** ![rbatte1](https://community.unix.com/letter_avatar/rbatte1/32/6_22734026db12803ebb3e059622ac3534.png) [@rbatte1](https://community.unix.com/u/rbatte1)\
**Post date:** [November 8, 2017, 11:07am UTC](https://community.unix.com/t/script-containing-curl-doesnt-run-with-crontab/369179/14 "2017-11-08T11:07:38Z")

</div>

If it's working, would you care to share the completed solution (or the relevant bits) for anyone finding this script with a similar problem in future?

Thanks, in advance,  
Robin

---

<div class="post-metadata">

**Author:** ![beautymind](https://community.unix.com/letter_avatar/beautymind/32/6_22734026db12803ebb3e059622ac3534.png) [@beautymind](https://community.unix.com/u/beautymind)\
**Post date:** [November 8, 2017, 11:19am UTC](https://community.unix.com/t/script-containing-curl-doesnt-run-with-crontab/369179/15 "2017-11-08T11:19:53Z")

</div>

But there's an other problem,  
when I get the csv file I try to put it via hdfs but the put fails  
and I get this message :

```nohighlight
Failed on local exception: java.io.IOException: javax.security.sasl.SaslException: GSS initiate failed 

```

---------- Post updated at 11:19 AM ---------- Previous update was at 11:13 AM ----------

Yes of course Robin ,

I just added the two links in the cron-run script :

```nohighlight
http_proxy="proxy.speria.saint-trop.net:3128"
HTTPS_PROXY="proxy.speria.saint-trop.net:3128"

```

As you mentioned in a post : "the cron-run script will not know how to get out of your network"

Hope that this helps