I'm trying to compile Linux for an embedded system. I'm using ptxdist command to build the system.
My colleague told me to use ptxdist get so we can get all dependencies downloaded before we start compiling.
We also sit behind a proxy and my /etc/environment is configured with the http/https/ftp proxy (same IP and port for all), as well as apt (meaning that wget works fine without extra parameters)
PROBLEM:
Running ptxdist get will download some of the packages, others it will just not download and the following error appears (example for 2 packages):
Here is an example when it worked fine:
---------------------------
target: cmake-3.13.4.tar.gz
---------------------------
--2019-07-23 20:54:28-- https://cmake.org/files/v3.13/cmake-3.13.4.tar.gz
Connecting to <PROXY IP HERE>:8080... connected.
Proxy request sent, awaiting response... 200 OK
Length: 8617881 (8.2M) [application/x-gzip]
Saving to: '/home/xxxxxxx/xxxxxxx/xxxxxxx/src/cmake-3.13.4.tar.gz.RSjrC7k7mC'
/home/xxxxxxx/x 100%[===================>] 8.22M 2.00MB/s in 4.7s
2019-07-23 20:54:34 (1.75 MB/s) - '/home/xxxxxxx/xxxxxxx/xxxxxxx/src/cmake-3.13.4.tar.gz.RSjrC7k7mC' saved [8617881/8617881]
And here an example when it failed:
-----------------------
target: lzo-2.08.tar.gz
-----------------------
--2019-07-23 20:55:13-- http://www.oberhumer.com/opensource/lzo/download/lzo-2.08.tar.gz
Connecting to <PROXY IP HERE>:8080... connected.
Proxy request sent, awaiting response... 502 Proxy Error ( The specified network name is no longer available. )
2019-07-23 20:55:13 ERROR 502: Proxy Error ( The specified network name is no longer available. ).
--2019-07-23 20:55:13-- http://www.pengutronix.de/software/ptxdist/temporary-src/lzo-2.08.tar.gz
Connecting to <PROXY IP HERE>:8080... connected.
Proxy request sent, awaiting response... 502 Proxy Error ( The specified network name is no longer available. )
2019-07-23 20:55:13 ERROR 502: Proxy Error ( The specified network name is no longer available. ).
make: *** [/home/xxxxxxxx/xxxxxxx/xxxxxxx/src/lzo-2.08.tar.gz] Error 1
Could not download package
URL: http://www.oberhumer.com/opensource/lzo/download/lzo-2.08.tar.gz
/usr/local/lib/ptxdist-2019.03.1/rules/post/ptxd_make_world_get.make:17: recipe for target '/home/xxxxxxx/xxxxxxx/xxxxxxx/src/lzo-2.08.tar.gz' failed
Even if I repeat ptxdist get, it stops at the same exact package that fails.
BUT:
If I just use
wget http://www.oberhumer.com/opensource/lzo/download/lzo-2.08.tar.gz
it downloads the file just fine, same happens to ANY other package that fails, using the URL that shows up above.
When I run ptxdist get then it moves on to other downloads as it sees the file already there, and then hangs on the next download that failes.
ALSO TRIED:
I also tried to run ptxdist setup and configure the proxy there. Nothing changes in the terminal, so no luck...
ALSO TRIED:
I also wrote a script that takes package names using ptxdist print PACKAGES, then gets the URL using ptxdist package-info <PKG> then downloads the file via wget <URL>, this solved most part of the errors while running ptxdist get as wget works fine, but it seems that there are many other packages that are needed and not listed with ptxdist print PACKAGES...
Any idea what is wrong and what can I do?
Since I'm supposed to automate this task, there is no way manual interaction (for using wget on failing packages) is an option...
Related
I HAVE A ERROR IN INSTALLING PYTORCH:
PLEASE HELP ME.
CondaHTTPError: HTT P000 CONNECTION FAILED for url https://conda.anaconda.org/pytorch/win-64/current_repodata.json
Elapsed: -
An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.
'https//conda.anaconda.org/pytorch/win-64'
An HTTP error occurred when trying to retrieve this URL. HTTP errors are often intermittent, and a simple retry will get you on your way
The possible reason for HTTP error could be an unstable network connection or a corporate firewall.
If it was an unstable network connection, as mentioned in the Error message, retry the installation steps that failed.
If you are behind a corporate firewall, you might need additional steps to add your proxy server to the .condarc file on your machine.
Since you are on Windows, you could open the Anaconda prompt and run conda info to figure out where the .condarc file is located.
Find the proxy by running echo "$http_proxy" in your prompt. Copy the proxy.
Open the .condarc file and paste the proxy under proxy_servers section
For more details see: Anaconda Docs: Configure conda for use behind a proxy server (proxy_servers)
(related and perhaps more simple problem to solve: proxy authentication by MSCHAPv2)
Summary: I am using a Ubuntu 18, the proxy is working with web-browser but not with terminal applications (wget, curl or apt update). Any clues? Seems the problem is to interpretate a proxy's "PAC file"... Is it? How to translate to Linux's proxy variables? ... Or the problem is simple: my proxy-config (see step-by-step procedure below) was wrong?
Details:
By terminal env | grep -i proxy we obtain
https_proxy=http://user:pass#pac._ProxyDomain_/proxy.pac:8080
http_proxy=http://user:pass#pac._ProxyDomain_/proxy.pac:8080
no_proxy=localhost,127.0.0.0/8,::1
NO_PROXY=localhost,127.0.0.0/8,::1
ftp_proxy=http://user:pass#pac._ProxyDomain_/proxy.pac:8080
and browser (Firefox) is working fine for any URL, but:
wget http://google.com say Resolving pac._ProxyDomain_ (pac._ProxyDomain_)... etc.etc.0.26 connecting to pac._ProxyDomain_ (pac._ProxyDomain_)|etc.etc.0.26|:80... conected.
Proxy request has been sent, waiting for response ... 403 Forbidden
2019-07-25 12:52:19 ERROR 403: Forbidden.
curl http://google.com say "curl: (5) Could not resolve proxy: pac._ProxyDomain_/proxy.pac"
Notes
(recent news here: purge exported proxy changes something and not tested all again...)
The proxy configuration procedures that I used (there are some plug-and-play PAC file generator? I need a PAC file?)
Config procedures used
All machine was running, with a direct non-proxy internet connection... Them the machine goes to the LAN with the proxy.
Add lines of "export *_proxy" (http, https and ftp) in my ~/.profile. The URL definitions are in the form http_proxy="http://user:pwd#etc" (supposing that is correct, because testesd before with user:pwd#http://pac.domain/proxy.pac syntax and Firefox promped proxy-login)(if the current proxy-password is using # character, need to change?)
Add lines of "export *_proxy" in my ~root/.profile.(need it?)
(can reboot and test with echo $http_proxy)
visudo procedure described here
reboot and navigate by Firefox without need of login, direct (good is working!). Testing env | grep -i proxy, it shows all correct values as expected.
Testing wget and curl as the begin of this report, proxy bug.
Testing sudo apt update, bug.
... after it more one step, supponing that for apt not exist a file, created by sudo nano /etc/apt/apt.conf.d/80proxy and add 3 lines for Acquire::*::proxy "value"; with value http://user:pass#pac._ProxyDomain_/proxy.pac:8080. where pass is etc%23etc, url-encoded.
Summary of tests performed
CONTEXT-1.1
(this was a problem but now ignoring it to focus on more relevant one)
After (the proxied) cable connection and proxy configurations in the system. (see above section "Config procedures used"). Proxy-password with special character.
curl http://google.com say "curl: (5) Could not resolve proxy..."
When change all .profile from %23 to # the error on wget changes, but curl not. Wget changes to "Error parsing proxy URL http://user:pass#pac._ProxyDomain_/proxy.pac:8080: Bad port number"
PS: when used $ on password the system (something in the internal export http_proxy command or use of http_proxy confused it with a variable).
CONTEXT-1.2
Same as context-1.1 above, but password with no special character. Good and clean proxy-password.
curl http://google.com say "curl: (5) Could not resolve proxy..."
CONTEXT-2
After (the proxied) cable connection and no proxy configurations in the system (but confirmed that connection is working on browser after automatic popup form login).
curl -x 192.168.0.1:8080 http://google.com "curl: (7) Failed to connect..."
curl --verbose -x "http://user:pass#pac._proxyDomain_/proxy.pac" http://google.com say "curl: (5) Could not resolve proxy..."
Other configs in use
As #Roadowl suggested to check:
files ~/.netrc and ~root/.netrc not exists
file more /etc/wgetrc exists, but all commented, exept by passive_ftp = on
When i run composer update it fails showing error:
[Composer\Downloader\TransportException]
The "https://packagist.org/packages.json" file could not be downloaded: failed to open stream:
Connection refused
If i visit the URL from my browser it goes through, when i try to wget or curl the URL from my terminal it shows error:
--2017-02-05 20:41:58-- https://packagist.org/packages.json
Connecting to 127.0.0.1:8888... failed:
Connection refused.
--2017-02-05 20:41:58-- https://packagist.org/packages.json
Connecting to 127.0.0.1:8888... failed: Connection refused.
I think its something with proxy connection, but then i can't seem to solve the issue.
Bellow is the full trace if i run composer update -vv:
Reading ./composer.json
Loading config file ./composer.json
Checked CA file /etc/ssl/certs/ca-certificates.crt: valid
Executing command (/var/www/gidiscrap): git branch --no-color --no- abbrev -v
Cannot create cache directory /home/oluwaslim/.composer/cache/repo/https---packagist.org/, or directory is not writable. Proceeding without cache
Cannot create cache directory /home/oluwaslim/.composer/cache/files/, or directory is not writable. Proceeding without cache
Failed to initialize global composer: Composer could not find the config file: /home/oluwaslim/.composer/composer.json
To initialize a project, please create a composer.json file as described in the https://getcomposer.org/ "Getting Started" section
Reading /var/www/gidiscrap/vendor/composer/installed.json
Running 1.3.2 (2017-01-27 18:23:41) with PHP 7.0.13-0ubuntu0.16.04.1 on Linux / 4.4.0-59-generic
Loading composer repositories with package information
Downloading https://packagist.org/packages.json using proxy 127.0.0.1:8888
Downloading https://packagist.org/packages.json using proxy 127.0.0.1:8888
Downloading https://packagist.org/packages.json using proxy 127.0.0.1:8888
[Composer\Downloader\TransportException]
The "https://packagist.org/packages.json" file could not be downloaded: failed to open stream: Connection refused
Exception trace:
() at phar:///usr/local/bin/composer/src/Composer/Util/RemoteFilesystem.php:489
Composer\Util\RemoteFilesystem->get() at phar:///usr/local/bin/composer/src/Composer/Util/RemoteFilesystem.php:101
Composer\Util\RemoteFilesystem->getContents() at phar:///usr/local/bin/composer/src/Composer/Repository/ComposerRepository.php:665
Composer\Repository\ComposerRepository->fetchFile() at phar:///usr/local/bin/composer/src/Composer/Repository/ComposerRepository.php:479
Composer\Repository\ComposerRepository->loadRootServerFile() at phar:///usr/local/bin/composer/src/Composer/Repository/ComposerRepository.php:258
Composer\Repository\ComposerRepository->hasProviders() at phar:///usr/local/bin/composer/src/Composer/DependencyResolver/Pool.php:99
Composer\DependencyResolver\Pool->addRepository() at phar:///usr/local/bin/composer/src/Composer/Installer.php:376
Composer\Installer->doInstall() at phar:///usr/local/bin/composer/src/Composer/Installer.php:223
Composer\Installer->run() at phar:///usr/local/bin/composer/src/Composer/Command/UpdateCommand.php:158
Composer\Command\UpdateCommand->execute() at phar:///usr/local/bin/composer/vendor/symfony/console/Command/Command.php:257
Symfony\Component\Console\Command\Command->run() at phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:850
Symfony\Component\Console\Application->doRunCommand() at phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:193
Symfony\Component\Console\Application->doRun() at phar:///usr/local/bin/composer/src/Composer/Console/Application.php:227
Composer\Console\Application->doRun() at phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:124
Symfony\Component\Console\Application->run() at phar:///usr/local/bin/composer/src/Composer/Console/Application.php:100
Composer\Console\Application->run() at phar:///usr/local/bin/composer/bin/composer:54
require() at /usr/local/bin/composer:24
Help would be appriciated! I can't seem to get past this error
Am on Ubuntu Linux 16.04, and below is what i had to do to fix the issue, as i mentiond earlier its some problem with proxy settings.
Step 1
Goto System settings
Network
Network proxy
Change it to none
finally click Apply system wide
Step 2
Open terminal and type the following command.
unset https_proxy && unset socks_proxy && unset ftp_proxy
Finally restart your system. The solution above got the issue solved for me.
The line using proxy 127.0.0.1:8888 suggest a problem with your system. I can guess that's a really aggressive antivirus. You should have no reason to connect to localhost on port 8888 unless you have to connect through proxy. What can you do? Check the antivirus settings and turn off checks for secure connections over HTTPS.
I am new to Cozy Cloud, and trying to setup on Ubuntu 14.04, as mentioned Install Cozy on Ubuntu:
and while installing the cozy I tried to run the command:
$ sudo apt-get update
I am getting the following error:
W: Failed to fetch
https://ubuntu.cozycloud.cc/debian/dists/trusty/main/binary-amd64/Packages
Received HTTP code 403 from proxy after CONNECT W: Failed to fetch
https://ubuntu.cozycloud.cc/debian/dists/trusty/main/binary-i386/Packages
Received HTTP code 403 from proxy after CONNECT E: Some index files
failed to download. They have been ignored, or old ones used instead.
Though the above urls are accessible in browser and downloads package file in a browser, but in the terminal with above command, I am getting an error.
How to resolve this issue?
Found the solution, my apt was configured to use a proxy server. I tried to run apt-config dump | grep -i proxy on command line and found an entry for proxy in the following file:
/etc/apt/apt.conf
Removed the entry and now it's working fine.
When I try to npm publish to a private npm repository on Artifactory I get a Failed PUT 302 error. I followed the directions in their video(https://www.youtube.com/watch?v=gyQ0riy3Hk8) and set up a remote repository that points to registry.npmjs.org, another that is the private repo(npm-local), and a virtual repo that aggregates the remote and private repos.
I set up my ~/.npmrc file to use the Artifactory virtual repo and added my credentials. I am able to install public packages fine using this setup. But when I try to publish directly to /npm-local I always get Failed PUT 302 - the repository responds with a generic nginx 302 Found page and npm can't parse the html.
Currently I am using npm#2.13.3 but I even tried reverting back to 1.4.x after reading npm publish to Artifactory not working and had the same issue.
Also I have tried publishing a directory with package.json in it and a tarball - both produce the same 302 error.
After several emails and a phone call I found the fix:
login to artifactoryonline
Goto Admin Tab
Under general settings set the Custom URL Base to: https://YOUR_ACCOUNT_NAME.artifactoryonline.com/YOUR_ACCOUNT_NAME
They said that they are working on having this set for you by default but until then you have to set it manually. Unfortunately it is not documented anywhere and not mentioned in any of the videos that I saw.
NPM v6 is not able to handle the HTTP Status 302. It process this code as successful printing PUT 302 into verbose output. It happens for example, if the Artifactory server expects the connection over https and redirects all the requests from port 80 to 443. NPM v8 handles redirects correctly.