I'm trying to take a script I have that updates my sytstem for me, and have it output it's text to a file, and the screen. This is fairly simple using tee, except my update manager, pacman, outputs the same kind of text like wget. Here's some sample output.
core 107.0 KiB 392K/s 00:00 [################################################################] 100%
extra 1531.8 KiB 719K/s 00:02 [################################################################] 100%
community 2.1 MiB 818K/s 00:03 [################################################################] 100%
When I try to redirect that kind of output to a file using tee, this is the output I get to my screen and file.
:: Synchronizing package databases...
downloading core.db...
downloading extra.db...
downloading community.db...
I understand this is because pacman uses a buffer to output to the screen, but I'm still hoping there's a way I can output the status to the screen and the file, and not just the "downloading xyz...".
Thank you in advance for any help.
Edit:
I have no current updates, but here's some sample output that's very similar of me installing the Opera browser. The lines with the percentages and the pound signs are buffers that update with the progress of the download.
resolving dependencies...
looking for inter-conflicts...
Packages (1): opera-12.16.1860-2
Total Download Size: 10.49 MiB
Total Installed Size: 45.03 MiB
:: Proceed with installation? [Y/n] y
:: Retrieving packages ...
opera-12.16.1860-2-x86_64 13.4 MiB 1151K/s 00:12 [################################################################] 100%
(1/1) checking keys in keyring [################################################################] 100%
(1/1) checking package integrity [################################################################] 100%
(1/1) loading package files [################################################################] 100%
(1/1) checking for file conflicts [################################################################] 100%
(1/1) checking available disk space [################################################################] 100%
(1/1) installing opera [################################################################] 100%
Optional dependencies for opera
gstreamer0.10-base-plugins: HTML5 Video support
gstreamer0.10-good: HTML5 Video support
This is what the above output looks like when redirected through tee.
warning: opera-12.16.1860-2 is up to date -- reinstalling
resolving dependencies...
looking for inter-conflicts...
Packages (1): opera-12.16.1860-2
Total Installed Size: 45.03 MiB
Net Upgrade Size: 0.00 MiB
:: Proceed with installation? [Y/n] y
checking keyring...
checking package integrity...
loading package files...
checking for file conflicts...
checking available disk space...
reinstalling opera...
As you can see, the download progess isn't shown. Just three periods.
Note sure if I fully understand the question, but I think possibly your script outputs to both stdout and stderr and you want both to be captured to file as well displayed on the terminal. In which case, you could do this by redirecting stderr of your script to stderr, before piping the whole thing to tee:
./myscript.sh 2>&1 | tee output.log
So I think you also want all the output with respect to progress bars, etc. You can use the script command to get this:
script -e -q -c "./myscript.sh" output.log
This works for me with wget:
Terminal output:
ubuntu#ubuntu:~$ script -e -q -c "wget http://stackoverflow.com" output.log
--2014-02-21 20:01:31-- http://stackoverflow.com/
Resolving stackoverflow.com (stackoverflow.com)... 198.252.206.140
Connecting to stackoverflow.com (stackoverflow.com)|198.252.206.140|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 214108 (209K) [text/html]
Saving to: `index.html.5'
100%[======================================>] 214,108 7.59K/s in 38s
Last-modified header invalid -- time-stamp ignored.
2014-02-21 20:02:13 (5.55 KB/s) - `index.html.5' saved [214108/214108]
ubuntu#ubuntu:~$
Contents of output.log:
Script started on Fri 21 Feb 2014 08:01:31 PM PST
--2014-02-21 20:01:31-- http://stackoverflow.com/
Resolving stackoverflow.com (stackoverflow.com)... 198.252.206.140
Connecting to stackoverflow.com (stackoverflow.com)|198.252.206.140|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 214108 (209K) [text/html]
Saving to: `index.html.5'
0% [ ] 0 --.-K/s
1% [ ] 3,367 3.66K/s
2% [> ] 5,863 5.09K/s
3% [> ] 7,111 3.54K/s
5% [> ] 10,855 3.73K/s
### Output omitted - I have a slow connection right now ###
97% [=====================================> ] 209,806 7.84K/s eta 1s
99% [=====================================> ] 212,302 7.44K/s eta 1s
100%[======================================>] 214,108 7.59K/s in 38s
Last-modified header invalid -- time-stamp ignored.
2014-02-21 20:02:13 (5.55 KB/s) - `index.html.5' saved [214108/214108]
Related
I am trying to install dlib in Anaconda so as to use it inside jupyter-notebook.
I tried using conda install -c menpo dlib and then dlib doesn't get installed rather it then says :
Downloading and Extracting Packages
py-boost-1.67.0 | 318 KB | ################################################################# | 100%
widgetsnbextension-1 | 1.1 MB | | 0%
ipywidgets-5.1.5 | 61 KB | | 0%
boost-1.59.0 | 11.9 MB | | 0%
dlib-19.9 | 2.3 MB | | 0%
CondaHTTPError: HTTP 000 CONNECTION FAILED for url <https://conda.anaconda.org/menpo/osx-64/widgetsnbextension-1.2.3-py35_1.tar.bz2>
Elapsed: -
An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.
CondaHTTPError: HTTP 000 CONNECTION FAILED for url <https://conda.anaconda.org/menpo/osx-64/ipywidgets-5.1.5-py35_0.tar.bz2>
Elapsed: -
An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.
CondaHTTPError: HTTP 000 CONNECTION FAILED for url <https://conda.anaconda.org/menpo/osx-64/boost-1.59.0-py35_0.tar.bz2>
Elapsed: -
An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.
CondaHTTPError: HTTP 000 CONNECTION FAILED for url <https://conda.anaconda.org/menpo/osx-64/dlib-19.9-py35_0.tar.bz2>
Elapsed: -
An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.
Then, I downloaded the dlib-19.16.0.tar.gz from here and saved it in /anaconda3/lib/python3.6/site-packages and then ran conda install -c conda-forge dlib-19.16.0.tar.gz but it shows :
PackagesNotFoundError: The following packages are not available from current channels:
- dlib-19.16.0.tar.gz
Current channels:
- https://conda.anaconda.org/conda-forge/osx-64
- https://conda.anaconda.org/conda-forge/noarch
- https://repo.anaconda.com/pkgs/main/osx-64
- https://repo.anaconda.com/pkgs/main/noarch
- https://repo.anaconda.com/pkgs/free/osx-64
- https://repo.anaconda.com/pkgs/free/noarch
- https://repo.anaconda.com/pkgs/r/osx-64
- https://repo.anaconda.com/pkgs/r/noarch
- https://repo.anaconda.com/pkgs/pro/osx-64
- https://repo.anaconda.com/pkgs/pro/noarch
To search for alternate channels that may provide the conda package you're
looking for, navigate to
https://anaconda.org
and use the search bar at the top of the page.
The first error is just saying you are not able to connect to the menpo repository of libraries. I recommend to check your proxy/internet connection.
In the second case you are trying to install a tar.gz file, that's a compressed file. Decompressed it(unzip it) first and then try to install it.
I've set up a packer template to generate vagrant base image of FreeBSD 10.3 and it was working well at least Mon Oct 3 00:34:41 2016 +0300.
Yesterday I was going to continue my work on this project and it turned out this is not working anymore. So here come details.
Packer does what it have to do, then runs my script to install FreeBSD by using bsdinstall(8) with the following script:
PARTITIONS="ada0 { 29G freebsd-ufs /, 5G freebsd-swap, 10G freebsd-ufs /var }"
DISTRIBUTIONS="base.txz kernel.txz"
#!/bin/sh
echo 'WITHOUT_X11="YES"' >> /etc/make.conf
echo 'OPTIONS_UNSET=X11' >> /etc/make.conf
echo 'nameserver 8.8.8.8' >> /etc/resolv.conf
cat >> /etc/rc.conf <<EOF
ifconfig_em0="DHCP"
sshd_enable="YES"
dumpdev="NO"
EOF
env ASSUME_ALWAYS_YES=1 pkg bootstrap # <<stops here
pkg update
pkg install -y sudo
[.....snip.....]
reboot
This stops at bootstrapping pkg with the message:
Bootstrapping pkg from pkg+http://pkg.FreeBSD.org/FreeBSD:10:amd64/quarterly, please wait...
Signature for pkg not available.
pkg: Error fetching http://pkg.FreeBSD.org/FreeBSD:10:amd64/quarterly/Latest/pkg.txz.sig: Connection reset by peer
A pre-built version of pkg could not be found for your system.
Consider changing PACKAGESITE or installing it from ports: 'ports-mgmt/pkg'.
If I stop the bsdinstall script and chroot /mnt /bin/sh I can fetch pkg.txz.sig from the above URL without any problems.
Any ideas what could be the reason of the "connection reset by peer"? Something was changed on the pkg.FreeBSD.org recently?
I couldn't find anything about the issue.
UPD1
Looking at the captured traffic -- the site really answers 200OK and then drops the connection for the pkg.txz.sig file.
But this 200OK packet contains the signature file and they are identical for both manual fetch (which succeeds) and pkg bootstrap (which fails)
Both sessions are identical, so this is likely not a networking problem.
UPD2
The truss was not helpful either.
So as a workaround I've just modified my bsdinstall script to fetch files manually:
[.....snip.....]
#env ASSUME_ALWAYS_YES=1 pkg bootstrap
fetch http://pkg.FreeBSD.org/FreeBSD:10:amd64/quarterly/Latest/pkg.txz
fetch http://pkg.FreeBSD.org/FreeBSD:10:amd64/quarterly/Latest/pkg.txz.sig
pkg add pkg.txz
pkg update
[.....snip.....]
PS: The only thing that I can suspect now is the virtualbox version update... anyway downgrading is not an option. (ISO checksum is hardcoded into the template, the template and scripts are in git repository, so accidential changes are impossible)
UPD3
I've set up a debugging environment, for the moment I only isolated the function where the error is raised.
It's the second buffer refill from the http connection (while the first one has already read 727 bytes - it should be EOF)...
Here is small gdb log with backtrace and breakpoints to get there.
Added tcpdump capture made on the system (wireshark compatible).
As I found out, partially the problem was with pkg -- they try to read 10240 bytes from the connection, expecting the EOF if file will be smaller, but somehow on my system EOF is not set when whole remote file was already read out.
# /release/10.3.0/usr.sbin/pkg/pkg.c
185 char buf[10240];
242 while ((r = fread(buf, 1, sizeof(buf), remote)) > 0) {
and the following loops twice -- first time reading the file, second time getting connection reset error instead of EOF
# /release/10.3.0/lib/libc/stdio/fread.c
94 resid = count * size; # == 10240 here
100 while (resid > (r = fp->_r)) {
101 (void)memcpy((void *)p, (void *)fp->_p, (size_t)r);
102 fp->_p += r;
103 /* fp->_r = 0 ... done in __srefill */
104 p += r;
105 resid -= r;
106 if (__srefill(fp)) {
107 /* no more input: return partial result */
108 return ((total - resid) / size);
109 }
110 }
While manual fetch succeeds because the size is adjusted for small chunks and they only ask 727 bytes to read:
# /release/10.3.0/usr.bin/fetch/fetch.c
720 if (us.size != -1 && us.size - count < B_size &&
721 us.size - count >= 0)
722 size = us.size - count;
723 else
724 size = B_size;
733 if ((readcnt = fread(buf, 1, size, f)) < size) {
...but why EOF is not set is still a question.
Posted this to freebsd-pkg mailing list.
UPD1
Downgraded Virtualbox from 5.028 to 5.026 and EOF is set, _sread() on libc/stdio/refill.c:135 returns 0 and it sets EOF on line 138.
So something was changed in Virtualbox networking too. Added pcap file for Virtualbox 5.026 to the gist. 5.028 really was the culprit of connection reset - here is captures comparison.
Virtualbox 5.1.8 has this bug too. Version 5.1.6 works ok.
Opened ticket #16141 in their bugtracker.
R version 3.0.2 (2013-09-25) -- "Frisbee Sailing"
Copyright (C) 2013 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
Natural language support but running in an English locale
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
> install.packages('devtools')
Installing package into ‘/home/kingslee/R/x86_64-pc-linux-gnu-library/3.0’
(as ‘lib’ is unspecified)
Warning in install.packages :
dependency ‘xml2’ is not available
also installing the dependency ‘rversions’
--2015-07-29 12:02:27--
https://cran.rstudio.com/src/contrib/rversions_1.0.2.tar.gz
Resolving cran.rstudio.com (cran.rstudio.com)... 54.230.188.51
Connecting to cran.rstudio.com (cran.rstudio.com)|54.230.188.51|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 4654 (4.5K) [application/x-gzip]
Saving to: ‘/tmp/Rtmpcd8fcf/downloaded_packages/rversions_1.0.2.tar.gz’
0K .... 100%
540M=0s
2015-07-29 12:02:29 (540 MB/s) -
‘/tmp/Rtmpcd8fcf/downloaded_packages/rversions_1.0.2.tar.gz’ saved [4654/4654]
--2015-07-29 12:02:29-- https://cran.rstudio.com/src/contrib/devtools_1.8.0.tar.gz
Resolving cran.rstudio.com (cran.rstudio.com)... 54.230.188.51
Connecting to cran.rstudio.com (cran.rstudio.com)|54.230.188.51|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 141487 (138K) [application/x-gzip]
Saving to: ‘/tmp/Rtmpcd8fcf/downloaded_packages/devtools_1.8.0.tar.gz’
0K .......... .......... .......... .......... .......... 36% 388K 0s
50K .......... .......... .......... .......... .......... 72% 487K 0s
100K .......... .......... .......... ........ 100% 440K=0.3s
2015-07-29 12:02:30 (434 KB/s) -
‘/tmp/Rtmpcd8fcf/downloaded_packages/devtools_1.8.0.tar.gz’ saved [141487/141487]
ERROR: dependency ‘xml2’ is not available for package ‘rversions’
* removing ‘/home/kingslee/R/x86_64-pc-linux-gnu-library/3.0/rversions’
Warning in install.packages :
installation of package ‘rversions’ had non-zero exit status
ERROR: dependency ‘rversions’ is not available for package ‘devtools’
* removing ‘/home/kingslee/R/x86_64-pc-linux-gnu-library/3.0/devtools’
Warning in install.packages :
installation of package ‘devtools’ had non-zero exit status
The downloaded source packages are in
‘/tmp/Rtmpcd8fcf/downloaded_packages’
You might to install that xml tool on the machine.
sudo apt-get -y install libxml2-dev
If that does indeed get rid of your error, you should follow this post for more instructions on how to get R properly set up on Ubuntu
I have a rhel 5.2 machine and I am trying to mount a ntfs file system harddisk. I know that we need ntfs and fuse rpms to mount it. By installing those rpms I mounted it in my rhel 5.4 machine. But in 5.2 I followed the same steps but failed to mount. Then after googling I found that 5.2 kernel doesn't support or have fuse kernel by default and then I followed this link.
Below is the output after doing the steps said in this link.
[root#racdb1 ~]# wget http://packages.sw.be/rpmforge-release/rpmforge-release-0. 3.6-1.el5.rf.i386.rpm
--14:14:20-- http://packages.sw.be/rpmforge-release/rpmforge-release-0.3.6-1.el 5.rf.i386.rpm
Resolving packages.sw.be... 78.46.17.228
Connecting to packages.sw.be|78.46.17.228|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: http://pkgs.repoforge.org/rpmforge-release/rpmforge-release-0.3.6-1.el 5.rf.i386.rpm [following]
--14:14:32-- http://pkgs.repoforge.org/rpmforge-release/rpmforge-release-0.3.6- 1.el5.rf.i386.rpm
Resolving pkgs.repoforge.org... 78.46.17.228
Connecting to pkgs.repoforge.org|78.46.17.228|:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: http://rpmforge.sw.be/redhat/el5/en/i386/rpmforge/RPMS/rpmforge-releas e-0.3.6-1.el5.rf.i386.rpm [following]
--14:14:39-- http://rpmforge.sw.be/redhat/el5/en/i386/rpmforge/RPMS/rpmforge-re lease-0.3.6-1.el5.rf.i386.rpm
Resolving rpmforge.sw.be... 78.46.17.228
Connecting to rpmforge.sw.be|78.46.17.228|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: http://tree.repoforge.org/redhat/el5/en/i386/rpmforge/RPMS/rpmforge-re lease-0.3.6-1.el5.rf.i386.rpm [following]
--14:14:51-- http://tree.repoforge.org/redhat/el5/en/i386/rpmforge/RPMS/rpmforg e-release-0.3.6-1.el5.rf.i386.rpm
Resolving tree.repoforge.org... 78.46.17.228
Connecting to tree.repoforge.org|78.46.17.228|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: http://apt.sw.be/redhat/el5/en/i386/rpmforge/RPMS/rpmforge-release-0.3 .6-1.el5.rf.i386.rpm [following]
--14:14:59-- http://apt.sw.be/redhat/el5/en/i386/rpmforge/RPMS/rpmforge-release -0.3.6-1.el5.rf.i386.rpm
Resolving apt.sw.be... 193.1.193.67
Connecting to apt.sw.be|193.1.193.67|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 16698 (16K) [application/x-redhat-package-manager]
Saving to: `rpmforge-release-0.3.6-1.el5.rf.i386.rpm'
100%[=======================================>] 16,698 14.4K/s in 1.1s
14:15:09 (14.4 KB/s) - `rpmforge-release-0.3.6-1.el5.rf.i386.rpm' saved [16698/1 6698]
[root#racdb1 ~]# rpm -Uhv rpmforge-release-0.3.6-1.el5.rf.i386.rpm
warning: rpmforge-release-0.3.6-1.el5.rf.i386.rpm: Header V3 DSA signature: NOKE Y, key ID 6b8d79e6
Preparing... ########################################### [100%]
1:rpmforge-release ########################################### [100%]
[root#racdb1 ~]# yum install fuse fuse-ntfs-3g dkms dkms-fuse
Loading "security" plugin
Loading "rhnplugin" plugin
This system is not registered with RHN.
RHN support will be disabled.
rpmforge 100% |=========================| 1.9 kB 00:00
primary.sqlite.bz2 100% |=========================| 7.3 MB 01:20
Setting up Install Process
Parsing package install arguments
No package fuse available.
No package dkms-fuse available.
Resolving Dependencies
--> Running transaction check
---> Package fuse-ntfs-3g.i386 0:2013.1.13-2.el5.rf set to be updated
--> Processing Dependency: fuse >= 2.6.3 for package: fuse-ntfs-3g
---> Package dkms.noarch 0:2.1.1.2-1.el5.rf set to be updated
--> Finished Dependency Resolution
**Error: Missing Dependency: fuse >= 2.6.3 is needed by package fuse-ntfs-3g**
I don't know what to do from the above error which I got. I searched in google but I didn't get the solution anywhere. How to solve the above issue?
I finally found the process to use ntfs file system in rhel 5.2. I followed the link.
http://oliphauntz.blogspot.in/2011/03/fix-how-to-mount-ntfs-filesystem-on.html#comment-form
In the above link the step " wget http://pbxinaflash.net/source/pogoplug/fuse-2.6.0.tar.gz" is not working. So I manually downloaded fuse-2.6.0 from internet and then installed and configured it. Try the same if anyone is facing the same problem which I faced.
I have ZTE Open with custom ROM Boot2Gecko 1.3.0.0-prerelease (Git 2013-10-19 22:09:07 d544afff51)
I'm building B2G v.1.2.
BRANCH=v1.2 ./config.sh inari
./build
Build finishes successful.
I'm flashing and see next:
./flash.sh
< waiting for device >
erasing 'cache'...
OKAY [ 0.530s]
finished. total time: 0.530s
erasing 'userdata'...
OKAY [ 1.405s]
finished. total time: 1.405s
sending 'userdata' (55044 KB)...
OKAY [ 5.074s]
writing 'userdata'...
OKAY [ 10.051s]
finished. total time: 15.125s
sending 'system' (81724 KB)...
OKAY [ 7.507s]
writing 'system'...
OKAY [ 14.973s]
finished. total time: 22.479s
rebooting...
finished. total time: 0.001s
Attempting to set the time on the device
time 1384896807 -> 1384896807.0
But my phone is frizzed on logoscreen
adb shell dmesg returns next: https://gist.github.com/blackbass1988/7559973
I'm building on MacosX 10.9
Strange, why build says that everything ok, but not ok
You used the adapted boot.img? Without this you will not be able to get a working system, just by using the build instructions on MDN. Here are some blogs that describe the build process:
https://blog.non.co.il/index.php/zte-open-phone-upgrading-to-firefoxos-1-1-how-to/
http://rowehl.com/blog/2013/10/24/firefoxos-1-dot-2-on-zte-open/
http://sl.edujose.org/2013/10/adapted-boot-image-for-use-with-b2g.html
Odd. Try reseting the cache from the system recovery.
ie : reboot, hold the power+volume up while it's restarting and you should get to the system recovery. move down to "wipe cache partition" w/ the volume down button and hit power button to confirm. Then reboot the device again and see if it starts up.