"Can you get cp to give a progress bar like wget?"

  • 24 January, 2008
< zed0> can you get cp to give a progress bar like wget?

Damn right you can.

#!/bin/sh
cp_p()
{
   strace -q -ewrite cp -- "${1}" "${2}" 2>&1 \
      | awk '{
        count += $NF
            if (count % 10 == 0) {
               percent = count / total_size * 100
               printf "%3d%% [", percent
               for (i=0;i<=percent;i++)
                  printf "="
               printf ">"
               for (i=percent;i<100;i++)
                  printf " "
               printf "]\r"
            }
         }
         END { print "" }' total_size=$(stat -c '%s' "${1}") count=0
}

In action:

% cp_p /mnt/raid/pub/iso/debian/debian-2.2r4potato-i386-netinst.iso /dev/null
76% [===========================================>                    ]

Comments (35)

**KadBmF...**

Kopieren auf der Bash mit Fortschrittsbalken!
gefunden bei Chris Lamb
Als root:
die Befehlsdatei anlegen:
touch /usr/bin/cp_p
und folgenden Inhalt mit dem Liebligs-Editor einf&#252;gen:
#!/bin/sh
cp_p()
{
set -e
strace -q -ewrite cp -- "${1}" ...

Jan. 24, 2008, 4:56 a.m. #
anon

Grade-A bad-assery! My hat is off to you, sir!

Jan. 24, 2008, 5:24 a.m. #

why don't you use pv ?
and, this method works with -r ?

Jan. 24, 2008, 9:05 a.m. #
Artur G. Sibagatullin

I'm trying to use this script but dot'n fully understand some things in it. Where does NF variable get from?

Jan. 24, 2008, 9:35 a.m. #
Joshua

But doesn't strace make anything a billion times slower anyway?

Jan. 24, 2008, 9:49 a.m. #

Be aware, however, that this adds quite some overhead (30% to 50% in my tests) to the time needed to copy a file. A probably better way would be to use <code>dd</code> under the hood and get its progress status by <code>kill -USR1 $PID</code>. You lose all switches that <code>cp</code> understands, however.
In the end it would probably be better to add a switch to <code>p</code>.

Unfinished sample code (maybe Bashistic):
<code>
#!/bin/sh

if [ -d "$1" ]; then
echo "$1" is a directory - aborting.
exit
fi

if [ -d "$2" ]; then
OUTFILE="$2/"`basename "$1"`
else
OUTFILE="$2"
fi

dd if="$1" of="$OUTFILE" 2> &amp;
PID=`pidof -s dd`

while [[ -x "/proc/$PID" ]]; do
kill -USR1 "$PID"
sleep 1
done
</code>

Jan. 24, 2008, 11:26 a.m. #

Great!

Jan. 24, 2008, 12:28 p.m. #
David Dumortier

Mmh good hack but a nasty flea, printing of progression bar is on a new line every printing.
And I must admit I know none solution to this. Somebody have ?

Jan. 24, 2008, 12:31 p.m. #

sweet! i will check it out.

right now i have this with a patch for coreutils: http://forums.…

Jan. 24, 2008, 12:49 p.m. #

Fantastic! Scary, but fantastic!

Jan. 24, 2008, 1:54 p.m. #

you could do something similar with the fab "pv" tool:

pv bar
2.69MB 0:00:00 [70.3MB/s] [=================================>] 100%

Jan. 24, 2008, 2:12 p.m. #

That looks great. But some usage instructions would be helpful. Where do you put the script? How do you use it? If I paste that into gedit do I need to make it executable? etc

Apart from that, its a great idea :)

Jan. 24, 2008, 2:36 p.m. #

This is cool and all, but the overhead of strace make this not very useful. rsync -av shows a progress bar. You can also get a progress bar if you use scp locally. Yes, scp supports local file transfers without using any network or connecting to a local sshd instance.

jeff@omniscience:~$ du -h windows.tar.bz2
822M windows.tar.bz2

jeff@omniscience:~$ time cp_p windows.tar.bz2{,.new}
99% [====================================================================================================>]

real 2m18.957s
user 0m11.589s
sys 0m23.469s
jeff@omniscience:~$ ls window*
windows.tar.bz2 windows.tar.bz2.new
jeff@omniscience:~$ time cp windows.tar.bz2{,.new2}

real 1m39.597s
user 0m0.096s
sys 0m5.092s

Jan. 24, 2008, 6:52 p.m. #
lamby

If you feel the need to point out an alternative solution, then you have missed the entire point by a wide margin.

Jan. 24, 2008, 7:10 p.m. #
Steve

"...missed the entire point.."

Too right! :-)
Damn fine hack.

"AWK. Is there anything you cannot do."

Jan. 24, 2008, 9:37 p.m. #
Jorge Luis

I copy the function into my ~/.bashrc file. The problem is that when i try to use th cp_p command from a gnome-terminal, Terminal, xterm, etc. the commando print the update of the progress into a new line, and should do it on the same line, like show you example of use. Any solution?

Jan. 25, 2008, 12:28 p.m. #
Branden Robinson

Bwa ha ha ha ha ha.

Quite possibly the most wicked thing I have ever seen on Planet Debian.

And I do mean "wicked" in both the en_US@Boston sense and the ordinary one.

Bravo!

Jan. 26, 2008, 3:12 a.m. #

Muchas gracias, me ha servido de mucho. Siempre me pregunte como se podría hacer eso. es excelente!

Saludos!

Jan. 26, 2008, 3:43 p.m. #

omg this is genius! :)

kudos

Jan. 26, 2008, 9:14 p.m. #
Dan

nice.. thanks for this :D

Jan. 29, 2008, 11:55 a.m. #
doki_pen

what about something like this:

#!/bin/sh
curl "file://$1" -o "$2"

of course there could be a lot of improvements, but the idea is to use a tool that does progress. Something similar could probably be done with wget.

Jan. 29, 2008, 2:43 p.m. #
Heartsjava

A very fine hack

Jan. 29, 2008, 2:52 p.m. #
Stu

David Dumortier:

To fix the flea use ANSI (escape codes that is)
The first time display an extra newline before drawing the bar.

Every time the bar is drawn; draw the ANSI to move the cursor UP one character.

Jan. 29, 2008, 6:08 p.m. #
sw

Fine hack. Thanx.

Jan. 29, 2008, 7:01 p.m. #
glen

you can add -s 1 to strace to lessen strace output

Jan. 29, 2008, 7:14 p.m. #

Very fast and dirty perl script to accomplish the same effect. It sure needs some polish.... but the effect is there.

Copy paste this text to a file named cp2 and do ./cp2
May need to change the location of perl on the first line, or do perl cp2 . 70+ Character terminal.

#!/bin/perl
use POSIX':sys_wait_h';
fork||exec('/bin/cp',@ARGV);
my $b1=(stat'calltree')[7];
select(undef,undef,undef,0.05);
while(!waitpid-1,WNOHANG){
$b2=(stat'calltree2')[7];
$sd=int(100*($b2/$b1));
$ss=(100-$sd)/2;
$ss++if$sd%2;
$|=1;
print" $sd% [".'='x($sd/2-1).'>'.' 'x($ss-1)."]
";
print"^H"x80;
select(undef,undef,undef,0.1);
}

Jan. 29, 2008, 7:33 p.m. #

[...]Linux: ???????? ??????? (? ???????? ?????). ??????????, ??... ???????! ?.?. ???????? ?? ?, ?????? ????????? ??? ?????.[...]

Jan. 30, 2008, 9:42 a.m. #

Nice , very Nice :-)

//Jadu
http://unstabl…

March 7, 2008, 6:31 a.m. #
personb

David Dumortier:
If you use 'echo -e "\r"', the cursor will move back to the beginning of the line. You could blank it out with:
echo -e "\r"
for i in `seq 1 $COLUMNS`; do
echo -n " "
done
echo -e "\r"

, although it would probably be better if you replaced $COLUMNS with a calculated number, rather than just setting 80 or the environment setting.

Also, if you would like to use @Stu 's solution, the following code /should/ work:
echo -e "\x1B[1A"
Where \x1B = ASCII ESC, and [1A is the ANSI code for move cursor up 1 line (see http://isthe.c…).

Remember, all the above require -e to echo - otherwise the escape sequences (\r, \x) just get printed out literally.

Dec. 30, 2008, 10:52 a.m. #
You probably want to use `printf` instead of `echo -e`.
Peter

Nice bit it trickery there!

Does anyone have ideas on how to modify the script to handle file names that have spaces in them?

Feb. 2, 2009, 9:17 a.m. #
...
sak

This is exactly what i wanted.Cheers mate.

Feb. 20, 2009, 9:37 a.m. #
Darep

Holy crap! This is awesome.

May 18, 2009, 2:12 p.m. #

hahaha :)

March 4, 2010, 1:15 a.m. #

I wonder if monitoring a running cp through /proc/<pid>/fdinfo/... wouldn't cause much less overhead than using strace.

March 4, 2010, 6:53 a.m. #
lamby

Comments are closed as they are just too depressing to receive anymore.

March 4, 2010, 2:24 p.m. #