Renaming multiple files: replacing or truncating varied file extensions

In the previous post, I ran into an issue where Wget saved files to disk verbatim, including query strings/parameters. The files on disk ended up looking like this:

wp-includes/js/comment-reply.min.js?ver=6.4.2
wp-includes/js/jquery/jquery-migrate.min.js?ver=3.4.1
wp-includes/js/jquery/jquery.min.js?ver=3.7.1
wp-includes/css/dist/block-library/style.min.css?ver=6.4.2

I wanted to find a way to rename all these files, and truncate the filename after and including the question mark. As an example, to convert jquery.min.js?ver=3.7.1 to jquery.min.js.

I ended up creating a setup script to help test this because I was trying several different methods. The following just creates a directory structure with some realistic examples several directories deep in the hierarchy:

setup.sh

#!/bin/sh

mkdir -p demo/wp-includes/js/jquery
mkdir -p demo/wp-includes/css/dist/block-library
touch demo/wp-includes/js/comment-reply.min.js?ver=6.4.2
touch demo/wp-includes/js/jquery/jquery-migrate.min.js?ver=3.4.1
touch demo/wp-includes/js/jquery/jquery.min.js?ver=3.7.1
touch demo/wp-includes/css/dist/block-library/style.min.css?ver=6.4.2

I used the find command to find all the files in the demo directory (and subdirectories below) that include a question mark in the filename:

find ./demo -type f -name '*\?*'

In case the find command is not something you use every day:

-type f indicates it will look only for regular files, not directories or symlinks
-name '*\?*' will look for any files with a question mark in the name (preceded or followed by zero or more characters)

Method 1: short shell script
One way to update the filenames is by creating a quick shell script, rename.sh:

rename.sh:

#!/bin/sh

NEWFILE=$(echo "$1" | cut -f1 -d?)
mv "$1" "$NEWFILE"

exit 0

Then I can call the rename.sh shell script from the find command using the -exec or -execdir options:

find ./demo -type f -name '*\?*' -execdir ~/rename.sh '{}' \;

After that, I ran find again to confirm the filenames were changed:

find ./demo -type f
./demo/wp-includes/js/jquery/jquery.min.js
./demo/wp-includes/js/jquery/jquery-migrate.min.js
./demo/wp-includes/js/comment-reply.min.js
./demo/wp-includes/css/dist/block-library/style.min.css

That works, but I felt like this is something that could be achieved with a one-liner.

Method 2: while loop
In this case I passed the output of the find command to a while loop that manipulates the filename and moves the file from the old name to the new name:

find ./demo -type f -name '*\?*' | while read OLDFILE; do NEWFILE=$(echo $OLDFILE | cut -f1 -d?); mv $OLDFILE $NEWFILE; done

While that’s technically a one-liner, I found it clunky. It’s hard to read. I don’t want to say to someone, “Just do this,” and give them 125 characters.

Method 3: Perl
I was previously unaware of this, but ran into Perl’s rename function somewhere on a Stack Exchange site:

find ./demo -type f -name '*\?*' | perl -lne '($old=$_) && s/\?ver=.+$// && rename($old,$_)'

That takes some deciphering:

The -l option tells Perl to chomp the input (remove whitespace characters, including the newline)
The -n option tells Perl to loop over the input one line at a time
The -e option tells Perl to execute or evaluate the given command string (rather than loading and executing Perl code from a file)
Recall that in Perl, the default variable is $_. And, if a variable is not specified for an operation, it will operate on the default variable.

Perl seems to have fallen out of favor in the past 10-20 years, but it is still very good at some operations and is preinstalled on most Linux distributions. Still, that’s 93 characters and it’s not immediately evident what the code does.

Method 4: the rename command
There are two (or maybe more) rename commands. One is part of the util-linux package from the Linux Kernel Organization. It is pretty basic, although it can be useful. Another is a Perl-based rename command, which can use regular expressions. You can’t have two commands with the same name, so on some systems the Perl-based rename command is file-rename, on others it is prename (pre = Perl Regular Expressions).

It’s kind of a mess. For more details and discussion, check out:

I find the Perl-based rename command the simplest solution to the problem. It may not be installed by default:

sudo apt install rename

Now the command looks like this:

find ./demo -type f -name '*\?*' -execdir file-rename 's/\?.+$//' '{}' \;

74 characters! That feels more like a one-liner.

On How do I change the extension of multiple files?, a comment suggests using + instead of ; to terminate the find command. Since file-rename can take multiple files, + would pass them all instead of running file-rename over and over:

find ./demo -type f -name '*\?*' -execdir file-rename 's/\?.+$//' '{}' +

In my tests that worked perfectly. The link above points out that for an exceedingly long list of files you may encounter the error message arg list too long. I think you are unlikely to run into that under normal circumstances.

If you have other favorite methods or improvements upon the above, let me know in the comments.

Leave a Reply