In the previous post, I ran into an issue where Wget saved files to disk verbatim, including query strings\/parameters. The files on disk ended up looking like this:<\/p>\n
wp-includes\/js\/comment-reply.min.js?ver=6.4.2<\/code><\/li>\nwp-includes\/js\/jquery\/jquery-migrate.min.js?ver=3.4.1<\/code><\/li>\nwp-includes\/js\/jquery\/jquery.min.js?ver=3.7.1<\/code><\/li>\nwp-includes\/css\/dist\/block-library\/style.min.css?ver=6.4.2<\/code><\/li>\n<\/ul>\nI wanted to find a way to rename all these files, and truncate the filename after and including the question mark. As an example, to convert jquery.min.js?ver=3.7.1<\/code> to jquery.min.js<\/code>.<\/p>\n<\/p>\n
I ended up creating a setup script to help test this because I was trying several different methods. The following just creates a directory structure with some realistic examples several directories deep in the hierarchy:<\/p>\n
setup.sh<\/strong><\/p>\n#!\/bin\/sh\r\n\r\nmkdir -p demo\/wp-includes\/js\/jquery\r\nmkdir -p demo\/wp-includes\/css\/dist\/block-library\r\ntouch demo\/wp-includes\/js\/comment-reply.min.js?ver=6.4.2\r\ntouch demo\/wp-includes\/js\/jquery\/jquery-migrate.min.js?ver=3.4.1\r\ntouch demo\/wp-includes\/js\/jquery\/jquery.min.js?ver=3.7.1\r\ntouch demo\/wp-includes\/css\/dist\/block-library\/style.min.css?ver=6.4.2<\/code><\/pre>\nI used the find<\/code> command to find all the files in the demo<\/code> directory (and subdirectories below) that include a question mark in the filename:<\/p>\nfind .\/demo -type f -name '*\\?*'<\/code><\/pre>\nIn case the find<\/code> command is not something you use every day:<\/p>\n\n-type f<\/code> indicates it will look only for regular files, not directories or symlinks<\/li>\n-name '*\\?*'<\/code> will look for any files with a question mark in the name (preceded or followed by zero or more characters)<\/li>\n<\/ul>\nMethod 1: short shell script<\/strong>
\nOne way to update the filenames is by creating a quick shell script, rename.sh<\/code>:<\/p>\nrename.sh:<\/strong><\/p>\n#!\/bin\/sh\r\n\r\nNEWFILE=$(echo \"$1\" | cut -f1 -d?)\r\nmv \"$1\" \"$NEWFILE\"\r\n\r\nexit 0<\/code><\/pre>\nThen I can call the rename.sh<\/code> shell script from the find<\/code> command using the -exec<\/code> or -execdir<\/code> options:<\/p>\nfind .\/demo -type f -name '*\\?*' -execdir ~\/rename.sh '{}' \\;<\/code><\/pre>\nAfter that, I ran find again to confirm the filenames were changed:<\/p>\n
find .\/demo -type f\r\n.\/demo\/wp-includes\/js\/jquery\/jquery.min.js\r\n.\/demo\/wp-includes\/js\/jquery\/jquery-migrate.min.js\r\n.\/demo\/wp-includes\/js\/comment-reply.min.js\r\n.\/demo\/wp-includes\/css\/dist\/block-library\/style.min.css<\/code><\/pre>\nThat works, but I felt like this is something that could be achieved with a one-liner.<\/p>\n
Method 2: while loop<\/strong>
\nIn this case I passed the output of the find command to a while loop that manipulates the filename and moves the file from the old name to the new name:<\/p>\nfind .\/demo -type f -name '*\\?*' | while read OLDFILE; do NEWFILE=$(echo $OLDFILE | cut -f1 -d?); mv $OLDFILE $NEWFILE; done<\/code><\/pre>\nWhile that’s technically a one-liner, I found it clunky. It’s hard to read. I don’t want to say to someone, “Just do this,” and give them 125 characters.<\/p>\n
Method 3: Perl<\/strong>
\nI was previously unaware of this, but ran into Perl’s rename<\/code> function somewhere on a Stack Exchange site:<\/p>\nfind .\/demo -type f -name '*\\?*' | perl -lne '($old=$_) && s\/\\?ver=.+$\/\/ && rename($old,$_)'<\/code><\/pre>\nThat takes some deciphering:<\/p>\n
\n- The
-l<\/code> option tells Perl to chomp the input (remove whitespace characters, including the newline)<\/li>\n- The
-n<\/code> option tells Perl to loop over the input one line at a time<\/li>\n- The
-e<\/code> option tells Perl to execute or evaluate the given command string (rather than loading and executing Perl code from a file)<\/li>\n- Recall that in Perl, the default variable is $_. And, if a variable is not specified for an operation, it will operate on the default variable.<\/li>\n<\/ul>\n
Perl seems to have fallen out of favor in the past 10-20 years, but it is still very good at some operations and is preinstalled on most Linux distributions. Still, that’s 93 characters and it’s not immediately evident what the code does.<\/p>\n
Method 4: the rename command<\/strong>
\nThere are two (or maybe more) rename<\/code> commands. One is part of the util-linux<\/code> package from the Linux Kernel Organization. It is pretty basic, although it can be useful. Another is a Perl-based rename<\/code> command, which can use regular expressions. You can’t have two commands with the same name, so on some systems the Perl-based rename command is file-rename<\/code>, on others it is prename<\/code> (pre = Perl Regular Expressions).<\/p>\nIt’s kind of a mess. For more details and discussion, check out:<\/p>\n
\n- What are the different versions of the rename command? How do I use the Perl version?<\/a><\/li>\n
- What’s with all the renames: prename, rename, file-rename?<\/a><\/li>\n<\/ul>\n
I find the Perl-based rename<\/code> command the simplest solution to the problem. It may not be installed by default:<\/p>\nsudo apt install rename<\/code><\/pre>\nNow the command looks like this:<\/p>\n
find .\/demo -type f -name '*\\?*' -execdir file-rename 's\/\\?.+$\/\/' '{}' \\;<\/code><\/pre>\n74 characters! That feels more like a one-liner.<\/p>\n