I was trying to access password-protected files via HTTPS using curl. The site required basic auth. For a demo, I created this example:
https://osric.com/chris/demo/admin/
Username: admin
Password: 123456
It’s trivial to access this interactively via curl
:
$ curl -u admin https://osric.com/chris/demo/admin/
Enter host password for user 'admin':
Or programmatically by providing the credentials in the URL:
$ curl https://admin:123456@osric.com/chris/demo/admin/
Or by providing a base64-encoded username:password
pair in an Authorization header:
$ curl -H "Authorization: Basic $(echo -n admin:123456 | base64)" https://osric.com/chris/demo/admin/
(Note that echo includes a trailing newline character by default, which we do not want to include in the base64-encoded value. Specify the -n
flag to echo
to eliminate the trailing newline.)
But I was manipulating files with a Bash script that was being stored in a Git repository, and I didn’t want to store the credentials in the repository. So I stored the credentials in a separate file:
$ echo -n 'admin:123456' > ~/admin-credentials
$ chmod 0600 ~/admin-credentials
Now I can read the credentials from the file:
$ curl -H "Authorization: Basic $(cat admin-credentials | base64)" https://osric.com/chris/demo/admin/
I ran into a problem when I tried to update the credentials file with vi
(or vim
). Vi automatically inserts an end-of-line (EOL) character, which is not apparent to the user. The base64-encoded value includes the EOL character, and therefore the above command would supply invalid credentials.
To eliminate this in vi
, use the following vi
commands:
:set binary
:set noeol
Alternately, just overwrite the file with the updated credentials:
$ echo -n 'admin:123456' > ~/admin-credentials
You can see the difference between the file with the EOL character and without in several ways:
(12 bytes vs 13 bytes)
Interesting to note that without the EOL,
wc
reports that the first file contains zero lines.Using
xxd
, you can see that the extra character is ASCII code0a
, the line feed (LF) character.