I was trying to access password-protected files via HTTPS using curl. The site required basic auth. For a demo, I created this example:
https://osric.com/chris/demo/admin/
Username: admin
Password: 123456
It’s trivial to access this interactively via curl:
$ curl -u admin https://osric.com/chris/demo/admin/
Enter host password for user 'admin':
Or programmatically by providing the credentials in the URL:
$ curl https://admin:123456@osric.com/chris/demo/admin/
Or by providing a base64-encoded username:password pair in an Authorization header:
$ curl -H "Authorization: Basic $(echo -n admin:123456 | base64)" https://osric.com/chris/demo/admin/
(Note that echo includes a trailing newline character by default, which we do not want to include in the base64-encoded value. Specify the -n flag to echo to eliminate the trailing newline.)
But I was manipulating files with a Bash script that was being stored in a Git repository, and I didn’t want to store the credentials in the repository. So I stored the credentials in a separate file:
$ echo -n 'admin:123456' > ~/admin-credentials
$ chmod 0600 ~/admin-credentials
Now I can read the credentials from the file:
$ curl -H "Authorization: Basic $(cat admin-credentials | base64)" https://osric.com/chris/demo/admin/
I ran into a problem when I tried to update the credentials file with vi (or vim). Vi automatically inserts an end-of-line (EOL) character, which is not apparent to the user. The base64-encoded value includes the EOL character, and therefore the above command would supply invalid credentials.
To eliminate this in vi, use the following vi commands:
:set binary
:set noeol
Alternately, just overwrite the file with the updated credentials:
$ echo -n 'admin:123456' > ~/admin-credentials
You can see the difference between the file with the EOL character and without in several ways:
(12 bytes vs 13 bytes)
Interesting to note that without the EOL,
wcreports that the first file contains zero lines.Using
xxd, you can see that the extra character is ASCII code0a, the line feed (LF) character.