curl basic auth using base64 encoded credentials

I was trying to access password-protected files via HTTPS using curl. The site required basic auth. For a demo, I created this example:
Username: admin
Password: 123456

It’s trivial to access this interactively via curl:

$ curl -u admin
Enter host password for user 'admin':

Or programmatically by providing the credentials in the URL:

$ curl

Or by providing a base64-encoded username:password pair in an Authorization header:

$ curl -H "Authorization: Basic $(echo -n admin:123456 | base64)"

(Note that echo includes a trailing newline character by default, which we do not want to include in the base64-encoded value. Specify the -n flag to echo to eliminate the trailing newline.)

But I was manipulating files with a Bash script that was being stored in a Git repository, and I didn’t want to store the credentials in the repository. So I stored the credentials in a separate file:

$ echo -n 'admin:123456' > ~/admin-credentials
$ chmod 0600 ~/admin-credentials

Now I can read the credentials from the file:

$ curl -H "Authorization: Basic $(cat admin-credentials | base64)"

I ran into a problem when I tried to update the credentials file with vi (or vim). Vi automatically inserts an end-of-line (EOL) character, which is not apparent to the user. The base64-encoded value includes the EOL character, and therefore the above command would supply invalid credentials.

To eliminate this in vi, use the following vi commands:

:set binary
:set noeol

Alternately, just overwrite the file with the updated credentials:

$ echo -n 'admin:123456' > ~/admin-credentials

One thought on “curl basic auth using base64 encoded credentials”

  1. You can see the difference between the file with the EOL character and without in several ways:

    $ ls -l admin*
    -rw-r--r--  1 chris  chris  12 Jul  6 09:16 admin-credentials
    -rw-r--r--  1 chris  chris  13 Jul  6 09:16 admin-credentials-eol

    (12 bytes vs 13 bytes)

    $ wc admin-credentials
           0       1      12 admin-credentials
    $ wc admin-credentials-eol
           1       1      13 admin-credentials-eol

    Interesting to note that without the EOL, wc reports that the first file contains zero lines.

    $ xxd admin-credentials
    00000000: 6164 6d69 6e3a 3132 3334 3536            admin:123456
    $ xxd admin-credentials-eol 
    00000000: 6164 6d69 6e3a 3132 3334 3536 0a         admin:123456.

    Using xxd, you can see that the extra character is ASCII code 0a, the line feed (LF) character.

Leave a Reply

Your email address will not be published. Required fields are marked *