As a general rule, when you don't know how to do something with LibVLC, but you know how to it with the VLC app (and it works), open the logging windows (Tools => Messages), enable debug logging and reproduce your operation. The logs will tell you a story.
Now for authentication matters, I think you can provide the credentials directly in the URL (see how VLC does it, from the logs) and you can also probably implement libvlc_dialog_post_login which is raised by LibVLC when auth creds are required.