OAuth support

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

OAuth support

t.r.stickland@sanger.ac.uk
I was doing a search of the documentation to see if there was any support for OAuth.

I was momentarily excited to see the "OauthUserAuthentication" option at https://genomearchitect.readthedocs.io/en/1.0.4/Database_setup/ until I noticed it was for a really old version.

I can find any mention in the current documentation, although I did find the "Other authentication strategies" section (https://genomearchitect.readthedocs.io/en/latest/Configure.html?highlight=remoteUserAuthenticatorService#other-authentication-strategies).  That mentions remoteUserAuthenticatorService which I guess is the successor to the RemoteUserAuthentication option in the old version -- but no mention of OAuth.    Also, this section is very short and I can't find any other information about enabling anything other than the built-in username & password authentication.

Am I missing something?  If anyone could point me in the right direction, I would be very grateful!

thanks

Tim

-- The Wellcome Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.

--
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: OAuth support

nathandunn

Tim, 

I updated the doc a little bit:  <a href="https://genomearchitect.readthedocs.io/en/latest/Configure.html?highlight=Remote User Authentication#other-authentication-strategies" class="">https://genomearchitect.readthedocs.io/en/latest/Configure.html?highlight=Remote%20User%20Authentication#other-authentication-strategies

Most of the folks that have used OAuth have an Apache proxy and the Remote User Authentication or have used the web services to inject names in directly: https://github.com/GMOD/Apollo/blob/develop/docs/Web_services.md#python-client

If you can’t do that, we can maybe meet offline to see how we can put together an OAuth solution that is somewhat generalizable (this has been on my todo-list for awhile - https://github.com/GMOD/Apollo/issues/136).    

Nathan


On Sep 9, 2019, at 9:08 AM, Tim Stickland <[hidden email]> wrote:

I was doing a search of the documentation to see if there was any support for OAuth.

I was momentarily excited to see the "OauthUserAuthentication" option at https://genomearchitect.readthedocs.io/en/1.0.4/Database_setup/ until I noticed it was for a really old version.

I can find any mention in the current documentation, although I did find the "Other authentication strategies" section (https://genomearchitect.readthedocs.io/en/latest/Configure.html?highlight=remoteUserAuthenticatorService#other-authentication-strategies).  That mentions remoteUserAuthenticatorService which I guess is the successor to the RemoteUserAuthentication option in the old version -- but no mention of OAuth.    Also, this section is very short and I can't find any other information about enabling anything other than the built-in username & password authentication.

Am I missing something?  If anyone could point me in the right direction, I would be very grateful!

thanks

Tim

-- The Wellcome Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.

--
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: OAuth support [EXT]

t.r.stickland@sanger.ac.uk
Thanks, those links are helpful.   The apache proxy/remote user authentication is probably the best place to start -- especially as I've been meaning to add an apache proxy anyway, as I know it's only a matter of time before I need rewrite rules, or have to serve static content, etc.

tim



On 09/09/2019 19:28, Nathan Dunn wrote:

Tim, 


Most of the folks that have used OAuth have an Apache proxy and the Remote User Authentication or have used the web services to inject names in directly: https://github.com/GMOD/Apollo/blob/develop/docs/Web_services.md#python-client [github.com]

If you can’t do that, we can maybe meet offline to see how we can put together an OAuth solution that is somewhat generalizable (this has been on my todo-list for awhile - https://github.com/GMOD/Apollo/issues/136 [github.com]).    

Nathan


On Sep 9, 2019, at 9:08 AM, Tim Stickland <[hidden email]> wrote:

I was doing a search of the documentation to see if there was any support for OAuth.

I was momentarily excited to see the "OauthUserAuthentication" option at https://genomearchitect.readthedocs.io/en/1.0.4/Database_setup/ [genomearchitect.readthedocs.io] until I noticed it was for a really old version.

I can find any mention in the current documentation, although I did find the "Other authentication strategies" section (https://genomearchitect.readthedocs.io/en/latest/Configure.html?highlight=remoteUserAuthenticatorService#other-authentication-strategies [genomearchitect.readthedocs.io]).  That mentions remoteUserAuthenticatorService which I guess is the successor to the RemoteUserAuthentication option in the old version -- but no mention of OAuth.    Also, this section is very short and I can't find any other information about enabling anything other than the built-in username & password authentication.

Am I missing something?  If anyone could point me in the right direction, I would be very grateful!

thanks

Tim

-- The Wellcome Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.



-- The Wellcome Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.

--
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: OAuth support [EXT]

t.r.stickland@sanger.ac.uk
In reply to this post by nathandunn
Nathan

I added an apache proxy, and then set up OIDC authentication using mod_auth_openidc, which was all tolerably straightforward -- but I can't get the Apollo remote user authentication working.

Unless I misread the galaxy documentation you referred me to,  the apache authentication should set a REMOTE_USER environment variable that contains the user's credentials.  I'm not quite clear what these are, but I think it should contain at least their openid, and  (if I configured the OIDC module correctly) their email address and some profile information should be present too?   Anyway, I get that this information has to be put into an HTTP header by the proxy so it is available to Apollo.

I think I have managed to do that correctly, but I can't really debug it as I'm not sure what the expected content of the header is.

I have added the apollo-config.groovy settings recommended in the documentation, except with Remote User Authentication active, i.e.

        [    "name":"Remote User Authenticator",
             "className":"remoteUserAuthenticatorService",
             "active":true,
        ]

I take it that the remoteUserAuthenticatorService expects a REMOTE_USER header?

I can't find any references than mention configuring this, an it seems too optimistic to hope it will Just Work -- at leats, I take my hat off to you if it does :)  Since the REMOTE_USER content could derive from a variety of apache modules,  and how it is presented in the HTTP header also seems to be variable, especially as the recommendations in the galaxy documentation suggest rewriting the environment variable.

Can you point me at any documentation that I've missed, or let me know what header(s) remoteUserAuthenticatorService expects, and what the contents should be?

Thanks again

tim



On 09/09/2019 19:28, Nathan Dunn wrote:

Tim, 


Most of the folks that have used OAuth have an Apache proxy and the Remote User Authentication or have used the web services to inject names in directly: https://github.com/GMOD/Apollo/blob/develop/docs/Web_services.md#python-client [github.com]

If you can’t do that, we can maybe meet offline to see how we can put together an OAuth solution that is somewhat generalizable (this has been on my todo-list for awhile - https://github.com/GMOD/Apollo/issues/136 [github.com]).    

Nathan


On Sep 9, 2019, at 9:08 AM, Tim Stickland <[hidden email]> wrote:

I was doing a search of the documentation to see if there was any support for OAuth.

I was momentarily excited to see the "OauthUserAuthentication" option at https://genomearchitect.readthedocs.io/en/1.0.4/Database_setup/ [genomearchitect.readthedocs.io] until I noticed it was for a really old version.

I can find any mention in the current documentation, although I did find the "Other authentication strategies" section (https://genomearchitect.readthedocs.io/en/latest/Configure.html?highlight=remoteUserAuthenticatorService#other-authentication-strategies [genomearchitect.readthedocs.io]).  That mentions remoteUserAuthenticatorService which I guess is the successor to the RemoteUserAuthentication option in the old version -- but no mention of OAuth.    Also, this section is very short and I can't find any other information about enabling anything other than the built-in username & password authentication.

Am I missing something?  If anyone could point me in the right direction, I would be very grateful!

thanks

Tim

-- The Wellcome Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.



-- The Wellcome Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.

--
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: OAuth support [EXT]

nathandunn

Tim, 

I think you add it explicitly in the apache config e.g., :


but I’ll see if I can’t bug the original authors to see if they have further insight.

Cheers,

Nathan


On Sep 17, 2019, at 9:42 AM, Tim Stickland <[hidden email]> wrote:

Nathan

I added an apache proxy, and then set up OIDC authentication using mod_auth_openidc, which was all tolerably straightforward -- but I can't get the Apollo remote user authentication working.

Unless I misread the galaxy documentation you referred me to,  the apache authentication should set a REMOTE_USER environment variable that contains the user's credentials.  I'm not quite clear what these are, but I think it should contain at least their openid, and  (if I configured the OIDC module correctly) their email address and some profile information should be present too?   Anyway, I get that this information has to be put into an HTTP header by the proxy so it is available to Apollo.

I think I have managed to do that correctly, but I can't really debug it as I'm not sure what the expected content of the header is.

I have added the apollo-config.groovy settings recommended in the documentation, except with Remote User Authentication active, i.e.

        [    "name":"Remote User Authenticator",
             "className":"remoteUserAuthenticatorService",
             "active":true,
        ]

I take it that the remoteUserAuthenticatorService expects a REMOTE_USER header?

I can't find any references than mention configuring this, an it seems too optimistic to hope it will Just Work -- at leats, I take my hat off to you if it does :)  Since the REMOTE_USER content could derive from a variety of apache modules,  and how it is presented in the HTTP header also seems to be variable, especially as the recommendations in the galaxy documentation suggest rewriting the environment variable.

Can you point me at any documentation that I've missed, or let me know what header(s) remoteUserAuthenticatorService expects, and what the contents should be?

Thanks again

tim



On 09/09/2019 19:28, Nathan Dunn wrote:

Tim, 


Most of the folks that have used OAuth have an Apache proxy and the Remote User Authentication or have used the web services to inject names in directly: https://github.com/GMOD/Apollo/blob/develop/docs/Web_services.md#python-client [github.com]

If you can’t do that, we can maybe meet offline to see how we can put together an OAuth solution that is somewhat generalizable (this has been on my todo-list for awhile - https://github.com/GMOD/Apollo/issues/136 [github.com]).    

Nathan


On Sep 9, 2019, at 9:08 AM, Tim Stickland <[hidden email]> wrote:

I was doing a search of the documentation to see if there was any support for OAuth.

I was momentarily excited to see the "OauthUserAuthentication" option at https://genomearchitect.readthedocs.io/en/1.0.4/Database_setup/ [genomearchitect.readthedocs.io] until I noticed it was for a really old version.

I can find any mention in the current documentation, although I did find the "Other authentication strategies" section (https://genomearchitect.readthedocs.io/en/latest/Configure.html?highlight=remoteUserAuthenticatorService#other-authentication-strategies [genomearchitect.readthedocs.io]).  That mentions remoteUserAuthenticatorService which I guess is the successor to the RemoteUserAuthentication option in the old version -- but no mention of OAuth.    Also, this section is very short and I can't find any other information about enabling anything other than the built-in username & password authentication.

Am I missing something?  If anyone could point me in the right direction, I would be very grateful!

thanks

Tim

-- The Wellcome Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.



-- The Wellcome Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.

--
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: OAuth support [EXT]

nathandunn


From one of our other users (thanks Cory):

We use a "RequestHeader set X-Remote-User %{REMOTE_USER}s" method to add the REMOTE USER header to our galaxy log in, and then the galaxy credentials are passed through https://github.com/erasche/gx-cookie-proxy that adds the expected header for Apollo. I think this ends up being the same header for our remote galaxy authentication but also adds the header for the galaxy managed users. 

I'm not sure if Oauth would require anything different, I'm not familiar enough with it.

I’m guessing you are using this:


It looks like it gets set automatically. 

Can you confirm / deny that the REMOTE_USER header is being sent to tomcat? 


Actually, I’m looking at the logging and you should see a lot of output related to logging in a remote user.  Can you share any logging info? 



Your configuration should look like this (so it uses RemoteUser first):

authentications = [
[
"name":"Remote User Authenticator",
"className":"remoteUserAuthenticatorService",
"active: true
],
[
"name":"Username Password Authenticator",
"className":"usernamePasswordAuthenticatorService",
"active": true
]
]

Thanks,

Nathan

On Sep 17, 2019, at 11:33 AM, Nathan Dunn <[hidden email]> wrote:


Tim, 

I think you add it explicitly in the apache config e.g., :


but I’ll see if I can’t bug the original authors to see if they have further insight.

Cheers,

Nathan


On Sep 17, 2019, at 9:42 AM, Tim Stickland <[hidden email]> wrote:

Nathan

I added an apache proxy, and then set up OIDC authentication using mod_auth_openidc, which was all tolerably straightforward -- but I can't get the Apollo remote user authentication working.

Unless I misread the galaxy documentation you referred me to,  the apache authentication should set a REMOTE_USER environment variable that contains the user's credentials.  I'm not quite clear what these are, but I think it should contain at least their openid, and  (if I configured the OIDC module correctly) their email address and some profile information should be present too?   Anyway, I get that this information has to be put into an HTTP header by the proxy so it is available to Apollo.

I think I have managed to do that correctly, but I can't really debug it as I'm not sure what the expected content of the header is.

I have added the apollo-config.groovy settings recommended in the documentation, except with Remote User Authentication active, i.e.

        [    "name":"Remote User Authenticator",
             "className":"remoteUserAuthenticatorService",
             "active":true,
        ]

I take it that the remoteUserAuthenticatorService expects a REMOTE_USER header?

I can't find any references than mention configuring this, an it seems too optimistic to hope it will Just Work -- at leats, I take my hat off to you if it does :)  Since the REMOTE_USER content could derive from a variety of apache modules,  and how it is presented in the HTTP header also seems to be variable, especially as the recommendations in the galaxy documentation suggest rewriting the environment variable.

Can you point me at any documentation that I've missed, or let me know what header(s) remoteUserAuthenticatorService expects, and what the contents should be?

Thanks again

tim



On 09/09/2019 19:28, Nathan Dunn wrote:

Tim, 


Most of the folks that have used OAuth have an Apache proxy and the Remote User Authentication or have used the web services to inject names in directly: https://github.com/GMOD/Apollo/blob/develop/docs/Web_services.md#python-client [github.com]

If you can’t do that, we can maybe meet offline to see how we can put together an OAuth solution that is somewhat generalizable (this has been on my todo-list for awhile - https://github.com/GMOD/Apollo/issues/136 [github.com]).    

Nathan


On Sep 9, 2019, at 9:08 AM, Tim Stickland <[hidden email]> wrote:

I was doing a search of the documentation to see if there was any support for OAuth.

I was momentarily excited to see the "OauthUserAuthentication" option at https://genomearchitect.readthedocs.io/en/1.0.4/Database_setup/ [genomearchitect.readthedocs.io] until I noticed it was for a really old version.

I can find any mention in the current documentation, although I did find the "Other authentication strategies" section (https://genomearchitect.readthedocs.io/en/latest/Configure.html?highlight=remoteUserAuthenticatorService#other-authentication-strategies [genomearchitect.readthedocs.io]).  That mentions remoteUserAuthenticatorService which I guess is the successor to the RemoteUserAuthentication option in the old version -- but no mention of OAuth.    Also, this section is very short and I can't find any other information about enabling anything other than the built-in username & password authentication.

Am I missing something?  If anyone could point me in the right direction, I would be very grateful!

thanks

Tim

-- The Wellcome Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.



-- The Wellcome Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.


--
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: OAuth support [EXT]

t.r.stickland@sanger.ac.uk
Thanks a lot for this.

You correct that I'm using mod_auth_openidc, and I can see a REMOTE_USER is set in the apache environment (plus various other variables, including the user's name).  For instance:
[hidden email]
OIDC_CLAIM_given_name=Tim
OIDC_CLAIM_iss=https://orcid.org
OIDC_CLAIM_family_name=Stickland
OIDC_CLAIM_id=https://orcid.org/0000-0003-3185-1433
OIDC_CLAIM_sub=0000-0003-3185-1433
plus various token values, timestamps and so on.

As you said, mod_auth_openidc should send a header as well as setting the REMOTE_USER environment variable, but I explicitly set the configuration option ("OIDCPassClaimsAs  both") anyway.      I also tried "RequestHeader  set   X-Remote-User  expr=%{REMOTE_USER}" in the apache proxy configuration.

The behaviour remains as before:  I  have to do an openid log in to get to apollo, but thereafter I get the usual username/password dialog within apollo.

I can't see any relevant output in the logs, except thesde 2 lines for each log in attempt:
2019-09-18 16:18:20,049 [http-nio-8080-exec-4] ERROR apollo.PermissionService  - Username not supplied so can not authenticate.
2019-09-18 16:18:20,172 [http-nio-8080-exec-1] ERROR apollo.PermissionService  - Username not supplied so can not authenticate.
(I think this is expected when an unauthorized user connects?)

 I'm not familiar with tomcat so maybe I'm looking in the wrong place?    The only logs (with content) are 'catalina' and 'localhost', but neither have any output after the usual server startup messages.   Any more tips on where to look, or logging options I can turn on?

I'm a bit stumped at the moment because I can see HTTP requests during the openid authentication process, and the data retrieved in the apache environment, but ATM I can't find a way to check what apollo (i.e. tomcat) is receiving, and I don't know what it requires :-/

tim


On 17/09/2019 20:30, Nathan Dunn wrote:


From one of our other users (thanks Cory):

We use a "RequestHeader set X-Remote-User %{REMOTE_USER}s" method to add the REMOTE USER header to our galaxy log in, and then the galaxy credentials are passed through https://github.com/erasche/gx-cookie-proxy [github.com] that adds the expected header for Apollo. I think this ends up being the same header for our remote galaxy authentication but also adds the header for the galaxy managed users. 

I'm not sure if Oauth would require anything different, I'm not familiar enough with it.

I’m guessing you are using this:


It looks like it gets set automatically. 

Can you confirm / deny that the REMOTE_USER header is being sent to tomcat? 


Actually, I’m looking at the logging and you should see a lot of output related to logging in a remote user.  Can you share any logging info? 



Your configuration should look like this (so it uses RemoteUser first):

authentications = [
    [
        "name":"Remote User Authenticator",
        "className":"remoteUserAuthenticatorService",
        "active: true
    ],
    [
        "name":"Username Password Authenticator",
        "className":"usernamePasswordAuthenticatorService",
        "active": true
    ]
]

Thanks,

Nathan

On Sep 17, 2019, at 11:33 AM, Nathan Dunn <[hidden email]> wrote:


Tim, 

I think you add it explicitly in the apache config e.g., :


but I’ll see if I can’t bug the original authors to see if they have further insight.

Cheers,

Nathan


On Sep 17, 2019, at 9:42 AM, Tim Stickland <[hidden email]> wrote:

Nathan

I added an apache proxy, and then set up OIDC authentication using mod_auth_openidc, which was all tolerably straightforward -- but I can't get the Apollo remote user authentication working.

Unless I misread the galaxy documentation you referred me to,  the apache authentication should set a REMOTE_USER environment variable that contains the user's credentials.  I'm not quite clear what these are, but I think it should contain at least their openid, and  (if I configured the OIDC module correctly) their email address and some profile information should be present too?   Anyway, I get that this information has to be put into an HTTP header by the proxy so it is available to Apollo.

I think I have managed to do that correctly, but I can't really debug it as I'm not sure what the expected content of the header is.

I have added the apollo-config.groovy settings recommended in the documentation, except with Remote User Authentication active, i.e.

        [    "name":"Remote User Authenticator",
             "className":"remoteUserAuthenticatorService",
             "active":true,
        ]

I take it that the remoteUserAuthenticatorService expects a REMOTE_USER header?

I can't find any references than mention configuring this, an it seems too optimistic to hope it will Just Work -- at leats, I take my hat off to you if it does :)  Since the REMOTE_USER content could derive from a variety of apache modules,  and how it is presented in the HTTP header also seems to be variable, especially as the recommendations in the galaxy documentation suggest rewriting the environment variable.

Can you point me at any documentation that I've missed, or let me know what header(s) remoteUserAuthenticatorService expects, and what the contents should be?

Thanks again

tim



On 09/09/2019 19:28, Nathan Dunn wrote:

Tim, 


Most of the folks that have used OAuth have an Apache proxy and the Remote User Authentication or have used the web services to inject names in directly: https://github.com/GMOD/Apollo/blob/develop/docs/Web_services.md#python-client [github.com]

If you can’t do that, we can maybe meet offline to see how we can put together an OAuth solution that is somewhat generalizable (this has been on my todo-list for awhile - https://github.com/GMOD/Apollo/issues/136 [github.com]).    

Nathan


On Sep 9, 2019, at 9:08 AM, Tim Stickland <[hidden email]> wrote:

I was doing a search of the documentation to see if there was any support for OAuth.

I was momentarily excited to see the "OauthUserAuthentication" option at https://genomearchitect.readthedocs.io/en/1.0.4/Database_setup/ [genomearchitect.readthedocs.io] until I noticed it was for a really old version.

I can find any mention in the current documentation, although I did find the "Other authentication strategies" section (https://genomearchitect.readthedocs.io/en/latest/Configure.html?highlight=remoteUserAuthenticatorService#other-authentication-strategies [genomearchitect.readthedocs.io]).  That mentions remoteUserAuthenticatorService which I guess is the successor to the RemoteUserAuthentication option in the old version -- but no mention of OAuth.    Also, this section is very short and I can't find any other information about enabling anything other than the built-in username & password authentication.

Am I missing something?  If anyone could point me in the right direction, I would be very grateful!

thanks

Tim

-- The Wellcome Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.



-- The Wellcome Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.




-- The Wellcome Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.

--
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: OAuth support [EXT]

nathandunn

Tim, 

Everything looks correct AFAICT.   For some reason its not reading the JSON through. 

Two things that would really help me.   

  (`debug grails.app` and debug ‘org.bbop.apollo’) and redo the login attempt. 

2 - If you “select * from grails_user;” what do you see?

Nathan


On Sep 18, 2019, at 8:28 AM, Tim Stickland <[hidden email]> wrote:

Thanks a lot for this.

You correct that I'm using mod_auth_openidc, and I can see a REMOTE_USER is set in the apache environment (plus various other variables, including the user's name).  For instance:
[hidden email]
OIDC_CLAIM_given_name=Tim
OIDC_CLAIM_iss=https://orcid.org
OIDC_CLAIM_family_name=Stickland
OIDC_CLAIM_id=https://orcid.org/0000-0003-3185-1433
OIDC_CLAIM_sub=0000-0003-3185-1433
plus various token values, timestamps and so on.

As you said, mod_auth_openidc should send a header as well as setting the REMOTE_USER environment variable, but I explicitly set the configuration option ("OIDCPassClaimsAs  both") anyway.      I also tried "RequestHeader  set   X-Remote-User  expr=%{REMOTE_USER}" in the apache proxy configuration.

The behaviour remains as before:  I  have to do an openid log in to get to apollo, but thereafter I get the usual username/password dialog within apollo.

I can't see any relevant output in the logs, except thesde 2 lines for each log in attempt:
2019-09-18 16:18:20,049 [http-nio-8080-exec-4] ERROR apollo.PermissionService  - Username not supplied so can not authenticate.
2019-09-18 16:18:20,172 [http-nio-8080-exec-1] ERROR apollo.PermissionService  - Username not supplied so can not authenticate.
(I think this is expected when an unauthorized user connects?)

 I'm not familiar with tomcat so maybe I'm looking in the wrong place?    The only logs (with content) are 'catalina' and 'localhost', but neither have any output after the usual server startup messages.   Any more tips on where to look, or logging options I can turn on?

I'm a bit stumped at the moment because I can see HTTP requests during the openid authentication process, and the data retrieved in the apache environment, but ATM I can't find a way to check what apollo (i.e. tomcat) is receiving, and I don't know what it requires :-/

tim


On 17/09/2019 20:30, Nathan Dunn wrote:


From one of our other users (thanks Cory):

We use a "RequestHeader set X-Remote-User %{REMOTE_USER}s" method to add the REMOTE USER header to our galaxy log in, and then the galaxy credentials are passed through https://github.com/erasche/gx-cookie-proxy [github.com] that adds the expected header for Apollo. I think this ends up being the same header for our remote galaxy authentication but also adds the header for the galaxy managed users. 

I'm not sure if Oauth would require anything different, I'm not familiar enough with it.

I’m guessing you are using this:


It looks like it gets set automatically. 

Can you confirm / deny that the REMOTE_USER header is being sent to tomcat? 


Actually, I’m looking at the logging and you should see a lot of output related to logging in a remote user.  Can you share any logging info? 



Your configuration should look like this (so it uses RemoteUser first):

authentications = [
    [
        "name":"Remote User Authenticator",
        "className":"remoteUserAuthenticatorService",
        "active: true
    ],
    [
        "name":"Username Password Authenticator",
        "className":"usernamePasswordAuthenticatorService",
        "active": true
    ]
]

Thanks,

Nathan

On Sep 17, 2019, at 11:33 AM, Nathan Dunn <[hidden email]> wrote:


Tim, 

I think you add it explicitly in the apache config e.g., :


but I’ll see if I can’t bug the original authors to see if they have further insight.

Cheers,

Nathan


On Sep 17, 2019, at 9:42 AM, Tim Stickland <[hidden email]> wrote:

Nathan

I added an apache proxy, and then set up OIDC authentication using mod_auth_openidc, which was all tolerably straightforward -- but I can't get the Apollo remote user authentication working.

Unless I misread the galaxy documentation you referred me to,  the apache authentication should set a REMOTE_USER environment variable that contains the user's credentials.  I'm not quite clear what these are, but I think it should contain at least their openid, and  (if I configured the OIDC module correctly) their email address and some profile information should be present too?   Anyway, I get that this information has to be put into an HTTP header by the proxy so it is available to Apollo.

I think I have managed to do that correctly, but I can't really debug it as I'm not sure what the expected content of the header is.

I have added the apollo-config.groovy settings recommended in the documentation, except with Remote User Authentication active, i.e.

        [    "name":"Remote User Authenticator",
             "className":"remoteUserAuthenticatorService",
             "active":true,
        ]

I take it that the remoteUserAuthenticatorService expects a REMOTE_USER header?

I can't find any references than mention configuring this, an it seems too optimistic to hope it will Just Work -- at leats, I take my hat off to you if it does :)  Since the REMOTE_USER content could derive from a variety of apache modules,  and how it is presented in the HTTP header also seems to be variable, especially as the recommendations in the galaxy documentation suggest rewriting the environment variable.

Can you point me at any documentation that I've missed, or let me know what header(s) remoteUserAuthenticatorService expects, and what the contents should be?

Thanks again

tim



On 09/09/2019 19:28, Nathan Dunn wrote:

Tim, 


Most of the folks that have used OAuth have an Apache proxy and the Remote User Authentication or have used the web services to inject names in directly: https://github.com/GMOD/Apollo/blob/develop/docs/Web_services.md#python-client [github.com]

If you can’t do that, we can maybe meet offline to see how we can put together an OAuth solution that is somewhat generalizable (this has been on my todo-list for awhile - https://github.com/GMOD/Apollo/issues/136 [github.com]).    

Nathan


On Sep 9, 2019, at 9:08 AM, Tim Stickland <[hidden email]> wrote:

I was doing a search of the documentation to see if there was any support for OAuth.

I was momentarily excited to see the "OauthUserAuthentication" option at https://genomearchitect.readthedocs.io/en/1.0.4/Database_setup/ [genomearchitect.readthedocs.io] until I noticed it was for a really old version.

I can find any mention in the current documentation, although I did find the "Other authentication strategies" section (https://genomearchitect.readthedocs.io/en/latest/Configure.html?highlight=remoteUserAuthenticatorService#other-authentication-strategies [genomearchitect.readthedocs.io]).  That mentions remoteUserAuthenticatorService which I guess is the successor to the RemoteUserAuthentication option in the old version -- but no mention of OAuth.    Also, this section is very short and I can't find any other information about enabling anything other than the built-in username & password authentication.

Am I missing something?  If anyone could point me in the right direction, I would be very grateful!

thanks

Tim

-- The Wellcome Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.



-- The Wellcome Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.




-- The Wellcome Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.

--
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: OAuth support [EXT]

nathandunn
Just wanted to share with the list in case anyone has more experience with remote-user forwarding through Apache.

Nathan

On Sep 23, 2019, at 9:01 AM, Tim Stickland <[hidden email]> wrote:

You  are right I think, but it's complicated slightly, as I was using REMOTE_USER (or trying to use it) in addition to X-Remote-User.  The value of X-Remote-User was being sent in the request, but the value of REMOTE_USER was not being send -- or at least, it wasn't making it to any place the Apollo code could read it from.

The problem seems to be that apache is dropping any header with an underscore.  I thought it was  only allowing me to set X-Remote-User headers because of the X- prefix, but actually it was dropping REMOTE_USER because of the underscore.    (It doesn't help to clarify things, that when headers are transformed into apache environment variables, all hyphens are transformed into underscores -- so you can't see the difference between these header names, when you look at the values that were sent...)

So:   I should be able to set REMOTE-USER (or, indeed X-Remote-User) but RemoteAurthenticatorService.groovy will need a change.  In my Apollo fork I tweaked it so that if REMOTE_USER isn't found, it will fall back on REMOTE-USER and then X-Remote-User.  If this works I'll send a PR, as this may be something other people encounter.

I have to go now, but I'll leave the build running and let you know tomorrow if it worked.

Fingers crossed :)

Tim


On 20/09/2019 17:41, Nathan Dunn wrote:

Tim,

I’m about 90% sure its because you are using HTTP_X_REMOTE_USER instead of REMOTE_USER. 

That being said, it's just a line of code:

If you change this to:


To 

remoteUser = request.getHeader(“HTTP_X_REMOTE_USER")

It should work.   If it doesn’t, I would do this right before it:

log.debug "headers "
request.getHeaderNames().each { log.debug it }
remoteUser = request.getHeader(“HTTP_X_REMOTE_USER")

Let me know the status when you are able to test it next week.

Have a great weekend.  

Nathan


On Sep 20, 2019, at 8:55 AM, Tim Stickland <[hidden email]> wrote:

On 19/09/2019 17:24, Nathan Dunn wrote:
On Sep 19, 2019, at 9:11 AM, Tim Stickland <[hidden email]> wrote:

... The only logs files recording anything are are the 'catalina' and 'localhost', but neither are catching any debug messages.  There's also the docker logs (we run tomcat in a docker container, so various messages are written to STDOUT which are captured in the docker logs) where the "username not supplied" messages appear, but there's no extra output there either :(

Ah . . .its going to be in the docker logs then.   You will have to edit the apollo-config.groovy and rebuild and deploy the war file.

turns out I had put the logging options in the wrong section on the groovy config file (doh!), so I moved them to the correct place and I'm now seeing the messages in the docker logs

I can also redirect successful openid authentication attempts at a CGI script to see what is being passed, which could help.   I was planning on doing this anyway, as if I start to add more identity providers then I need to be able to check up on what's being returned!

This is not a bad idea.

This was quite enlightening.  I can see that X-Remote_User is passed.   I added two virtual hosts, with script to display details of the requests.  One was just served directed from apache, and the other was proxied (the same proxy settings as used upstream of tomcat, but proxying onto my script) so I could see what differed downstream of the proxy.

The good news in the proxy doesn't seem to be doing anything it shouldn't.  It adds some HTTP_X_FORWARDED_<stuff> headers, as expected, but otherwise it's the same as upstream of the proxy.

I did notice that I can't set a REMOTE_USER header, but I can set X_REMOTE_USER.  I guess that's an apache constraint, only allowing headers with the X- prefix to be added?  Hopefully this doesn't matter, as it is what Cory said they were using.   FWIW I'm seeing
HTTP_X_REMOTE_USER=[hidden email]
Now here the odd part; the messages I see when I access Apollo are...
 2019-09-20 16:51:34,262 [http-nio-8080-exec-2] DEBUG apollo.AnnotatorController  - loading the index
 2019-09-20 16:51:35,538 [http-nio-8080-exec-4] DEBUG apollo.PreferenceService  - PS: getCurrentOrganismForCurrentUser 2052387963773638283506968216
 2019-09-20 16:51:35,567 [http-nio-8080-exec-4] DEBUG apollo.PreferenceService  - No session found
 2019-09-20 16:51:35,567 [http-nio-8080-exec-6] DEBUG apollo.PermissionService  - No session found
 2019-09-20 16:51:35,571 [http-nio-8080-exec-4] DEBUG apollo.PreferenceService  - found organism in session null so returning
 2019-09-20 16:51:35,582 [http-nio-8080-exec-5] DEBUG apollo.PermissionService  - dataObject does not contain organism (may not be needed)
 2019-09-20 16:51:35,589 [http-nio-8080-exec-4] WARN  apollo.PreferenceService  - No user present, so using the client token
 2019-09-20 16:51:35,601 [http-nio-8080-exec-4] DEBUG apollo.PreferenceService  - token for org 2052387963773638283506968216
 2019-09-20 16:51:35,609 [http-nio-8080-exec-4] DEBUG apollo.PreferenceService  - is NOT long 
 2019-09-20 16:51:35,606 [http-nio-8080-exec-7] DEBUG apollo.PreferenceService  - Evaluating saves: true
 2019-09-20 16:51:35,614 [http-nio-8080-exec-7] DEBUG apollo.PreferenceService  - Saving with time diff: 70607
 2019-09-20 16:51:35,620 [http-nio-8080-exec-5] DEBUG apollo.PermissionService  - creating session with found json object null, null
 2019-09-20 16:51:35,623 [http-nio-8080-exec-5] ERROR apollo.PermissionService  - Username not supplied so can not authenticate.
 2019-09-20 16:51:35,624 [http-nio-8080-exec-5] DEBUG apollo.PermissionService  - User not logged in
 2019-09-20 16:51:35,645 [http-nio-8080-exec-7] INFO  apollo.PermissionService  - Authenticating with remoteUserAuthenticatorService
 2019-09-20 16:51:35,693 [http-nio-8080-exec-7] WARN  authenticator.RemoteUserAuthenticatorService  - Remote user found []
 2019-09-20 16:51:35,694 [http-nio-8080-exec-7] WARN  authenticator.RemoteUserAuthenticatorService  - No remote user passed in header!
 2019-09-20 16:51:35,694 [http-nio-8080-exec-7] INFO  apollo.PermissionService  - Authenticating with usernamePasswordAuthenticatorService
 2019-09-20 16:51:35,713 [http-nio-8080-exec-7] WARN  apollo.PermissionService  - Failed to authenticate user
 2019-09-20 16:51:35,726 [http-nio-8080-exec-8] DEBUG apollo.PermissionService  - dataObject does not contain organism (may not be needed)
 2019-09-20 16:51:35,727 [http-nio-8080-exec-8] DEBUG apollo.PermissionService  - creating session with found json object null, null
 2019-09-20 16:51:35,727 [http-nio-8080-exec-8] ERROR apollo.PermissionService  - Username not supplied so can not authenticate.
 2019-09-20 16:51:35,728 [http-nio-8080-exec-8] DEBUG apollo.PermissionService  - User not logged in
My interpretation is that it is reading a header, hence the "Remote user found []" message, but that it's an empty string, hence the "No remote user passed in header!" message, and the subsequent fall back to usernamePasswordAuthenticatorService.

Would be curious to see how this works with multiple identity providers.   Not sure if apache, or a simple node client might be easiest with all of the pre-baked solutions there. 

mod_auth_openidc claims to support multiple providers, though I haven't tried it yet...

It's getting late here (for a Friday), so this is me signing off until next week.

Any thoughts would be very welcome...

thanks again

tim



-- The Wellcome Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.



-- The Wellcome Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.

--
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: OAuth support [EXT]

t.r.stickland@sanger.ac.uk


On 26/09/2019 09:50, Helena Rasche wrote:

>>>
>>> The problem seems to be that apache is dropping any header with an
>>> underscore.  I thought it was  only allowing me to set X-Remote-User
>>> headers because of the X- prefix, but actually it was dropping
>>> REMOTE_USER because of the underscore.
>
> In my experience, the only reason that header was dropped, was if
> there was another proxy between your apache server which sets it, and
> your tomcat. That header is "protected" somewhat, and anything that is
> responsible for proxying will drop it if it's incoming? Don't know if
> that's the case for you, but it sounds to me quite odd that you can't
> set that header, and only an X- prefixed one.
>

There's definitely only one proxy.  These are services on a docker
network; there's only one tomcat and one httpd on the network, and the
httpd is the only container with a port exposed to the outside world.

I'm no longer sure whether the problem is with the RequestHeader in my
httpd configuration, or whether it's the expression I am using
(expr=%{REMOTE_USER}) to assign the remote user value to the header
(%{REMOTE_USER}e doesn't work for me, possibly as it's httpd 2.4.41).

>>> (It doesn't help to clarify things, that when headers are
>>> transformed into apache environment variables, all hyphens are
>>> transformed into underscores -- so you can't see the difference
>>> between these header names, when you look at the values that were
>>> sent...)
>
> Have you used tcpdump? I've found it super useful for debugging these
> sort of issues. e.g.
>
> On your machine with tomcat, as root, run tcpdump -i any -vvv -A 'dst
> port 8080', changing 8080 to whichever port your tomcat is listening
> on. Then if you access a URL like .../annotator/getAppState in a
> browser (so you just see one request at a time, loading the normal
> apollo interface can produce an overwhelming amount of logs:
>

Great, thanks, that's a helpful tip.  I'll try it and report back.

tim



--
 The Wellcome Sanger Institute is operated by Genome Research
 Limited, a charity registered in England with number 1021457 and a
 company registered in England with number 2742969, whose registered
 office is 215 Euston Road, London, NW1 2BE.

--
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].

Reply | Threaded
Open this post in threaded view
|

Re: OAuth support [EXT]

t.r.stickland@sanger.ac.uk
On 26/09/2019 14:33, Tim Stickland wrote:
On 26/09/2019 09:50, Helena Rasche wrote:

Have you used tcpdump? I've found it super useful for debugging these sort of issues. e.g.

On your machine with tomcat, as root, run tcpdump -i any -vvv -A 'dst port 8080', changing 8080 to whichever port your tomcat is listening on. Then if you access a URL like .../annotator/getAppState in a browser (so you just see one request at a time, loading the normal apollo interface can produce an overwhelming amount of logs:


Great, thanks, that's a helpful tip.  I'll try it and report back.

Following a bit of a fight to get tcpdump working on a docker network, this revealed the problem :)

Summary: it should all work fine if your entire Apollo installation is under access control, but is more complicated if you want parts (/jbrowse, say) to be public.

The ugly details...

Passing REMOTE_USER to tomcat from an a apache httpd proxy is not a problem.  It wasn't working when I had an apache httpd downstream of the proxy (which, ironically, I had added to help me debug this:-/), but that's something to do with the downstream httpd, not the proxy httpd.     So for the time being at least,  it looks like it's fine to use an apache httpd proxy with tomcat downstream of it.   This is worth watching though;  if you google it, you'll see quite a lot of people have encountered problems with REMOTE_USER, in nginx as well as apache httpd, so that's one of the first places I'd look if you upgrade a web server/proxy/load balancer and your authentication breaks.

I'm using mod_auth_openidc in my apache proxy.  This will sets the REMOTE_USER environment variable for all requests under access control, and I can pass this downstream to tomcat with this:

    RequestHeader set REMOTE_USER "expr=%{REMOTE_USER}"

Of course this means that REMOTE_USER is always sent, but it will have no content unless REMOTE_USER has a value in the apache environment on the proxy.  That is only the case when the request is under access control, of course.    If you have public content, mod_auth_openidc isn't invoked when that is requested, and REMOTE_USER is never set.

This seems like it wouldn't be an issue.   After all, if something isn't under access control, then the user won't be asked to log in, so there's no need for REMOTE_USER to be set?

The complication is that the OAuth callback request (/annotator/index) does not cause the grails app calls RemoteAuthenticatorService.   That's done by a request to /user, which is an API call (so you have to look pretty closely to find it).

If you have the entire server under access control, that shouldn't matter -- but if you want to keep some parts open (like /jbrowse) then you need to be careful about setting up access control, so that it includes not just what you want to restrict, but also the API calls which require authentication-related headers.

I hope this explanation makes some sort of sense.

This is the apache configuration I have ATM...
   LoadModule auth_openidc_module /usr/lib/apache2/modules/mod_auth_openidc.so

   # these OIDC... headers are all standard mod_auth_openidc settings
   OIDCProviderMetadataURL <oauth provider metadata URL>
   OIDCClientID            <my client ID>
   OIDCClientSecret        <my client secret>
   OIDCScope               "openid email profile"
   # vanity URL, must point to a path protected by this module but must NOT point to any content
   OIDCRedirectURI         <my apollo host>/annotator/openid/foo
   OIDCCryptoPassphrase    <random password

   # this prompts a log in when the user accesses apollo
   <Location /annotator>
      AuthType       openid-connect
      Require        valid-user
   </Location>

   # required to get mod_auth_openid to pass headers with /user requests
   <Location /user>
      AuthType       openid-connect
      Require        valid-user
   </Location>

   RequestHeader     set REMOTE_USER "expr=%{REMOTE_USER}"

   # <tomcat host> is accessible only on the internal network
   ProxyPass         /stomp/info    http://<tomcat host>:8080/stomp/info
   ProxyPassReverse  /stomp/info    http://<tomcat host>:8080/stomp/info
   ProxyPass         /stomp         ws://<tomcat host>:8080/stomp
   ProxyPassReverse  /stomp         ws://<tomcat host>:8080/stomp
   ProxyPass         /              http://<tomcat host>:8080/
   ProxyPassReverse  /              http://<tomcat host>:8080/

So far I think this is OK, but I'm not done testing yet.    I'm minded to switch to locking down the entire server and enabling open access just where it needs to be public  (like /jbrowse) but I haven't done that yet as I don't know everything that needs to be accessible (/stomp?  static content?).   Any thoughts on this would be welcome.

Thanks for all the hints & tips

Tim


-- The Wellcome Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.

--
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: OAuth support [EXT]

t.r.stickland@sanger.ac.uk

Right, I found time to try this, and I think I have found the apache configuration needed to secure the entire apollo service apart from the public genome browsing (i.e. plain jbrowse) that we want to enable.

First off all, restrict access to the entire server.  This is simple enough, and exactly what you would do if you wanted all of your apollo instance to be behind access control.#
   <Location />
      AuthType openid-connect
      Require  valid-user
   </Location>
Next, the public areas need to be opened up with Location directives.  These should appear in the apache config  after the one just above.
   <Location /jbrowse>
      AuthType None
      Require  all granted
   </Location>
(the "Require all granted" being the rather unfriendly apache 2.4 expression for allowing full access to everyone.)

As well as /jbrowse, there are other paths you need to open up, so add Location directives as above for these paths:
   /assets
   /support
   /favicon.ico
...plus of course any static content etc. that you choose to serve from apache rather than tomcat, and wish to make freely available.

Lastly, there are requests that begin with arbitrary alphanumeric identifiers, e.g.  "/11/jbrowse..." for requests that display organism 11.  If you have only a few organisms then you could enable these as above, but if (like us) you have lots then it is easier to add a LocationMatch to enable access based on a suitable regular expressions:
   <LocationMatch "^/[0-9]+/jbrowse">
      AuthType None
      Require  all granted
   </LocationMatch>
I can't be certain yet I have caught everything.

The really obvious problem with this solution is that it's likely to be unstable, and need tweaking for new releases.   It's far simpler to leave the whole server open and then restrict access only to /annotator and /user.

The thing to remember is that with this apache module (mod_auth_openidc), the authentication-related HTTP headers are passed only with requests for restricted content, so you must restrict everything in apollo that requires these headers.   That's why (rewinding this thread a bit) I was getting authentication failures when I restricted only /annotator.   I thought what I had done was right, as the only content I wanted to restrict under /annotator.   But the the API request that actually does the authentication is under /user, and (because this wasn't restricted) the needful headers were not being passed with those requests.

Hopefully this information may be of use to someone.

Now my next job is to see how I can bypass the OIDC authentication entirely when I want to log in as administrator...

tim



On 27/09/2019 22:09, Nathan Dunn wrote:

Definitely post more details (including client fails, etc.) back to the list when you’re done with testing.  

Nathan


On Sep 27, 2019, at 7:37 AM, Tim Stickland <[hidden email]> wrote:

On 26/09/2019 14:33, Tim Stickland wrote:
On 26/09/2019 09:50, Helena Rasche wrote:

Have you used tcpdump? I've found it super useful for debugging these sort of issues. e.g.

On your machine with tomcat, as root, run tcpdump -i any -vvv -A 'dst port 8080', changing 8080 to whichever port your tomcat is listening on. Then if you access a URL like .../annotator/getAppState in a browser (so you just see one request at a time, loading the normal apollo interface can produce an overwhelming amount of logs:


Great, thanks, that's a helpful tip.  I'll try it and report back.

Following a bit of a fight to get tcpdump working on a docker network, this revealed the problem :)

Summary: it should all work fine if your entire Apollo installation is under access control, but is more complicated if you want parts (/jbrowse, say) to be public.

The ugly details...

Passing REMOTE_USER to tomcat from an a apache httpd proxy is not a problem.  It wasn't working when I had an apache httpd downstream of the proxy (which, ironically, I had added to help me debug this:-/), but that's something to do with the downstream httpd, not the proxy httpd.     So for the time being at least,  it looks like it's fine to use an apache httpd proxy with tomcat downstream of it.   This is worth watching though;  if you google it, you'll see quite a lot of people have encountered problems with REMOTE_USER, in nginx as well as apache httpd, so that's one of the first places I'd look if you upgrade a web server/proxy/load balancer and your authentication breaks.

I'm using mod_auth_openidc in my apache proxy.  This will sets the REMOTE_USER environment variable for all requests under access control, and I can pass this downstream to tomcat with this:

    RequestHeader set REMOTE_USER "expr=%{REMOTE_USER}"

Of course this means that REMOTE_USER is always sent, but it will have no content unless REMOTE_USER has a value in the apache environment on the proxy.  That is only the case when the request is under access control, of course.    If you have public content, mod_auth_openidc isn't invoked when that is requested, and REMOTE_USER is never set.

This seems like it wouldn't be an issue.   After all, if something isn't under access control, then the user won't be asked to log in, so there's no need for REMOTE_USER to be set?

The complication is that the OAuth callback request (/annotator/index) does not cause the grails app calls RemoteAuthenticatorService.   That's done by a request to /user, which is an API call (so you have to look pretty closely to find it).

If you have the entire server under access control, that shouldn't matter -- but if you want to keep some parts open (like /jbrowse) then you need to be careful about setting up access control, so that it includes not just what you want to restrict, but also the API calls which require authentication-related headers.

I hope this explanation makes some sort of sense.

This is the apache configuration I have ATM...
   LoadModule auth_openidc_module /usr/lib/apache2/modules/mod_auth_openidc.so

   # these OIDC... headers are all standard mod_auth_openidc settings
   OIDCProviderMetadataURL <oauth provider metadata URL>
   OIDCClientID            <my client ID>
   OIDCClientSecret        <my client secret>
   OIDCScope               "openid email profile"
   # vanity URL, must point to a path protected by this module but must NOT point to any content
   OIDCRedirectURI         <my apollo host>/annotator/openid/foo
   OIDCCryptoPassphrase    <random password

   # this prompts a log in when the user accesses apollo
   <Location /annotator>
      AuthType       openid-connect
      Require        valid-user
   </Location>

   # required to get mod_auth_openid to pass headers with /user requests
   <Location /user>
      AuthType       openid-connect
      Require        valid-user
   </Location>

   RequestHeader     set REMOTE_USER "expr=%{REMOTE_USER}"

   # <tomcat host> is accessible only on the internal network
   ProxyPass         /stomp/info    http://<tomcat host>:8080/stomp/info
   ProxyPassReverse  /stomp/info    http://<tomcat host>:8080/stomp/info
   ProxyPass         /stomp         ws://<tomcat host>:8080/stomp
   ProxyPassReverse  /stomp         ws://<tomcat host>:8080/stomp
   ProxyPass         /              http://<tomcat host>:8080/
   ProxyPassReverse  /              http://<tomcat host>:8080/

So far I think this is OK, but I'm not done testing yet.    I'm minded to switch to locking down the entire server and enabling open access just where it needs to be public  (like /jbrowse) but I haven't done that yet as I don't know everything that needs to be accessible (/stomp?  static content?).   Any thoughts on this would be welcome.

Thanks for all the hints & tips

Tim


-- The Wellcome Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.



-- The Wellcome Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.

--
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: OAuth support [EXT]

t.r.stickland@sanger.ac.uk
The final piece of the puzzle was to enable access to local user authentication (mainly so I could log in as admin) when the Apollo tomcat was behind an Apache proxy using mod_auth_openidc -- so access to Apollo was blocked without an OIDC (OAuth) log in.

First of all, it's easy if you have access direct to tomcat.   Just access it directly on http://your.host:8080/ and that bypasses the apache module.   But that doesn't work for me, as I run the tomcat in a docker container  that's accessible only on the docker network, and I don't really want to expose that container to the external network just for this purpose -- I want the only way in to be via the httpd (in a separate container).

I initially thought I'd do this by tweaking proxy settings to enable access via a separate path (/admin, say) which was outside access control (bearing in mind that bypassing access control in the httpd doesn't give open access to apollo -- it just makes it fall back on to local user authentication).  This gets a bit complicated as apollo makes a lot of API calls, and you have proxy all of these appropriately to prevent access control catching them.  It was simpler  just to have the httpd listen to another port, and add a virtual host to handle requests on that port.

With a separate virtual host, it's trivial to use only the regular apollo proxy settings, and use no access control.   e.g. to give access on host apollo-httpd port 8000, where apollo-tomcat is the host with tomcat on 8080:
Listen 8000

<VirtualHost *:8000>

   ServerAdmin       apollo@apollo-httpd
   ServerName        apollo-httpd
   ProxyPreserveHost On
   ProxyRequests     Off

   <Location />
      AuthType None
      Require  all granted
   </Location>

   ProxyPass         /stomp/info    http://apollo-tomcat:8080/stomp/info
   ProxyPassReverse  /stomp/info    http://apollo-tomcat:8080/stomp/info
   ProxyPass         /stomp         <a class="moz-txt-link-freetext" href="ws://apollo-tomcat:8080/stomp">ws://apollo-tomcat:8080/stomp
   ProxyPassReverse  /stomp         <a class="moz-txt-link-freetext" href="ws://apollo-tomcat:8080/stomp">ws://apollo-tomcat:8080/stomp
   ProxyPass         /              http://apollo-tomcat:8080/
   ProxyPassReverse  /              http://apollo-tomcat:8080/

</VirtualHost>

It would be easy and probably a good idea to add restrictions to that Location directive (e.g. to restrict access to your local network), though in my case it's unnecessary as only port 80 can be reached from outside (corporate firewall) so it's already restricted to our private network.

tim



On 08/10/2019 16:16, Nathan Dunn wrote:

Cool.  Share that final solution.  I’ll try to add it as a cookbook doc when you’re done. 

Nathan


On Oct 8, 2019, at 7:31 AM, Tim Stickland <[hidden email]> wrote:


Right, I found time to try this, and I think I have found the apache configuration needed to secure the entire apollo service apart from the public genome browsing (i.e. plain jbrowse) that we want to enable.

First off all, restrict access to the entire server.  This is simple enough, and exactly what you would do if you wanted all of your apollo instance to be behind access control.#
   <Location />
      AuthType openid-connect
      Require  valid-user
   </Location>
Next, the public areas need to be opened up with Location directives.  These should appear in the apache config  after the one just above.
   <Location /jbrowse>
      AuthType None
      Require  all granted
   </Location>
(the "Require all granted" being the rather unfriendly apache 2.4 expression for allowing full access to everyone.)

As well as /jbrowse, there are other paths you need to open up, so add Location directives as above for these paths:
   /assets
   /support
   /favicon.ico
...plus of course any static content etc. that you choose to serve from apache rather than tomcat, and wish to make freely available.

Lastly, there are requests that begin with arbitrary alphanumeric identifiers, e.g.  "/11/jbrowse..." for requests that display organism 11.  If you have only a few organisms then you could enable these as above, but if (like us) you have lots then it is easier to add a LocationMatch to enable access based on a suitable regular expressions:
   <LocationMatch "^/[0-9]+/jbrowse">
      AuthType None
      Require  all granted
   </LocationMatch>
I can't be certain yet I have caught everything.

The really obvious problem with this solution is that it's likely to be unstable, and need tweaking for new releases.   It's far simpler to leave the whole server open and then restrict access only to /annotator and /user.

The thing to remember is that with this apache module (mod_auth_openidc), the authentication-related HTTP headers are passed only with requests for restricted content, so you must restrict everything in apollo that requires these headers.   That's why (rewinding this thread a bit) I was getting authentication failures when I restricted only /annotator.   I thought what I had done was right, as the only content I wanted to restrict under /annotator.   But the the API request that actually does the authentication is under /user, and (because this wasn't restricted) the needful headers were not being passed with those requests.

Hopefully this information may be of use to someone.

Now my next job is to see how I can bypass the OIDC authentication entirely when I want to log in as administrator...

tim



On 27/09/2019 22:09, Nathan Dunn wrote:

Definitely post more details (including client fails, etc.) back to the list when you’re done with testing.  

Nathan


On Sep 27, 2019, at 7:37 AM, Tim Stickland <[hidden email]> wrote:

On 26/09/2019 14:33, Tim Stickland wrote:
On 26/09/2019 09:50, Helena Rasche wrote:

Have you used tcpdump? I've found it super useful for debugging these sort of issues. e.g.

On your machine with tomcat, as root, run tcpdump -i any -vvv -A 'dst port 8080', changing 8080 to whichever port your tomcat is listening on. Then if you access a URL like .../annotator/getAppState in a browser (so you just see one request at a time, loading the normal apollo interface can produce an overwhelming amount of logs:


Great, thanks, that's a helpful tip.  I'll try it and report back.

Following a bit of a fight to get tcpdump working on a docker network, this revealed the problem :)

Summary: it should all work fine if your entire Apollo installation is under access control, but is more complicated if you want parts (/jbrowse, say) to be public.

The ugly details...

Passing REMOTE_USER to tomcat from an a apache httpd proxy is not a problem.  It wasn't working when I had an apache httpd downstream of the proxy (which, ironically, I had added to help me debug this:-/), but that's something to do with the downstream httpd, not the proxy httpd.     So for the time being at least,  it looks like it's fine to use an apache httpd proxy with tomcat downstream of it.   This is worth watching though;  if you google it, you'll see quite a lot of people have encountered problems with REMOTE_USER, in nginx as well as apache httpd, so that's one of the first places I'd look if you upgrade a web server/proxy/load balancer and your authentication breaks.

I'm using mod_auth_openidc in my apache proxy.  This will sets the REMOTE_USER environment variable for all requests under access control, and I can pass this downstream to tomcat with this:

    RequestHeader set REMOTE_USER "expr=%{REMOTE_USER}"

Of course this means that REMOTE_USER is always sent, but it will have no content unless REMOTE_USER has a value in the apache environment on the proxy.  That is only the case when the request is under access control, of course.    If you have public content, mod_auth_openidc isn't invoked when that is requested, and REMOTE_USER is never set.

This seems like it wouldn't be an issue.   After all, if something isn't under access control, then the user won't be asked to log in, so there's no need for REMOTE_USER to be set?

The complication is that the OAuth callback request (/annotator/index) does not cause the grails app calls RemoteAuthenticatorService.   That's done by a request to /user, which is an API call (so you have to look pretty closely to find it).

If you have the entire server under access control, that shouldn't matter -- but if you want to keep some parts open (like /jbrowse) then you need to be careful about setting up access control, so that it includes not just what you want to restrict, but also the API calls which require authentication-related headers.

I hope this explanation makes some sort of sense.

This is the apache configuration I have ATM...
   LoadModule auth_openidc_module /usr/lib/apache2/modules/mod_auth_openidc.so

   # these OIDC... headers are all standard mod_auth_openidc settings
   OIDCProviderMetadataURL <oauth provider metadata URL>
   OIDCClientID            <my client ID>
   OIDCClientSecret        <my client secret>
   OIDCScope               "openid email profile"
   # vanity URL, must point to a path protected by this module but must NOT point to any content
   OIDCRedirectURI         <my apollo host>/annotator/openid/foo
   OIDCCryptoPassphrase    <random password

   # this prompts a log in when the user accesses apollo
   <Location /annotator>
      AuthType       openid-connect
      Require        valid-user
   </Location>

   # required to get mod_auth_openid to pass headers with /user requests
   <Location /user>
      AuthType       openid-connect
      Require        valid-user
   </Location>

   RequestHeader     set REMOTE_USER "expr=%{REMOTE_USER}"

   # <tomcat host> is accessible only on the internal network
   ProxyPass         /stomp/info    http://<tomcat host>:8080/stomp/info
   ProxyPassReverse  /stomp/info    http://<tomcat host>:8080/stomp/info
   ProxyPass         /stomp         ws://<tomcat host>:8080/stomp
   ProxyPassReverse  /stomp         ws://<tomcat host>:8080/stomp
   ProxyPass         /              http://<tomcat host>:8080/
   ProxyPassReverse  /              http://<tomcat host>:8080/

So far I think this is OK, but I'm not done testing yet.    I'm minded to switch to locking down the entire server and enabling open access just where it needs to be public  (like /jbrowse) but I haven't done that yet as I don't know everything that needs to be accessible (/stomp?  static content?).   Any thoughts on this would be welcome.

Thanks for all the hints & tips

Tim


-- The Wellcome Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.



-- The Wellcome Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.



-- The Wellcome Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.

--
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].