hive - Hue Beeswax / HCat no longer working (kerberos default user) after migration to HDP2.2 -
i've done migration of secure hdp2.1 hdp2.2 hadoop cluster. seems work (including hive in command line), hue. if file browser, job browser, pig interface , oozie interface working, not case of beeswax & webhcat interface. (nb : working before migration, same hue.ini file).
the error : could not start sasl: error in sasl_client_start (-1) sasl(-1): generic failure: gssapi error: unspecified gss failure. minor code may provide more information (server krbtgt/localdomain@hadoop.dev not found in kerberos database)
it seems thrift trying authenticate default user krbtgt/localdomain instead of configured ones.
i've tried log happens in python file failed see gets default user : kerberos principal short name hive, impersonification enabled. hue & hive proxies configured in hdfs conf files.
the complete stack trace :
[11/may/2015 06:10:40 +0000] access info 172.20.43.39 alinz - "get /beeswax/ http/1.0" [11/may/2015 06:10:40 +0000] hive_server2_lib info use_sasl=true, mechanism=gssapi, kerberos_principal_short_name=hive, impersonation_enabled=true [11/may/2015 06:10:40 +0000] thrift_util info thrift exception; retrying: not start sasl: error in sasl_client_start (-1) sasl(-1): generic failure: gssapi error: unspecified gss failure. minor code may provide more information (server krbtgt/localdomain@hadoop.dev not found in kerberos database) [11/may/2015 06:10:40 +0000] thrift_util info thrift exception; retrying: not start sasl: error in sasl_client_start (-1) sasl(-1): generic failure: gssapi error: unspecified gss failure. minor code may provide more information (server krbtgt/localdomain@hadoop.dev not found in kerberos database) [11/may/2015 06:10:40 +0000] thrift_util warning out of retries thrift call: opensession [11/may/2015 06:10:40 +0000] thrift_util info thrift saw transport exception: not start sasl: error in sasl_client_start (-1) sasl(-1): generic failure: gssapi error: unspecified gss failure. minor code may provide more information (server krbtgt/localdomain@hadoop.dev not found in kerberos database) [11/may/2015 06:10:40 +0000] middleware info processing exception: not start sasl: error in sasl_client_start (-1) sasl(-1): generic failure: gssapi error: unspecified gss failure. minor code may provide more information (server krbtgt/localdomain@hadoop.dev not found in kerberos database) (code thrifttransport): ttransportexception('could not start sasl: error in sasl_client_start (-1) sasl(-1): generic failure: gssapi error: unspecified gss failure. minor code may provide more information (server krbtgt/localdomain@hadoop.dev not found in kerberos database)',): traceback (most recent call last): file "/usr/lib/hue/build/env/lib/python2.6/site-packages/django-1.2.3-py2.6.egg/django/core/handlers/base.py", line 100, in get_response response = callback(request, *callback_args, **callback_kwargs) file "/usr/lib/hue/apps/beeswax/src/beeswax/views.py", line 69, in index return execute_query(request) file "/usr/lib/hue/apps/beeswax/src/beeswax/views.py", line 526, in execute_query databases = _get_db_choices(request) file "/usr/lib/hue/apps/beeswax/src/beeswax/views.py", line 1849, in _get_db_choices dbs = _get_databases(request) file "/usr/lib/hue/apps/beeswax/src/beeswax/views.py", line 1844, in _get_databases dbs = db.get_databases() file "/usr/lib/hue/apps/beeswax/src/beeswax/server/dbms.py", line 110, in get_databases return self.client.get_databases() file "/usr/lib/hue/apps/beeswax/src/beeswax/server/hive_server2_lib.py", line 746, in get_databases return [table[col] table in self._client.get_databases()] file "/usr/lib/hue/apps/beeswax/src/beeswax/server/hive_server2_lib.py", line 445, in get_databases res = self.call(self._client.getschemas, req) file "/usr/lib/hue/apps/beeswax/src/beeswax/server/hive_server2_lib.py", line 408, in call session = self.open_session(self.user) file "/usr/lib/hue/apps/beeswax/src/beeswax/server/hive_server2_lib.py", line 382, in open_session res = self._client.opensession(req) file "/usr/lib/hue/desktop/core/src/desktop/lib/thrift_util.py", line 329, in wrapper raise structuredthrifttransportexception(e, error_code=502) structuredthrifttransportexception: not start sasl: error in sasl_client_start (-1) sasl(-1): generic failure: gssapi error: unspecified gss failure. minor code may provide more information (server krbtgt/localdomain@hadoop.dev not found in kerberos database) (code thrifttransport): ttransportexception('could not start sasl: error in sasl_client_start (-1) sasl(-1): generic failure: gssapi error: unspecified gss failure. minor code may provide more information (server krbtgt/localdomain@hadoop.dev not found in kerberos database)',) any idea wrong?
krb5.conf :
[libdefaults] renew_lifetime = 7d forwardable = true default_realm = hadoop.dev ticket_lifetime = 24h dns_lookup_realm = false dns_lookup_kdc = false [logging] default = file:/var/log/krb5kdc.log admin_server = file:/var/log/kadmind.log kdc = file:/var/log/krb5kdc.log [realms] hadoop.dev = { admin_server = bt1svlmy kdc = bt1svlmy } and sudo klist -e /tmp/hue_krb5_ccache gives:
ticket cache: file:/tmp/hue_krb5_ccache default principal: hue/bt1svlmy.bpa.bouyguestelecom.fr@hadoop.dev valid starting expires service principal 05/11/15 15:10:34 05/12/15 15:10:34 krbtgt/hadoop.dev@hadoop.dev renew until 05/11/15 15:10:34, etype (skey, tkt): aes256-cts-hmac-sha1-96, aes256-cts-hmac-sha1-96 05/11/15 15:49:52 05/12/15 15:10:34 http/bt1svlmy.bpa.bouyguestelecom.fr@ renew until 05/11/15 15:10:34, etype (skey, tkt): aes256-cts-hmac-sha1-96, aes256-cts-hmac-sha1-96 05/11/15 15:49:52 05/12/15 15:10:34 http/bt1svlmy.bpa.bouyguestelecom.fr@hadoop.dev renew until 05/11/15 15:10:34, etype (skey, tkt): aes256-cts-hmac-sha1-96, aes256-cts-hmac-sha1-96
i have krbtgt/hadoop.dev@hadoop.devticket no krbtgt/localdomain@hadoop.dev ; maybe it's cause of issue?
kerberos log file :
may 11 16:12:35 bt1svlmy krb5kdc[12636](info): tgs_req (4 etypes {18 17 16 23}) 172.19.115.50: unknown_server: authtime 0, hue/bt1svlmy.bpa.bouyguestelecom.fr@hadoop.dev hive/localhost.localdomain@hadoop.dev, server not found in kerberos database may 11 16:12:35 bt1svlmy krb5kdc[12636](info): tgs_req (4 etypes {18 17 16 23}) 172.19.115.50: unknown_server: authtime 0, hue/bt1svlmy.bpa.bouyguestelecom.fr@hadoop.dev krbtgt/localdomain@hadoop.dev, server not found in kerberos database may 11 16:12:35 bt1svlmy krb5kdc[12636](info): tgs_req (4 etypes {18 17 16 23}) 172.19.115.50: unknown_server: authtime 0, hue/bt1svlmy.bpa.bouyguestelecom.fr@hadoop.dev hive/localhost.localdomain@hadoop.dev, server not found in kerberos database may 11 16:12:35 bt1svlmy krb5kdc[12636](info): tgs_req (4 etypes {18 17 16 23}) 172.19.115.50: unknown_server: authtime 0, hue/bt1svlmy.bpa.bouyguestelecom.fr@hadoop.dev krbtgt/localdomain@hadoop.dev, server not found in kerberos database may 11 16:12:35 bt1svlmy krb5kdc[12636](info): tgs_req (4 etypes {18 17 16 23}) 172.19.115.50: unknown_server: authtime 0, hue/bt1svlmy.bpa.bouyguestelecom.fr@hadoop.dev hive/localhost.localdomain@hadoop.dev, server not found in kerberos database may 11 16:12:35 bt1svlmy krb5kdc[12636](info): tgs_req (4 etypes {18 17 16 23}) 172.19.115.50: unknown_server: authtime 0, hue/bt1svlmy.bpa.bouyguestelecom.fr@hadoop.dev krbtgt/localdomain@hadoop.dev, server not found in kerberos database it seems me missed default hostname in conf somewhere, not find documentation entry it.
okay, found (had debug full python stack understand). it's not advertised, hue.ini parameter names have changed:
beeswax_server_host-->hive_server_hostbeeswax_server_port-->hive_server_port
it defaulting hive_server_host localhost, not correct on secure cluster.
Comments
Post a Comment