yaooqinn/spark-ranger

spark RangerAdminRESTClient cant get resources from hive service def on ranger

Closed this issue · 4 comments

brucemen711 commented

I try using spark 2.3 + ranger 1.2 and spark 2.4 + ranger 2.0. It working if i put some custom json policy to policyCache folder.
But RangerAdminRESTClient cant get full servicepolicies from Ranger, it always get empty resources so anyone get denied.
I try and catch some error:

ERROR policyresourcematcher.RangerDefaultPolicyResourceMatcher: RangerDefaultPolicyResourceMatcher.init() failed: policyResources is null or empty, or serviceDef is null. (serviceDef=hive, policyResourceKeys=
20/03/03 17:05:42 ERROR policyresourcematcher.RangerDefaultPolicyResourceMatcher: RangerDefaultPolicyResourceMatcher.init() failed: policyResources is null or empty, or serviceDef is null. (serviceDef=hive, policyResourceKeys=
20/03/03 17:05:42 ERROR policyresourcematcher.RangerDefaultPolicyResourceMatcher: RangerDefaultPolicyResourceMatcher.init() failed: policyResources is null or empty, or serviceDef is null. (serviceDef=hive, policyResourceKeys=
20/03/03 17:05:42 ERROR policyresourcematcher.RangerDefaultPolicyResourceMatcher: RangerDefaultPolicyResourceMatcher.init() failed: policyResources is null or empty, or serviceDef is null. (serviceDef=hive, policyResourceKeys=
20/03/03 17:05:42 ERROR policyresourcematcher.RangerDefaultPolicyResourceMatcher: RangerDefaultPolicyResourceMatcher.init() failed: policyResources is null or empty, or serviceDef is null. (serviceDef=hive, policyResourceKeys=
20/03/03 17:05:42 ERROR policyresourcematcher.RangerDefaultPolicyResourceMatcher: RangerDefaultPolicyResourceMatcher.init() failed: policyResources is null or empty, or serviceDef is null. (serviceDef=hive, policyResourceKeys=

brucemen711 commented

@yaooqinn i follow your guide, anything misses ?
And with ranger 2.0, i faced error:
java.util.set is an interface, and jaxb can't handle interfaces.
Maybe all issues come from RangerAdminRESTClient class and ServicePolicies class ?

yaooqinn commented

probably jar conflicts, I made #8, can you try?

brucemen711 commented

probably jar conflicts, I made #8, can you try?

I managed to fix it with a custom class.
But i will try yours.

Thanks you.

brucemen711 commented

Fixed by @yaooqinn commit

Thanks.