Configuring the Multi-tenant Mode¶
Scenarios¶
In multi-tenant mode, JDBCServers are bound with tenants. Each tenant corresponds to one or more JDBCServers, and a JDBCServer provides services for only one tenant. Different tenants can be configured with different Yarn queues to implement resource isolation.
Configuration Description¶
Log in to Manager, choose Cluster > Name of the desired cluster > Services > Spark2x > Configurations, click All Configurations, and search for and modify the following parameters.
Parameter | Description | Default Value |
|---|---|---|
spark.proxyserver.hash.enabled | Specifies whether to connect to ProxyServer using the Hash algorithm.
| true Note After this parameter is modified, you need to download the client again. |
spark.thriftserver.proxy.enabled | Specifies whether to use the multi-tenant mode.
| true |
spark.thriftserver.proxy.maxThriftServerPerTenancy | Specifies the maximum number of JDBCServer instances that can be started by a tenant in multi-tenant mode. | 1 |
spark.thriftserver.proxy.maxSessionPerThriftServer | Specifies the maximum number of sessions in a single JDBCServer instance in multi-tenant mode. If the number of sessions exceeds this value and the number of JDBCServer instances does not exceed the upper limit, a new JDBCServer instance is started. Otherwise, an alarm log is output. | 50 |
spark.thriftserver.proxy.sessionWaitTime | Specifies the wait time before a JDBCServer instance is stopped when it has no session connections in multi-tenant mode. | 180000 |
spark.thriftserver.proxy.sessionThreshold | In multi-tenant mode, when the session usage (formula: number of current sessions/spark.thriftserver.proxy.maxSessionPerThriftServer x number of current JDBCServer instances) of the JDBCServer instance reaches the threshold, a new JDBCServer instance is automatically added. | 100 |
spark.thriftserver.proxy.healthcheck.period | Specifies the period of JDBCServer health checks conducted by the JDBCServer proxy in multi-tenant mode. | 60000 |
spark.thriftserver.proxy.healthcheck.recheckTimes | Specifies the number of JDBCServer health check retries conducted by the JDBCServer proxy in multi-tenant mode. | 3 |
spark.thriftserver.proxy.healthcheck.waitTime | Specifies the wait time for JDBCServer to respond to a health check request sent by the JDBCServer proxy. | 10000 |
spark.thriftserver.proxy.session.check.interval | Specifies the period of JDBCServer proxy sessions in multi-tenant mode. | 6h |
spark.thriftserver.proxy.idle.session.timeout | Specifies the idle time interval of a JDBCServer proxy session in multi-tenant mode. If no operation is performed within this period, the session is closed. | 7d |
spark.thriftserver.proxy.idle.session.check.operation | Specifies whether to check that operations still exist on a JDBCServer proxy session when the session is checked for expiration in multi-tenant mode. | true |
spark.thriftserver.proxy.idle.operation.timeout | Specifies the timeout interval of an operation in multi-tenant mode. An operation that times out is closed. | 5d |