代码拉取完成，页面将自动刷新
spark-sql> select a.address from (select name,address,age from customers where 1=1 and age > 5) a where a.age<30;
21/11/17 00:19:02 WARN PlanChangeLogger: Batch Substitution has no effect.
21/11/17 00:19:02 WARN PlanChangeLogger: Batch Disable Hints has no effect.
21/11/17 00:19:02 WARN PlanChangeLogger: Batch Hints has no effect.
21/11/17 00:19:02 WARN PlanChangeLogger: Batch Simple Sanity Check has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger:
=== Applying Rule org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations ===
'Project ['a.address] 'Project ['a.address]
+- 'Filter ('a.age < 30) +- 'Filter ('a.age < 30)
+- 'SubqueryAlias a +- 'SubqueryAlias a
+- 'Project ['name, 'address, 'age] +- 'Project ['name, 'address, 'age]
+- 'Filter ((1 = 1) AND ('age > 5)) +- 'Filter ((1 = 1) AND ('age > 5))
! +- 'UnresolvedRelation [customers], [], false +- 'SubqueryAlias spark_catalog.default.customers
! +- 'UnresolvedCatalogRelation `default`.`customers`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, [], false
21/11/17 00:19:03 WARN PlanChangeLogger:
=== Applying Rule org.apache.spark.sql.execution.datasources.FindDataSourceTable ===
'Project ['a.address] 'Project ['a.address]
+- 'Filter ('a.age < 30) +- 'Filter ('a.age < 30)
+- 'SubqueryAlias a +- 'SubqueryAlias a
+- 'Project ['name, 'address, 'age] +- 'Project ['name, 'address, 'age]
+- 'Filter ((1 = 1) AND ('age > 5)) +- 'Filter ((1 = 1) AND ('age > 5))
! +- 'SubqueryAlias spark_catalog.default.customers +- SubqueryAlias spark_catalog.default.customers
! +- 'UnresolvedCatalogRelation `default`.`customers`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, [], false +- HiveTableRelation [`default`.`customers`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [name#40, age#41, address#42], Partition Cols: []]
21/11/17 00:19:03 WARN PlanChangeLogger:
=== Applying Rule org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveReferences ===
!'Project ['a.address] Project [address#42]
!+- 'Filter ('a.age < 30) +- Filter (age#41 < 30)
! +- 'SubqueryAlias a +- SubqueryAlias a
! +- 'Project ['name, 'address, 'age] +- Project [name#40, address#42, age#41]
! +- 'Filter ((1 = 1) AND ('age > 5)) +- Filter ((1 = 1) AND (age#41 > 5))
+- SubqueryAlias spark_catalog.default.customers +- SubqueryAlias spark_catalog.default.customers
+- HiveTableRelation [`default`.`customers`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [name#40, age#41, address#42], Partition Cols: []] +- HiveTableRelation [`default`.`customers`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [name#40, age#41, address#42], Partition Cols: []]
21/11/17 00:19:03 WARN PlanChangeLogger:
=== Result of Batch Resolution ===
!'Project ['a.address] Project [address#42]
!+- 'Filter ('a.age < 30) +- Filter (age#41 < 30)
! +- 'SubqueryAlias a +- SubqueryAlias a
! +- 'Project ['name, 'address, 'age] +- Project [name#40, address#42, age#41]
! +- 'Filter ((1 = 1) AND ('age > 5)) +- Filter ((1 = 1) AND (age#41 > 5))
! +- 'UnresolvedRelation [customers], [], false +- SubqueryAlias spark_catalog.default.customers
! +- HiveTableRelation [`default`.`customers`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [name#40, age#41, address#42], Partition Cols: []]
21/11/17 00:19:03 WARN PlanChangeLogger: Batch Apply Char Padding has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger:
=== Applying Rule org.apache.spark.sql.hive.DetermineTableStats ===
Project [address#42] Project [address#42]
+- Filter (age#41 < 30) +- Filter (age#41 < 30)
+- SubqueryAlias a +- SubqueryAlias a
+- Project [name#40, address#42, age#41] +- Project [name#40, address#42, age#41]
+- Filter ((1 = 1) AND (age#41 > 5)) +- Filter ((1 = 1) AND (age#41 > 5))
+- SubqueryAlias spark_catalog.default.customers +- SubqueryAlias spark_catalog.default.customers
+- HiveTableRelation [`default`.`customers`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [name#40, age#41, address#42], Partition Cols: []] +- HiveTableRelation [`default`.`customers`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [name#40, age#41, address#42], Partition Cols: []]
21/11/17 00:19:03 WARN PlanChangeLogger:
=== Result of Batch Post-Hoc Resolution ===
Project [address#42] Project [address#42]
+- Filter (age#41 < 30) +- Filter (age#41 < 30)
+- SubqueryAlias a +- SubqueryAlias a
+- Project [name#40, address#42, age#41] +- Project [name#40, address#42, age#41]
+- Filter ((1 = 1) AND (age#41 > 5)) +- Filter ((1 = 1) AND (age#41 > 5))
+- SubqueryAlias spark_catalog.default.customers +- SubqueryAlias spark_catalog.default.customers
+- HiveTableRelation [`default`.`customers`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [name#40, age#41, address#42], Partition Cols: []] +- HiveTableRelation [`default`.`customers`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [name#40, age#41, address#42], Partition Cols: []]
21/11/17 00:19:03 WARN PlanChangeLogger: Batch Normalize Alter Table has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger: Batch Remove Unresolved Hints has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger: Batch Nondeterministic has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger: Batch UDF has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger: Batch UpdateNullability has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger: Batch Subquery has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger: Batch Cleanup has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger:
=== Metrics of Executed Rules ===
Total number of runs: 218
Total time: 0.211820244 seconds
Total number of effective runs: 4
Total time of effective runs: 0.156881334 seconds
21/11/17 00:19:03 WARN PlanChangeLogger: Batch Eliminate Distinct has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger:
=== Applying Rule org.apache.spark.sql.catalyst.analysis.EliminateSubqueryAliases ===
Project [address#42] Project [address#42]
+- Filter (age#41 < 30) +- Filter (age#41 < 30)
! +- SubqueryAlias a +- Project [name#40, address#42, age#41]
! +- Project [name#40, address#42, age#41] +- Filter ((1 = 1) AND (age#41 > 5))
! +- Filter ((1 = 1) AND (age#41 > 5)) +- HiveTableRelation [`default`.`customers`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [name#40, age#41, address#42], Partition Cols: []]
! +- SubqueryAlias spark_catalog.default.customers
! +- HiveTableRelation [`default`.`customers`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [name#40, age#41, address#42], Partition Cols: []]
21/11/17 00:19:03 WARN PlanChangeLogger:
=== Result of Batch Finish Analysis ===
Project [address#42] Project [address#42]
+- Filter (age#41 < 30) +- Filter (age#41 < 30)
! +- SubqueryAlias a +- Project [name#40, address#42, age#41]
! +- Project [name#40, address#42, age#41] +- Filter ((1 = 1) AND (age#41 > 5))
! +- Filter ((1 = 1) AND (age#41 > 5)) +- HiveTableRelation [`default`.`customers`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [name#40, age#41, address#42], Partition Cols: []]
! +- SubqueryAlias spark_catalog.default.customers
! +- HiveTableRelation [`default`.`customers`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [name#40, age#41, address#42], Partition Cols: []]
21/11/17 00:19:03 WARN PlanChangeLogger: Batch Union has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger: Batch OptimizeLimitZero has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger: Batch LocalRelation early has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger: Batch Pullup Correlated Expressions has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger: Batch Subquery has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger: Batch Replace Operators has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger: Batch Aggregate has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger:
=== Applying Rule org.apache.spark.sql.catalyst.optimizer.PushDownPredicates ===
Project [address#42] Project [address#42]
!+- Filter (age#41 < 30) +- Project [name#40, address#42, age#41]
! +- Project [name#40, address#42, age#41] +- Filter (((1 = 1) AND (age#41 > 5)) AND (age#41 < 30))
! +- Filter ((1 = 1) AND (age#41 > 5)) +- HiveTableRelation [`default`.`customers`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [name#40, age#41, address#42], Partition Cols: []]
! +- HiveTableRelation [`default`.`customers`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [name#40, age#41, address#42], Partition Cols: []]
21/11/17 00:19:03 WARN PlanChangeLogger:
=== Applying Rule org.apache.spark.sql.catalyst.optimizer.ColumnPruning ===
Project [address#42] Project [address#42]
!+- Project [name#40, address#42, age#41] +- Project [address#42]
+- Filter (((1 = 1) AND (age#41 > 5)) AND (age#41 < 30)) +- Filter (((1 = 1) AND (age#41 > 5)) AND (age#41 < 30))
+- HiveTableRelation [`default`.`customers`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [name#40, age#41, address#42], Partition Cols: []] +- HiveTableRelation [`default`.`customers`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [name#40, age#41, address#42], Partition Cols: []]
21/11/17 00:19:03 WARN PlanChangeLogger:
=== Applying Rule org.apache.spark.sql.catalyst.optimizer.CollapseProject ===
Project [address#42] Project [address#42]
!+- Project [address#42] +- Filter (((1 = 1) AND (age#41 > 5)) AND (age#41 < 30))
! +- Filter (((1 = 1) AND (age#41 > 5)) AND (age#41 < 30)) +- HiveTableRelation [`default`.`customers`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [name#40, age#41, address#42], Partition Cols: []]
! +- HiveTableRelation [`default`.`customers`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [name#40, age#41, address#42], Partition Cols: []]
21/11/17 00:19:03 WARN PlanChangeLogger:
=== Applying Rule org.apache.spark.sql.catalyst.optimizer.ConstantFolding ===
Project [address#42] Project [address#42]
!+- Filter (((1 = 1) AND (age#41 > 5)) AND (age#41 < 30)) +- Filter ((true AND (age#41 > 5)) AND (age#41 < 30))
+- HiveTableRelation [`default`.`customers`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [name#40, age#41, address#42], Partition Cols: []] +- HiveTableRelation [`default`.`customers`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [name#40, age#41, address#42], Partition Cols: []]
21/11/17 00:19:03 WARN PlanChangeLogger:
=== Applying Rule org.apache.spark.sql.catalyst.optimizer.BooleanSimplification ===
Project [address#42] Project [address#42]
!+- Filter ((true AND (age#41 > 5)) AND (age#41 < 30)) +- Filter ((age#41 > 5) AND (age#41 < 30))
+- HiveTableRelation [`default`.`customers`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [name#40, age#41, address#42], Partition Cols: []] +- HiveTableRelation [`default`.`customers`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [name#40, age#41, address#42], Partition Cols: []]
21/11/17 00:19:03 WARN PlanChangeLogger:
=== Result of Batch Operator Optimization before Inferring Filters ===
Project [address#42] Project [address#42]
!+- Filter (age#41 < 30) +- Filter ((age#41 > 5) AND (age#41 < 30))
! +- Project [name#40, address#42, age#41] +- HiveTableRelation [`default`.`customers`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [name#40, age#41, address#42], Partition Cols: []]
! +- Filter ((1 = 1) AND (age#41 > 5))
! +- HiveTableRelation [`default`.`customers`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [name#40, age#41, address#42], Partition Cols: []]
21/11/17 00:19:03 WARN PlanChangeLogger:
=== Applying Rule org.apache.spark.sql.catalyst.optimizer.InferFiltersFromConstraints ===
Project [address#42] Project [address#42]
!+- Filter ((age#41 > 5) AND (age#41 < 30)) +- Filter (isnotnull(age#41) AND ((age#41 > 5) AND (age#41 < 30)))
+- HiveTableRelation [`default`.`customers`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [name#40, age#41, address#42], Partition Cols: []] +- HiveTableRelation [`default`.`customers`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [name#40, age#41, address#42], Partition Cols: []]
21/11/17 00:19:03 WARN PlanChangeLogger:
=== Result of Batch Infer Filters ===
Project [address#42] Project [address#42]
!+- Filter ((age#41 > 5) AND (age#41 < 30)) +- Filter (isnotnull(age#41) AND ((age#41 > 5) AND (age#41 < 30)))
+- HiveTableRelation [`default`.`customers`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [name#40, age#41, address#42], Partition Cols: []] +- HiveTableRelation [`default`.`customers`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [name#40, age#41, address#42], Partition Cols: []]
21/11/17 00:19:03 WARN PlanChangeLogger: Batch Operator Optimization after Inferring Filters has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger: Batch Push extra predicate through join has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger: Batch Early Filter and Projection Push-Down has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger: Batch Join Reorder has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger: Batch Eliminate Sorts has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger: Batch Decimal Optimizations has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger: Batch Distinct Aggregate Rewrite has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger: Batch Object Expressions Optimization has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger: Batch LocalRelation has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger: Batch Check Cartesian Products has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger:
=== Applying Rule org.apache.spark.sql.catalyst.optimizer.RewritePredicateSubquery ===
Project [address#42] Project [address#42]
!+- Filter (isnotnull(age#41) AND ((age#41 > 5) AND (age#41 < 30))) +- Filter ((isnotnull(age#41) AND (age#41 > 5)) AND (age#41 < 30))
+- HiveTableRelation [`default`.`customers`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [name#40, age#41, address#42], Partition Cols: []] +- HiveTableRelation [`default`.`customers`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [name#40, age#41, address#42], Partition Cols: []]
21/11/17 00:19:03 WARN PlanChangeLogger:
=== Result of Batch RewriteSubquery ===
Project [address#42] Project [address#42]
!+- Filter (isnotnull(age#41) AND ((age#41 > 5) AND (age#41 < 30))) +- Filter ((isnotnull(age#41) AND (age#41 > 5)) AND (age#41 < 30))
+- HiveTableRelation [`default`.`customers`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [name#40, age#41, address#42], Partition Cols: []] +- HiveTableRelation [`default`.`customers`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [name#40, age#41, address#42], Partition Cols: []]
21/11/17 00:19:03 WARN PlanChangeLogger: Batch NormalizeFloatingNumbers has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger: Batch ReplaceUpdateFieldsExpression has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger: Batch Optimize Metadata Only Query has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger: Batch PartitionPruning has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger: Batch Pushdown Filters from PartitionPruning has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger: Batch Cleanup filters that cannot be pushed down has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger: Batch Extract Python UDFs has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger: Batch User Provided Optimizers has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger:
=== Metrics of Executed Rules ===
Total number of runs: 187
Total time: 0.099935515 seconds
Total number of effective runs: 8
Total time of effective runs: 0.039378119 seconds
21/11/17 00:19:03 WARN PlanChangeLogger:
=== Applying Rule org.apache.spark.sql.execution.CollapseCodegenStages ===
!Project [address#42] *(1) Project [address#42]
!+- Filter ((isnotnull(age#41) AND (age#41 > 5)) AND (age#41 < 30)) +- *(1) Filter ((isnotnull(age#41) AND (age#41 > 5)) AND (age#41 < 30))
+- Scan hive default.customers [address#42, age#41], HiveTableRelation [`default`.`customers`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [name#40, age#41, address#42], Partition Cols: []] +- Scan hive default.customers [address#42, age#41], HiveTableRelation [`default`.`customers`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [name#40, age#41, address#42], Partition Cols: []]
21/11/17 00:19:03 WARN PlanChangeLogger:
=== Result of Batch Preparations ===
!Project [address#42] *(1) Project [address#42]
!+- Filter ((isnotnull(age#41) AND (age#41 > 5)) AND (age#41 < 30)) +- *(1) Filter ((isnotnull(age#41) AND (age#41 > 5)) AND (age#41 < 30))
+- Scan hive default.customers [address#42, age#41], HiveTableRelation [`default`.`customers`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [name#40, age#41, address#42], Partition Cols: []] +- Scan hive default.customers [address#42, age#41], HiveTableRelation [`default`.`customers`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [name#40, age#41, address#42], Partition Cols: []]
21/11/17 00:19:03 WARN PlanChangeLogger: Batch CleanExpressions has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger:
=== Metrics of Executed Rules ===
Total number of runs: 1
Total time: 1.0415E-5 seconds
Total number of effective runs: 0
Total time of effective runs: 0.0 seconds
21/11/17 00:19:03 WARN PlanChangeLogger: Batch CleanExpressions has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger:
=== Metrics of Executed Rules ===
Total number of runs: 1
Total time: 1.2525E-5 seconds
Total number of effective runs: 0
Total time of effective runs: 0.0 seconds
21/11/17 00:19:03 WARN PlanChangeLogger: Batch CleanExpressions has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger:
=== Metrics of Executed Rules ===
Total number of runs: 1
Total time: 2.8678E-5 seconds
Total number of effective runs: 0
Total time of effective runs: 0.0 seconds
21/11/17 00:19:03 WARN PlanChangeLogger: Batch CleanExpressions has no effect.
21/11/17 00:19:03 WARN PlanChangeLogger:
=== Metrics of Executed Rules ===
Total number of runs: 1
Total time: 7.015E-5 seconds
Total number of effective runs: 0
Total time of effective runs: 0.0 seconds
21/11/17 00:19:03 WARN SessionState: METASTORE_FILTER_HOOK will be ignored, since hive.security.authorization.manager is set to instance of HiveAuthorizerFactory.
Time taken: 2.098 seconds
spark-sql>
此处可能存在不合适展示的内容，页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容，可点击提交进行申诉，我们将尽快为您处理。
马建仓 AI 助手