Spark2 DataFrame数据框常用操作(七)之统计指标:mean均值,variance方差,stddev标准差,corr(Pearson相关系数),skewness偏度,kurtosis峰度

val df4=spark.sql("SELECT mean(age),variance(age),stddev(age),corr(age,yearsmarried),skewness(age),kurtosis(age) FROM Affairs")

df4.show
+--------+------------------+------------------+-----------------------+-----------------+--------------------+
|avg(age)|     var_samp(age)|  stddev_samp(age)|corr(age, yearsmarried)|    skewness(age)|       kurtosis(age)|
+--------+------------------+------------------+-----------------------+-----------------+--------------------+
|    34.0|173.33333333333334|13.165611772087667|     0.7456766124552038|0.965388004190285|-0.43417159763313595|
+--------+------------------+------------------+-----------------------+-----------------+--------------------+

 

文章来自:http://www.cnblogs.com/wwxbi/p/6102545.html
© 2021 jiaocheng.bubufx.com  联系我们
ICP备案:鲁ICP备09046678号-3