13 Star 65 Fork 18

DataManagement / Spark-Autotuning

加入 Gitee
与超过 1200万 开发者一起发现、参与优秀开源项目,私有仓库也完全免费 :)
免费加入
克隆/下载
HIT_tiny_cluster_SearchDistrict.json 1.85 KB
一键复制 编辑 原始数据 按行查看 历史
wengchangsu 提交于 2019-09-30 09:23 . v0.1
[
{
"name": "spark.task.cpus",
"upperBound":1,
"lowerBound":1,
"changeFactor": 1
},
{
"name": "spark.executor.cores",
"upperBound":8,
"lowerBound":1,
"changeFactor": 1
},
{
"name": "spark.executor.memory",
"upperBound":4096,
"lowerBound":512,
"changeFactor": 256
},
{
"name": "spark.driver.cores",
"upperBound":4,
"lowerBound":1,
"changeFactor": 1
},
{
"name": "spark.driver.memory",
"upperBound":4096,
"lowerBound":1024,
"changeFactor": 512
},
{
"name": "spark.reducer.maxSizeInFlight",
"upperBound":72,
"lowerBound":24,
"changeFactor": 6
},
{
"name": "spark.shuffle.compress",
"upperBound":1,
"lowerBound":0,
"changeFactor": 1
},
{
"name": "spark.shuffle.spill.compress",
"upperBound":1,
"lowerBound":0,
"changeFactor": 1
},
{
"name": "spark.shuffle.file.buffer",
"upperBound":128,
"lowerBound":16,
"changeFactor": 16
},
{
"name": "spark.broadcast.compress",
"upperBound":1,
"lowerBound":0,
"changeFactor": 1
},
{
"name": "spark.broadcast.blockSize",
"upperBound":14,
"lowerBound":2,
"changeFactor": 2
},
{
"name": "spark.rpc.message.maxSize",
"upperBound":256,
"lowerBound":64,
"changeFactor": 64
},
{
"name": "spark.memory.fraction",
"upperBound":80,
"lowerBound":10,
"changeFactor": 10
},
{
"name": "spark.memory.storageFraction",
"upperBound":70,
"lowerBound":30,
"changeFactor": 10
},
{
"name": "spark.rdd.compress",
"upperBound":1,
"lowerBound":0,
"changeFactor": 1
},
{
"name": "spark.io.compression.codec",
"upperBound":2,
"lowerBound":0,
"changeFactor": 1
}
]
//"_comment": "comment text goes here...",
//这种注释是非法的,但是还是用了…………
//毕竟是自己解析
//值域为[lowerBound,upperBound]
Java
1
https://gitee.com/HITMassiveData/Spark-Autotuning.git
git@gitee.com:HITMassiveData/Spark-Autotuning.git
HITMassiveData
Spark-Autotuning
Spark-Autotuning
master

搜索帮助