[SPARK-19991][CORE][YARN] FileSegmentManagedBuffer performance improv…

…ement ## What changes were proposed in this pull request? Avoid `NoSuchElementException` every time `ConfigProvider.get(val, default)` falls back to default. This apparently causes non-trivial overhead in at least one path, and can easily be avoided. See #17329 ## How was this patch tested? Existing tests Author: Sean Owen <[email protected]> Closes #17567 from srowen/SPARK-19991.
apache · Apr 9, 2017 · 1f0de3c · 1f0de3c
1 parent 34fc48f
commit 1f0de3c
Show file tree

Hide file tree

Showing 3 changed files with 13 additions and 1 deletion.
diff --git a/common/network-common/src/main/java/org/apache/spark/network/util/MapConfigProvider.java b/common/network-common/src/main/java/org/apache/spark/network/util/MapConfigProvider.java
@@ -42,6 +42,12 @@ public String get(String name) {
     return value;
   }
 
+  @Override
+  public String get(String name, String defaultValue) {
+    String value = config.get(name);
+    return value == null ? defaultValue : value;
+  }
+
   @Override
   public Iterable<Map.Entry<String, String>> getAll() {
     return config.entrySet();

diff --git a/...n/network-yarn/src/main/java/org/apache/spark/network/yarn/util/HadoopConfigProvider.java b/...n/network-yarn/src/main/java/org/apache/spark/network/yarn/util/HadoopConfigProvider.java
@@ -41,6 +41,12 @@ public String get(String name) {
     return value;
   }
 
+  @Override
+  public String get(String name, String defaultValue) {
+    String value = conf.get(name);
+    return value == null ? defaultValue : value;
+  }
+
   @Override
   public Iterable<Map.Entry<String, String>> getAll() {
     return conf;

diff --git a/core/src/main/scala/org/apache/spark/network/netty/SparkTransportConf.scala b/core/src/main/scala/org/apache/spark/network/netty/SparkTransportConf.scala
@@ -60,7 +60,7 @@ object SparkTransportConf {
 
     new TransportConf(module, new ConfigProvider {
       override def get(name: String): String = conf.get(name)
-
+      override def get(name: String, defaultValue: String): String = conf.get(name, defaultValue)
       override def getAll(): java.lang.Iterable[java.util.Map.Entry[String, String]] = {
         conf.getAll.toMap.asJava.entrySet()
       }