Skip to content

Commit

Permalink
TensorFlow: Upstream changes to git.
Browse files Browse the repository at this point in the history
Change 109240606
	Fix typo
Change 109240358
	Fix bug in Concat's shape inference due to legacy scalar handling.

	The shape function was inadvertently converting outputs of unknown
	shape (rank=None) to vectors of unknown length (rank=1), due to
	inability to distinguish between legacy scalars and vectors, because
	`max(1, None)` is 1.
Change 109237152
	Remove numarray requirement in python_config.
Change 109234003
	Fix typo in elu documentation.
Change 109232946
	Python must now be configured via ./configure script
Change 109232134
	Backported fixes to the tensor comparison operators from the public Eigen repository
Change 109231761
	Test invalid inputs to softmax_cross_entropy_with_logits.
Change 109230218
	Backported fixes to the tensor comparison operators from the public Eigen repository
Change 109229915
	Correct comments in seq2seq to show the right input types for embedding models.
	(Thanks to hugman@github for bringing this up.)
Change 109229118
	Fix resize_images example in documentation and allow resize_images to run on a single image with partially-known shape.
Change 109228940
	Fix demo and node add/remove button spacing
Change 109227909
	Include Elu in the NN docs.
Change 109227059
	Adds variable_op_scope and makes variable_scope always add a name_scope.

	This creates an op scope for variables that makes it easy to create independent
	operations with a default name by making that name unique for the current scope
	and it allows explicit names that are not made unique.

Change 109224492
	Streamline yuv -> rgb conversion to be done in one pass in native code.

	The entire process now takes ~2ms (including the ByteBuffer.get() calls), down from 10+ ms when the arrays were being interleaved in Java prior to conversion.

	Also abstracting common yuv->rgb color conversion into helper method.
Change 109224389
	Add ability to move nodes in and out of auxiliary nodes in graph.
Change 109217177
	Update generated Op docs.
Change 109215030
	Implementation of the ELU activation function: http://arxiv.org/abs/1511.07289
Change 109209848
	When GPUBFCAllocator runs out of memory, also log a summary
	of chunks in use by size.
Change 109206569
	Switched to the public version of the Eigen::sign method since it supports complex numbers.
Change 109199813
	Modify tensorflow.SequenceExample to support multiple-length sequences.

Base CL: 109241553
  • Loading branch information
Vijay Vasudevan committed Dec 2, 2015
1 parent fa095c5 commit bf6b536
Show file tree
Hide file tree
Showing 57 changed files with 1,923 additions and 338 deletions.
34 changes: 30 additions & 4 deletions configure
Original file line number Diff line number Diff line change
@@ -1,12 +1,38 @@
#!/bin/bash

## Set up python-related environment settings
while true; do
fromuser=""
if [ -z "$PYTHON_BIN_PATH" ]; then
default_python_bin_path=$(which python)
read -p "Please specify the location of python. [Default is $default_python_bin_path]: " PYTHON_BIN_PATH
fromuser="1"
if [ -z "$PYTHON_BIN_PATH" ]; then
PYTHON_BIN_PATH=$default_python_bin_path
fi
fi
if [ -e "$PYTHON_BIN_PATH" ]; then
break
fi
echo "Invalid python path. ${PYTHON_BIN_PATH} cannot be found" 1>&2
if [ -z "$fromuser" ]; then
exit 1
fi
PYTHON_BIN_PATH=""
# Retry
done

# Invoke python_config and set up symlinks to python includes
(./util/python/python_config.sh --setup "$PYTHON_BIN_PATH";) || exit -1

## Set up Cuda-related environment settings

while [ "$TF_NEED_CUDA" == "" ]; do
read -p "Do you wish to build TensorFlow with GPU support? [y/n] " INPUT
read -p "Do you wish to build TensorFlow with GPU support? [y/N] " INPUT
case $INPUT in
[Yy]* ) echo -e "GPU support will be enabled for TensorFlow\n"; TF_NEED_CUDA=1;;
[Nn]* ) echo -e "No GPU support will be enabled for TensorFlow\n"; TF_NEED_CUDA=0;;
[Yy]* ) echo "GPU support will be enabled for TensorFlow"; TF_NEED_CUDA=1;;
[Nn]* ) echo "No GPU support will be enabled for TensorFlow"; TF_NEED_CUDA=0;;
"" ) echo "No GPU support will be enabled for TensorFlow"; TF_NEED_CUDA=0;;
* ) echo "Invalid selection: " $INPUT;;
esac
done
Expand Down Expand Up @@ -77,7 +103,7 @@ CUDNN_INSTALL_PATH="$CUDNN_INSTALL_PATH"
EOF

function UnofficialSetting() {
echo -e "\nWARNING: You are configuring unofficial settings in TensorFlow. Because some external libraries are not backward compatible, these settings are largely untested and unsupported. \n"
echo -e "\nWARNING: You are configuring unofficial settings in TensorFlow. Because some external libraries are not backward compatible, these settings are largely untested and unsupported. \n" 1>&2

# Configure the compute capabilities that TensorFlow builds for.
# Since Cuda toolkit is not backward-compatible, this is not guaranteed to work.
Expand Down
21 changes: 20 additions & 1 deletion tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc
Original file line number Diff line number Diff line change
Expand Up @@ -342,6 +342,7 @@ size_t GPUBFCAllocator::AllocatedSize(void* ptr) {

void GPUBFCAllocator::DumpMemoryLog(size_t num_bytes) {
// For each bin: tally up the total number of chunks and bytes.
// Note that bins hold only free chunks.
for (auto bit : bins_) {
Bin* b = bit.second;

Expand Down Expand Up @@ -389,6 +390,24 @@ void GPUBFCAllocator::DumpMemoryLog(size_t num_bytes) {
LOG(INFO) << c->DebugString(true);
}
}
}

// Next show the the chunks that are in use, and also summarize their
// number by size.
std::map<size_t, int> in_use_by_size;
for (auto& it : ptr_to_chunk_map_) {
const Chunk& c = *it.second;
in_use_by_size[c.size]++;
LOG(INFO) << "Chunk at " << it.first << " of size " << c.size;
}

LOG(INFO) << " Summary of in-use Chunks by size: ";
size_t total_bytes = 0;
for (auto& it : in_use_by_size) {
LOG(INFO) << it.second << " Chunks of size " << it.first << " totalling "
<< strings::HumanReadableNumBytes(it.first * it.second);
total_bytes += (it.first * it.second);
}
LOG(INFO) << "Sum Total of in-use chunks: "
<< strings::HumanReadableNumBytes(total_bytes);
}
} // namespace tensorflow
14 changes: 7 additions & 7 deletions tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.h
Original file line number Diff line number Diff line change
Expand Up @@ -115,14 +115,14 @@ class GPUBFCAllocator : public VisitableAllocator {
};

Chunk* AllocateNewChunk(size_t num_bytes);
void SplitChunk(Chunk* c, size_t num_bytes);
void Merge(Chunk* c1, Chunk* c2);
void FreeAndMaybeCoalesce(Chunk* c);
void InsertFreeChunkIntoBin(Chunk* c);
void SplitChunk(Chunk* c, size_t num_bytes) EXCLUSIVE_LOCKS_REQUIRED(lock_);
void Merge(Chunk* c1, Chunk* c2) EXCLUSIVE_LOCKS_REQUIRED(lock_);
void FreeAndMaybeCoalesce(Chunk* c) EXCLUSIVE_LOCKS_REQUIRED(lock_);
void InsertFreeChunkIntoBin(Chunk* c) EXCLUSIVE_LOCKS_REQUIRED(lock_);
void RemoveFreeChunkFromBin(Chunk* c);
void DeleteChunk(Chunk* c);
void DeleteChunk(Chunk* c) EXCLUSIVE_LOCKS_REQUIRED(lock_);

void DumpMemoryLog(size_t num_bytes);
void DumpMemoryLog(size_t num_bytes) EXCLUSIVE_LOCKS_REQUIRED(lock_);

// A Bin is a collection of similar-sized free chunks.
struct Bin {
Expand Down Expand Up @@ -163,7 +163,7 @@ class GPUBFCAllocator : public VisitableAllocator {
// Structures mutable after construction
mutable mutex lock_;
// Chunk * owned.
std::unordered_map<void*, Chunk*> ptr_to_chunk_map_;
std::unordered_map<void*, Chunk*> ptr_to_chunk_map_ GUARDED_BY(lock_);

// Called once on each region, ASAP.
std::vector<Visitor> region_visitors_;
Expand Down
201 changes: 187 additions & 14 deletions tensorflow/core/example/example.proto
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,21 @@ import "tensorflow/core/example/feature.proto";

package tensorflow;

// Example for a movie recommendation application:
// An Example is a mostly-normalized data format for storing data for
// training and inference. It contains a key-value store (features); where
// each key (string) maps to a Feature message (which is oneof packed BytesList,
// FloatList, or Int64List). This flexible and compact format allows the
// storage of large amounts of typed data, but requires that the data shape
// and use be determined by the configuration files and parsers that are used to
// read and write this format. That is, the Example is mostly *not* a
// self-describing format. In TensorFlow, Examples are read in row-major
// format, so any configuration that describes data with rank-2 or above
// should keep this in mind. For example, to store an M x N matrix of Bytes,
// the BytesList must contain M*N bytes, with M rows of N contiguous values
// each. That is, the BytesList value must store the matrix as:
// .... row 0 .... .... row 1 .... // ........... // ... row M-1 ....
//
// An Example for a movie recommendation application:
// features {
// feature {
// key: "age"
Expand Down Expand Up @@ -58,7 +72,7 @@ package tensorflow;
// }
// }
//
// A conformant data set obeys the following conventions:
// A conformant Example data set obeys the following conventions:
// - If a Feature K exists in one example with data type T, it must be of
// type T in all other examples when present. It may be omitted.
// - The number of instances of Feature K list data may vary across examples,
Expand All @@ -72,23 +86,182 @@ message Example {
Features features = 1;
};

// Example representing a ranking instance.
message RankingExample {
Features context = 1;
repeated Features positive = 2;
repeated Features negative = 3;
};
// A SequenceExample is an Example representing one or more sequences, and
// some context. The context contains features which apply to the entire
// example. The feature_lists contain a key, value map where each key is
// associated with a repeated set of Features (a FeatureList).
//
// A SequenceExample for a movie recommendation application:
//
// context: {
// feature: {
// key : "locale"
// value: {
// bytes_list: {
// value: [ "pt_BR" ]
// }
// }
// }
// feature: {
// key : "age"
// value: {
// float_list: {
// value: [ 19.0 ]
// }
// }
// }
// feature: {
// key : "favorites"
// value: {
// bytes_list: {
// value: [ "Majesty Rose", "Savannah Outen", "One Direction" ]
// }
// }
// }
// }
// feature_lists: {
// feature_list: {
// key : "movie_ratings"
// value: {
// feature: {
// float_list: {
// value: [ 4.5 ]
// }
// }
// feature: {
// float_list: {
// value: [ 5.0 ]
// }
// }
// }
// }
// feature_list: {
// key : "movie_names"
// value: {
// feature: {
// bytes_list: {
// value: [ "The Shawshank Redemption" ]
// }
// }
// feature: {
// bytes_list: {
// value: [ "Fight Club" ]
// }
// }
// }
// }
// }
//
// A conformant SequenceExample data set obeys the following conventions:
//
// Context:
// - All conformant context features K must obey the same conventions as
// a conformant Example's features (see above).
// Feature lists:
// - A FeatureList L may be missing in an example; it is up to the
// parser configuration to determine if this is allowed or considered
// an empty list (zero length).
// - If a FeatureList L exists, it may be empty (zero length).
// - If a FeatureList L is non-empty, all features within the FeatureList
// must have data type T, and all features within the FeatureList must
// have the same size.
// - If a FeatureList L exists in one example with data type T,
// it must be of type T in all other examples when present.
// - If a FeatureList L exists in one example having features' sizes all S,
// these sizes must be S in all other examples when present.
//
// Examples of conformant and non-conformant examples' FeatureLists:
//
// Conformant FeatureLists:
// feature_lists: { feature_list: {
// key: "movie_ratings"
// value: { feature: { float_list: { value: [ 4.5 ] } }
// feature: { float_list: { value: [ 5.0 ] } } }
// } }
//
// Non-conformant FeatureLists (mismatched types):
// feature_lists: { feature_list: {
// key: "movie_ratings"
// value: { feature: { float_list: { value: [ 4.5 ] } }
// feature: { int64_list: { value: [ 5 ] } } }
// } }
//
// Non-conformant FeatureLists (mismatched sizes):
// feature_lists: { feature_list: {
// key: "movie_ratings"
// value: { feature: { float_list: { value: [ 4.5 ] } }
// feature: { float_list: { value: [ 5.0, 6.0 ] } } }
// } }
//
// Conformant pair of SequenceExample
// feature_lists: { feature_list: {
// key: "movie_ratings"
// value: { feature: { float_list: { value: [ 4.5 ] } }
// feature: { float_list: { value: [ 5.0 ] } } }
// } }
// and:
// feature_lists: { feature_list: {
// key: "movie_ratings"
// value: { feature: { float_list: { value: [ 4.5 ] } }
// feature: { float_list: { value: [ 5.0 ] } }
// feature: { float_list: { value: [ 2.0 ] } } }
// } }
//
// Conformant pair of SequenceExample
// feature_lists: { feature_list: {
// key: "movie_ratings"
// value: { feature: { float_list: { value: [ 4.5 ] } }
// feature: { float_list: { value: [ 5.0 ] } } }
// } }
// and:
// feature_lists: { feature_list: {
// key: "movie_ratings"
// value: { }
// } }
//
// Conditionally conformant pair of SequenceExample, the parser configuration
// determines if the second feature_lists is consistent (zero-length) or
// invalid (missing "movie_ratings"):
// feature_lists: { feature_list: {
// key: "movie_ratings"
// value: { feature: { float_list: { value: [ 4.5 ] } }
// feature: { float_list: { value: [ 5.0 ] } } }
// } }
// and:
// feature_lists: { }
//
// Non-conformant pair of SequenceExample (mismatched types)
// feature_lists: { feature_list: {
// key: "movie_ratings"
// value: { feature: { float_list: { value: [ 4.5 ] } }
// feature: { float_list: { value: [ 5.0 ] } } }
// } }
// and:
// feature_lists: { feature_list: {
// key: "movie_ratings"
// value: { feature: { int64_list: { value: [ 4 ] } }
// feature: { int64_list: { value: [ 5 ] } }
// feature: { int64_list: { value: [ 2 ] } } }
// } }
//
// Non-conformant pair of SequenceExample (mismatched sizes)
// feature_lists: { feature_list: {
// key: "movie_ratings"
// value: { feature: { float_list: { value: [ 4.5 ] } }
// feature: { float_list: { value: [ 5.0 ] } } }
// } }
// and:
// feature_lists: { feature_list: {
// key: "movie_ratings"
// value: { feature: { float_list: { value: [ 4.0, 5.0 ] } }
// feature: { float_list: { value: [ 5.0, 3.0 ] } }
// } }

// Example representing a sequence.
// The context contains features which apply to the entire sequence.
// Each element in example represents an entry in the sequence.
message SequenceExample {
Features context = 1;
repeated Features features = 2;
FeatureLists feature_lists = 2;
};

// Example representing a list of feature maps.
// The context contains features which apply to all feature maps.
message InferenceExample {
Features context = 1;
repeated Features features = 2;
Expand Down
Loading

0 comments on commit bf6b536

Please sign in to comment.