Add Docker image to run code formatter

CONTRIBUTING.md

@@ -51,48 +51,10 @@ The code was originally contributed by Google so it conforms to the
 [Google C++ Style Guide](https://google.github.io/styleguide/cppguide.html) and
 [Google Java Style Guide](https://google.github.io/styleguide/javaguide.html).
 
-The `format_code.sh` script is the convinent way to format your staged files. It uses the
-[clang-format](https://clang.llvm.org/docs/ClangFormat.html) for checking your C/C++ files,
-[buildifier](https://github.com/bazelbuild/buildtools/tree/master/buildifier) for bazel files and
-[google-java-format](https://github.com/google/google-java-format) for Javafiles.
-
-The `format_all_code.sh` script is the convinent way to format all files.
-
-
-##### Using docker image
-These are installed in the docker image. To make the docker image run:
-```
-make output/mlperf_mobile_docker_1_0.stamp
-```
-
-You can format your staged files by running:
-```bash
-bash android/formatters/docker_run.sh android/format_code.sh
-```
-
-You can format all files by running:
-```bash
-bash android/formatters/docker_run.sh android/format_all_code.sh
-```
-
-##### Using native OS
-
-To install its dependencies, do:
-
-```bash
-sudo apt install clang-format-10
-go get github.com/bazelbuild/buildtools/buildifier
-mkdir /opt/formatters/
-curl --output /opt/formatters/google-java-format-1.9-all-deps.jar -L https://github.com/google/google-java-format/releases/download/google-java-format-1.9/google-java-format-1.9-all-deps.jar
-```
-
-Then you can format your staged files by running:
-```bash
-bash android/format_code.sh
-```
-
-Optionally, you can add it to pre-commit hook by running:
-```bash
-sed -i -e '$a\' -e 'bash `git rev-parse --show-toplevel`/android/format_code.sh && git add *' \
--e "#android/format_code.sh#d" .git/hooks/pre-commit
-```
+From the root directory you can run the command `make format` to format all files or `make format/<bazel|java|clang|dart>` to format only certain files in the directory.
+See [format.mk](tools/formatter/format.mk) for more commands.
+
+Running `make format` requires you have all the tools installed locally on your computer.
+For your convenience you can use a Docker image to format code files by running `make docker/format`.
+Note: if you have an error related to flutter when running `make docker/format`, 
+try to run `cd flutter && flutter clean` to clear flutter cache, then try `make docker/format` again.

Makefile

@@ -18,7 +18,7 @@
 
 all: app
 
-include format.mk
+include tools/formatter/format.mk
 
 # TFLite is the default backend
 include mobile_back_tflite/tflite_backend.mk

android/README.md

@@ -93,14 +93,14 @@ make WITH_QTI=1 WITH_MEDIATEK=1 app
 Note: Follow the instruction of the backend vendor (in `mobile_back_*` folders at root level) to build for that backend.
 Some backends may contain proprietary code. In that case, please contact MLCommons to get help.
 
-# Install the app with the command:
+Install the app with the command:
+```
 adb install -r build/mlperf_app.apk
 ```
 
 If you want to build an instrumented test APK you should add target `//androidTest:mlperf_test_app`, i.e. :
 ```bash
 make test_app
-
 adb install -r build/mlperf_test_app.apk
 ```
 
@@ -134,7 +134,7 @@ The following steps are required to add your backend:
 2. Implement all the C API backend interfaces defined in android/cpp/c/backend_c.h
 3. Add a bazel BUILD file to create lib[vendor]backend.so shared library
 
-You can look at the [default TFLite backend implementation](../../mobile_back_tflite) for a reference.
+You can look at the [default TFLite backend implementation](../mobile_back_tflite) for a reference.
 The following steps are required to add your backend:
 
 ### Unified app changes

android/formatters/README.md

@@ -2,9 +2,9 @@
 
 ### Prerequisites
 You should have in $PATH:
- - `java` to run Java code formating
- - `clang-format` to run C++ code formating
- - `buildifier` ro run bazel congiguration files formating
+ - `java` to run Java code formatting
+ - `clang-format` to run C++ code formatting
+ - `buildifier` ro run bazel configuration files formatting
 
 
 ## Development
@@ -15,6 +15,50 @@ Example:
 $ sh android/formatters/docker_run.sh android/format_code.sh
 ```
 
+The `format_code.sh` script is the convenient way to format your staged files. It uses the
+[clang-format](https://clang.llvm.org/docs/ClangFormat.html) for checking your C/C++ files,
+[buildifier](https://github.com/bazelbuild/buildtools/tree/master/buildifier) for bazel files and
+[google-java-format](https://github.com/google/google-java-format) for Java files.
+
+The `format_all_code.sh` script is the convenient way to format all files.
+
+### Using docker image
+These are installed in the docker image. To make the docker image run:
+```
+make output/mlperf_mobile_docker_1_0.stamp
+```
+
+You can format your staged files by running:
+```bash
+bash android/formatters/docker_run.sh android/format_code.sh
+```
+
+You can format all files by running:
+```bash
+bash android/formatters/docker_run.sh android/format_all_code.sh
+```
+
+### Using native OS
+
+To install its dependencies, do:
+
+```bash
+sudo apt install clang-format-10
+go get github.com/bazelbuild/buildtools/buildifier
+mkdir /opt/formatters/
+curl --output /opt/formatters/google-java-format-1.9-all-deps.jar -L https://github.com/google/google-java-format/releases/download/google-java-format-1.9/google-java-format-1.9-all-deps.jar
+```
+
+Then you can format your staged files by running:
+```bash
+bash android/format_code.sh
+```
+
+Optionally, you can add it to pre-commit hook by running:
+```bash
+sed -i -e '$a\' -e 'bash `git rev-parse --show-toplevel`/android/format_code.sh && git add *' \
+-e "#android/format_code.sh#d" .git/hooks/pre-commit
+```
 
 ## CI
 The scripts in the current directory are intended to be used primarily in the CI process. "CI" should be used as the only parameter in that case.

flutter/pubspec.yaml

@@ -68,3 +68,5 @@ flutter:
 
 import_sorter:
   comments: false
+  ignored_files:
+    - \/.pub-cache\/*

tools/formatter/Dockerfile

@@ -0,0 +1,68 @@
+# Copyright 2021 The MLPerf Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##########################################################################
+
+# This Docker image is used to run code formatter
+
+FROM ubuntu:20.04
+
+ENV DEBIAN_FRONTEND="noninteractive"
+
+# Install dev packages
+RUN apt-get update && apt-get upgrade -y && apt-get install -y --no-install-recommends \
+    gcc make git curl wget zip unzip tar gzip
+
+ENV FORMATTER_HOME /opt/formatters/bin
+ENV PATH=${PATH}:${FORMATTER_HOME}
+RUN mkdir -p ${FORMATTER_HOME}
+
+# Add Java formatter
+RUN apt-get update && apt-get install -y \
+    openjdk-11-jdk-headless
+RUN curl --output `dirname ${FORMATTER_HOME}`/google-java-format-1.9-all-deps.jar \
+         -L https://github.com/google/google-java-format/releases/download/google-java-format-1.9/google-java-format-1.9-all-deps.jar
+
+# Add buildifier
+RUN apt-get update && apt-get install -y \
+    golang-1.16-go
+ENV PATH=$PATH:/usr/lib/go-1.16/bin:~/go/bin
+RUN /usr/lib/go-1.16/bin/go get github.com/bazelbuild/buildtools/buildifier
+RUN cp ~/go/bin/buildifier ${FORMATTER_HOME}/
+
+# Add CLang formatter
+RUN apt-get update && apt-get install -y \
+    clang-format-10
+RUN cp $(which clang-format-10) ${FORMATTER_HOME}/clang-format
+
+# Add dos2unix to format line endings
+RUN apt-get update && apt-get install -y \
+    dos2unix
+
+# Setup permission to run Flutter as non-root user
+# this is needed to share Flutter cache with host machine
+ARG UNAME=mlperf
+ARG UID=1000
+ARG GID=1000
+RUN groupadd -g $GID -o $UNAME
+RUN useradd -m -u $UID -g $GID -o -s /bin/bash $UNAME
+USER $UNAME
+
+# Add Flutter SDK
+RUN git clone --branch "2.5.3" --single-branch --depth 1 \
+    https://github.com/flutter/flutter.git /home/$UNAME/flutter
+ENV PATH=$PATH:/home/$UNAME/flutter/bin:/home/$UNAME/flutter/bin/cache/dart-sdk/bin
+RUN flutter doctor -v
+
+
+CMD /bin/bash

format.mk → tools/formatter/format.mk

@@ -74,3 +74,21 @@ lint/line-endings:
 		echo -e "found files with CRLF line endings: \n$$_files"; false; \
 	else echo all files have unix line endings; \
 	fi
+
+output/docker_mlperf_formatter.stamp: tools/formatter/Dockerfile
+	docker build --progress=plain \
+		--build-arg UID=`id -u` --build-arg GID=`id -g` \
+		-t mlperf/formatter tools/formatter
+	# need to clean flutter cache first else we will have error when running `dart run import_sorter:main` later in docker
+	cd flutter && flutter clean
+	touch $@
+
+.PHONY: docker/format
+docker/format: output/docker_mlperf_formatter.stamp
+	docker run -it --rm \
+    	-v ~/.pub-cache:/home/mlperf/.pub-cache \
+    	-v ~/.config/flutter:/home/mlperf/.config/flutter \
+    	-v $(CURDIR):/home/mlperf/mobile_app_open \
+    	-w /home/mlperf/mobile_app_open \
+    	-u `id -u`:`id -g` \
+    	mlperf/formatter bash -c "make format"