Skip to content

Commit

Permalink
Added basic script for combining JSON datasets
Browse files Browse the repository at this point in the history
Currently very inflexible, only supports datasets in a folder named
after the dataset, with the data split into files named after the areas.
  • Loading branch information
jtsymon committed Jul 30, 2016
0 parents commit 3b04793
Showing 1 changed file with 20 additions and 0 deletions.
20 changes: 20 additions & 0 deletions combiner.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
#!/usr/bin/env sh

mkdir -p combined-datasets
for area in $(find processed-datasets/ -type f -printf '%f\n' | sort -u); do
(
echo '{'
first=1
for dataset in $(find processed-datasets/ -name "$area"); do
if [[ $first -eq 1 ]]; then
first=0
else
echo ","
fi
echo -n '"'$(basename $(dirname $dataset))'":'
cat $dataset
done
echo
echo '}'
) > combined-datasets/$area
done

0 comments on commit 3b04793

Please sign in to comment.