Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Syncing to latest updates. #1

Merged
merged 121 commits into from
Feb 4, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
121 commits
Select commit Hold shift + click to select a range
0fe15d2
Merge pull request #3 from hackforla/dev
ryanmswan Dec 10, 2019
162bdb4
Merge pull request #4 from hackforla/dev
ryanmswan Dec 11, 2019
112030e
bring up to date with dev
ryanmswan Dec 11, 2019
9c36f78
Setup eslint
Dec 11, 2019
6417d7d
Fixed footer
Dec 11, 2019
e25bc1c
Cleaned up constants
Dec 11, 2019
ae1a101
Added CSV dump functionality
ryanmswan Dec 11, 2019
d750755
Merge pull request #125 from ryanmswan/dev
ryanmswan Dec 11, 2019
693bab0
fixing header datafilter state
Dec 11, 2019
9cd7aca
added onboarding document draft
ryanmswan Dec 11, 2019
22047e4
Merge pull request #126 from ryanmswan/dev
ryanmswan Dec 11, 2019
774168c
Updated data location
Dec 11, 2019
a89388c
Added first iteration of onboarding script, does not auto-install
sellnat77 Dec 13, 2019
8a2be6d
Adding reporting service WIP need sqlalchemy query
sellnat77 Dec 13, 2019
7da3d47
updated .gitignore for datasets and docker compose
rgao Dec 13, 2019
38d8fbb
Created reporting worker that retrieves data from postgres
sellnat77 Dec 14, 2019
e10b829
Merge pull request #127 from hackforla/onboardingScript
sellnat77 Dec 14, 2019
38bbcfa
Merge pull request #128 from hackforla/reporting
sellnat77 Dec 14, 2019
a9dd13a
Updated onboarding doc
ryanmswan Dec 15, 2019
d0198d9
Merge pull request #130 from ryanmswan/dev
ryanmswan Dec 15, 2019
24d3004
further updates to onboarding doc
ryanmswan Dec 15, 2019
71fba00
Merge pull request #131 from ryanmswan/dev
ryanmswan Dec 15, 2019
13ed3da
Merge branch 'dev' of https://github.com/hackforla/311-data into dev
rgao Dec 16, 2019
8b3bb38
set an empty string as placeholder for the value of the dataset param…
rgao Dec 16, 2019
630239d
added framework and placeholder queries for timetoclose and frequency…
rgao Dec 16, 2019
989bea2
cleaned up empty files and deprecated code in dataAnalysis folder; ke…
rgao Dec 17, 2019
7abd936
combined all deprecated dataAnalysis code into utils.py, and added do…
rgao Dec 18, 2019
b9c3f79
Merge pull request #132 from rgao/dev
sellnat77 Dec 18, 2019
b5c92f7
updated index filename
Dec 18, 2019
e0df146
Cleaned up code
Dec 18, 2019
325e26f
cleaned marker key
Dec 18, 2019
11fdcba
Delete 2018_full.csv
gennaer Dec 18, 2019
3057074
Merge pull request #134 from gennaer/dev
sellnat77 Dec 18, 2019
d3ec81d
Merge pull request #135 from hackforla/dev
gennaer Dec 18, 2019
1067ee2
Removed csv files
sellnat77 Dec 21, 2019
141d153
Merge branch 'dev' of github.com:hackforla/311-data into dev
sellnat77 Dec 21, 2019
af3ab81
Removed LFS attr file:
sellnat77 Dec 21, 2019
3ebffc0
Merge branch 'master' of github.com:hackforla/311-data into dev
sellnat77 Dec 21, 2019
6edc5cf
First revision of getting started doc
sellnat77 Dec 22, 2019
695469e
Fixed build and test scripts
brodly Dec 24, 2019
68001a7
Merge pull request #140 from hackforla/fix_scripts
sellnat77 Dec 24, 2019
03ce5a8
Merge pull request #139 from hackforla/getting_started
sellnat77 Dec 24, 2019
263ec01
Merge branch 'dev' into eslint
sellnat77 Dec 24, 2019
ec96659
Update Continuous_Delivery.yml
brodly Dec 24, 2019
8369a95
Update Continuous_Integration_Backend.yml
brodly Dec 24, 2019
5e8beeb
Merge pull request #141 from hackforla/brodly-patch-1
sellnat77 Dec 24, 2019
a941a2a
Merge branch 'master' into dev
sellnat77 Dec 24, 2019
955c464
Merge pull request #136 from hackforla/dev
sellnat77 Dec 24, 2019
05891b9
Merge branch 'dev' into eslint
sellnat77 Dec 24, 2019
c52d9b8
Merge branch 'master' of https://github.com/hackforla/311-data into e…
brodly Dec 24, 2019
949fde9
Merge pull request #133 from hackforla/eslint
sellnat77 Dec 24, 2019
d7c1497
fixed build issues
brodly Dec 24, 2019
2ff8fe6
Merge pull request #142 from hackforla/eslint
sellnat77 Dec 24, 2019
b20fb60
successfully returns data with sql->pandas df->json conversion
rgao Dec 26, 2019
4784482
successfully converted dates to a consistent human-readable format as…
rgao Dec 29, 2019
ad680c6
added function for returning time to close/service for all data
rgao Dec 29, 2019
9197555
Merge pull request #146 from rgao/dev
sellnat77 Dec 29, 2019
c4f2d39
Removed git-lfs, onboarding docs, onboarding scripts
sellnat77 Dec 22, 2019
1635333
Merge pull request #144 from hackforla/onboarding_Fix
sellnat77 Dec 29, 2019
bc1ee57
Fixed webpack side of dockerfile for front end
sellnat77 Dec 29, 2019
929b604
Merge pull request #147 from hackforla/FE_Dockerfile_Fix
sellnat77 Dec 29, 2019
b1269cf
Adjusted environments and scripts to clean up onboarding
sellnat77 Dec 29, 2019
926a04e
Merge pull request #148 from hackforla/onboarding-adjustments
sellnat77 Dec 29, 2019
ebbe972
Merge pull request #150 from hackforla/dev
sellnat77 Dec 30, 2019
c8f05b2
completed first iteration of time to close module; returns summary da…
rgao Jan 5, 2020
4cf2f4c
Merge pull request #152 from rgao/dev
sellnat77 Jan 5, 2020
b93fddf
updated timetoclose to add viewing all entries as an option to summar…
rgao Jan 6, 2020
5cd8ed2
Merge pull request #5 from hackforla/dev
ryanmswan Jan 7, 2020
214e22c
added redux
Jan 8, 2020
c8b7ec4
implemented basic functionality for frequency request module
rgao Jan 13, 2020
2776c16
Merge pull request #168 from rgao/dev
sellnat77 Jan 15, 2020
ff892b5
Updated Sanic app to use multiple workers.
Jan 15, 2020
6a2e5a3
Fix #105: Updated Sanic app to use multiple workers.
Jan 15, 2020
3b6258c
Merge branch '105-BACK-Async' of github.com:GeorgeSu/311-data into 10…
Jan 15, 2020
5b2b635
Removed test hello_world function.
Jan 15, 2020
659bcb3
Merge branch 'dev' of https://github.com/hackforla/311-data into 105-…
Jan 15, 2020
2df04b7
Updated test API endpoint name.
Jan 15, 2020
fcbe38b
Ingestion pipeline updated to functional
ryanmswan Jan 15, 2020
b173189
completed frequency module: added date selection function and aggrega…
rgao Jan 15, 2020
418f9f9
Merge pull request #176 from GeorgeSu/105-BACK-Async
sellnat77 Jan 15, 2020
d568276
Update actions to build on PR instead of push
sellnat77 Jan 15, 2020
144c3f5
Merge pull request #6 from hackforla/dev
ryanmswan Jan 15, 2020
1b13543
Merge pull request #177 from ryanmswan/dev
sellnat77 Jan 15, 2020
4debba0
Merge branch 'dev' into PRActions
sellnat77 Jan 15, 2020
dbe33a4
Added sqlIngest documentation notebook
ryanmswan Jan 15, 2020
49c4200
Minor changes to fetchSocrataFull method
ryanmswan Jan 15, 2020
69623d0
completed frequency module; added functionality to filter by council
rgao Jan 19, 2020
33edc49
Merge remote-tracking branch 'upstream/dev' into dev
rgao Jan 19, 2020
048eb4e
updated docstring for freqency module
rgao Jan 19, 2020
9c42a85
Merge pull request #181 from hackforla/PRActions
sellnat77 Jan 19, 2020
993360b
Merge branch 'dev' into dev
sellnat77 Jan 19, 2020
0a299bc
Merge branch 'dev' of github.com:hackforla/311-data into feature_redux
sellnat77 Jan 19, 2020
0124c05
Merge pull request #182 from hackforla/feature_redux
sellnat77 Jan 19, 2020
a4410e7
Merge branch 'dev' into dev
sellnat77 Jan 19, 2020
1f4011f
Merge pull request #186 from rgao/dev
sellnat77 Jan 19, 2020
b187cff
Updated webpack to support the CD scripts
sellnat77 Jan 19, 2020
6d94097
Merge pull request #187 from hackforla/restoreCD
sellnat77 Jan 19, 2020
9c4661f
Merge branch 'master' into dev
sellnat77 Jan 19, 2020
e1b1e85
Added sodapy to requirements.txt
sellnat77 Jan 19, 2020
4e36878
Merge pull request #189 from hackforla/sodapy
sellnat77 Jan 19, 2020
4e3ff73
Making sure CI runs on all PRs
sellnat77 Jan 19, 2020
9caa697
Merge pull request #190 from hackforla/CIonPR
sellnat77 Jan 19, 2020
e372cd0
Merge pull request #188 from hackforla/dev
ryanmswan Jan 19, 2020
024bba4
Merge pull request #8 from hackforla/dev
ryanmswan Jan 19, 2020
cc8ebb7
Added linting and correctly implemented the backend github action
sellnat77 Jan 19, 2020
a0df5d6
Merge pull request #192 from hackforla/101-BCK-Tests
ryanmswan Jan 20, 2020
e5db4a9
No Flake errors/warnings allowed
sellnat77 Jan 22, 2020
18734bf
Merge pull request #194 from hackforla/CIFix
ryanmswan Jan 22, 2020
a2ce5a9
added changed in local fork to feature modules
rgao Jan 23, 2020
843a2ea
Updated sqlIngest pipeline
ryanmswan Jan 25, 2020
d0f3338
Fixing conflicts in sqlIngest before merge
ryanmswan Jan 26, 2020
784a285
Merge branch 'hackforla-dev' into dev
ryanmswan Jan 26, 2020
a61e163
Updated sqlIngest documentation
ryanmswan Jan 26, 2020
3a693ea
Removed newline
sellnat77 Jan 26, 2020
4ea4b6c
Merge pull request #206 from hackforla/feature_modules
sellnat77 Jan 26, 2020
9dd9c52
Fixed linting issues
ryanmswan Jan 26, 2020
9850837
Merge branch 'dev' into dev
sellnat77 Jan 26, 2020
6bcda51
Merge pull request #204 from ryanmswan/dev
sellnat77 Jan 26, 2020
34781fc
Merge branch 'master' of github.com:hackforla/311-data into devUpdate
sellnat77 Jan 29, 2020
52d999a
Merge pull request #209 from hackforla/devUpdate
sellnat77 Jan 29, 2020
5ead2e3
Merge pull request #208 from hackforla/dev
sellnat77 Jan 29, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions .eslintrc.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
module.exports = {
env: {
browser: true,
es6: true,
node: true,
},
extends: [
'airbnb',
'eslint:recommended',
'plugin:react/recommended',
],
globals: {
Atomics: 'readonly',
SharedArrayBuffer: 'readonly',
},
parser: 'babel-eslint',
parserOptions: {
ecmaFeatures: {
jsx: true,
},
ecmaVersion: 2018,
sourceType: 'module',
},
plugins: [
'react',
],
rules: {
'linebreak-style': 'off',
},
};
2 changes: 2 additions & 0 deletions .flake8
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
[flake8]
exclude = node_modules/*, dataAnalysis/*
2 changes: 0 additions & 2 deletions .gitattributes

This file was deleted.

18 changes: 9 additions & 9 deletions .github/workflows/Continuous_Delivery.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,17 +19,17 @@ jobs:
- name: Install Packages
run: npm install
- name: Build project
run: npm run-script build
run: npm run build
- name: Run Tests
run: export CI=true && npm run test -- --coverage
env:
MAPBOX_TOKEN: ${{ secrets.MAPBOX_TOKEN }}
# DISABLING DEPLOYMENT TO PRESERVE "PRODUCTION"
# - name: Deploy to GH pages
# uses: JamesIves/github-pages-deploy-action@master
# env:
# ACCESS_TOKEN: ${{ secrets.ACCESS_TOKEN }}
# BASE_BRANCH: master # The branch the action should deploy from.
# BRANCH: gh-pages # The branch the action should deploy to.
# FOLDER: build # The folder the action should deploy.
# BUILD_SCRIPT: echo REACT_APP_MAPBOX_TOKEN=${{secrets.MAPBOX_TOKEN}} > .env && npm rebuild node-sass && npm install && npm run-script build # The build script the action should run prior to deploying.
- name: Deploy to GH pages
uses: JamesIves/github-pages-deploy-action@master
env:
ACCESS_TOKEN: ${{ secrets.ACCESS_TOKEN }}
BASE_BRANCH: master # The branch the action should deploy from.
BRANCH: gh-pages # The branch the action should deploy to.
FOLDER: dist # The folder the action should deploy.
BUILD_SCRIPT: echo REACT_APP_MAPBOX_TOKEN=${{secrets.MAPBOX_TOKEN}} > .env && npm rebuild node-sass && npm install && npm run-script build # The build script the action should run prior to deploying.
34 changes: 18 additions & 16 deletions .github/workflows/Continuous_Integration_Backend.yml
Original file line number Diff line number Diff line change
@@ -1,26 +1,28 @@
name: Dev_CI_Backend
on:
push:
branches-ignore:
- master
on: [pull_request]

jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [10.x, 11.x]
python-version: [3.6, 3.7, 3.8]
steps:
- uses: actions/checkout@v1
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v1
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v1
with:
node-version: ${{ matrix.node-version }}
- name: Install Packages
run: npm install
- name: Build project
run: npm run-script build
- name: Run Tests
run: export CI=true && npm run test -- --coverage
env:
MAPBOX_TOKEN: ${{ secrets.MAPBOX_TOKEN }}
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
sudo apt install libpq-dev python3-dev
python -m pip install --upgrade pip
pip install -r server/requirements.txt
- name: Lint with flake8
run: |
# stop the build if there are Python syntax errors or undefined names
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
flake8 . --count --max-complexity=10 --max-line-length=127 --statistics
- name: Test with pytest
run: pytest server
9 changes: 3 additions & 6 deletions .github/workflows/Continuous_Integration_Frontend.yml
Original file line number Diff line number Diff line change
@@ -1,8 +1,5 @@
name: Dev_CI_Frontend
on:
push:
branches-ignore:
- master
on: [pull_request]

jobs:
build:
Expand All @@ -19,8 +16,8 @@ jobs:
- name: Install Packages
run: npm install
- name: Build project
run: npm run-script build
run: npm run build
- name: Run Tests
run: npm run-script test -- --coverage
run: npm run test -- --coverage
env:
MAPBOX_TOKEN: ${{ secrets.MAPBOX_TOKEN }}
10 changes: 7 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -33,11 +33,15 @@ config.js
.env
settings.cfg

docker-compose.yml
Orchestration/docker-compose.yml

__pycache__/

# csv files
/dataAnalysis/rawdata
/dataAnalysis/ryanAnalysis.ipynb
# csv files
/dataAnalysis/datasets
/server/src/static

# checkpoints
/dataAnalysis/.ipynb_checkpoints
server/src/static/temp
18 changes: 6 additions & 12 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,20 +1,14 @@
# base image
FROM node:12.2.0-alpine
RUN npm install webpack webpack-cli -g

WORKDIR /tmp
COPY package.json /tmp/
RUN npm config set registry http://registry.npmjs.org/ && npm install

WORKDIR /app
COPY . /app/
RUN cp -a /tmp/node_modules /app/

WORKDIR /usr/src/app
COPY . /usr/src/app/
RUN cp -a /tmp/node_modules /usr/src/app/
RUN webpack

ENV NODE_ENV=production
ENV PORT=8080
ENV PORT=3000
RUN ls
CMD [ "npm", "run-script", "dev" ]

EXPOSE 8080
CMD [ "node", "server.js" ]
EXPOSE 3000
Loading