Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

M d #272

Merged
merged 85 commits into from
Feb 21, 2020
Merged

M d #272

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
85 commits
Select commit Hold shift + click to select a range
bc29208
icon components
ainneo Feb 5, 2020
7d34fbe
Cleaned up menu for readability. Added Content section
brodly Feb 9, 2020
d1175c5
Created NC Selector Layout and search
brodly Feb 9, 2020
9481435
Merge branch 'dev' of https://github.com/hackforla/310-data into 155_…
brodly Feb 12, 2020
ed536c5
Add proptypes for checked prop
brodly Feb 12, 2020
0c6764d
Added react-datepicker dependency.
adamkendis Feb 12, 2020
57bd94c
Updated babel
brodly Feb 12, 2020
a1a204c
testing checkbox functionality
brodly Feb 12, 2020
2081e00
Checkboxes point to redux
brodly Feb 12, 2020
7ba271b
tabs use anchor tags
brodly Feb 12, 2020
4a6f6f5
CustomDateInput component.
adamkendis Feb 12, 2020
843612b
Modal component with content rendered via props.
adamkendis Feb 13, 2020
3cf8b24
Fixed keyCode typo.
adamkendis Feb 13, 2020
a76ac8d
Added checked prop to Checkbox proptypes.
adamkendis Feb 13, 2020
c78a4e8
Added forwardRef to CustomDateInput. Modified several props.
adamkendis Feb 13, 2020
fe1f98c
Custom DateRangePicker component to render in Modal content.
adamkendis Feb 13, 2020
7f25e93
Style changes. Replaced standard button with Button component.
adamkendis Feb 13, 2020
aea2011
Added very high zIndex to Footer and Header to keep them on top of mo…
adamkendis Feb 13, 2020
0dcccd4
Moved CustomDateInput and DateRangePicker to menu/DateSelector.
adamkendis Feb 13, 2020
862f601
Moved date selector components.
adamkendis Feb 13, 2020
f91b28f
DateRangePicker styling.
adamkendis Feb 13, 2020
09d4b3e
Minor changes to Dropdown/Item props, styles.
adamkendis Feb 13, 2020
9de6ce1
DateSelector main component. getDates not yet implemented.
adamkendis Feb 13, 2020
3c149ba
Added moment dependency.
adamkendis Feb 13, 2020
204af44
Styles on DateRangePicker.
adamkendis Feb 13, 2020
18dfd65
DateSelector component with getDate implemented. Need to connect to r…
adamkendis Feb 13, 2020
d5fced2
Replaced redux date-related action types and updated reducer.
adamkendis Feb 13, 2020
8dba2cc
Date-related redux action creators and reducers.
adamkendis Feb 13, 2020
5fb250a
DateSelector connected to redux. All working except custom date range.
adamkendis Feb 13, 2020
68c4124
DateSelector fully hooked up to redux.
adamkendis Feb 13, 2020
af89b90
Redux data.js cleanup.
adamkendis Feb 13, 2020
a01ac67
Connected Modal to redux. Modal closes on esc keydown.
adamkendis Feb 13, 2020
5b0f13e
Connected DateRangePicker to redux.
adamkendis Feb 13, 2020
5a0a128
Comments in CustomDateInput.
adamkendis Feb 13, 2020
2b9b994
Fixed broken year selection dropdown.
adamkendis Feb 14, 2020
0b242d4
Renamed to Icon, added extra props.
adamkendis Feb 14, 2020
3efee03
Merge branch 'dev' into icons
adamkendis Feb 14, 2020
24970fe
Removed empty line.
adamkendis Feb 14, 2020
4d34e91
Merge pull request #234 from hackforla/icons
brodly Feb 15, 2020
7551cb7
Component handles select logic instead of redux
brodly Feb 16, 2020
0cfbf6a
Removed select logic for NC. Component now handles
brodly Feb 16, 2020
f31dc01
Merge branch 'dev' of https://github.com/hackforla/311-data into 155_…
brodly Feb 16, 2020
a956ffd
Added button icon
brodly Feb 16, 2020
ecd194e
Updated with button icon
brodly Feb 16, 2020
6010cc1
Updated search icon
brodly Feb 16, 2020
7a8d3f2
Redux removed from modal. Modal open state is now controlled by paren…
adamkendis Feb 16, 2020
8c99388
Removed modal-related actions from redux.
adamkendis Feb 16, 2020
e30896f
Removed modal-related redux from DateRangePicker.
adamkendis Feb 16, 2020
a8d42cf
Added className to Dropdown props.
adamkendis Feb 16, 2020
530b027
Moved modal and modal open control to DateSelector. Removed modal-rel…
adamkendis Feb 16, 2020
1b942b8
Merge branch 'dev' into 166_FRONT_dateFilter
adamkendis Feb 16, 2020
92b896f
Added Icon component to Dropdown.
adamkendis Feb 16, 2020
3fcd903
Replaced default close button with Icon component.
adamkendis Feb 16, 2020
2a095de
Removed blank line.
adamkendis Feb 16, 2020
1b6a9ad
Merge pull request #255 from adamkendis/166_FRONT_dateFilter
brodly Feb 17, 2020
dd53485
Merge branch 'dev' of https://github.com/hackforla/311-data into 155_…
brodly Feb 17, 2020
3c2d251
Disabled jsx-a11y for now
brodly Feb 17, 2020
8ca7edf
Merge pull request #259 from hackforla/155_nc_selector
adamkendis Feb 17, 2020
05e7bc8
Added REQUEST_TYPES (type, abbrev, color) to CONSTANTS.
adamkendis Feb 17, 2020
113da70
Changed default color prop in Icon to undefined.
adamkendis Feb 17, 2020
62526b9
Re-ordered REQUEST_TYPES.
adamkendis Feb 18, 2020
d0e4566
Added style prop to Checkbox.
adamkendis Feb 18, 2020
37d4085
Dropdown icon color change.
adamkendis Feb 18, 2020
7152189
DateSelector style changes.
adamkendis Feb 18, 2020
bdd3974
RequestTypeSelector component mocked up.
adamkendis Feb 18, 2020
022796e
Request type redux actions, reducers, select/deselect all states.
adamkendis Feb 18, 2020
8a871c0
Connected RequestTypeSelector to redux.
adamkendis Feb 18, 2020
af94d3c
Removed unused requestType field from initialState.
adamkendis Feb 18, 2020
c10a409
Added DateSelector and RequestTypeSelector to Menu.
adamkendis Feb 18, 2020
fd42ea5
Moved column arrays outside of RequestTypeSelector function body.
adamkendis Feb 18, 2020
f18d88e
Minor hack for imports
sellnat77 Feb 19, 2020
32bc0dd
Merge pull request #262 from adamkendis/156_FRONT_requestTypeSelector
brodly Feb 19, 2020
e35e9c2
Added sass/css/webpack dependencies.
adamkendis Feb 19, 2020
91c7331
Fixes #265 Added pinService and started data layer abstraction
sellnat77 Feb 19, 2020
661dd00
Merge pull request #266 from hackforla/265_BE_Pinmap
sellnat77 Feb 19, 2020
44a0bce
Merge branch 'dev' into DBOrmFix
sellnat77 Feb 19, 2020
594912a
Updated webpack config for Bulma sass customization.
adamkendis Feb 19, 2020
01f27af
Added styles.scss import to index. Removed bulma direct imports.
adamkendis Feb 19, 2020
bc6ef66
Main styles.scss stylesheet.
adamkendis Feb 19, 2020
9502036
Added comments to styles.scss.
adamkendis Feb 20, 2020
825675f
Merge branch 'dev' into 254_FRONT_sassSetup
adamkendis Feb 20, 2020
ddd1a15
Merge pull request #263 from hackforla/DBOrmFix
ryanmswan Feb 20, 2020
4e6084a
Merge branch 'dev' into 254_FRONT_sassSetup
sellnat77 Feb 21, 2020
34ac1ed
Merge pull request #267 from adamkendis/254_FRONT_sassSetup
brodly Feb 21, 2020
c35a3c3
Merge branch 'master' of github.com:hackforla/311-data into m-d
sellnat77 Feb 21, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 8 additions & 4 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,11 @@
"html-webpack-plugin": "^3.2.0",
"jest": "^24.9.0",
"leaflet": "^1.5.1",
"moment": "^2.24.0",
"proptypes": "^1.1.0",
"react": "^16.8.6",
"react-burger-menu": "^2.6.13",
"react-datepicker": "^2.12.1",
"react-dom": "^16.8.6",
"react-leaflet": "^2.4.0",
"react-leaflet-choropleth": "^2.0.0",
Expand All @@ -39,10 +41,10 @@
"deploy": "gh-pages -d dist"
},
"devDependencies": {
"@babel/core": "^7.7.2",
"@babel/core": "^7.8.4",
"@babel/plugin-proposal-class-properties": "^7.7.0",
"@babel/preset-env": "^7.7.1",
"@babel/preset-react": "^7.7.0",
"@babel/preset-env": "^7.8.4",
"@babel/preset-react": "^7.8.3",
"babel-eslint": "^10.0.3",
"babel-loader": "^8.0.6",
"css-loader": "^3.2.0",
Expand All @@ -54,12 +56,14 @@
"eslint-plugin-jsx-a11y": "^6.2.3",
"eslint-plugin-react": "^7.17.0",
"eslint-plugin-react-hooks": "^1.7.0",
"extract-text-webpack-plugin": "^4.0.0-beta.0",
"file-loader": "^4.2.0",
"mini-css-extract-plugin": "^0.9.0",
"node-sass": "^4.12.0",
"sass-loader": "^8.0.0",
"style-loader": "^1.0.0",
"webpack": "^4.41.2",
"webpack-cli": "^3.3.10",
"webpack-cli": "^3.3.11",
"webpack-dev-server": "^3.9.0"
}
}
1 change: 1 addition & 0 deletions public/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.3.1/css/all.css" integrity="sha384-mzrmE5qonljUremFsqc01SB46JvROS7bZs3IO2EmfFsd15uHvIt+Y8vEf7N7fWAU" crossorigin="anonymous">
<link rel="stylesheet" href="https://unpkg.com/react-vis/dist/style.css">
<link rel="stylesheet" href="https://unpkg.com/[email protected]/dist/leaflet.css"
integrity="sha512-xwE/Az9zrjBIphAcBb3F6JVqxf46+CDLwfLMHloNu6KEQCAWi6HcDUbeOfBIptF7tcCzusKFjFw2yuvEpDL9wQ=="
Expand Down
30 changes: 24 additions & 6 deletions server/src/app.py
Original file line number Diff line number Diff line change
@@ -1,15 +1,16 @@
import os
from sanic import Sanic
from sanic.response import json
from services.time_to_close import time_to_close
from services.frequency import frequency
from services.ingress_service import ingress_service
from configparser import ConfigParser
from threading import Timer
from multiprocessing import cpu_count
from services.sqlIngest import DataHandler
from datetime import datetime
from multiprocessing import cpu_count

from services.time_to_close import time_to_close
from services.frequency import frequency
from services.pinService import PinService
from services.ingress_service import ingress_service
from services.sqlIngest import DataHandler

app = Sanic(__name__)

Expand Down Expand Up @@ -77,7 +78,8 @@ async def ingest(request):
return json({"error": "'years' parameter is required."})
years = set([int(year) for year in request.args.get("years").split(",")])
if not all(year in ALLOWED_YEARS for year in years):
return json({"error": f"'years' parameter values must be one of {ALLOWED_YEARS}"})
return json({"error":
f"'years' param values must be one of {ALLOWED_YEARS}"})
loader = DataHandler()
loader.loadConfig(configFilePath='./settings.cfg')
loader.populateFullDatabase(yearRange=years)
Expand All @@ -99,6 +101,22 @@ async def delete(request):
return json(return_data)


@app.route('/pins', methods=["POST"])
async def pinMap(request):
pin_worker = PinService(app.config['Settings'])
postArgs = request.json
start = postArgs.get('startDate', '2015-01-01')
end = postArgs.get('endDate', '2015-12-31 01:01:01')
ncs = postArgs.get('ncList', ['SHERMAN OAKS NC'])
requests = postArgs.get('requestTypes', ['Bulky Items'])

return_data = await pin_worker.get_base_pins(startDate=start,
endDate=end,
ncList=ncs,
requestTypes=requests)
return json(return_data)


@app.route('/test_multiple_workers')
async def test_multiple_workers(request):
Timer(10.0, print, ["Timer Test."]).start()
Expand Down
39 changes: 39 additions & 0 deletions server/src/services/dataService.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
import sqlalchemy as db
import pandas as pd


class DataService(object):
def includeMeta(func):
def inner1(*args, **kwargs):
dataResponse = func(*args, **kwargs)

withMeta = {'lastPulled': 'NOW', 'data': dataResponse}
print(withMeta)
return withMeta

return inner1

def __init__(self, config=None, tableName="ingest_staging_table"):
self.config = config
self.dbString = None if not self.config \
else self.config['Database']['DB_CONNECTION_STRING']

self.table = tableName
self.data = None
self.engine = db.create_engine(self.dbString)

@includeMeta
def query(self, queryItems=[], queryfilters=[], limit=None):
items = ', '.join(queryItems)
query = 'SELECT {} FROM {}'.format(items, self.table)
if queryfilters:
filters = ' AND '.join(queryfilters)
query += ' WHERE {}'.format(filters)
if limit:
query += ' LIMIT {}'.format(limit)

df = pd.read_sql_query(query, con=self.engine)
return df.to_dict(orient='records')

def storedProc(self):
pass
48 changes: 48 additions & 0 deletions server/src/services/pinService.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
from .dataService import DataService


class PinService(object):
def __init__(self, config=None, tableName="ingest_staging_table"):
self.dataAccess = DataService(config, tableName)

async def get_base_pins(self,
startDate='',
endDate='',
ncList=[],
requestTypes=[]):
"""
Returns the base pin data given times, ncs, and request filters
{
'LastPulled': 'Timestamp',
'data': [
{
'ncname':'String',
'requesttype':'String',
'srnumber':'String',
'latitude': 'String',
'longitude': 'String',
'address': 'String',
'createddate': 'Timestamp'
}
]
}
"""

items = ['ncname',
'requesttype',
'srnumber',
'latitude',
'longitude',
'address',
'createddate']

ncs = '\'' + '\', \''.join(ncList) + '\''
requests = '\'' + '\', \''.join(requestTypes) + '\''

filters = ['createddate > \'{}\''.format(startDate),
'createddate < \'{}\''.format(endDate),
'ncname IN ({})'.format(ncs),
'requesttype IN ({})'.format(requests)]
result = self.dataAccess.query(items, filters)

return result
18 changes: 9 additions & 9 deletions server/src/services/sqlIngest.py
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
import os
import sqlalchemy as db
import pandas as pd
from configparser import ConfigParser
import time
import numpy as np
import pandas as pd
import sqlalchemy as db
from sodapy import Socrata
import time
import databaseOrm # Contains database specs and field definitions
from .databaseOrm import tableFields, insertFields, readFields # Contains db specs and field definitions
from configparser import ConfigParser


class DataHandler:
Expand All @@ -17,9 +17,9 @@ def __init__(self, config=None, configFilePath=None, separator=','):
self.filePath = None
self.configFilePath = configFilePath
self.separator = separator
self.fields = databaseOrm.tableFields
self.insertParams = databaseOrm.insertFields
self.readParams = databaseOrm.readFields
self.fields = tableFields
self.insertParams = insertFields
self.readParams = readFields
self.dialect = None

def loadConfig(self, configFilePath):
Expand Down Expand Up @@ -177,7 +177,7 @@ def populateFullDatabase(self, yearRange=range(2015, 2021)):
Default operation is to fetch data from 2015-2020
!!! Be aware that each fresh import will wipe the
existing staging table'''
print('Performing fresh ' + self.dialect + ' population from Socrata data sources')
print('Performing {} population from data source'.format(self.dialect))
tableInit = False
globalTimer = time.time()
for y in yearRange:
Expand Down
15 changes: 14 additions & 1 deletion src/components/common/Button.jsx
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ const Button = ({
isStatic,
disabled,
style,
icon,
iconStyle,
}) => {
// Dynamically generates button className from props to comply with Bulma styling modifiers.
const buttonClassName = classNames('button', {
Expand Down Expand Up @@ -53,7 +55,14 @@ const Button = ({
className={buttonClassName}
style={style}
>
{label}
{icon && (
<span className="icon is-small" style={iconStyle}>
<i className={`fas fa-${icon}`} />
</span>
)}
<span>
{label}
</span>
</button>
);
};
Expand All @@ -78,6 +87,8 @@ Button.propTypes = {
isStatic: PropTypes.bool,
disabled: PropTypes.bool,
style: PropTypes.shape({}),
iconStyle: PropTypes.shape({}),
icon: PropTypes.string,
};

Button.defaultProps = {
Expand All @@ -97,4 +108,6 @@ Button.defaultProps = {
isStatic: false,
disabled: false,
style: undefined,
iconStyle: undefined,
icon: undefined,
};
63 changes: 63 additions & 0 deletions src/components/common/CONSTANTS.js
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,69 @@ export const MONTHS = [
'December',
];

export const REQUEST_TYPES = [
{
type: 'Dead Animal',
abbrev: 'DAN',
color: '#4FEFEF',
},
{
type: 'Homeless Encampment',
abbrev: 'HLE',
color: '#ECB800',
},
{
type: 'Single Streetlight',
abbrev: 'SSL',
color: '#AD7B56',
},
{
type: 'Multiple Streetlight',
abbrev: 'MSL',
color: '#F7ADAD',
},
{
type: 'Feedback',
abbrev: 'FBK',
color: '#FFE6B7',
},
{
type: 'Bulky Items',
abbrev: 'BLK',
color: '#FF0000',
},
{
type: 'E-Waste',
abbrev: 'EWT',
color: '#DDEC9F',
},
{
type: 'Metal/Household Appliances',
abbrev: 'MHA',
color: '#B8D0FF',
},
{
type: 'Graffiti',
abbrev: 'GFT',
color: '#2368D0',
},
{
type: 'Illegal Dumping',
abbrev: 'ILD',
color: '#6A8011',
},
{
type: 'Other',
abbrev: 'OTH',
color: '#6D7C93',
},

/*
* Is 'Report Water Waste' still a valid request type?
* If so, we're missing it on the front end mockups.
*/
];

export const REQUESTS = [
'Bulky Items',
'Dead Animal Removal',
Expand Down
7 changes: 6 additions & 1 deletion src/components/common/Checkbox.jsx
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ const Checkbox = ({
name,
value,
checked,
style,
/*
* Props below correspond with Bulma modifiers.
* wikiki.github.io/form/checkradio/
Expand Down Expand Up @@ -48,7 +49,7 @@ const Checkbox = ({
checked={checked}
disabled={disabled}
/>
<label htmlFor={checkboxId}>
<label htmlFor={checkboxId} style={style}>
{label}
</label>
</>
Expand All @@ -64,6 +65,8 @@ Checkbox.propTypes = {
label: PropTypes.string,
name: PropTypes.string,
value: PropTypes.string,
checked: PropTypes.bool,
style: PropTypes.shape({}),
rightToLeft: PropTypes.bool,
color: PropTypes.string,
size: PropTypes.oneOf(['small', '', 'medium', 'large']),
Expand All @@ -80,6 +83,8 @@ Checkbox.defaultProps = {
label: null,
name: null,
value: undefined,
checked: false,
style: undefined,
rightToLeft: false,
color: 'primary',
size: '',
Expand Down
Loading