Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove duplicate color for waterwaste #1226

Closed
wants to merge 3 commits into from
Closed

Remove duplicate color for waterwaste #1226

wants to merge 3 commits into from

Conversation

joshuayhwu
Copy link
Contributor

@joshuayhwu joshuayhwu commented Jun 2, 2022

Fixes #{1223}

Added color "#DCED08" for water waste in DISCRETE_COLOR_MAP of design.py and update the reference file.

  • Update color mapping references seed file
  • Up to date with dev branch
  • All PR Status checks are successful
  • Peer reviewed and approved

@joshuayhwu joshuayhwu linked an issue Jun 2, 2022 that may be closed by this pull request
3 tasks
@joshuayhwu joshuayhwu requested a review from nichhk June 2, 2022 01:30
Copy link
Member

@nichhk nichhk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Josh! One small comment.

Besides the code changes themselves, I think we'll also need to run something to upgrade the database with this new color. More background: https://github.com/hackforla/311-data/tree/dev/server/api/alembic

I'm not sure how to do this right now. Are you interested in looking into this?

server/dash/design.py Outdated Show resolved Hide resolved
@joshuayhwu
Copy link
Contributor Author

joshuayhwu commented Jun 2, 2022

Thanks Josh! One small comment.

Besides the code changes themselves, I think we'll also need to run something to upgrade the database with this new color. More background: https://github.com/hackforla/311-data/tree/dev/server/api/alembic

I'm not sure how to do this right now. Are you interested in looking into this?

I'll take a look - will give you updates later today. Thanks for the suggestion!

@nichhk
Copy link
Member

nichhk commented Jun 4, 2022

FYI, this has been on my mind since I've been looking at the databases. I think we'd either need to manually update this row by SSHing into the database (we currently don't have access to do this right now), or we'd need to write a new alembic script that drops the request_types table and creates it again given the new data.

@joshuayhwu
Copy link
Contributor Author

joshuayhwu commented Jun 5, 2022

FYI, this has been on my mind since I've been looking at the databases. I think we'd either need to manually update this row by SSHing into the database (we currently don't have access to do this right now), or we'd need to write a new alembic script that drops the request_types table and creates it again given the new data.

Thanks for the suggestions Nich! I was curious regarding the point on alembic script - does simply changing the color field in the request_types table not work? See dummy script below:

# flake8: noqa

"""update request type color

Revision ID: N/A
Revises: f605be47c1ec
Create Date: 2022-06-05 10:37:43.545716

"""
from os.path import join, dirname

from alembic import op

# revision identifiers, used by Alembic.
revision = ' N/A'
down_revision = 'f605be47c1ec'
branch_labels = None
depends_on = None

DATA_FOLDER = join(dirname(__file__), '../seeds/')

def upgrade():
    op.execute('UPDATE request_types SET color = "#DCED08" WHERE type_name = "Water Waste"')


def downgrade():
    op.execute('UPDATE request_types SET color = #EDAD08 WHERE type_name = "Water Waste"')

Reference: request_types table definition

P.S.

  • Tried to make this script through alembic but there seems to be some geopandas problem. Currently invesitgating how to get around it.

@nichhk
Copy link
Member

nichhk commented Jun 6, 2022

Yes, I think your solution works too!

@nichhk
Copy link
Member

nichhk commented Jul 20, 2022

@joshuayhwu : is this something we can finish up?

@joshuayhwu
Copy link
Contributor Author

@joshuayhwu : is this something we can finish up?

Isn't this the one that required SSH from Matt because we need to make update with alembic? If we ignore the db part then the rest is indeed completed

@joshuayhwu
Copy link
Contributor Author

Trying to get an empty migration script with alembic revision with docker after using bash entry point - but failed at running docker containers. Seems to be some issue with the async functions. Any thoughts on this?

PS C:\Users\Joshua Wu\Downloads\311_Data\311-data\server\api> docker run -t dup
msgpack not installed, MsgPackSerializer unavailable
INFO:     Started server process [8]
INFO:     Waiting for application startup.
Cannot connect to the database; max retries reached.
ERROR:    Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/starlette/routing.py", line 526, in lifespan
    async for item in self.lifespan_context(app):
  File "/usr/local/lib/python3.7/site-packages/starlette/routing.py", line 467, in default_lifespan
    await self.startup()
  File "/usr/local/lib/python3.7/site-packages/starlette/routing.py", line 502, in startup
    await handler()
  File "/usr/local/lib/python3.7/site-packages/gino_starlette.py", line 183, in startup
    **config["kwargs"],
  File "/usr/local/lib/python3.7/site-packages/gino_starlette.py", line 226, in set_bind
    return await super().set_bind(bind, loop=loop, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/gino/api.py", line 417, in set_bind
    bind = await create_engine(bind, loop=loop, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/gino/strategies.py", line 55, in create
    pool = await dialect.init_pool(u, loop, pool_class=pool_class)
  File "/usr/local/lib/python3.7/site-packages/gino/dialects/asyncpg.py", line 465, in init_pool
    return await pool_class(url, loop, init=self.on_connect(), **self._pool_kwargs)
  File "/usr/local/lib/python3.7/site-packages/gino/dialects/asyncpg.py", line 218, in _init
    self._pool = await asyncpg.create_pool(**args)
  File "/usr/local/lib/python3.7/site-packages/asyncpg/pool.py", line 407, in _async__init__
    await self._initialize()
  File "/usr/local/lib/python3.7/site-packages/asyncpg/pool.py", line 435, in _initialize
    await first_ch.connect()
  File "/usr/local/lib/python3.7/site-packages/asyncpg/pool.py", line 127, in connect
    self._con = await self._pool._get_new_connection()
  File "/usr/local/lib/python3.7/site-packages/asyncpg/pool.py", line 482, in _get_new_connection
    **self._connect_kwargs)
  File "/usr/local/lib/python3.7/site-packages/asyncpg/connection.py", line 1997, in connect
    max_cacheable_statement_size=max_cacheable_statement_size,
  File "/usr/local/lib/python3.7/site-packages/asyncpg/connect_utils.py", line 677, in _connect
    raise last_error
  File "/usr/local/lib/python3.7/site-packages/asyncpg/connect_utils.py", line 668, in _connect
    record_class=record_class,
  File "/usr/local/lib/python3.7/site-packages/asyncpg/connect_utils.py", line 634, in _connect_addr
    tr, pr = await compat.wait_for(connector, timeout=timeout)
  File "/usr/local/lib/python3.7/site-packages/asyncpg/compat.py", line 103, in wait_for
    return await asyncio.wait_for(fut, timeout)
  File "/usr/local/lib/python3.7/asyncio/tasks.py", line 442, in wait_for
    return fut.result()
  File "/usr/local/lib/python3.7/site-packages/asyncpg/connect_utils.py", line 547, in _create_ssl_connection
    host, port)
  File "uvloop/loop.pyx", line 2020, in create_connection
OSError: Multiple exceptions: [Errno 111] Connection refused, [Errno 99] Cannot assign requested address

ERROR:    Application startup failed. Exiting.

@nichhk
Copy link
Member

nichhk commented Jul 22, 2022

Hey @joshuayhwu, have you tried running docker-compose up first from "server/"? Then, you should be able to run the upgrade script using docker-compose run api alembic upgrade head. More info here.

@joshuayhwu
Copy link
Contributor Author

Ah - now I am facing the same docker error I faced before you came where the API where the prefect container is not starting :(

PS C:\Users\Joshua Wu\Downloads\311_Data\311-data\server> docker-compose up
Starting 311-postgres ... done
Starting 311-redis    ... done
Starting 311-api      ... done
Starting 311-prefect  ... done
Attaching to 311-redis, 311-postgres, 311-prefect, 311-api
311-postgres |
311-postgres | PostgreSQL Database directory appears to contain a database; Skipping initialization
311-postgres |
311-postgres | 2022-07-24 19:43:18.312 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
311-postgres | 2022-07-24 19:43:18.312 UTC [1] LOG:  listening on IPv6 address "::", port 5432
311-postgres | 2022-07-24 19:43:18.319 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
311-redis  | 1:C 24 Jul 2022 19:43:18.102 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
311-redis  | 1:C 24 Jul 2022 19:43:18.102 # Redis version=5.0.14, bits=64, commit=00000000, modified=0, pid=1, just started
311-redis  | 1:C 24 Jul 2022 19:43:18.102 # Warning: no config file specified, using the default config. In order to specify a config file 
use redis-server /path/to/redis.conf
311-postgres | 2022-07-24 19:43:18.337 UTC [27] LOG:  database system was shut down at 2022-07-24 19:43:04 UTC
311-redis  | 1:M 24 Jul 2022 19:43:18.103 * Running mode=standalone, port=6379.
311-redis  | 1:M 24 Jul 2022 19:43:18.103 # Server initialized
311-redis  | 1:M 24 Jul 2022 19:43:18.103 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To 
fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
311-postgres | 2022-07-24 19:43:18.347 UTC [1] LOG:  database system is ready to accept connections
311-redis  | 1:M 24 Jul 2022 19:43:18.103 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create 
latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.      
311-redis  | 1:M 24 Jul 2022 19:43:18.104 * DB loaded from disk: 0.001 seconds
311-redis  | 1:M 24 Jul 2022 19:43:18.105 * Ready to accept connections
311-prefect exited with code 0

The issue sort of resolved itself when I started manually starting containers on the docker UI, but after using docker-compose up again it seems to get back to the same buggy state.

@nichhk
Copy link
Member

nichhk commented Jul 26, 2022

Hmm could you explain why it seems like the prefect container is not starting? It seems like it is starting, but it just exits early?

Are you using Docker Desktop? Can you check if there's a container called 311-prefect?

Also, I think you can just directly run docker-compose run api alembic upgrade head after running docker-compose up.

@joshuayhwu
Copy link
Contributor Author

You're right - the container starts and immediately exits.

I am currently using Docker Desktop on my PC with wsl2 backend. There is container called 311-prefect.

I tried running the docker-compose up, but the prefect container exit early again. Then I tried running the docker-compose run api ... line, and return the following. API still unavailable?

PS C:\Users\Joshua Wu\Downloads\311_Data\311-data\server> docker-compose run api alembic upgrade head
Creating 311_data_api_run ... done
msgpack not installed, MsgPackSerializer unavailable
INFO  [alembic.runtime.migration] Engine(postgresql://311_user:***@db:5432/311_db_test)
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.  
INFO  [alembic.runtime.migration] Will assume transactional DDL.

@ryanfchase ryanfchase mentioned this pull request Nov 19, 2023
9 tasks
@Skydodle
Copy link
Member

Closed without merge due to outdated PR & irrelevant to current working branch main.

@Skydodle Skydodle closed this Jan 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Finding new Request type color for multiple streetlights or waterwaste color
3 participants